Google Stax Aims to Make AI Model Evaluation Accessible for Developers
Google Stax is a framework designed to replace subjective evaluations of AI models with an objective, data-driven, and repeatable process for measuring model output quality. Google says this will allow AI developers to tailor the evaluation process to their specific use cases rather than relying on generic benchmarks.
Dreamer 4: Learning to Achieve Goals from Offline Data Through Imagination Training
Researchers from DeepMind have described a new approach for teaching intelligent agents to solve complex, long-term tasks by training them exclusively on video footage rather than through direct interaction with the environment. Their new agent, called Dreamer 4, demonstrated the ability to mine diamonds playing Minecraft after being trained on videos, without ever actually playing the game.
The New Data Commons MCP Server Unlocks a Wealth of Public Datasets for AI Developers
Google has recently introduced the Data Commons Model Context Protocol (MCP) Server, a tool that enables AI developers and researchers to easily access the public dataset collection available through Data Commons.
OpenAI Study Investigates the Causes of LLM Hallucinations and Potential Solutions
In a recent research paper, OpenAI suggested that the tendency of LLMs to hallucinate stems from the way standard training and evaluation methods reward guessing over acknowledging uncertainty. According to the study, this insight could pave the way for new techniques to reduce hallucinations and build more trustworthy AI systems, but not all agree on what hallucinations are in the first place.