evals (evaluations)
evals (evaluations) on Beyond Market Intelligence: a running collection of 2 stories we have gathered and hand-picked because they are worth your time. Every post here touches on evals (evaluations) in some way — the news, the analysis, the deep dives, and the occasional surprise find. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work with data. New stories are added to this page as we find them, so check back if you want to keep up with what is happening around evals (evaluations), or subscribe to the RSS feed to get them as soon as they are published. Browse the collection below, or head back to the homepage to see everything Beyond Market Intelligence is covering right now.
Intuit scrapped its own AI agent architecture twice in four months. At VB Transform 2026, its AI VP called that the fast path
Intuit’s journey with agentic AI highlights a crucial truth: rapid iteration is essential. The company initially built a fleet of specialist agents, then pivoted to an orchestration layer, only to rebuild the entire architecture within 60 days after encountering limitations in context retention. This experience, shared at VB Transform 2026, underscores the challenges of scaling agent-based systems and the importance of prioritizing customer outcomes. As Brex demonstrated, observing agent behavior can be a powerful tool in policy creation.
QCon AI Boston: Production AI Moves Beyond Prompts to Platforms, Harnesses, and Evals
QCon AI Boston 2026 addressed a critical shift: Production AI moving beyond initial prompt-based exploration to robust platforms, harnessed agents, and rigorous evaluations. The conference centered on the operational challenges of deploying AI agents at scale, emphasizing improved context management and robust security measures—including a "harness" approach to contain agent access. Attendees explored a comprehensive engineering model for AI, recognizing the need for mature infrastructure. For further insight into agent security concerns, see our recent article, "The agent security gap."