MMLU

MMLU at Beyond Market Intelligence is a file of one story. The newest of them: “Three Small AI Models Falter When Told Their Answers Are Wrong”. A simple "that's wrong" does more damage to small AI models than you might expect. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work… The list below is every MMLU story on Beyond Market Intelligence, newest first.

Towards Data Science

Three Small AI Models Falter When Told Their Answers Are Wrong

A simple "that's wrong" does more damage to small AI models than you might expect. In a recent test, three open models faced 150 MMLU questions each with five styles of pushback. Naming a wrong option was nearly fatal for their answers. Qwen2.5-1.5B flipped its correct responses two out of three times when challenged with "are you sure?" This fragility raises real questions about how we trust these systems. For deeper context, see our piece on matching data retrieval to AI coding tasks.