Beyond Market Intelligence/long-horizon agent evals

long-horizon agent evals

Beyond Market Intelligence keeps long-horizon agent evals in one place: one story so far. The section currently leads with “How frontier models master the real-world grind of materials research”. Seven frontier AI models, across twelve runs, scored between 97.8 and 100 on a PhD-level materials research test that included unit traps, hidden crystal structures, and a manuscript with planted errors. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work… The list below is every long-horizon agent evals story on Beyond Market Intelligence, newest first.

How frontier models master the real-world grind of materials research
Machine Learning

How frontier models master the real-world grind of materials research

Seven frontier AI models, across twelve runs, scored between 97.8 and 100 on a PhD-level materials research test that included unit traps, hidden crystal structures, and a manuscript with planted errors. Every model caught the false premise and the fabricated paper. Three runs even found a real mistake the test designer hadn't planted. This is not hype, it's measurement. Bounded, procedure-driven work is saturated. The real frontier lies in long-horizon, open-ended tasks.