solve rate
2 stories filed under solve rate on Beyond Market Intelligence. The newest of them: “When eight systems tie for first, the real story is how close they all are” and “Testing 49 LLMs on Nonograms Reveals Where AI Logic Still Falls Short”. Eight systems share the top score on the Agent Memory Leaderboard's coding board, but the real insight is how tightly clustered the entire field has become. Nonograms look like simple logic puzzles, but they've exposed a sharp limit in how today's best AI models reason. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work… The list below is every solve rate story on Beyond Market Intelligence, newest first.
When eight systems tie for first, the real story is how close they all are
Eight systems share the top score on the Agent Memory Leaderboard's coding board, but the real insight is how tightly clustered the entire field has become. With 43 methods spanning just 6.67 percentage points, a single task moves a contender by 0.67 points. That convergence tells us different designs land in the same productive neighborhood. For anyone reporting scores this close, raw counts or tier groupings often resonate more than strict ranks.
Testing 49 LLMs on Nonograms Reveals Where AI Logic Still Falls Short
Nonograms look like simple logic puzzles, but they've exposed a sharp limit in how today's best AI models reason. Nonobench tested 49 LLMs on these grid puzzles, giving each model the clues once and letting it fill the full grid in one attempt, no tools, no retries. Solve rates dropped fast as grids grew: 85% on 5x5 puzzles fell to just 20% on 15x15. Even GPT-6 Astra solved all 30 standard puzzles, but on hard mode, Claude Opus 5.