SWE-bench
SWE-bench on Beyond Market Intelligence: a running collection of 3 stories we have gathered and hand-picked because they are worth your time. Every post here touches on swe-bench in some way — the news, the analysis, the deep dives, and the occasional surprise find. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work with data. New stories are added to this page as we find them, so check back if you want to keep up with what is happening around swe-bench, or subscribe to the RSS feed to get them as soon as they are published. Browse the collection below, or head back to the homepage to see everything Beyond Market Intelligence is covering right now.
Top 10 Open-Source Benchmarks for AI Coding Agents in 2026
Evaluating AI coding agents demands rigorous benchmarks. In 2026, several open-source options will be essential for developers. Explore the top 10, including SWE-bench, Terminal-Bench, SlopCodeBench, and ProgramBench, alongside emerging contenders. These benchmarks offer critical insight into agent capabilities across diverse coding tasks. For deeper context on related AI research and development, see our discussion thread for EMNLP 2026 Notifications/Results. Discover how these tools empower informed decisions in the rapidly evolving landscape of AI-powered software engineering.
Poolside drops Laguna S 2.1, an open-weight coding model that beats rivals 10x its size
Poolside's Laguna S 2.1 introduces a compelling new option in the open-weight coding model landscape. This 118-billion-parameter system, activating just 8 billion parameters per token, impressively matches or surpasses models many times its size on agentic coding tasks, achieving top scores on benchmarks like Terminal-Bench 2.1. With a permissive OpenMDW-1.1 license and broad ecosystem support, Laguna S 2.1 represents a strategic move to empower Western users with trustworthy, self-hostable AI.
Training a harness for model-agnostic and task-environment-agnostic capability improvements with PyTorch-like framework [P]
Introducing Harness Training, a novel framework designed to unlock general capability improvements across diverse language models. Developed with a PyTorch-like structure, this project reframes "Agent-driven Self-improving Harness" to a more streamlined approach: training a harness once against a specific task environment, then leveraging it to evaluate *any* model on *any* new environment. Initial results, detailed in a recent blog post (https://www.henrypan.com/blog/2026-07-18-harness-training), demonstrate significant gains on benchmarks like Terminal Bench 2.0, as explored in a related article discussing agent evaluation challenges.