A Billion-Dollar Bet That Autoregressive Models Can't Plan

LeCun's recent $1 billion seed round for his startup, Logical Intelligence, raises intriguing questions about the future of autoregressive language models (LLMs) in formal reasoning.

3 min readMachine Learning

Yann LeCun's $1 billion bet on Energy-Based Models is either a stroke of genius or a very expensive physics experiment. We lean toward the latter, but the ambition deserves a closer look.

The core claim is straightforward: autoregressive models, for all their fluency, cannot truly plan. They guess the next token based on probability, not logic. LeCun has been making this argument for years, and he's not wrong. When you need mathematically verified code, for application security, for critical infrastructure, for systems where a hallucinated library could cause real harm, probabilistic guessing is a liability. Logical Intelligence wants to treat constraints as an energy minimization problem. Instead of asking "what word comes next most often," the model asks "which output satisfies all the rules with the least energy." In theory, that yields provably correct code.

In practice, Energy-Based Models are notoriously difficult to train. They require stabilizing a continuous energy landscape, and mapping that landscape onto discrete, rigid outputs like code is computationally expensive at inference time. The elegance of the theory collides with the messiness of engineering. We have seen this pattern before: a beautiful mathematical idea that works in a lab but buckles under real-world scale and latency demands.

What does this mean for you, the developer or security engineer evaluating tools today? It means you should watch this space, but not hold your breath. If Logical Intelligence succeeds, it could offer a genuine alternative to LLMs for high-stakes tasks where correctness is non-negotiable. But the more likely near-term path is that a sufficiently large autoregressive model, wrapped in a good symbolic solver and brute-forced through enough compute, will match or beat EBM performance for most practical applications. The $1 billion is a bet on a paradigm shift, but paradigms rarely shift on a single funding round. They shift when the engineering catches up to the theory. That day is not here yet.

From Machine Learning

I’m still trying to wrap my head around the Bloomberg news from a couple of weeks ago. A $1 billion seed round is wild enough, but the actual technical bet they are making is what's really keeping me up.

LeCun has been loudly arguing for years that next-token predictors are fundamentally incapable of actual planning. Now, his new shop, Logical Intelligence, is attempting to completely bypass Transformers to generate mathematically verified code using Energy-Based Models. They are essentially treating logical constraints as an energy minimization problem rather than a probabilistic guessing game.

Read the original at Machine Learning