[D] Is LeCun’s $1B seed round the signal that autoregressive LLMs have actually hit a wall for formal reasoning?
Our take
I’m still trying to wrap my head around the Bloomberg news from a couple of weeks ago. A $1 billion seed round is wild enough, but the actual technical bet they are making is what's really keeping me up.
LeCun has been loudly arguing for years that next-token predictors are fundamentally incapable of actual planning. Now, his new shop, Logical Intelligence, is attempting to completely bypass Transformers to generate mathematically verified code using Energy-Based Models. They are essentially treating logical constraints as an energy minimization problem rather than a probabilistic guessing game.
It sounds beautiful in theory for AppSec and critical infrastructure where you absolutely cannot afford a hallucinated library. But practically? We all know how notoriously painful EBMs are to train and stabilize. Mapping continuous energy landscapes to discrete, rigid outputs like code sounds incredibly computationally expensive at inference time.
Are we finally seeing a genuine paradigm shift away from LLMs for rigorous, high-stakes tasks, or is this just a billion-dollar physics experiment that will eventually get beaten by a brute-forced GPT-5 wrapped in a good symbolic solver? Curious to hear from anyone who has actually tried forcing EBMs into discrete generation tasks lately.
[link] [comments]
Read on the original site
Open the publisher's page for the full experience
Related Articles
- We are hitting a wall trying to force transformers to do actual logic [D]seriously losing my mind a bit at work lately. my tech lead keeps telling us to just "refine the system prompt" to stop our production LLM from failing basic multi-step logic tasks. like, no amount of prompt engineering is going to magically turn a probabilistic next-token predictor into a discrete reasoning engine. it's so frustrating watching the entire industry just burn millions on compute trying to brute force logic out of architectures that literally can't do exact math reliably Was watching a Milken Conference panel on deterministic AI earlier this week (mostly cause im trying to keep track of what the hardware guys like ASML are predicting for compute demand) and they got into this whole discussion about Energy-Based Models vs standard LLMs. and honestly it just reinforced my burnout with our current approach. we keep stacking RAG and "chain of thought" hacks like they're a permanent fix for the fact that the underlying model has zero concept of hard constraints or correctness tbh it feels like we're just building increasingly expensive dictionaries and hoping a calculator emerges if we make the book big enough. it's exhausting trying to explain to stakeholders that "scaling" doesn't fix a fundamental lack of reasoning architecture. Im really starting to think we need a total pivot toward something more grounded, otherwise we're just going to keep hitting these weird edge-case failures in production forever. submitted by /u/TheBr14n [link] [comments]
- [D] Probabilistic Neuron Activation in Predictive Coding Algorithm using 1 Bit LLM ArchitectureIf we use Predictive Coding architecture we wouldn't need backpropogation anymore which would work well for a non deterministic system that depends on randomness. Since each neuron just activates or doesn't activate we could use the 1 bit LLM architecture and control the activations with calculated chance. This would increase efficiency and memory used with the proper stochastic hardware. Instead of expecting AI to generate a proper output in 1 attempt we could make it constantly re prompt itself to generate outputs from the input. We could store the memory in Ram and let the AI pull the neccesary information from it to retrain its weights for that specific question until the answer is satisfied. This would also avoid catastrophic forgetting and with the increased efficiency of this proposed architecture could actually be viable. Now I understand that using the modern hardwares for this is inefficient, so why not make a new hardware that computes non diterminestically? If we could create a way of simulating randomness in transistor level and control it then each componant of that hardware can act as a neuron. The physics of the metal itself would activate the neuron or not activate it. Technically we could use heat as a noise source that would allow this, but nobody is attempting it. The closest thing I saw to this idea for hardware is Extropic's TSU, but nobody is really attempting this idea. Why? Why are we wasting resources knowing that the AI Bubble will pop without new advancments in hardware? Scaling clearly isn't working as expected. It's just stagnating. submitted by /u/Sevdat [link] [comments]