We are hitting a wall trying to force transformers to do actual logic [D]

seriously losing my mind a bit at work lately. my tech lead keeps telling us to just "refine the system prompt" to stop our production LLM from failing basic multi-step logic tasks. like, no amount of prompt engineering is going to magically turn a probabilistic next-token predictor into a discrete reasoning engine. it's so frustrating watching the entire industry just burn millions on compute trying to brute force logic out of architectures that literally can't do exact math reliably

Was watching a Milken Conference panel on deterministic AI earlier this week (mostly cause im trying to keep track of what the hardware guys like ASML are predicting for compute demand) and they got into this whole discussion about Energy-Based Models vs standard LLMs. and honestly it just reinforced my burnout with our current approach. we keep stacking RAG and "chain of thought" hacks like they're a permanent fix for the fact that the underlying model has zero concept of hard constraints or correctness

tbh it feels like we're just building increasingly expensive dictionaries and hoping a calculator emerges if we make the book big enough. it's exhausting trying to explain to stakeholders that "scaling" doesn't fix a fundamental lack of reasoning architecture. Im really starting to think we need a total pivot toward something more grounded, otherwise we're just going to keep hitting these weird edge-case failures in production forever.