I tried to make a real fly connectome learn to play Pong. It didn't — and auditing why turned out to be way more interesting than if it had worked [p]
Our take
The recent flurry of excitement surrounding the MaleCNS v1.0 connectome release – those captivating clips of a fly brain playing Doom, Minecraft, and Beat Saber – has understandably captured the imagination. These demonstrations, showcasing a reconstruction of 166,000 neurons based on real electron microscopy, represent a significant achievement in computational neuroscience. However, as Jonatas Peraza’s recent investigation reveals, the initial enthusiasm needs a healthy dose of critical examination. It's crucial to discern genuine emergent behavior from clever engineering, particularly as the proliferation of connected devices raises privacy concerns, as highlighted in "Comcast adds motion sensing to millions of its newer routers, with a privacy catch." Similarly, the pursuit of increasingly complex AI simulations echoes the challenges of understanding and mitigating bias in machine learning models, a topic explored in The Ethics of AI: Navigating Bias and Fairness. Peraza’s decision to test the MaleCNS model with the deceptively simple game of Pong—a "least forgiving test bed" as he puts it—was a brilliant choice, exposing critical flaws that a more forgiving environment would have masked.
Peraza's meticulous audit of the system revealed a series of significant issues, far beyond a simple failure to learn. From a regex bug silently zeroing out neuron populations to a fundamental disconnect between photoreceptors and motion detectors, the investigation provided a compelling case study in the complexities of accurately modeling biological systems. The fact that even the viral projects – Doom, Minecraft, and Beat Saber – exhibited similar limitations, relying on hand-injected behaviors or overfitted data, underscores a broader challenge within the field. It’s tempting to celebrate the spectacle of a fly brain “playing” a video game, but Peraza's work reminds us that true understanding requires rigorous validation and a willingness to confront uncomfortable truths about the accuracy of our models. This isn't about dismissing the MaleCNS project; rather, it’s a call for more critical engagement and a deeper focus on the underlying biological mechanisms.
The implications of this work extend beyond the specific case of the fly connectome. It highlights the crucial distinction between simulation and emulation. While simulations can produce impressive results, they often lack the fidelity required to truly understand biological processes. Peraza’s iterative process, initially attempting to replicate a known pathway and then pivoting to a different biological hypothesis based on the data, exemplifies a scientific approach that prioritizes accuracy over superficial success. The identification of motor neurons with zero synaptic connections from sensory pathways, a purely coincidental artifact of the model's construction, is a stark reminder of the potential for subtle errors to propagate through complex systems. This underscores the need for more robust validation methods and a greater emphasis on biological realism in computational neuroscience. It's a valuable lesson for anyone attempting to model complex systems, whether they are brains, ecosystems, or economies.
Ultimately, Peraza’s findings suggest that the real value of these connectome projects may not lie in their ability to mimic human behavior but in their potential to illuminate the intricacies of biological circuits. His call to focus on the central complex and steering circuits, rather than circumventing them, is a particularly insightful suggestion. As we move forward, it will be fascinating to see how researchers respond to this challenge, and whether the pursuit of understanding the *why* behind circuit failures proves to be more rewarding than celebrating the fleeting successes of emergent behavior. What new analytical techniques will emerge to rigorously test these increasingly complex models, and will we witness a shift in focus from dazzling demonstrations to a deeper appreciation for the subtle nuances of biological computation?
You've probably seen the fly-brain-plays-Doom / Minecraft / Beat Saber clips going around this week, from the new MaleCNS v1.0 connectome release (166k neurons, real EM reconstruction, not a toy model).
Cool clips. Nobody seemed to be checking whether any of it works though, versus just producing motion in a game engine generous enough to make anything look alive. So I picked the least forgiving test bed around: Pong, one binary hit or miss signal measured every frame, nowhere to hide a null result, and tried to get a small real subgraph of the connectome to track the ball via dopamine-style plasticity.
Short version: it didn't learn. Working out why took auditing individual synapses, and it turned into a decent case study in a circuit not working being more informative than it working:
- Fixed a neuPrint regex bug that silently zeroed out two entire neuron populations (full-match vs substring semantics, not obvious from the docs).
- Found the original neuron selection had no path at all from photoreceptors to anything else. Real photoreceptors don't synapse directly onto motion detectors, there's a whole intermediate layer missing.
- Got a working pipeline, turned learning on vs off, and got bit-for-bit identical results in both conditions across multiple seeds, even though the weights were verifiably changing under the hood.
- Traced that to half of the 4 available motor neurons having zero synapses from any sensory pathway in the model. Not weak signal, zero. They'd been assigned to the "paddle down" group by array index, purely by coincidence, and could never have fired no matter what the learning rule did.
- Rebuilt the circuit around a better biological hypothesis (swapped a threat-detection pathway for one tied to visual target tracking during courtship pursuit), got that literal hypothesis refuted by the data, then followed the trail to a different descending neuron that actually connected end to end.
- Finally got learning-on vs learning-off to diverge for the first time, except the effect looks like the learning rule quieting the whole system down rather than anything resembling skill improvement (misses outnumber hits, so punishment dominates and shrinks the motor response).
Then I checked whether the bigger, viral projects had actually solved this. They hadn't either: the Doom project's own repo says it failed its own validation gates after six iterations, the Minecraft mod's own limitations section admits the real motion-detection pathway stays silent and the escape and foraging behaviors are hand-injected or reflex-layer fallbacks rather than emergent, and the Beat Saber creator's own replies admit it's overfit to one track with replay data mixed into the input.
Full writeup with the gory audit details, the connectivity numbers, and the comparison to the other projects is here: https://jonatasperaza.medium.com/i-made-a-real-fly-brain-play-pong-it-didnt-learn-and-that-s-the-interesting-part-80b8560695fe.
Curious if anyone here has poked at MaleCNS v1.0 directly and hit similar walls, especially around the central complex and steering circuits. That seems like the obvious next thing to simulate properly instead of routing around it.
[link] [comments]
Read on the original site
Open the publisher's page for the full experience