Science moves forward when someone asks the question the field skipped. That is exactly what happened here. A solo, self-funded researcher took TRIBE v2, a high-profile multimodal model that predicts fMRI from video, audio, and text, and ran the one test its own paper never did: whether it reproduces the brain's classic topographic maps. The results are a masterclass in why validation matters, and they reveal a model that gets motion selectivity spectacularly right while failing entirely on retinotopy and tonotopy. For anyone building or using brain-encoding models, this is not an edge case. It is a direct challenge to how we evaluate whether these systems actually understand the data they claim to predict.
The model's authors admitted in their methods that video features are spatially averaged before prediction, which should destroy position information. Yet nobody checked. The auditor ran the standard mapping stimuli, checkerboard wedges, hemifield flicker, moving dots, tone sweeps, and fitted the same models used on real fMRI data. Retinotopy was flat (d=0.05, not significant). Tonotopy was worse (ρ=-0.07, not significant). But motion selectivity landed precisely in area MT+ with a d=4.31 and 52x enrichment over chance. That is a striking dissociation: the model encodes *what* is moving but not *where* it is in space or *what frequency* a sound carries. And the eccentricity result is the real warning. It looked null until three separate stimulus-geometry controls flipped it to significant and "correct", a textbook image-statistics confound, not a real positional code. That is the kind of artifact that can mislead an entire research line if no one looks.
This is not an isolated concern. The field of brain-encoding models has grown fast, and evaluation resolution often stays coarse. A 218-million-parameter model mapping brain activity from films may capture broad responses without ever testing whether it preserves the spatial organization that defines real sensory cortex. Similarly, how we evaluate brain-like learning rules can reshape what we think the model has learned. The lesson here is concrete: if your model spatially averages video features, you cannot claim it predicts retinotopy. And if you never check, you are publishing a black box that passes plausibility checks but fails fundamental neuroscience.
The practical takeaway is direct. Any group training or publishing a brain-encoding model should run these topographic mapping stimuli before claiming their model captures spatial or frequency representations. The auditor has released code and a paper. There is no excuse to skip this step. The open question is whether this WHERE-versus-WHAT dissociation generalizes to other video models and multimodal architectures. If it does, the field may need to revisit what "predicting brain activity" actually means, and whether we have been measuring correlation with image statistics instead of neural computation. That is a question worth answering before the next model ships without a map.