The evaluation resolution has been shown to have a significant impact on the identification of the "learning rule" that exhibits the most brain-like characteristics at V1. [R]

The recent preprint by Leutwyler, available on arXiv, throws a significant wrench into a frequently cited observation in the burgeoning field of computational neuroscience: the surprising ability of untrained convolutional neural networks (CNNs) to rival, and sometimes even surpass, backpropagation-trained CNNs in their representational similarity to the early visual cortex (V1). This finding, often touted as evidence for the brain's inherent efficiency or the limitations of backpropagation, is now revealed to be largely an artifact of evaluation resolution. This challenges existing interpretations of model-brain comparisons and highlights a crucial methodological consideration. The work builds on previous explorations of weight-space perception, such as those discussed in How much of the weight-space perception gap is actually symmetry? Evidence from ~1.8M fitted SIRENs, which also investigates nuances in how models learn and represent information.

Leutwyler's study meticulously examines the impact of evaluation resolution on the observed alignment between untrained and trained CNNs and the V1. By systematically varying the resolution at which a small CNN, trained on a subset of CIFAR-10, is evaluated against fMRI data from human subjects, the author demonstrates a monotonic widening of the gap between the two. This means the initial, often-cited, observation of untrained networks matching or exceeding trained ones disappears as the evaluation resolution increases. Crucially, the finding isn't unique to untrained networks; similarly trained models, including ResNet-50 and Swin-Tiny, also peak at low resolution, ruling out a training/evaluation mismatch as the sole cause. This rigorous approach, which systematically eliminates alternative explanations like low-level image structure or convergence toward global brightness, strengthens the conclusion that the initial observation was indeed an artifact. The work’s meticulous troubleshooting, including identifying and correcting a batch-norm evaluation-mode bug in previous preprints, underscores the importance of careful validation in this rapidly evolving field. The methodology employed also resonates with efforts to build more efficient spiking language models, as explored in Continued development of the model based on the SSN, demonstrating a broader interest in optimizing neural network architectures.

The implications of this work extend beyond simply debunking a specific finding. It serves as a potent reminder of the critical role of methodological rigor in model-brain comparisons. The tendency to draw sweeping conclusions about the brain’s computational principles from the behavior of relatively simple neural networks is tempered by this demonstration of how easily spurious correlations can arise from subtle experimental choices. While the study acknowledges a separate, somewhat unsettling finding regarding the limitations of current comparison styles – that even a single scalar luminance value can unexpectedly align with V1 responses – the primary takeaway is a call for greater caution and more nuanced interpretation of these comparisons. The fact that learning *does* leave a mark on representations, just not necessarily in the areas typically scrutinized, is a valuable insight in itself.

Looking forward, this research compels us to re-evaluate existing literature claiming the superiority of untrained networks and encourages more careful consideration of evaluation parameters in future studies. The author's suggestion of exploring receptive-field matching, referencing Laskar et al. (2018), offers a promising avenue for further investigation. The question remains: if the readily observed similarities between untrained networks and V1 are largely an artifact of evaluation resolution, what *does* training truly contribute to the emergence of brain-like representations? And perhaps more importantly, what other subtle methodological choices are shaping our understanding of the brain’s computational mechanisms?