linearly decodable spatial map

Gemma 3 builds spatial maps from text but answers operate on a separate layer

Gemma 3 builds detailed spatial maps from text, yet never consults them when answering questions. An independent preregistered study found the model encodes every object's position, even across combined sentences (R² 0.74), but erasing that map costs no more than erasing random directions. The answers run on a separate 2-D code per sentence instead, acting like a switch rather than a coordinate. Reading answers off the unused map actually outperforms the model on chain questions: 80% versus 73%.