product-hash routing

Explore how learned routing simplifies sparse attention for modern data workflows.

Monodratic takes a fresh angle on sparse attention by learning where to look, not just when. An independent researcher, its product-hash routing assigns source blocks to bounded posting lists after RoPE, letting queries probe product addresses, rerank candidates, and run exact causal softmax over a fixed remote set plus local blocks. The results are compelling: 99.35% mean accuracy on associative recall, with all 768/768 recovered when forcing the target block under the same budget. It is synthetic and PyTorch-based, yet the scaling exponent near 1.