Huginn

Looped transformers share weights but memory limits keep them from staying on chip

Shared weights sound like a free lunch, but memory bandwidth has other plans. A chain of eight matmuls with one 72 MiB weight ran 24% faster than eight identical copies, reading almost nothing from HBM. Scale that weight to 288 MiB, larger than L2 cache, and the advantage vanishes. Eight iterations of a full Huginn Core took eight times as long and read eight times the data. Sharing saves storage, but the GPU still fetches those weights from HBM every single time.