Beyond Market Intelligence/HBM traffic

HBM traffic

One story filed under HBM traffic on Beyond Market Intelligence. The newest of them: “Looped transformers share weights but memory limits keep them from staying on chip”. Shared weights sound like a free lunch, but memory bandwidth has other plans. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work… The list below is every HBM traffic story on Beyond Market Intelligence, newest first.

Looped transformers share weights but memory limits keep them from staying on chip
Machine Learning

Looped transformers share weights but memory limits keep them from staying on chip

Shared weights sound like a free lunch, but memory bandwidth has other plans. A chain of eight matmuls with one 72 MiB weight ran 24% faster than eight identical copies, reading almost nothing from HBM. Scale that weight to 288 MiB, larger than L2 cache, and the advantage vanishes. Eight iterations of a full Huginn Core took eight times as long and read eight times the data. Sharing saves storage, but the GPU still fetches those weights from HBM every single time.