self-modulating attention

Looping a small AI model beats a model over six times its size

Looping a small AI model through its own transformer blocks beats a model over six times its size. That's the finding from the Looped Diffusion Transformer paper, and it's a genuinely practical insight. The authors fixed naive looping with deep supervision and self-modulating attention, then proved a 260M-parameter looped model outperforms a 1.7B-parameter one using 4.9x less compute. Depth is chosen at sampling time, letting you trade compute for refinement per image.