EMA teacher
2 stories filed under EMA teacher on Beyond Market Intelligence. The newest of them: “Scaling AI training with lightweight adapters instead of doubling models” and “Exploring alternative teacher models for stable self-distillation”. Training large models with self-distillation usually means doubling up, teacher and student both on GPU, eating memory and slowing everything down. Traditional self-distillation leans on an EMA teacher to keep student training stable. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work… The list below is every EMA teacher story on Beyond Market Intelligence, newest first.
Scaling AI training with lightweight adapters instead of doubling models
Training large models with self-distillation usually means doubling up, teacher and student both on GPU, eating memory and slowing everything down. A smarter approach? Share the frozen backbone and swap lightweight LoRA adapters instead. One adapter trains as the student; another tracks its EMA as the teacher. It took some careful engineering to make this work cleanly under TRL and ZeRO-3, but the results speak for themselves. The repo is live, tested on Qwen3.5, and ready to explore.
Exploring alternative teacher models for stable self-distillation
Traditional self-distillation leans on an EMA teacher to keep student training stable. But that's not the only path worth exploring. The community's search for alternative teacher models, ones that evolve alongside the student without letting it drift, deserves attention. It's a practical question with real implications for how we build more robust systems. If you're curious about where AI methods hit their limits, our piece on testing a browser agent's blind spots offers a parallel look at stability.