cross-attention
cross-attention at Beyond Market Intelligence is a file of 3 stories. The newest of them: “One transformer model replaced 15 specialized generators in Yandex Music's A/B test”, “Transforming LLM Efficiency: A DSP-Inspired Semantic Vocoder Approach”, and “What one GPU reveals about training a text-to-image diffusion transformer”. One transformer replaced 15 specialized generators in Yandex Music's production stack. The idea of treating text like an audio signal is a sharp pivot. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work… The list below is every cross-attention story on Beyond Market Intelligence, newest first.
One transformer model replaced 15 specialized generators in Yandex Music's A/B test
One transformer replaced 15 specialized generators in Yandex Music's production stack. That's the kind of result that makes you rethink how we build recommenders. Sona, their single-model generative recommender, achieved a +4.53% lift in Active Users and +6.30% in Total Listening Time on smart speakers. History Compression cuts inference cost roughly in half while retaining most of the quality. For deeper context on how models manage long contexts efficiently, see our article on smarter token compression techniques.
Transforming LLM Efficiency: A DSP-Inspired Semantic Vocoder Approach
The idea of treating text like an audio signal is a sharp pivot. By splitting generation into a slow-rate planner and a fast-rate vocoder, this work tackles the inefficiency where predicting every token costs the same compute, regardless of its weight in meaning. The TinyStories results are compelling, showing faster convergence. However, the conditioning over-reliance is a real bottleneck. When the base model leans on semantic hashes, grammar suffers. This is a thoughtful exploration of a genuine problem.
What one GPU reveals about training a text-to-image diffusion transformer
Training a 210M text-to-image DiT from scratch on a single GPU is a bold experiment, and the numbers are worth reading closely. The finding that two learned null attention slots absorb roughly 90% of cross-attention mass at mid-noise is striking, especially since it quietly sidelines the usual EOS sink. That is the kind of concrete detail most write-ups skip. Flow-matching loss tracks training health, not output quality.