tokens per second

New open source diffusion LLM hits 2,788 tokens per second on a single RTX 5090

Two thousand seven hundred eighty-eight tokens per second on a single RTX 5090. That's not a benchmark you see every day. The team behind ORL has open-sourced a diffusion language model that learns to refine its own predictions in fewer passes, bypassing the usual trade-off between speed and quality. Their custom inference engine dlmrt handles the entire denoising loop on the GPU, no CPU interruptions. If you're curious how small models can outpace speculative decoding without sacrificing accuracy, the results speak for themselves.