Reinforcement learning from verifiable rewards (RLVR) is producing a pattern that looks like a trade-off, but a new theoretical paper proves it is actually a structural constraint. Every gain in pass@1 comes with a hidden cost: a strictly lower success rate on some prompts. This is not a bug that more training will fix. It is a mathematical consequence of how RLVR tilts the base policy, and it forces a crossover where the base model catches up at larger sampling budgets. The finding reframes what we thought we knew about model improvement, and it connects directly to discussions about Why AI labs are betting on simulated environments for smarter models and the practical limits of Why one successful agent run doesn't mean the database agrees.
The paper formalizes RLVR as an exponential tilt of the base policy over shared solution strategies. Under exclusive specialization, where each prompt has its own best strategy, any update that lifts pass@1 must lower success on at least one other prompt. The authors prove this with a Stiemke duality condition: there is a strictly positive price on prompts that equalizes strategy fitness. Even being the best strategy on every prompt is not enough to escape the trap, a 3x3 counterexample shows a free lunch is provably impossible. The practical implication is immediate: if you are tracking pass@1 gains in your eval logs, you already have the data to compute the crossing count, the bound, and the worst pass@k loss. On published GRPO numbers for Omni-MATH, the method finds exactly one crossover between k=64 and k=256.
This is not a reason to abandon RLVR. It is a reason to understand its frontier. The paper shows that with a per-prompt floor, the constrained update traces the optimal trade-off between pass@1 and pass@k. Forbidding any per-prompt loss removes the crossover but also removes the entire pass@1 gain. The trade is structural, not accidental. This mirrors the insight from Rethinking the compute demands behind LLM post-training research: the resources you allocate to training must account for these hidden costs, or you will misinterpret your results.
The paper also derives an entropy-performance law with an identifiable exponent, tying the tilt directly to model behavior at scale. The authors expect pushback on the shared-strategy abstraction and the coverage assumption that forces the crossover, but they provide the regime where the assumption fails. The open question is whether anyone can produce per-prompt logs showing two or more crossings. That would be the signature of RL reordering prompt difficulty, and it would push the theory into new territory. For now, the takeaway is clear: do not treat pass@1 gains as free. Every improvement has a price, and the price is a prompt left behind. Watch for the crossover, and measure it directly from your own logs.