Does telling an LLM to "be concise" actually save you money? We measured it across 9 models. Compressing the output can save you money and keep accuracy, compressing the input prompt does not. [R]

The burgeoning world of Large Language Models (LLMs) presents a fascinating paradox: we wield powerful tools increasingly shrouded in opacity, and controlling costs becomes a crucial, yet often elusive, goal. Recent research, detailed in a new paper and accompanying code, sheds light on a simple, yet profound, truth: instructing an LLM to be concise yields significant cost savings, while attempting to constrain its input proves counterproductive. This finding arrives at a timely moment, particularly as providers like Claude introduce "concise output styles," The LLM Judge That Kept Agreeing With Itself— highlighting the ongoing challenges in trusting and managing these complex models— and the broader effort to optimize AI workflows. The study’s meticulous testing across nine models, five datasets, and eleven languages underscores the universality of this principle, moving beyond anecdotal observations to offer concrete data-driven insights for practitioners.

The core takeaway—that shrinking output tokens is demonstrably cheaper than shrinking input tokens—challenges conventional prompting strategies. It’s intuitive to think that a shorter, more focused prompt would lead to a more efficient response, but the research reveals the opposite: LLMs often compensate for reduced input by generating longer, more verbose outputs to fill the perceived information gap. This phenomenon, coupled with the fact that output tokens typically cost more than input tokens, creates a significant financial disincentive for this approach. The study's finding that output token compression can save up to 3x on API costs is particularly compelling, especially considering the ongoing search for cost-effective solutions in an increasingly competitive AI landscape. Relatedly, TrueFoundry's open source AI agent harness, TrueForge, TrueFoundry's open source AI agent harness TrueForge boasts 30%-75% cheaper task completion than Claude Managed Agents, is a testament to the growing desire for optimized and cheaper AI workflows.

Beyond the immediate cost implications, this research raises interesting questions about the nature of LLM reasoning. The observation that shortened outputs sometimes deviate from the model’s unconstrained reasoning path, though often still yielding the correct answer, highlights a potential trade-off between efficiency and transparency. While the pragmatic user might prioritize accuracy over a complete understanding of the model’s thought process, this disconnect underscores the “black box” nature of LLMs and the inherent limitations of our control. The study rightly points out that the lack of clarity around how providers are charging for "concise output styles" adds another layer of complexity, further incentivizing users to directly manage prompting strategies via APIs for maximum cost control. This reinforces the need for greater transparency and granular control over LLM behavior.

Ultimately, this research serves as a valuable reminder that optimizing LLM usage isn’t about complex architectures or groundbreaking algorithms, but rather about understanding the fundamental economics of token consumption. As AI continues to permeate various aspects of our lives, the ability to efficiently manage these costs will become increasingly critical. What remains to be seen is how providers will adapt their pricing models and prompting interfaces to reflect these findings, and whether the drive for conciseness will ultimately lead to more transparent and controllable LLM systems. Will we see a future where LLM providers offer granular control over output length and style, allowing users to fine-tune both cost and accuracy?