Writer's AI harness cuts token spend nearly 40% — without sacrificing accuracy
Our take
The current explosion of enterprise AI adoption is hitting a predictable snag: the ROI paradox. Initial product experiments often thrive on brute-force compute, feeding ever-larger context windows to the most powerful foundation models. However, the costs associated with this “tokenmaxxing” approach quickly become unsustainable when scaling to production environments. This reality is neatly illustrated by a new paper from Writer, which offers a pragmatic solution accessible to engineering teams: optimizing the AI harness, the orchestration layer that sits between the model and the application. The research demonstrates significant cost reductions—up to 61%—without sacrificing accuracy, suggesting a shift in focus from simply throwing resources at the problem to strategically engineering the system around it. This mirrors the findings in "Safety guardrails blocked Hugging Face's defenders, not the attacker, when an AI agent breached its systems," where inadequate system design, not solely model vulnerability, contributed to a security incident, highlighting the need for robust operational infrastructure.
The core of the issue, as Writer’s CTO Waseem AlShikh points out, is that teams often prioritize short-term fixes—stuffing more context into the window—over building elegant workflows. This approach masks underlying inefficiencies, particularly as token prices remain relatively high compared to the cost savings of improved architecture. The paper's focus on the harness—system prompt caching, interaction history compaction, tool management, retrieval strategies, and error management—represents a crucial shift in perspective. Rather than treating the harness as disposable “glue code,” the researchers argue it should be considered a primary software artifact, demanding rigorous design and testing. This echoes the principles explored in "Loop Engineering with Adaptive Parsing in Action: Parsing Flat Tables with Azure and Figures with a Vision LLM," which showcases how intelligently structured loops and agent interactions can significantly enhance performance and efficiency. The implications are profound, essentially reframing the “own versus rent” decision for enterprises, as control over the harness translates directly into control over unit economics.
The experimental results are compelling: a 41% reduction in cost per task and a 38% decrease in tokens per task, alongside steady quality metrics. Critically, the study also identifies limitations, noting that sub-agent orchestration—delegating tasks to specialized sub-agents—is most effective with stronger models like Writer’s own Palmyra X6 and Claude Sonnet 4.6. This highlights a key tradeoff: while harness optimization is valuable across the board, its full potential remains tied to model capabilities. The actionable takeaways presented—implementing "Two-Zone Prompts," context offloading, and establishing robust error management—provide a practical roadmap for engineering teams. However, the authors caution against unnecessary complexity, emphasizing that optimizing the harness isn't "free" and requires careful consideration of the model’s limitations.
Ultimately, this research signals a maturing of the enterprise AI landscape. The era of indiscriminate tokenmaxxing is drawing to a close, replaced by a focus on architectural efficiency and governance. As foundation models continue to evolve, the harness will likely transition from a compensatory layer to an enforcement mechanism, defining the boundaries of what the model can access and spend. The question now becomes: how will enterprises adapt their development processes and organizational structures to prioritize harness optimization and ensure that AI investments deliver sustainable ROI, and will those who fail to do so find themselves increasingly priced out of the AI revolution?
Enterprise AI is facing an ROI paradox. While throwing more compute at the strongest foundation model works well in product experiments, the costs become unbearable when the product is deployed in production.
A new paper from researchers at Writer provides a solution that is accessible to engineering teams. The study takes a systematic look at optimizing the different components of the orchestration layer that wraps around the foundation model, aka the AI harness.
By optimizing the harness, the researchers show dramatic reductions in tokens per task, a drop in cost-per-successful-task by up to 61%, and quality that holds steady, all without changing the underlying foundation model.
Because the harness is fully under the developer's control and requires no model fine-tuning, engineering teams can apply these findings to build highly cost-efficient AI applications.
The ROI crisis of tokenmaxxing
The current state of AI engineering is plagued by "tokenmaxxing," an industry trend where developers rely on massive context windows and brute-force token consumption as a substitute for good system design.
Rather than engineering elegant workflows, developers have imported a reflex from traditional software development: generate, run, fail, stuff the error and more context back into the window, and retry.
"Teams tokenmaxx because it's the cheapest fix in the moment, and because it's literally how most engineers work today," Waseem AlShikh, CTO and co-founder of Writer, told VentureBeat. Because this approach succeeds often enough on coding tasks, it has become the default reflex for every other agentic workload. The danger is that per-token price drops mask the underlying inefficiency.
"Your invoice is tokens-per-task times price-per-token, and most teams only watch the second number," AlShikh said. "In agentic workloads, tokens-per-task compounds — every loop iteration re-transmits the growing context — and it compounds faster than prices fall. The price cut becomes an anesthetic. It masks the fact that the loop itself is bleeding."
Tokenmaxxing leads to several enterprise failure modes. Teams route simple tasks to premium frontier models by default. They use the LLM as a lazy search index, stuffing the context window with raw documents instead of retrieving exact answers. Most destructively, they build unconstrained agentic loops that spiral out of control when the model encounters an error. Because output tokens cost significantly more than input tokens across all major model providers, inefficient task execution acts as a silent budget killer.
The industry has introduced several efficiency techniques to curb these costs, but they largely fall short because they treat the model in isolation:
Prompt compression condenses input text to save space, but ignores how the system sequences those inputs across complex workflows.
Budgeted reasoning caps the computational steps a model can take, which often degrades output quality if the workflow isn't intelligently routed.
Terse coding forces models to output minimal code to save output tokens, but does nothing to solve inefficient tool calling.
Speculative decoding uses a smaller draft model to speed up a larger model's text generation, optimizing inference speed while failing to address bloated agent architectures.
These efforts fail because they optimize the engine while ignoring the transmission. They do not look at the orchestration layer, leaving underlying architectural inefficiencies unresolved.
Unpacking the harness: the levers of efficiency
The harness is the orchestration layer that routes, formats, and turns the underlying LLM into a working system.
The core levers of harness optimization include system prompt caching, interaction history compaction, tool management, retrieval strategies, and error management. These are the most accessible intervention points for engineering teams looking to improve AI performance.
As the Writer researchers note in the study: “If the harness is the layer that composes model calls into work, it is also the layer that sets the price of work.”
Historically, developers have treated the harness as disposable glue code designed simply to connect an API to a user interface. The study signals that the harness must now be treated as a first-class object: a primary software artifact that requires its own testing, versioning, and rigorous design.
For enterprises, this reframes the "own-versus-rent" decision.
"Enterprises spend months on model evaluations and then rent their orchestration off the shelf — which means they're optimizing the smaller lever and outsourcing the bigger one," AlShikh said. "Whoever owns the harness owns your unit economics, and an open framework tuned for demos is not tuned for your invoice."
Inside the experiments
To isolate the impact of the orchestration layer, the researchers ran experiments on six foundation models spanning multiple vendors and weight classes: Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, and Writer’s own model, Palmyra X6.
Their experiments compared a frozen, conventional production agent loop against the finished Writer Agent Harness on the same 22 locked enterprise tasks, spanning capabilities like grounding and retrieval, multi-step workflows, tool use, and content generation. By holding the models and tasks constant, they could isolate the effects of the orchestration layer itself.
The optimized harness drove a significant drop in costs, cutting the blended cost per task by 41%, from 21 cents to 12 cents. This was largely achieved by slashing token consumption, with the number of tokens per task falling 38%, from 14.2k to 8.8k.
The harness is designed to delegate tasks like search to specialized sub-agents. A sub-agent receives only the tool and the specific query it needs, retrieves the exact data, and returns a capped, clean summary to the main agent — keeping the primary context window from filling up with raw search results.
Task success rates held steady even as token use fell — moving from 78% to 81%, a gain the researchers describe as directional rather than statistically significant at their sample size, meaning quality didn't suffer even as costs dropped.
End-to-end task latency also dropped significantly, reducing the median wall-clock time by 44%, from 48 seconds to 27 seconds, due to prompt caching and the elimination of dead-end reasoning loops.
However, the researchers also found limits to multi-agent orchestration. Smaller models like Gemini Flash 3.5 and Qwen 3.6 scored well below a usable reliability threshold on sub-agent delegation tasks (0.45 and 0.42, respectively) — the capability simply isn't dependable yet on lighter-weight models.
Sub-agent orchestration only crossed a usable reliability threshold on the two strongest models tested: Writer's own Palmyra X6 (0.86) and Claude Sonnet 4.6 (0.85).
The developer’s playbook: actionable takeaways and tradeoffs
The findings from the study translate into a playbook for enterprise developers building agentic workflows at scale. The first step is to implement what AlShikh calls the "Two-Zone Prompt" and "Context Offloading."
Structure for system prompt caching (The Two-Zone Prompt): Modern LLM APIs offer prompt caching, but developers must structure their payloads correctly to trigger it. Developers must separate the "stable zone" from the "volatile zone." Place static, unchanging elements (e.g., core rules, large tool schemas, and standard operating procedures) at the top of the prompt. Dynamic elements, such as the specific user query or recent conversational task state, must be appended at the bottom. This ordering allows the harness to reuse the cached prefix across hundreds of calls. "That single separation makes prompt caching actually work and stops you from re-paying for the same instructions on every one of an agent's thirty steps," AlShikh said.
Manage context with Context Offloading: Avoid context stuffing, where every turn of a loop is appended into a monolithic prompt until the window maxes out. Instead, move history and intermediate artifacts out of the window into retrievable storage, and pull back only what the current step needs. If possible, delegate tasks to single-purpose sub-agents to avoid context bloat. As AlShikh points out, "the biggest line item in agent spend isn't reasoning — it's re-sending things the model has already seen."
Build resilient loops and redefine KPIs: Unmanaged agent loops drain API budgets rapidly. Teams must begin tracking Completions Per Million tokens (CPM) to understand their true task costs, but the harness itself must contain physical guardrails. "The core principle is that you never ask the model to police its own spending," AlShikh said. "The fence has to live below the model, in code, on your side of the API." This requires three hard checks:
Hard per-task token budgets: The run terminates when the budget is spent, no exceptions.
Generation fencing: Caps on steps, tool calls, and recursion depth to stop non-converging agents.
Failure-spend governance: Cap what a run can spend after its first failed validation so a failing task doesn't become your most expensive task.
Avoid unnecessary complexity: Optimizing the orchestration layer comes with engineering overhead. If you're in the prototyping and exploration stage, that overhead isn't justified — iterate fast with a strong model and a light harness. Once you're scaling to millions of requests a day, the savings from harness optimization become substantial.
However, teams must be aware of "harness leverage." Adding structural scaffolding requires the model to hold and obey that context. If a model is too small, it will spend its limited capacity parsing the scaffolding instead of doing the task, causing accuracy to drop and tokens to rise. The rule for adding complex orchestration features is strictly mathematical: "If a feature adds more coordination tokens than it removes task tokens for that specific model, cut it," AlShikh said. "Nothing in the harness is free."
The future of the enterprise harness
The era of tokenmaxxing and treating context windows like bottomless buckets is coming to an end. Throwing more compute at poorly designed systems is not a viable strategy for companies that need to demonstrate a return on their AI investments.
As foundation models evolve to absorb planning, tool selection, and multi-step reasoning natively into their weights, the role of the harness will shift from compensating for model weakness to enforcing enterprise policy.
"What never moves into the model is the 'allowed': budgets, permissions, data boundaries, audit trails, deterministic kill-switches," AlShikh said. "Five years from now, the harness will be thinner but more important. There will be less scaffolding and more governance. However capable the model gets, someone external to it still has to define what it may spend, see, and touch. That layer belongs to the enterprise, and it should never be rented."
Read on the original site
Open the publisher's page for the full experience