AI companies often spend months comparing language models to cut costs. New research suggests they may be optimizing the wrong thing.
A recent study found that improving an AI agent’s orchestration layer, or “harness”, cut token usage by 38%, reduced costs by 41%, and completed tasks 44% faster. The model itself never changed.
Each model ran through two setups: a standard production agent loop, and Writer’s optimized agent harness. The models, tasks, and pricing stayed fixed. Only the orchestration layer changed.
| Measurement | Standard harness | Optimized harness | Change |
|---|---|---|---|
| Tokens per task | 14,200 | 8,800 | 38% lower |
| Average task cost | $0.21 | $0.12 | 41% lower |
| Median completion time | 48 seconds | 27 seconds | 44% faster |
| Task-completion quality | 0.78 | 0.81 | Broadly stable |
| Completions per million tokens | 54.9 | 92.0 | Nearly doubled |
Every model tested got cheaper under the optimized harness, with savings ranging from 33% to 61%.
The quality score moved from 0.78 to 0.81. Researchers called this directional rather than conclusive, since the study only covered 22 tasks. The safer read: quality held steady while cost and token use dropped.
Why this matters: Two apps can run the exact same model and still have very different bills. The difference comes down to how prompts, memory, and tools are managed, not which model is doing the thinking.
What Is an AI Agent Harness?
A harness is the software layer that turns a raw language model into a working application.
The model handles reasoning and language. The harness controls everything around it:
- What information the model sees
- Which tools it can use
- What it remembers from earlier steps
- When a task should stop
Two apps can use an identical model and still perform completely differently, depending on how the harness is built.
Why AI Agents Waste Tokens
The study points to a common habit it calls “token maxing.” This happens when developers try to improve performance simply by feeding the model more of everything:
- Longer prompts
- Wider tool descriptions
- Larger search results
- More conversation history
- Extra retries
This can work fine in a quick test. It becomes expensive once it’s the permanent design.
A few ways this plays out in practice:
- The full conversation gets resent at every step, so the system keeps re-paying for information it already processed.
- Long documents get dumped into the prompt even when only a few lines matter.
- An agent forgets what it already found and re-runs the same search.
- A failed tool call spirals into a long retry chain.
- Multiple subagents each receive the same bulky context, duplicating cost.
Why this matters: Falling model prices can hide this problem. A company might celebrate a lower price per million tokens while the tokens needed per task keep climbing, so the bill never actually drops.
A related analysis from CIO found that Claude Code and OpenCode, running the same underlying model, still had different baseline token overhead. The cause: differences in system prompts, tool definitions, and how each system coordinated agents. Same model, different bill.
How the Optimized Harness Reduced Costs
Researchers traced the savings to five specific design choices.
Prompt Caching
Many providers charge less when the start of a prompt stays identical across calls.
An optimized harness keeps stable content, system instructions, rules, tool definitions, at the front of the prompt. Changing content, like the current request or new documents, goes at the end.
That stable “prefix” gets served from cache instead of being reprocessed at full price each time. In one measured layout, 99.9% of prompt tokens were served as cache reads.
Caching only works if the prefix stays untouched. If the harness keeps rewriting the start of the prompt, the cache breaks.
Context Compaction
A basic agent resends its entire conversation history every turn, which snowballs fast on long workflows.
A better-designed harness compresses older messages into structured checkpoints instead, keeping key decisions and task state without re-sending every prior sentence.
Context Offloading
Large documents, search results, and tool outputs don’t need to sit inside the model’s active context the whole time.
Instead, they can be stored outside the prompt and pulled in only when needed. A search subagent, for example, can scan several documents but hand back a short summary instead of the raw data.
The same applies to tools: the harness can show only the tools relevant to the current step, rather than the entire toolbox.
Specialized Subagents
Rather than cramming search, billing, policy checks, and writing into one agent’s context, the harness assigns each subagent a narrow job and a short result to report back.
This cuts bloat, but only when kept tight. Spinning up agents for simple tasks can add more coordination overhead than it saves.
The Slackbot MCP integration shows how enterprise agents can connect with CRM data, dashboards, APIs and business workflows. However, each connection can also add tool instructions and context that the harness must manage carefully.
Failure Controls
An unrestricted agent can retry a failed action endlessly, adding cost with every attempt and no guarantee of progress.
Hard limits, on token budgets, tool calls, retries, and recursion, stop a failing run before it becomes one of the most expensive tasks in the system.
Why this matters: None of these techniques require a different model. They’re engineering choices any team can apply to what they already have.
Why This Matters for Enterprise AI
Most companies compare AI systems by model benchmarks and published token prices. Those numbers miss the full cost picture.
A cheaper model might need more retries to finish a task. A pricier model might finish faster and end up cheaper overall. The only way to know is to measure the model and the harness together.
In this study, the orchestration layer affected task cost more than the price difference between models.
Questions worth asking instead of “which model is cheapest”:
- How many tokens does the whole system use per successful task?
- How often does it retry?
- How many tools does it call?
- How much of the prompt is served from cache?
- What does a failed task cost?
- How often does it need human review?
Many companies can’t currently answer these questions, since harness design, prompts, and tool integrations are often split across different teams. That’s driving calls for better cost tracking as agents move into production, an AI version of FinOps.
Important AI cost metrics to track:
- Cost per successful task
- Tokens per task
- Cache hit rate
- Retry costs
- Tool calls per task
- Completion latency
- Failure rate
Why this matters: Harness design is also a governance question. It controls what data an agent touches, what it can spend, and which actions need human approval.
Limitations of the Study
These results are strong, but they’re not a guaranteed 41% saving for every AI application.
- The evaluation covered only 22 tasks, a useful controlled comparison, but not representative of every workflow.
- The optimized harness was compared against one fixed baseline. A different, already-efficient system might show smaller gains.
- The quality improvement (0.78 to 0.81) wasn’t statistically conclusive.
- Prompt caching works best with large, stable prompts and repeated tasks. A one-off short prompt won’t benefit much.
- Stronger models gained more from advanced orchestration, an effect the researchers called “harness leverage.” Weaker models got cheaper but didn’t always handle complex delegation reliably.
Why this matters: A sophisticated harness can backfire if the model spends more effort understanding the orchestration than the task itself. For simple, predictable jobs, a basic workflow can still beat an elaborate multi-agent setup.
What Businesses Should Do Next
Nobody needs to rebuild their AI stack overnight. Start by measuring what already exists.
Step 1: Build a baseline. Track average token use, cost, success rate, latency, retries, and tool calls, including for failed tasks, which often hide the biggest costs.
Step 2: Review prompt structure. Keep stable rules and tool instructions in a cache-friendly section up front. Put current requests and retrieved data in the changing section at the end.
Step 3: Use checkpoints, not full replays. Long workflows should compress history instead of resending it every turn.
Step 4: Offload large content. Keep big files and search results outside the active context until the model actually needs them.
Step 5: Set hard spending limits. Token limits, tool-call limits, and retry rules should be enforced in code, not left to the model to self-regulate.
Step 6: Test models and harnesses together. Evaluate a model inside its real production workflow, using the same tasks and pricing method across comparisons.
The study challenges a common assumption in enterprise AI: that switching to a cheaper model is the best way to cut costs.
The evidence points elsewhere, toward the entire system: prompts, memory management, tool orchestration, caching, and failure controls.
The reported 41% savings still need validation across more workloads. But the broader message holds up: smarter orchestration may save more money than switching models.
