Back to insightsThe knowledge journal

Harness Engineering: Why the Same AI Model Wins in One Agent and Fails in Another

The model gets the headline; the harness decides whether the work gets done. A practical guide to agent tools, context, execution, verification, and enterprise architecture decisions.

15 min read

The model gets the headline. The harness often determines whether the work gets done. A practical guide to agent tools, context, execution, verification, and the architecture decisions behind reliable enterprise AI.

TL;DR

  • An agent harness is the software around a model that manages context, tools, execution, state, permissions, and feedback.

  • The same model can perform differently when those surrounding systems change. A model leaderboard cannot tell you how your complete agent will perform.

  • Diagnose failed runs before upgrading models. Missing evidence, unclear tools, broken handoffs, and weak verification require different fixes.

  • Evaluate complete systems against real tasks, with comparable budgets and independent acceptance checks.

  • Keep permission checks and spending limits outside the model, even as stronger models absorb more planning and coordination.

Why harness engineering matters now

Two teams choose the same frontier model. One builds an agent that completes useful work. The other builds an expensive loop that searches, retries, loses context, and confidently announces that it is finished.

The difference can sit entirely in the software surrounding the model.

In February 2026, LangChain reported improving its coding agent from 52.8% to 66.5% on Terminal-Bench 2.0, a benchmark of terminal-based tasks, while keeping GPT-5.2-Codex fixed. That is a 13.7 percentage-point gain from harness changes. It is a vendor-reported benchmark result, not a forecast for another workload, but it directly illustrates the distinction. Source

Google’s 13 November Antigravity research offers a more recent architectural signal. Its /teamwork systems combine parallel exploration, constrained context for individual agents, and verification through tools such as unchanged test suites and proof checkers. Google also reports that tightly coupled coding work benefited from sequential implementation and revision rather than parallel splits. These are system-design findings, not a controlled demonstration that any model will improve by a particular amount. Source

The implication for technology leaders is practical: an agent purchase is a system purchase, even when the sales conversation revolves around the model.

Before asking which model tops the leaderboard, ask what the system lets it observe, what it allows it to change, and how it determines whether the task succeeded.

What an agent harness actually does

An agent harness is the software that turns a model’s proposed actions into a managed process. It assembles information, exposes tools, executes permitted actions, handles results, preserves state, and decides when to continue, stop, or ask for help.

The term has fuzzy boundaries. Some teams use it narrowly for the model-and-tool loop. Others include the execution environment, evaluation hooks, and operational controls. For enterprise architecture, the broader definition is more useful because failures cross those boundaries.

A framework supplies components. A runtime hosts execution. A harness is the assembled behavior your agent actually runs with. Buying a framework or runtime does not settle every harness decision.

DisciplineMain questionTypical work
Prompt engineeringWhat instructions should the model receive?Task wording, examples, output requirements
Context engineeringWhat information should it see at this step?Retrieval, memory selection, tool-result filtering
Harness engineeringHow should the complete agent operate?Execution loops, tool contracts, recovery, checks, budgets
Model trainingWhat capabilities should the model learn?Changes to model weights through training

These disciplines overlap. Context engineering shapes the model’s immediate view; the harness manages how that view changes as work proceeds.

There is also an important distinction between an agent harness, which runs the work, and an evaluation harness, which measures performance. If the working agent can rewrite the tests that determine its score, those responsibilities have become dangerously entangled.

This is why a model name is an incomplete description of an agent. It leaves out most of the machinery that determines the path from a request to an accepted result.

Seven decisions that shape agent performance

1. Context selection

Consider a research agent answering a market-sizing question. It might need a definition, a geography, a reporting period, and the relevant source tables. Loading an entire document collection introduces competing definitions and irrelevant detail.

Anthropic’s context-engineering guidance emphasizes selective information delivery and warns that larger context can reduce effective recall. It describes loading references and retrieving details as needed rather than placing every available object into the prompt. Source

A useful context policy identifies what is mandatory, what can be retrieved later, and what must remain attached to a claim. For enterprise research, source dates and units often matter as much as the quoted number.

Practical test: inspect the model input at the first consequential mistake. Was the necessary information present, current, and distinguishable from conflicting material?

2. Tool design

A valid application programming interface (API) can still be a poor interface for an agent.

Imagine exposing dozens of overlapping search functions, each returning hundreds of records with inconsistent identifiers. The model has to solve an interface puzzle before it can solve the user’s problem.

Anthropic’s tool-design research recommends clear tool purposes, meaningful responses, and evaluation against representative tasks. Its examples favor targeted search over dumping entire collections into context. Source

For an enterprise document-search tool, I would require stable document identifiers, source dates, access-filtered results, relevant excerpts, and an explicit indication when results are truncated. An empty response should distinguish “no matching records” from “the search service failed.”

The Model Context Protocol (MCP) provides a way to connect tools and agents. A standard connection still needs a well-designed tool contract.

3. Execution flow

How much control should the model have over the next step?

For an exploratory research task, choosing where to search next may be useful autonomy. For a refund, mandatory approval should be an explicit workflow condition. The answer depends on the consequences of the action.

Start with the simplest flow that can complete the task. A fixed sequence may be sufficient. Add a model-directed loop when the path genuinely depends on discoveries made along the way.

Use multiple agents when work separates cleanly or independent review adds measurable value. Shared files, tightly coupled reasoning, and repeated handoffs can erase the benefit of parallelism.

Practical test: can you explain which dependency or information constraint each additional agent resolves?

4. State and recovery

A conversation summary describes what the agent believes happened. Durable application state records what actually happened.

Keep those separate.

Anthropic’s long-running-agent work describes using an initial setup phase, incremental execution, progress artifacts, and version history to maintain continuity across sessions. The pattern addresses work being left incomplete or declared finished prematurely. Source

For production systems, also record confirmed external actions and unresolved outcomes. If a request to create a folder times out, the next run should check whether the folder exists before trying again.

An idempotency key, where the target service supports it, lets repeated requests refer to the same intended operation. Where it does not, recovery needs another reconciliation mechanism.

Practical test: terminate a run immediately after an external write. Can it resume without creating a duplicate?

5. Verification

An agent’s declaration of success is a claim that needs evidence.

Choose acceptance checks before the run. A coding task might require unchanged tests and an independent test set. A research task might require source support, consistent units, and explicit treatment of conflicting evidence. A workflow task might require confirmation from the system where the change was made.

Another model can help judge quality, but it can also share the original model’s blind spots. Prefer direct checks where the outcome is mechanically observable, and calibrate model-based review against human judgments where it is not.

Protect the acceptance criteria. The agent should not silently weaken them to finish the task.

Practical test: what would stop a plausible but incorrect output from being accepted?

6. Permissions and isolation

A system prompt can describe an access policy. The tool service must enforce it.

Red Hat’s 12 November analysis draws a useful boundary: stronger models may absorb more cognitive scaffolding, while identity, isolation, permissions, and budgets still require external enforcement. Source

For a document agent, filter access before returning content. For a write operation, authorize the actual resource and action at execution time. Bind an approval to the specific action being approved so it cannot become permission for an unrelated change.

Retrieved documents and tool responses should remain untrusted inputs. Instructions found inside them must not grant additional authority.

Practical test: deliberately request a forbidden action. Does the surrounding system reject it even if the model proposes the tool call?

7. Budgets and stopping rules

A capable agent can still spend its entire allowance on an unproductive path.

Set limits for elapsed time, model consumption, tool calls, and repeated failures. Reserve enough budget to verify and package the result. Define what happens when completion is impossible: a partial artifact, a useful escalation, or a clear failure.

LangChain’s experiment also illustrates the reasoning-budget tradeoff: its highest reasoning setting performed worse than a lower setting under benchmark timeouts. More deliberation can consume time needed for execution and checking. Source

Practical test: when the agent stops, can you distinguish verified completion, budget exhaustion, missing information, and a blocked action?

The same model in two enterprise agents

Consider an illustrative assignment:

Compare three technology providers using approved internal research and recent public disclosures. Produce a cited recommendation and save the draft to the project workspace.

Both implementations use the same model and comparable spending limits.

Agent A receives a broad instruction, unrestricted search, a generic document reader, and a save function. It appends every search result to the conversation. Its completion condition is the model deciding that the answer looks finished.

It can produce an impressive recommendation while comparing different reporting periods, mixing customer segments, and citing a source that mentions a provider without supporting the claim. A failed save may go unnoticed.

Agent B starts with an explicit comparison contract: scope, time period, criteria, evidence requirements, and destination. Retrieval enforces the user’s access. Results include source dates and stable references. Calculations run in a suitable computation tool. Unsupported claims are flagged before drafting. The save operation returns a persistent identifier, and the workflow confirms the artifact exists.

Design choiceAgent AAgent B
Comparison scopeInferred during draftingRecorded before evidence collection
Evidence handlingLong transcript of search resultsClaim-to-source records with dates and units
Conflicting figuresReconciled implicitlyFlagged for resolution or explicit qualification
CompletionModel says it is doneRequired checks and save confirmation pass
RecoveryRetry from conversationReconcile recorded state and resume

This is an architecture example, not a measured performance comparison. Agent B can still misunderstand a source or make a poor recommendation. Its advantage is that important assumptions and failure points are exposed to inspection.

The model’s capability matters in both systems. Only one design makes the surrounding work sufficiently explicit to evaluate.

How to diagnose a harness failure

When an agent fails, inspect the earliest point where its trajectory diverged from a valid path. The final wrong answer may be several steps downstream.

SymptomInvestigate firstCandidate intervention
It invents a factMissing evidence, retrieval failure, or ignored evidenceImprove retrieval and enforce claim support
It chooses the wrong toolOverlapping descriptions or missing task contextClarify tool contracts and test selection
It repeats the same failed actionError feedback and retry policyClassify failures and trigger a different path or stop
It forgets a constraintContext selection, compaction, or handoff lossPreserve critical constraints in structured state
It reports success after a failed writeCompletion checksRequire confirmation from the target system
It exceeds the budgetSearch breadth, loops, and verification allowanceApply runtime limits and reserve completion budget
It remains wrong with the right evidence and working toolsModel reasoning or task ambiguityClarify the task and test a stronger model

These are diagnostic hypotheses. Several can coexist in one run.

Record the model input, tool arguments, tool results, state transitions, policy decisions, and verification outcomes, with appropriate redaction and access controls. Observable events are more dependable debugging evidence than an agent’s retrospective explanation.

A useful failure report identifies a failed decision, its available evidence, and the system change most likely to prevent recurrence. “The model hallucinated” rarely gives an engineer enough to act on.

How to evaluate models and harnesses separately

Replacing the model and rewriting the execution loop in the same experiment makes attribution difficult. Use a small comparison matrix.

ConfigurationModelHarnessWhat it tests
BaselineCurrentCurrentStarting performance
Model changeCandidateCurrentEffect of the model change in the existing setup
Harness changeCurrentCandidateEffect of the harness change with the current model
Combined changeCandidateCandidateInteraction between the two changes

Run all four against the same representative tasks and data snapshots. Hold tool access and budgets comparable. Record model versions, provider settings, prompts, tool definitions, runtime configuration, and acceptance-check versions.

A shared harness may favor the model it was designed around. Therefore, answer two separate questions: which model improves the current system with minimal adaptation, and which tuned model-and-harness combination performs best within the same engineering and operating budget?

Keep development tasks separate from held-out evaluation tasks. Repeat tasks enough to expose variability. Report uncertainty, particularly when the difference is small. Preserve difficult cases involving missing evidence, contradictory records, access denials, timeouts, and interrupted writes.

The scorecard should cover:

  • Verified completion: accepted outcomes divided by all eligible tasks, including failed runs.

  • Quality: correctness and usefulness against task-specific criteria.

  • Safety: unauthorized actions attempted, blocked, and executed, reported separately.

  • Efficiency: total operating cost and human effort per accepted outcome.

  • Latency: the distribution of completion times, including slow cases.

  • Recovery: whether interrupted work resumes without corruption or duplicated effects.

Report correct abstention separately. An agent that refuses impossible work is behaving well; an agent that refuses every task is not delivering the service.

Measure the cost of accepted work

Suppose two systems each attempt 100 tasks. In this illustrative example, System A costs $500 and produces 80 accepted outcomes. System B costs $300 and produces 40.

Their direct cost per accepted outcome is $6.25 and $7.50, respectively. The cheaper experiment produced more expensive successful work.

That calculation excludes human review, remediation, and downstream losses. Add them when comparing production economics, alongside comparable task coverage and quality thresholds.

Token price is one input to the calculation. The business buys completed work.

What changes for enterprise buyers and engineering leaders

Buyers need evidence from the complete product

A benchmark score for the underlying model does not establish the quality of a vendor’s retrieval, tools, recovery, or approval flow.

Ask vendors to demonstrate your workload, including an incomplete request, a denied action, a failed dependency, and an interrupted run. Inspect whether they can export meaningful traces and preserve configuration history. Ask what changes when they upgrade the model and whether you can validate or roll back the change.

A particularly revealing question is: “Show us a failed run, how you diagnosed it, and what changed afterward.”

Platform teams need a smaller shared foundation

Centralize capabilities that should behave consistently across products: identity, authorization, secret handling, execution isolation, trace collection, and evaluation infrastructure.

Keep task-specific evidence rules and acceptance criteria close to the product team. A research agent and an account-maintenance agent can share infrastructure while needing different definitions of success.

OpenAI’s February account of harness engineering describes making repository knowledge discoverable and enforcing architectural rules mechanically. That offers a useful example of engineering effort moving into the environment agents work within. Source

The defensible layer may move toward workflow knowledge

My expectation is that generic agent loops will become easier to acquire. Durable differentiation is more likely to come from domain-specific tools, permission-aware information access, representative evaluations, and accumulated knowledge of failure modes.

That changes the build-versus-buy question. Buy interchangeable infrastructure where it meets your requirements. Invest internal effort where your organization’s workflow, evidence, and accountability determine value.

It also changes ownership. Product teams should define acceptable outcomes. Engineering should own execution and recovery. Security should own the access model and isolation requirements. Domain experts should help judge correctness. Someone must own the complete service when a failure crosses those boundaries.

A practical starting plan

You do not need to create a harness-engineering department. Start with one valuable workflow and a repeatable improvement loop.

Week 1: Establish the baseline. Define the task boundary and acceptance criteria. Instrument the existing system. Collect representative tasks and preserve failed traces. Identify actions that require external enforcement.

Week 2: Fix the largest failure category. If required documents never arrive, improve retrieval. If tools return ambiguous errors, fix their contracts. If the agent finishes prematurely, strengthen completion checks. Change one major variable at a time.

Week 3: Test recovery and limits. Introduce dependency failures, interrupted execution, unavailable evidence, and denied actions. Verify that retries do not duplicate external effects. Confirm that spending and time limits actually stop execution.

Week 4: Compare alternatives. Test another model and the improved harness using the comparison matrix. Evaluate held-out tasks. Release gradually, preserve the previous configuration, and monitor accepted outcomes alongside cost and human effort.

The cadence is illustrative. The discipline matters more than the calendar: inspect failures, form a hypothesis, change the system, and test for regressions.

Remove complexity when it stops earning its cost. A newer model may make an elaborate planning stage redundant. Permission enforcement and independently maintained acceptance checks still serve a different purpose.

Frequently asked questions

What is harness engineering in AI?

Harness engineering is the design and improvement of the software surrounding an AI model so an agent can perform tasks reliably. It includes context delivery, tool interfaces, execution flow, state, recovery, verification, permissions, and resource limits.

How is it different from context engineering?

Context engineering focuses on the information available to the model at each step. Harness engineering also manages actions and their consequences: what executes, what state persists, what happens after failure, and what counts as completion.

Does a better harness make a weaker model better than a stronger model?

It can improve a system enough to change the ranking on a particular workload. That does not eliminate capability limits. Compare complete configurations on your tasks and budgets before drawing a broader conclusion.

Do I need multiple agents?

Only when decomposition, separate context, or independent review justifies the coordination overhead. A single agent with effective tools and clear acceptance checks is a useful baseline.

Should we build our own agent framework?

Usually, start with existing components. Own your workflow contract, authorization requirements, evaluation data, and acceptance criteria. Build additional infrastructure when a demonstrated gap warrants its maintenance cost.

The architecture decision behind the model decision

The model sets important limits on what an agent can understand and reason through. The harness determines the information, actions, feedback, and recovery paths available within those limits.

Teams should improve both. But they should know which one they are changing and what evidence would establish that the change worked.

The next time two agents using the same model deliver different results, inspect the path each one took: what it could see, which tools it could use, what happened after failure, and who checked the outcome.

Choose the model with evidence. Design the system so its work can be trusted.

Thanks for reading.
More on enterprise ai & ai agents, platform engineering and software & systems architecture

Back to top ↑
Find a new perspective

Explore the site

↑↓ to move ↵ to openSearch the entire collection