What is Harness Engineering
A better model does not automatically make a better agent. The missing piece is the system around the model: the harness.
How We Got Here
The first coding assistants were thin wrappers around a model. They passed a prompt, returned text, and left the engineer to figure out what to do with it.
Then agents got tools. They could read a repository, edit files, run commands, and try again. The demos got more convincing. The failures did not disappear.
An agent could still inspect the wrong files, run the wrong test, lose the task halfway through, or declare victory with a broken patch. The fix was not always a smarter model. It was better scaffolding around the model.
That scaffolding is the harness.
Models are powerful and unreliable. They can invent facts, misread a codebase, and follow a local instruction that conflicts with a system-level constraint. You do not solve that by treating the model like a careful colleague. You give it the right context, restrict what it can touch, show it what happened, and require evidence before calling the work done.
Agent = Model + Harness
The formula is simple:
The model is the learned capability: reasoning, language, and code generation. It can propose a plan and produce a patch, but a model invocation has no direct access to your repository, test runner, or production systems.
The harness is the runtime around it. It manages context, tools, memory, task state, feedback, verification, permissions, and traces. It gives the model a way to act, a way to observe the result, and boundaries around both.
The agent is the composition. The plus sign is the part people skip.
Harness engineering is the practice of designing that second half so a capable model becomes a more reliable system.
The Autonomy Gap
The useful question is not “Can the model write this function?” It is “Can the system make a correct, maintainable change and show me why it is correct?”
That distance between local coding ability and complete software work is the autonomy gap. A model can write correct logic and still fail the task because it chose the wrong files, missed a requirement, misunderstood a test failure, or left the repository in a worse state.
The harness closes the gap by turning implicit human knowledge into runtime support: repository guides, tool interfaces, project memory, feedback loops, permission boundaries, and verification gates.
The Anatomy of a Harness
There is no single canonical checklist. The recent research framework by Zhong and Zhu names eleven runtime responsibilities, while practitioner accounts tend to group them into a smaller set of working subsystems.
This five-part view is useful because it maps directly to the questions an engineer has to answer:
Context — what the model can see. This includes the system instructions, conversation history, repository structure, relevant files, retrieved knowledge, and current task state. Context quality sets the ceiling for everything that follows.
Tools — what the agent can do. File access, code execution, search, APIs, and databases are not just integrations. Their interfaces define the actions the agent can take and the mistakes it can make.
Memory — what survives across steps and sessions. The harness decides what to keep, what to retrieve, and what to discard. A longer context window is not the same thing as useful memory.
Observation — what you can inspect. Tool calls, outputs, decisions, retries, and interventions should leave traces. Without them, debugging an agent is guesswork.
Verification — what counts as done. Tests, type checks, static analysis, requirement-level checks, and human review turn “the agent says it worked” into evidence.
These subsystems overlap. A test result is both feedback and verification. A repository guide is both context and a control. The point is not the taxonomy. The point is making the runtime support explicit.
Why Agent Failures Are Often Harness Failures
When an agent makes a mistake, the reflex is to blame the model. Often the model was given a bad operating environment.
Context failure: it had stale, incomplete, or irrelevant information.
Tool failure: the required action was unavailable, ambiguous, too powerful, or poorly handled when it failed.
Memory failure: it forgot an earlier decision or retrieved the wrong project knowledge.
Observation failure: the system recorded too little to explain what happened.
Verification failure: it had no reliable way to distinguish a finished change from a plausible-looking patch.
The distinction matters. If the failure is in context, a larger model may not help. If the failure is in verification, another prompt is not a safety mechanism. Name the failure before you try to fix it.
What This Means in Practice
The harness is the part you can shape. You can improve the repository structure, expose better tools, write down architectural decisions, add feedback sensors, tighten permissions, and move cheap checks earlier in the workflow.
This work is not free. Making a codebase easier for agents to use often means making it easier for humans to use too: clearer modules, better documentation, consistent conventions, and tests that say what the system is supposed to do.
A good harness does not make a model infallible. It makes failures visible, bounded, and cheaper to recover from. That is the real promise of harness engineering.
The best agent is not simply the one with the best model. It is the one embedded in the best system.
Further Reading
- Hailin Zhong and Shengxin Zhu — AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents
- Birgitta Böckeler — Harness engineering for coding agent users
- Thoughtworks — Harness engineering