Harness engineering: what a harness is, what it is not, and the minimum viable harness

A harness is the runtime that couples a model to the world. The model produces a possible next move; the harness decides whether that move becomes work, evidence, a retry, an escalation, or nothing at all. In full: a harness assembles what the model sees, exposes what it may attempt, executes what policy permits, keeps the task's state, coordinates any workers, observes what happened, and decides when to continue, stop, recover or hand off to a person. Harness engineering is the discipline of building that runtime on purpose. It is not prompt engineering (which changes what the model is told), not a workflow (which fixes the route in advance), and not a framework (which is a library you build with). And it is not EleutherAI's lm-evaluation-harness, which is a benchmark runner that happens to share the word.

The phrase entered common use in early 2026, after Mitchell Hashimoto described his practice as engineering a fix into the surrounding system every time an agent made a mistake, and OpenAI's Codex team published an essay under that title in February describing a product built with no hand-written code. The idea is older than the name. Every team that has shipped an agent has a harness, whether or not they drew it, and most of the failures that get blamed on the model live in it. This article gives the precise definition, the anatomy, the smallest harness worth building for consequential work, and the twelve properties you can test a harness against. It is the hub for our engineering series on the subject.

Take two systems using the same model. The first gets a task, a terminal, broad user-level access and a large context window, and runs until it says it is done. The second starts clean, separates reads from writes, exposes a handful of typed tools, requires independent approval for anything dangerous, checkpoints its state, verifies the final artifact, enforces a budget, and attributes every action to an identity and a version. If it cannot prove success, it retries, escalates or stops. The model is identical and the systems are not; the harness is the whole difference, and it is the part a model benchmark cannot see.

That is why "which model?" is the least interesting question in an architecture review, and why the same team can ship a good agent and a dangerous one in the same quarter with the same API key.

A precise definition, and three equations

The short version fits in a sentence: a harness is the runtime and control system that couples a model to an environment. The longer version is a list of responsibilities, and every one of them is a place where a real system has failed:

Three equations organise the field, and the first is the one everyone quotes:

Agent = Model + Harness

It corrects the habit of attributing every outcome to the model. A model without tools, state and a loop does not inspect a repository, update a CRM record or reconcile an invoice. For engineering, a second equation is more useful, because the terms interact:

Agentic outcome
  = f(model, harness, task, environment, budget, policy)

Not additive. A better tool helps one model and confuses another. A verification hook prevents premature "done" on easy tasks and exhausts the turn budget on hard ones. The same harness is competent with read-only access and unacceptable the day you let it write. And for trust, a third:

Trustable agentic system
  = model + harness + independent evidence + accountable owner

The harness controls behaviour. Evidence qualifies the claims you make about that behaviour. An owner accepts the residual risk and has the authority to stop the system. Without independent evidence, "trust" is confidence. Without an owner, "governance" is a dashboard nobody is obliged to act on.

Not a prompt, not a workflow, not a framework

The word gets stretched, so here is where the edges are.

The prompt case is the one worth internalising. Suppose an agent keeps declaring a task complete before it has run the relevant test. The prompt intervention is one line: "always run the test before finishing." The harness intervention makes the test an explicit tool with a success contract, records whether it ran against the current artifact version, rejects a completion action that lacks fresh test evidence, reserves budget for the verification step, checkpoints the pre-test state, and logs the evidence used to authorise completion. The prompt asks. The harness arranges.

There is now experimental support for the distinction. In a 2026 study that evolved a coding harness automatically over ten iterations, changes to tools, middleware and long-term memory carried the gains; replacing only the system prompt made results worse. The evolved harness lifted Terminal-Bench 2 from 69.7 to 77.0 percent and transferred to three other model families. That is not proof that prompts do not matter. It is evidence that prose cannot substitute for structure.

The disambiguation nobody asks for but everybody needs: lm-evaluation-harness is EleutherAI's benchmark runner for language models. Good tool, different thing. If you arrived here looking for it, the link is right there.

Coding agents made the harness visible, but they do not define it

Harness engineering became legible through coding agents for a practical reason: software makes the surrounding system unusually easy to see. Give a model a blank chat window and ask it to fix a bug, and it describes a plausible patch. Give the same model a repository, an editor, a shell, version history, tests, logs, a clean worktree, permission rules and a stopping condition, and it can attempt the work, observe what happened, recover, and produce evidence another engineer can inspect. The capability did not appear inside the model between those two experiments. The environment changed.

A source-code study of eleven production coding harnesses published in July, covering about four million lines of Python, TypeScript and Rust, found the same anatomy everywhere: a loop, tools, context management, safety controls, orchestration and extension surfaces. Two details from it are worth repeating at dinner. None of the eleven imports a general-purpose agent framework, and none retrieves code with vector embeddings. The field runs on hand-rolled async loops and deterministic retrieval.

Coding offered five unusually favourable conditions: the world is inspectable, state survives the conversation, actions are executable, feedback is cheap (compilers reject bad states in seconds), and recovery is usually available (discard the worktree, revert the commit). None of that makes the harness a coding-only idea. It makes coding the laboratory. Replace the repository with a customer case and the mapping holds:

What changes across that table is the evidence economy. A failed test is immediate and cheap. A customer who does not reply is ambiguous. A payment may be accepted while the acknowledgement is lost. A hiring recommendation may affect someone before its error is visible. The farther a task sits from executable truth and cheap rollback, the more the harness has to invest in identity, authority, independent checks, reconciliation and redress. Coding agents showed how to surround intelligence with an environment. Everyone else has to surround it with responsibility.

The anatomy: six planes

A harness that will survive an incident review owns six planes. This is a separation of authority and meaning, not a deployment diagram; a small harness runs all six in one process.

The last row is the one that separates a harness from a self-modifying toy. Adaptive systems are real now: a harness can pick a different model, load a specialised tool, spend more reasoning effort, spawn a verifier, and an outer loop can propose changes to its own prompts and tools for next time. The question is what the system may change about itself. The answer that holds up is a split: the adaptive runtime may optimise tactics inside an approved boundary, and an assurance kernel controls the boundary itself. A better tactic must remain a permitted tactic. The system may generate its successor. It may not appoint it.

The minimum viable harness

A harness can be tiny. The eleven-system study points at roughly one-hundred-line coding agents (a loop, one model call, one tool, message history, limits) as an existence proof. But "minimum viable" depends on consequence. A local read-only experiment and a production payment agent do not share a minimum. This is the architectural floor for work that has consequences, in plain Python:

state = initialize(task, principal, harness_version)

while True:
    if budget.exhausted(state):
        return terminate("budget_exhausted", state)

    context = context_policy.assemble(state)  # what the model sees
    proposal = model.propose(context)         # the model proposes

    # policy decides what the proposal may become
    decision = policy.evaluate(
        proposal, state.principal,
        state.authority, state.evidence)
    if decision.requires_independent_approval:
        return escalate_out_of_band(decision, state)
    if decision.denied:
        state.record_denial(decision)
        if recovery_policy.can_continue(state):
            state = recovery_policy.constrain_and_continue(state)
            continue
        return terminate("policy_denied", state)

    # execution happens in a declared environment
    observation = executor.run(
        decision.action, sandbox=decision.sandbox,
        identity=state.agent_identity)
    state = state.transition(decision.action, observation)
    checkpoint(state)  # durable, resumable

    # stopping depends on evidence, not on the model saying done
    verdict = stop_policy.evaluate(state)
    if verdict == "verify":
        state.attach(verifier.check(state.artifact))
    elif verdict in TERMINAL_STATES:
        return terminate(verdict, state)

The point is not this code. It is the separation of authority the code makes visible: the model proposes; policy decides what is admissible; execution happens in a declared environment; state transitions are durable; stopping depends on evidence; and escalation uses a channel the requesting agent does not control. Each of those six is a line you can point at. If your framework hides one of them, find where it went.

The terminal states matter more than they look. A run should be able to end in complete, abstain, escalate, budget_exhausted, policy_denied, failed_recoverably or failed_terminally, and every one of those needs an owner. The run that ends in a state nobody is paged for is the run that becomes an incident three weeks later.

Twelve properties you can test a harness against

"Best practice" is advice. An invariant is stronger: an execution that violates it is evidence of a defect, and you can write a property test for it. These twelve are the ones we check in architecture reviews.

1. Attributable effect. Every external side effect traces to a principal, an agent identity, a task contract, a permission grant, a policy decision and a harness version.
2. Monotonic delegation. A child's authority is at most the parent's remaining grant intersected with the task's ceiling. Delegation cannot create authority.
3. Complete mediation. Every consequential action passes through policy and the effect protocol. No skill, plugin, recovery path or operator shortcut reaches the capability another way.
4. Evidence-bound completion. A run cannot be marked complete unless every required obligation has fresh evidence tied to the current artifact and environment versions.
5. Unknown is not failed. A timed-out effect stays "unknown" until reconciled. Recovery cannot repeat it unless idempotency or non-occurrence is established.
6. A retry changes something. Evidence, context, model, strategy or constraint. Identical repetition is sampling, not recovery, and is accounted for as such.
7. Budgets only shrink. Time, cost, steps and delegated resources decrease during a run unless an authorised external principal expands them. Spawning children mints nothing.
8. Artifact lineage. Every derived artifact carries its inputs, producer, version and inherited restrictions. A summary of a restricted document is restricted.
9. Memory quarantine. No unverified model inference writes directly to durable cross-run memory. Promotion is a separate, attributable step with scope, provenance and a way to revoke it.
10. Assurance is immutable within the run. The adaptive plane cannot change its own permission ceilings, evaluators, completion definition, rollback or approval channel while the run is live.
11. Replay distinguishes proposal from effect. Replaying a trace may reproduce decisions without re-firing side effects. Effect replay needs an explicit simulation.
12. Every terminal state has an owner. Completion, abstention, denial, exhaustion, failure, cancellation and unresolved effects each route to a named operational or business owner.

Test them against normal execution and then against the adverse conditions that actually happen: a process crash mid-effect, a delayed tool response, a duplicated delivery, a stale read, two concurrent writers, a model substitution, a context compaction, a poisoned skill. The harness that passes those is the one you can defend to an auditor, and the one the Engineering Series is built to teach, course by course: foundations, trust, accountable agents, context, security, and the enterprise operating system that ties them together.

Before you pick a framework: the boundary canvas

The mistake we see most is choosing the machinery first. A team picks a framework, adds a second agent, and only then discovers that nobody can say who may publish, what "done" means, or what happens when the payment provider times out. Fill in the boundary first. If a row has no owner or no enforcement point, it is not a boundary. It is a hope.

harness:            name, version, technical owner, business owner
purpose:            user outcome, qualified task classes,
                    explicitly out of scope
environment:        data sources, external systems, mutable state,
                    trust boundaries
model_integration:  qualified models, routing, fallback,
                    what a model change requires
context_and_memory: sources, provenance and freshness rules,
                    permission filter, compaction policy,
                    durable memory policy
tools_and_actions:  read capabilities, write capabilities,
                    irreversible actions, sandbox boundary,
                    idempotency and compensation
control_loop:       step / time / cost budgets, retry policy,
                    checkpoint policy, stop evidence,
                    terminal states and their owners
coordination:       default single_agent; when and how to delegate
evidence:           current eval, known limitations,
                    residual risks, next review date

Then test the boundary against the smallest credible baseline before adopting anything. Hold the model, the cases, the fixtures, the budgets and the grader constant, and compare four rungs: model plus prompt (can the task be done as advice?), a single bounded loop (does acting add value over advice?), a recoverable harness with durable state, effect identities, reconciliation, independent verification and rollback (do recovery and evidence justify their cost?), and only then a coordinated or adaptive harness (does the extra machinery beat the recoverable one under matched resources?). For a read-only drafting task the answer may be the first rung. For a payment that can become ambiguous after dispatch it cannot be.

One adverse condition is worth running on every harness that moves money or sends messages: the acknowledgement that never arrived. The agent proposes one authorised refund with an operation key. The provider commits it, and the connection drops before the acknowledgement. In a single bounded loop, the transcript holds a timeout and no durable effect record; a retry may refund twice, and stopping leaves the customer uninformed. In a recoverable harness, intent and operation identity were persisted before dispatch; the timeout produces an "unknown effect" state, not a failure; the model cannot issue a fresh refund; a reconciler queries the provider with the same key and either attaches evidence or, after the provider's finality window, permits a same-key retry. Completion stays blocked until evidence binds the effect, the customer message and the case version. That is what the extra rung buys. If it changes no outcome on a genuinely read-only task class, keep the simpler rung for that class. Harness engineering is not a mandate for maximal infrastructure. It is a method for finding the minimum system that can bear the claim.

Where evals, memory, security and coordination sit inside it

Once the harness is the unit of design, the topics that usually get separate conference tracks turn out to be planes of the same object, and we have written each one up:

• Evaluation is a second harness around the first. It decides what claims may be made about the production harness and which permissions those claims support. A score does not grant authority; qualification does. See how to evaluate AI agents.
• Memory is the state plane's hardest problem: a governed write with provenance, scope and expiry, never a side effect of a helpful model. See agent memory architecture.
• Security is control over how information may influence authority, action, persistence and evidence; the prompt is one carrier among several. See MCP security and tool poisoning.
• Coordination is a resource policy the harness implements, not a cast of personalities: one agent until the structure of the work earns another. See orchestrator agents.
• Infrastructure is where the harness runs, and self-hosting moves the risk to you without removing it. See the self-hosted production stack.

The six open-source libraries we published after sixty deployments, guardrails, agent auth, context routing, monitoring and reliability certification, are the assurance plane of this picture as code. And if you want to watch a real harness loop unrolled step by step, the Codex agent loop is a good afternoon.

What I'd do

Draw your current harness before you change anything: the loop, the tools, where state lives, who approves what, how a run can end. Most teams discover in that hour that they have one, that nobody owns two of the six planes, and that "complete" has no evidence attached. Then fill in the boundary canvas and hold the answers to the twelve invariants against it, honestly; write down the three you fail. Fix those in the runtime, not in the prompt, one at a time, and run the acknowledgement-that-never-arrived scenario before the next deploy. Keep the harness as small as the consequences allow, and keep the canvas as a versioned document the team actually runs on, because the prompt was never the bottleneck and the document is the operating layer, the same idea we teach operators as the AI Operating System. If you want to build the whole thing with us rather than assemble it alone, that is the Engineering Foundations course, and the rest of the series follows the planes.

FAQ

What is harness engineering?

The discipline of designing the runtime around a model: what it sees, what it may attempt, what policy permits, how state is kept, how workers are coordinated, how outcomes are observed, and how a run ends. The model proposes; the harness decides what a proposal becomes. The phrase became common in early 2026, popularised by Mitchell Hashimoto and an OpenAI essay on the Codex team's practice.

What is an AI agent harness?

The runtime and control system that couples a model to an environment: loop, tools, context assembly, state, safety controls, coordination and the stop policy. Agent = model + harness. A source-code study of eleven production coding harnesses found the same anatomy in all of them, built on hand-rolled loops with no general-purpose framework.

Harness vs framework vs prompt: what is the difference?

A prompt changes what the model is told and can only ask. A framework is a library you build with and does not own your loop, state or permissions. A harness is the runtime the agent works in and decides the conditions under which model output becomes consequential. A platform sits one level above, sharing identity, connectors and monitoring across many harnesses.

What is the minimum viable harness?

For consequential work: a loop in which the model proposes, policy evaluates the proposal against the principal's authority and current evidence, execution runs in a declared sandbox under an identity, state transitions are checkpointed, stopping requires verifier evidence, and escalation uses a channel the agent does not control, with typed terminal states that each have an owner. Around a hundred lines is enough for a read-only experiment; the minimum grows with the consequence.

Is lm-evaluation-harness the same thing?

No. EleutherAI's lm-evaluation-harness is a benchmark runner for evaluating language models on standard tasks. An agent harness is the production runtime around a model. They share a word and nothing else.

Who coined harness engineering?

The phrase is usually credited to Mitchell Hashimoto, who described engineering a fix into the surrounding system every time an agent made a mistake, and it spread after OpenAI published an essay titled "Harness Engineering" in February 2026 about building a product with Codex and no hand-written code. The practice predates the name by years.

Back to the two systems with the same model. One of them will be in an incident review within a quarter, and the review will spend its first hour arguing about the model. The other will be boring. Boring is the goal. The difference between them was never intelligence.

The model proposes. Everything you are actually accountable for happens after that.

Charafeddine MouzouniSeptember 13, 2026