Orchestrator agents: what they do, when you need one, and the coordination tax nobody budgets

An orchestrator agent is the agent that owns a piece of work on behalf of other agents. It breaks a task into smaller jobs, hands each job to a worker along with a contract and a share of one shared budget, keeps track of what has been done, and owns the join: the moment where the workers' results become a single accepted answer. Three jobs, then: split the work, hold the state, hold the budget. A workflow engine does the same three jobs with steps you wrote in advance. A router picks one specialist and steps aside. An orchestrator decides at runtime how many branches exist, which is exactly why it is useful and exactly why it is dangerous.

When do you need one? When the branches of the work only become visible while doing it, when the useful information does not fit in one context window, or when several searches genuinely cover different ground. Anthropic's research system, an orchestrator with subagents, beat a single Claude Opus 4 by about 90 percent on their internal evaluation, using roughly fifteen times the tokens of a chat. A controlled study across 260 configurations, published in Nature Machine Intelligence this July, found that the average gain from multi-agent designs across six benchmarks was zero: plus 81 percent on financial analysis, minus 70 percent on sequential planning. Both results are true. The rule that reconciles them is what this article is about: one agent until the structure of the work earns another, and a coordination tax you measure before you pay it.

Here is the review we run most often at Cohorte, and it always starts with a slide that gets applause. A duplicate-charge workflow, redrawn as a team: an identity agent, a billing agent, a subscription agent, a policy agent, a refund agent, a critic and a supervisor. Seven boxes, seven role prompts, a supervisor that "convenes a discussion and announces a consensus". It looks like an organization, and the people who drew it did real work. Then we trace one ticket through it. Three of the seven fetch the same account record. The policy agent summarizes the policy and drops the effective date. The critic reviews the summary, not the source. The supervisor picks the most confident answer. And the refund agent, spawned fresh, receives a tool budget and a write permission that its parent never had.

Nothing important has improved. The system has more language, more latency, more places to lose track of where a fact came from, and one new way for a permission to appear out of nowhere. Which is why every one of those reviews opens with the same sentence: an agent role is not an independent source of evidence.

What an orchestrator agent is, and what people call one by mistake

Three words carry most of this article, so let me pin them down. A contract is what a worker receives: the job, what "done" means, what it may touch, how much it may spend. A join is where several results become one answer. A budget is everything a run may consume: tokens, money, tool calls, time, retries. With those in hand, the word "orchestrator" gets used for four different things, and they fail in four different ways.

OpenAI's Agents SDK makes the last two visible as API choices: an agent exposed as a tool keeps the manager in control, while a handoff transfers control, and by default the prior conversation history, to the receiver. That is not a cosmetic difference. It changes who owns the final answer, what crosses the trust boundary, and where your guardrails run. IBM's glossary sorts orchestration into centralized, decentralized, hierarchical and federated. Those are org charts: they describe who reports to whom. The questions that decide whether the system works are narrower. Who owns the join? Who owns the budget? And which evidence survives the trip from worker to answer?

The three jobs, with code

Strip the framework away and an orchestrator is a short loop. The version below is deliberately plain Python with no dependencies, because the point is the shape, not the library. Read the three comments.

def orchestrate(task, budget, model, tools):
    # job 2: state lives in a ledger, not in the transcript
    ledger = Ledger(task)

    while ledger.open_jobs() and budget.remaining() > 0:
        # the model proposes; the harness caps the branches
        plan = model.propose_jobs(task, ledger.summary())
        for job in plan[: ledger.branch_cap()]:
            if ledger.is_duplicate(job):
                continue  # no duplicate workers
            contract = Contract(
                job=job,
                done_when=job.acceptance,
                # permissions: never wider than the parent's
                may_use=tools.subset(job.needs) & task.grant,
                # job 3: carved from the parent, never created
                budget=budget.carve(job.estimate),
            )
            # workers return artifacts with sources, not chat
            artifact = run_worker(contract)
            ledger.record(job, artifact, artifact.sources)

    # job 1 ends here: one join, one decision rule
    return join(ledger.artifacts(), task.decision_rule)

Job one: split the work into contracts, not requests. "Check policy and approve a refund" is a message. A contract says what the worker must produce, when it is done, what it may touch and how much it may spend. Anthropic reported that early versions of their system sent far too many subagents at simple questions and duplicated searches, until the orchestrator was taught to size the effort to the query. That is the branch cap and the duplicate check in the loop above.

Job two: hold the state in a ledger, not in the transcript. Workers return artifacts: a report, an evidence bundle, a proposal, with the sources attached. The orchestrator tracks which jobs are open, which are done, and which sources have been used. A transcript is a presentation of messages, with stale values, contradictions and summaries nobody can reproduce. It is fine for the model to read. It is not a database.

Job three: hold the budget, and never mint it. The carve call is the whole point. A worker's budget comes out of the parent's remainder. A worker's permissions are at most the parent's permissions intersected with the task's ceiling. Spawning a worker does not refill tokens, money, retries or tool calls. The most common multi-agent bug we find in audits is a parent with a two-euro ceiling spawning four children with two euros each, and most frameworks make that the default because nobody wrote the carve. Budget is authority. If your orchestrator can create either one out of nothing, it is not an orchestrator, it is a mint.

Four inequalities keep this honest, and they are worth checking on every edge of your agent graph:

child.permissions  ⊆ parent.remaining ∩ task.ceiling
sum(child.budgets) ≤ parent.remaining_budget
child.artifacts    keep the source restrictions they came with
join.permissions   ⊆ permissions of whoever acts on it

The architecture that enforces those rules outside the prompt, with identity, permissions and action approval running as platform services a model cannot talk its way past, is what we build in the Accountable Agents course.

The coordination tax

This is the case we use in our harness engineering playbook, because it is the cleanest version of the problem. A refund reaches the join with three matching reports. The billing worker says the second charge is refundable. The policy worker says the same. The critic agrees. The dashboard shows three out of three, confidence rises, the case moves toward approval. All three depend on the same cached policy passage, and the cache is stale. The policy worker retrieved it directly, the billing worker received it inside its context, the critic read the policy worker's summary. Three messages and three model calls, all standing on one source. The system did not obtain three independent reasons. It copied one reason through three interfaces, and the copies made the error look safer than it was.

That gap, between how confident the system looks and how confident it has earned the right to be, is the coordination tax. Tokens are the easy part of it. Coordination can also eat useful context, average away the one worker who actually knew the answer, drop correct evidence at the join, wait on the slowest branch, multiply the checking work, create cleanup work when a branch fails halfway, expose more data to more components, and add one more thing that can be wrong. Some of those costs you can price. Some of them have to stay hard limits.

So do not compress this into one "efficiency" score. A single number hides a permission violation inside an average quality gain. Run three gates, in this order:

1. Safety first. Does the candidate keep every hard rule: permissions, evidence requirements, who is allowed to do what, auditability, recovery? If not, stop here. No benchmark number pays for an unauthorized refund.
2. New information second. Does the extra worker, message or debate round supply evidence, capability, coverage or checking that the system did not already have, and does that contribution survive the join? If not, remove it. Fluent redundancy is still redundancy.
3. Economics last. Only now: does the value gained, counted on outcomes that meet your bar, exceed the complete cost and the expected cost of new failures? Complete cost includes the checking, the human review, the cleanup, and the permanent overhead of one more prompt, schema, dashboard and on-call question.

Written as one line, and this is the one to keep: gain = value added on outcomes that meet the bar, minus complete cost, minus expected cost of new failures. Most multi-agent demos report the first term and call it a result.

One piece of the tax that almost nobody budgets: parallel work buys less waiting by paying more total work. If your join needs every branch, the finish time is the slowest branch, so adding workers can make your slowest ten percent of requests slower even as the average worker gets faster. And the parts that stay sequential no matter what, verifying identity, reconciling records, getting approval, executing the action, do not speed up at all. Ten parallel readers make the reading faster and the waiting longer.

What the evidence actually says in 2026

The published results disagree with each other. That disagreement is the finding.

• Anthropic's multi-agent research system (June 2025): an orchestrator with subagents beat single-agent Opus 4 by 90.2 percent on their internal research evaluation, at about fifteen times the tokens of a chat, and token use alone explained 80 percent of the performance differences on BrowseComp. The same post says the pattern fits poorly for coding and for any task where agents must share context or depend on each other.
• Cognition, "Don't Build Multi-Agents" (June 2025, the same week): parallel subagents make conflicting implicit decisions that the final agent cannot reconcile. Their advice is to share full traces rather than messages, and to prefer one single-threaded agent with compressed history.
• "Why Do Multi-Agent LLM Systems Fail?" (UC Berkeley, 2025): 1,600 annotated traces across seven frameworks, fourteen failure modes in three families: system design, disagreement between agents, and verification of the task. Most of them are boundary failures, not model failures.
• "Capable language models can outgrow the benefits of collaboration" (Nature Machine Intelligence, July 2026): 260 configurations, five architectures, three model families, six benchmarks, matched reasoning budgets. Central coordination lifted financial analysis by 80.8 percent; every multi-agent design degraded sequential planning by 39 to 70 percent. Mean improvement across the board: zero. The strength of the single agent was the only thing that reliably predicted the outcome.
• Tran and Kiela (Stanford, April 2026): with the thinking-token budget held equal, a single agent matched or beat multi-agent systems on multi-hop reasoning. Their explanation is information-theoretic: every handoff between agents is a channel, and channels lose information.

Read together, these say something specific. Coordination pays when the work has breadth that can be searched independently, or exceeds one context. It costs when the critical path is sequential, when the branches share sources and therefore share mistakes, or when the join can only summarize what it receives. The topology is not a preference. It has to be chosen from the shape of the work, and tested against a single agent that was given the same resources.

Delegation: the four things that break at the boundary

"Here is the context, please check policy and approve a refund" is concise, and it is unusable. Which customer identity was verified? Which charges are in scope? Which version of the billing system was read? Which policy date applies? Who said this worker may approve anything? What if a human edits the case in the meantime? The worker will fill those gaps with plausible assumptions, because that is what models are good at, and you will get gorgeous garbage: internally coherent, impossible to audit. The failure did not start in the specialist. It started at the boundary.

Four boundary failures account for most of what we see in traces:

• Five different moves hiding inside one word. Delegating a job, transferring control of a conversation, passing an artifact, escalating to a human, and resuming an interrupted job move different things. A framework method called handoff that treats all five the same hides the question that decides correctness: who owns the outcome now?
• Paraphrase instead of artifact. Anthropic's own fix was to let subagents write persistent artifacts and return lightweight references, because passing summaries through the coordinator was a game of telephone. Move the artifact, not another model's description of it.
• The transcript used as the database. Some things need one defensible order: who owns a job, how much budget is left, whether an approval was used, whether an action was taken. Two workers adding different sources can proceed without coordinating. Two workers spending from one hard budget cannot. Two workers closing the same case cannot. The rule you must not break decides how strictly you coordinate, not the framework default.
• Debate as conflict resolution. When the payment provider and the ledger disagree on an amount, the answer is a rule about which source is authoritative and a reconciliation step, not a vote. "Last write wins" is a storage convenience that throws meaning away. Keep the disagreement visible until something with the authority to resolve it does. False consensus is data loss.

A handoff that survives an audit is mostly references, not one giant context field. Stripped to its bones:

handoff:
  job: reconcile-provider-ledger (v3)
  done_when: discrepancy on charge-17 is classified;
             unresolved disagreement is preserved
  inputs:
    - evidence://provider/charge-17@9
    - evidence://ledger/charge-17@42
  untrusted: [customer_attachment]   # cannot instruct
  may_use: [payments.read, ledger.read]
  may_never: [payments.refund, case.close]
  budget: carved from task-882
    tokens: 8000; tool_calls: 10; retries: 1
  owners:
    job: resolution_controller
    execution: conflict_worker_b
    refund: refund_service
  on_failure: return a typed result, not a guess

Notice what is in there that a prompt never carries: what the worker may not do, who still owns the job if the worker vanishes, and what counts as done.

The decision table: single agent, orchestrator, or something in between

Six questions, in order, pick a row. Can ordinary software do this job? Is the critical reasoning path sequential? Is there independent read work you can run as parallel tool calls before adding another reasoning node? Does each branch add distinct context, capability, coverage or a different way of being wrong? Can the join check branch results against the original evidence, or only summarize them? And does the candidate beat a strong single agent at the same complete budget and latency? Most teams stop at the second question and pick a team anyway, because a team looks like progress on a slide.

Evaluate against the single-agent baseline, or you have not evaluated

No multi-agent candidate should reach a ship review without climbing a short ladder first. Rung zero: can a rule, a query or a parser do it? Rung one: the strongest single model you can afford, with the relevant tools, well-prepared context and the same guardrails. A deliberately weak baseline is not a baseline, it is a sales prop. Rung two: that same agent with parallel reads, better retrieval, structured outputs and deterministic checks. Rung three: several independent samples from that agent at the total budget the team would consume. Only then does a team get evaluated, and it gets evaluated at the margin: n workers against n minus one, on the tasks the extra worker is actually routed to, with the thing it is supposed to add written down before the run. "More perspectives" is not a mechanism, whereas unique source coverage, a specialization that has been tested, or an independent implementation are.

The playbook's worked case shows the shape, and the shape is common. Candidate S: one bounded investigator issuing four authorized reads in parallel, a separate verifier, refunds approved by a human. Candidate M: a supervisor and three workers for billing, subscription and policy, with the same verifier downstream. M sounds more specialized. The task graph says the three branches are retrieval, not distinct reasoning, and S can do the reads in parallel. The team declares in advance that M must improve resolved-to-standard cases by five points to ship. The numbers are constructed to demonstrate the method, not production claims:

M does not ship. The team keeps one idea from it: route the rare cross-system conflicts to a specialist, after deterministic reconciliation fails. The result is one agent with one exception path, not a permanent committee. That is what harness engineering looks like in practice. The experiment does not crown an architecture. It finds the smallest mechanism that earns its place. Building that experiment, the graders, the paired trials and the statistics that stop you shipping noise as signal, is the subject of the Trust Engineering course, and the short version is free in The Agent Eval Playbook.

Choosing a framework rather than a topology? Our orchestrator landscape guide compares the platforms. This article is about the decision that comes before the vendor.

What I'd do

Draw the task graph before the agent graph. Ship the bounded single agent with parallel tool calls first, with a ledger for state and a budget it cannot exceed, and let it run on real tickets for two weeks. That is the baseline, and there is no skipping it. If a measured, remaining error splits cleanly into independent branches, add an orchestrator with three caps from day one: total workers, a novelty check so no two workers chase the same evidence, and budgets carved from the parent's remainder rather than granted fresh. Make every worker return an artifact, never a paraphrase. Run the marginal experiment before the second worker, not after the fifth. Write the topology decision down, with the six questions and the numbers, in a versioned document, because the document is what the team actually runs on; that is the same idea behind the AI Operating System we teach operators. And if a stronger single model at the same budget beats the team, ship the model and keep the slide as a souvenir.

FAQ

What is the purpose of an orchestrator agent?

To own a task on behalf of the workers: split it into bounded jobs, allocate one shared budget, track what has been covered, and own the join where the workers' artifacts become one accepted result. It decides at runtime how many branches exist. It should never create permissions or budget that its parent did not have.

What is the role of the orchestrator agent in IBM's architecture for agentic AI?

In IBM's framing, the orchestrator is the coordinator that activates the right specialized agent at the right time, in a centralized, decentralized, hierarchical or federated arrangement. That describes who reports to whom. The production questions are who owns the join, who owns the budget, and what evidence survives each handoff.

What is the difference between an orchestrator agent and a workflow engine?

A workflow engine follows steps you wrote in advance, so the set of possible outcomes is small and gradable. An orchestrator decides the branches at runtime. Use the engine when the steps are known; use the orchestrator only when branches genuinely appear during the work.

When is a single agent better than a multi-agent system?

When the critical path is sequential or depends on one evolving state, when one context can hold the decisive evidence, when the branches would share the same sources and therefore the same mistakes, or when the join can only summarize. Under matched budgets, studies published in 2026 found the single agent matched or beat multi-agent systems on multi-hop reasoning and on sequential planning.

What is the coordination tax?

Everything a coordination mechanism costs beyond tokens: consumed context, the expert averaged away by the group, evidence dropped at the join, waiting on the slowest branch, extra checking and cleanup work, more data exposure, and one more component that can be wrong. The decision rule: value added on outcomes that meet the bar, minus complete cost, minus the expected cost of new failures, after the hard rules have passed.

How do you audit a multi-agent system?

Give every run and worker its own identity, log the boundary (contracts out, artifacts back, budget consumed, join decisions), and check four inequalities on every edge: a child's permissions inside its parent's, children's budgets summing inside the parent's remainder, source restrictions preserved on artifacts, and the join never holding more authority than whoever is allowed to act on the result. Then compare the whole thing against the single-agent baseline at the same budget.

The seven-box slide usually survives the review, by the way. It goes on a wall. What ships is one agent with four parallel reads, a verifier, and one specialist that wakes up when two systems disagree. It resolves more tickets than the committee would have, faster, for a fraction of the cost, and when it is wrong there is one place to look.

A team of agents is not an architecture. It is a bill, until the work proves otherwise.

Charafeddine MouzouniSeptember 5, 2026