The Graph Tax — AI Agent Graphs Are an Inefficient Way to Work

The fashionable architecture of supervisors, routers, sub-agents and hand-offs does not create free intelligence.

The fashionable architecture of supervisors, routers, sub-agents and hand-offs does not create free intelligence. It creates an expensive bureaucracy made from inference calls, repeated context and management overhead.

Keywords: artificial intelligence, large language models, AI agents, multi-agent systems, LangGraph, agent orchestration, token efficiency, supervisor agents, inference cost, context windows, AI productivity, human oversight

There is a fashionable picture of artificial intelligence that looks suspiciously like an organisation chart. A user gives a task to a “supervisor”. The supervisor decomposes it. A researcher is dispatched. A planner is consulted. A critic reviews the planner. A verifier reviews the critic. A writer converts the result into prose. Another agent checks citations. Messages travel along edges in a graph. Nodes light up. State moves from one box to another. The interface looks controlled, systematic and sophisticated.

The problem is that the graph itself does not create intelligence or perform the substantive reasoning. The models and tools at its nodes do, and every time the graph asks another model to plan, route, summarise, translate, verify, hand off, reread or reconcile the work of another model, it spends tokens on management rather than on the task. In many common knowledge-work settings, that management layer is not merely a small overhead. It is the dominant inefficiency.

My argument is therefore deliberately stronger than the usual qualification that “multi-agent systems have trade-offs”. General-purpose LLM agent graphs are inefficient. They burn tokens, replicate context, introduce translation errors, expand latency, create new failure points and shift work from producing the requested output to administering the system that is supposed to produce it. Their apparent sophistication is frequently organisational theatre implemented with probabilistic text generators.

That is not an argument that every graph data structure is bad, that deterministic workflow engines are useless, or that no multi-agent architecture can ever outperform a single model call. Those would be silly claims. The target here is the current fashion for graph-based LLM orchestration as a default method for research, coding, writing, analysis and other work that a capable user can often divide between a handful of focused model windows directly. When the same task can be performed by several independent windows, with the human deciding what each window should do and combining the results, adding an LLM supervisor often means paying a model to imitate the coordination the human can perform more cheaply, more transparently and with less loss of information.

The evidence for this is no longer anecdotal. Anthropic reports that its agents typically use about four times as many tokens as ordinary chat interactions and that its multi-agent systems use about fifteen times as many tokens as chats (Hadfield et al., 2025). A 2026 controlled comparison of single-agent and prompt-defined multi-agent orchestration reported roughly five- to six-fold increases in both token consumption and latency for the multi-agent design, without a corresponding improvement in aggregate benchmark performance (Sanabria, 2026). LangChain’s own benchmark of multi-agent architectures found that a supervisor architecture consumed more tokens than a swarm architecture and lost performance through the supervisor’s “translation” of sub-agent outputs (Fu-Hinthorn, 2025). Peer-reviewed work at ICLR, ACL, NAACL and AAAI now treats redundant inter-agent communication as a problem serious enough to require pruning, sparsification, dynamic agent elimination and explicit resource rationing (Fan et al., 2026; Z. Wang et al., 2025; Zeng et al., 2025; Zhang et al., 2025).

The graph tax is not a theoretical possibility. It is what the architecture is built to do.

The graph is a machine for paying for coordination

The easiest mistake is to treat a supervisor call as though it were free control logic. It is not. A conventional program can route a task with an if statement, a queue, a state machine or a deterministic scheduler at negligible marginal inference cost. An LLM supervisor routes a task by performing another probabilistic inference. It must receive instructions, ingest state, reason about what should happen next, emit a decision and often later ingest the worker’s answer before deciding again.

That distinction matters. In ordinary computing, management logic is cheap relative to expensive computation. In an LLM graph, the manager may be using the same class of expensive model as the worker. One is therefore spending reasoning capacity to decide who should reason, spending more reasoning capacity to interpret the reasoning, and then spending still more reasoning capacity to convert that interpretation into a final answer.

LangGraph’s supervisor pattern makes this structure explicit. The supervisor receives the user input, delegates work to sub-agents, receives their responses back and is the only component permitted to answer the user in the standard supervisor architecture tested by LangChain. LangChain found that this extra translation layer both cost tokens and introduced errors. Its engineers improved the system partly by removing hand-off messages from agent state, reducing context clutter, and adding a forwarding tool so the supervisor could send a worker’s answer directly to the user without regenerating it (Fu-Hinthorn, 2025). That is revealing. The system became better by doing less of the orchestration that defined the system.

The same pattern appears in the current LangGraph supervisor reference implementation. It exposes an output_mode that can return either a worker’s full_history or only its last_message, and add_handoff_messages controls whether additional hand-off messages are inserted into the workflow history (LangChain, n.d.). These are sensible engineering controls. They also expose the underlying cost structure. A graph that must be carefully configured not to repeatedly pass, store and reread its own internal management chatter is managing overhead created by its own abstraction.

LangChain itself has made the point more directly. Chase (2025) wrote that general-purpose supervisor and swarm architectures use many LLM calls and communicate inefficiently by design because they are not optimised for a particular use case. LangChain’s 2026 architecture guidance likewise recommends beginning with a single agent and moving to multi-agent patterns only when clear limits are reached; it explicitly identifies the extra model call, latency and token cost imposed by centralised sub-agent orchestration (Runkle, 2026).

This is where the graph metaphor becomes misleading. A diagram shows an edge as a line. Economically, an edge is not a line. It is a communication event. In an LLM system, a communication event can mean serialising state, moving text into another context, paying for the recipient to process that text, generating a response, adding the response to shared state, and later paying another model to process the accumulated result. The visual representation hides the cost precisely because the line in the diagram has no apparent weight.

Coase’s (1937) analysis of the firm remains unexpectedly useful here. Coordination is not costless simply because it occurs inside an organisation rather than through a market. Internal organisation has costs, and those costs can rise as more transactions are brought under managerial direction. An LLM agent graph recreates that problem in miniature. It substitutes a hierarchy of artificial managers for direct task execution and then assumes that the hierarchy is productive because it is organised. But organisation is only valuable when the reduction in other costs exceeds the cost of organisation itself.

Too many graph demonstrations omit that accounting.

Figure 1 uses Anthropic’s own reported operational figures. Ordinary chat is normalised to 1×. Agents use about 4× as many tokens, while multi-agent systems use about 15× as many as chat interactions (Hadfield et al., 2025). Anthropic is not hostile to multi-agent systems; it built one and reports substantial benefits for broad, open-ended research. That makes the admission more important, not less. Its engineering team states plainly that multi-agent architectures “burn through tokens fast” and require tasks whose value is high enough to justify the additional expenditure (Hadfield et al., 2025). THEY DO THIS AND ACHIEVE LESS.

A fifteen-fold token multiplier does not mean that every graph wastes exactly fifteen times as many tokens as a direct window. It means that the claim that direct windows can be roughly an order of magnitude more token-efficient is not rhetorical excess. It lies inside the observed range of current production and experimental systems. In some designs the multiplier is smaller. In others, communication-pruning results imply that the removable overhead is enormous. The relevant conclusion is not that the universal constant is 10.000. The conclusion is that the order of magnitude is plausible and that anyone presenting graph orchestration as an efficiency improvement has the burden of proving it for the actual workload.

“More agents” is usually another way of saying “more inference”

One reason agent graphs can look impressive is that they increase the amount of computation spent on a problem. This can raise benchmark scores. It does not follow that the architecture is efficient.

Anthropic’s analysis is unusually candid on this point. In its BrowseComp work, token usage alone explained 80% of observed performance variance, while token usage, tool calls and model choice together explained 95% (Hadfield et al., 2025). Anthropic’s description of its multi-agent system is therefore not magic emerging from a clever organisational chart. The system works in large part because it spends far more inference on the task. Multi-agent architecture is a mechanism for scaling token expenditure.

That can be worthwhile. It is not the same thing as efficiency.

Suppose one method spends one unit of computation and achieves a score of 70, while another spends fifteen units and achieves a score of 85. The second method may be preferable if the extra 15 points are sufficiently valuable. It would nevertheless be nonsense to say that it is more computationally efficient merely because its absolute score is higher. Efficiency requires a denominator.

A recent controlled study makes this point directly. Leins et al. (2026) compared several orchestration methods with task-only and chain-of-thought single-call baselines across competitive programming, chess puzzles and mathematics. The best average gains from orchestration were moderate, topping out at about 4.6 percentage points over optimised chain-of-thought inference and 4.5 points over task-only inference, while orchestration required roughly two to four times the mean total tokens of the task-only baseline. The authors’ conclusion is appropriately restrained: orchestration choices need to be model-specific and should be judged by whether modest accuracy gains justify additional inference cost (Leins et al., 2026).

That finding matters because the current discourse often reverses the burden of proof. The graph is treated as the advanced architecture and the single call as the naive baseline. It should be the other way around. The direct system is the economically preferred baseline because it introduces fewer paid operations, fewer state transitions and fewer opportunities for error. An orchestration layer should be added only when there is measurable evidence that it produces enough incremental value to pay its coordination tax.

The same principle appears in Amdahl’s (1967) analysis of parallel computing. Parallelism does not make sequential work disappear. If a fraction of a job cannot be parallelised, that serial component limits total speed-up. LLM graphs make the problem worse because they add a new serial component: orchestration.

A useful adaptation of Amdahl’s law is:

S(k) = 1 / [(1 − p) + p/k + h(k)]

Here, p is the fraction of useful work that can genuinely be parallelised across k workers, while h(k) is non-negative orchestration overhead expressed as a fraction of the one-worker baseline runtime. If h(k) were zero, the familiar benefit of parallelism would remain. In a real graph, however, h(k) can include planning, routing, hand-offs, shared-state processing, duplicated context, retries, verification, synthesis and waiting for dependencies. If h(k) grows quickly enough, adding workers makes the system busier without making the user proportionately more productive.

That is exactly what users experience when a graph spends its time narrating its own organisation.

The token accounting is worse than the diagram suggests

A direct-window workflow can be represented simply. Give each window a focused task, receive an answer, and then decide what to do next. If window i consumes Pᵢ input tokens and emits Oᵢ output tokens, the direct token expenditure is approximately:

T_direct = Σᵢ(Pᵢ + Oᵢ)

There is no need for another LLM to decide that the source-checking window should check sources or that the drafting window should draft. The human already made that decision when opening the window.

Now add a supervisor graph. Let T_work denote the tokens spent on the graph’s substantive worker calls. The graph then adds planning, routing, hand-off, rereading, verification, synthesis and retry costs:

T_graph = T_work + T_plan + T_route + T_handoff + T_reread + T_verify + T_synthesis + T_retry

The correct comparison with direct windows is therefore:

ΔT = T_graph − T_direct = (T_work − T_direct) + T_plan + T_route + T_handoff + T_reread + T_verify + T_synthesis + T_retry

This matters because a well-designed partition can sometimes reduce T_work by isolating context, as the clinical-scale counterexample discussed later demonstrates. But unless those worker-side savings are large enough to offset the added coordination terms, the graph consumes more tokens. Every added term must therefore be justified by lower worker cost, higher output quality, lower latency or reduced human effort. If it is not, it is waste.

The most damaging term is often T_reread. Multi-agent systems repeatedly process overlapping context. Ye et al. (2025), in work published at NeurIPS, identify this repeated reprocessing explicitly: when one agent receives a message from another, the receiving model may have to process prior context again, and ordinary key-value caching cannot simply be reused because agent-specific prefixes diverge. Their KVCOMM system is designed to recover efficiency by reusing cross-context cache information, achieving more than 70% reuse across tested multi-agent workloads and, in one five-agent configuration, up to a 7.8× speed-up in time to first token compared with standard prefilling (Ye et al., 2025).

Again, the optimisation is instructive. The researchers did not discover that communication was free. They discovered that large amounts of computation were being wasted by processing overlapping text repeatedly and designed a system to avoid doing so.

Q. Wang et al. (2025) describe the same phenomenon as a “communication tax”. Their AgentTaxo analysis attributes substantial multi-agent inference latency and token cost to repeated LLM calls and duplicated tokens, including reasoning results that are passed around and reused for validation. The phrase is exactly right. Communication in a graph is taxed in tokens.

In a naive sequential supervisor with full history, this can grow particularly badly. Assume k worker outputs of roughly r tokens each and suppose the supervisor rereads the accumulated worker outputs after each return. Ignoring other prompt material, the repeated reading of prior worker output is approximately:

R(k) ≈ r × k(k − 1) / 2

That is quadratic in k. The first worker’s output can be paid for repeatedly as later turns accumulate; the second is reread almost as often; and so forth. Modern frameworks provide trimming, summary memory, last-message modes and other mitigations precisely because naive history propagation is expensive. Those mitigations can turn the worst case into something more reasonable, but they do so by reducing, compressing or bypassing the graph’s internal communication.

This is why separate windows are so often superior for a skilled user. Each context can remain narrow. The research window does not need the drafting window’s stylistic chatter. The mathematical checker does not need the marketing copy. The source verifier does not need every abandoned line of reasoning produced by the planner. The human performs high-information routing by deciding what each window needs to know.

The graph, by contrast, is tempted to preserve state because state feels like safety. It therefore accumulates exactly the material that later calls must process.

Long context does not remove the problem. Liu et al. (2024) showed in Transactions of the Association for Computational Linguistics that models do not necessarily use long context robustly; performance can vary substantially depending on where relevant information appears, with important material in the middle often used less effectively. A larger context window makes it possible to send more information. It does not make sending irrelevant or duplicative information free, nor does it guarantee that the model will use the additional information well.

Context capacity and context quality are different things.

The supervisor is often a lossy translator

There is another cost that token accounting alone misses. Every time one model paraphrases another model, information can be lost, distorted or inflated.

LangChain’s 2025 multi-agent benchmark is unusually useful because it comes from the organisation building one of the most prominent graph-oriented agent frameworks. In its comparison of single-agent, swarm and supervisor architectures, the swarm slightly outperformed the supervisor across the benchmark. LangChain attributed the difference to the “translation” performed by the supervisor: workers could not answer the user directly, so the supervisor had to relay their output. The supervisor also consistently used more tokens than the swarm for the same reason (Fu-Hinthorn, 2025).

The engineering response was to reduce the translation. LangChain removed hand-off messages from sub-agent state to reduce clutter and added a forward_message capability so that a worker’s answer could be passed through without the supervisor regenerating it. The forwarding change reduced errors caused by the supervisor paraphrasing the worker incorrectly (Fu-Hinthorn, 2025).

That is the graph tax in one experiment. The manager consumes tokens to rewrite the specialist and then sometimes makes the specialist’s work worse.

Anthropic reports similar problems from production. Its early multi-agent research agents spawned far too many sub-agents for simple queries, repeated each other’s searches, pursued missing sources indefinitely and required explicit prompting rules to prevent overinvestment in simple tasks (Hadfield et al., 2025). Its engineers found that detailed delegation instructions were needed because vague tasks caused duplicate work and gaps. They introduced explicit effort budgets, source-selection rules, context management, state recovery, observability and safeguards against agents spiralling out of control (Hadfield et al., 2025).

These are impressive engineering solutions to difficult production problems. They are not evidence that the underlying architecture is cheap. They show how much engineering is required to prevent an LLM bureaucracy from behaving like a bureaucracy.

The analogy should not be pushed too far, but the economic structure is familiar. A specialist produces information. A manager requires a report. The report is reformatted. Another manager reads it. The information is summarised again for the person who originally wanted the answer. Each layer exists to coordinate the other layers, and each layer consumes resources. In human organisations, hierarchy may still be rational because a single manager can coordinate large amounts of labour at relatively low marginal cost. In an LLM graph, the manager is often another billable inference process consuming the same scarce resource as the worker.

The management process therefore competes directly with production for tokens.

The empirical overhead can be five or six times before we even reach the fifteen-times case

A particularly clean recent comparison comes from Sanabria (2026), who tested a single-agent architecture against a prompt-defined multi-agent orchestration system on 100 synthetic Mars-rover decision-support scenarios, with repeated evaluations using GPT-4o and GPT-5.5. The task was deliberately compact: all relevant scenario information could be supplied in a single context. The multi-agent version split analysis among specialist roles and then used an orchestrator to synthesise the result.

For GPT-4o, the single-agent condition averaged about 458 tokens and 2.32 seconds per evaluation. The multi-agent condition averaged about 2,273 tokens and 11.83 seconds. For GPT-5.5, the corresponding figures were about 548 versus 3,160 tokens and 6.06 versus 35.59 seconds (Sanabria, 2026). That translates into approximately 4.96× and 5.77× token overhead, and approximately 5.10× and 5.87× latency overhead, respectively.

The cost would be easier to defend if the multi-agent system clearly performed better. It did not. Decision accuracy was numerically higher for the single-agent architecture under both model configurations, although most decision-quality differences did not remain statistically significant after scenario-level correction. The efficiency differences, by contrast, were large and highly significant. The multi-agent system also generated more false-positive hazards (Sanabria, 2026).

This is precisely the class of problem for which graph orchestration should be rejected. All relevant information fits in one coherent context. The task does not require genuinely independent data access. The specialist agents do not possess different proprietary tools. The orchestrator must reconcile analyses that were artificially separated. The decomposition creates communication that the original task did not require.

The graph takes a compact problem and manufactures a coordination problem around it.

That pattern is common in writing and research workflows. A user asks for an essay. A graph creates a planner, researcher, outline agent, drafter, critic, rewriter and citation agent. But the planner, drafter and critic are all language models operating on heavily overlapping text. They are not seven independent laboratories. They are seven inference passes with role labels. If the resulting essay is better, the first question should be whether it is better because the graph discovered a superior organisation of reasoning or simply because the system spent several times the tokens.

The distinction matters because the latter can often be achieved more cheaply by the user opening several focused windows and deciding which outputs are worth carrying forward.

The pruning literature is an indictment of the default graph

The strongest evidence against indiscriminate graph communication may come from the literature designed to make multi-agent systems more efficient. Again and again, the way to improve the graph is to remove large portions of the graph.

Zhang et al. (2025), in an ICLR paper with the unusually appropriate title “Cut the Crap”, formalised communication redundancy in LLM multi-agent systems and introduced AgentPrune. Across six benchmarks, the method reduced tokens by between 28.1% and 72.8% while maintaining strong performance. The authors report comparable results to state-of-the-art topologies at a cost of $5.60 compared with $43.70 for the compared systems (Zhang et al., 2025). That is a cost ratio of roughly 7.8 to one.

Zeng et al. (2025), in a NAACL long paper on multi-agent debate, went further. Their sparsification method reduced token cost by as much as 94.5% while keeping performance degradation below 2%. If nearly nineteen out of every twenty tokens in a communication-heavy debate scheme can be removed for less than a two-percentage-point performance loss in the reported setting, then “more inter-agent discussion” cannot be treated as intrinsically valuable. Much of it is computational noise.

Z. Wang et al. (2025) reached a similar conclusion through dynamic agent elimination. AgentDropout identifies redundant agents and communication across rounds and removes them. The authors report average reductions of 21.6% in prompt tokens and 18.4% in completion tokens alongside an improvement in task performance. The graph becomes both cheaper and better when unnecessary participants disappear.

Li et al. (2025) developed Adaptive Graph Pruning to vary both the number of agents and the communication topology according to the task. They report token reductions exceeding 90% in their experiments while improving performance across several benchmarks. Fan et al. (2026), writing in the AAAI proceedings, describe unrestricted multi-agent communication as inefficient “free-for-all” communication associated with rapidly growing token costs and low signal-to-noise ratios. Their proposed solution treats communication bandwidth as scarce and makes agents compete for the right to speak.

The common lesson is not subtle. Communication has a price. Most graphs communicate too much. Static topologies allocate communication without sufficient evidence that the recipient needs it. The system then spends additional computation discovering, after the fact, that a large fraction of its own communication was unnecessary.

A direct-window workflow begins from the opposite assumption. Communication does not occur unless the user decides it is useful. A source-verification window may return three corrected references. The user can paste those three corrections into the drafting window rather than transferring the entire verification transcript. A mathematical checker may identify one invalid step. The user can carry the correction, not the checker’s complete chain of interaction. A critical reader may identify two weak claims. Only those claims need to return to the writer.

This is manual information compression, but that phrase understates what is happening. The human is performing semantic routing using knowledge of the actual goal. Graph systems try to automate that routing by adding more model calls. When those calls are imperfect, they either move too much information, omit important information, or require another layer to check the routing.

It is entirely possible to build a sophisticated graph that approximates the human’s decisions. The question is why one would pay to reproduce a decision one can make directly when working interactively.

The human-management tax is the part benchmarks frequently miss

Token counts are visible. Human attention is harder to measure, and that makes it easier to pretend it is free.

A graph-based workflow creates a new job for the user: manager of the agents. The user must define roles, decide how much autonomy to grant, specify hand-off rules, diagnose duplicated work, inspect failures, decide whether the supervisor routed correctly, determine whether a stalled agent should be restarted, reconcile incompatible outputs, adjust context and sometimes recover from a graph that has confidently spent a large budget doing the wrong thing.

For a developer building a production system, that work may be legitimate engineering. For an individual trying to produce a paper, analyse evidence, write code or prepare a report, it is frequently pure displacement. Time that would have been spent checking the substance is spent administering the mechanism.

Anthropic’s production account illustrates the scale of this hidden work. The company had to develop explicit delegation heuristics, effort-scaling rules, tool-selection guidance, observability, resumable execution, state management, failure recovery and guardrails against runaway behaviour. It describes the gap between prototype and production as wider than anticipated and notes that minor failures can cascade through a stateful agent system into very different trajectories (Hadfield et al., 2025). That is not a criticism of Anthropic’s engineering. It is evidence that autonomous coordination itself is a hard problem.

The problem becomes more perverse when users feel faster while objectively becoming slower. Becker et al. (2025) conducted a randomised controlled trial with experienced open-source developers working on real issues in repositories they knew well. Developers using early-2025 AI tools took 19% longer to complete tasks than those working without AI assistance. Before the study, participants expected AI to speed them up by 24%; after experiencing the workflow, they still believed it had sped them up by 20% (Becker et al., 2025). METR later warned that its subsequent 2026 data were affected by severe selection problems and provided an unreliable signal of the current productivity effect, so the 19% figure should not be generalised to every modern AI coding workflow (Becker et al., 2026). The important point is narrower and robust: subjective impressions of AI productivity can be badly miscalibrated.

That matters enormously for graph systems because graph interfaces often provide constant visible activity. Agents are planning. Tasks are being assigned. Nodes are changing state. Logs are scrolling. Critiques are being generated. The user receives an abundance of evidence that the system is busy. Busyness is psychologically easy to confuse with progress.

The right productivity metric is not how many agents executed. It is useful output per unit of total cost, where total cost includes tokens, latency, user attention, debugging time, verification and correction.

For my own intensive use, the management burden of agentic workflows can become several times larger than the effort required to operate focused windows directly. I would not present a five- to seven-fold personal-time multiple as a universal experimental estimate; there is no general population study establishing that number. It is a workflow observation and should be labelled as such. But the mechanism behind it is independently documented: agent systems require coordination, state management, delegation, debugging and human evaluation, and current empirical work shows that perceived acceleration can diverge sharply from measured acceleration (Becker et al., 2025; Hadfield et al., 2025).

This is also why benchmark success does not settle the question. Whitfill et al. (2026) had maintainers review 296 AI-generated pull requests from SWE-bench Verified tasks. They found that roughly half of test-passing AI pull requests in their sample would not be merged into the main repository after adjusting for noise in maintainer decisions, and maintainer merge decisions were about 24 percentage points lower on average than automated benchmark scores. The authors correctly warn against interpreting this as a fundamental inability of agents; iterative feedback could improve many patches. What it demonstrates is that “the agent completed the benchmark” and “the human received production-ready work” are different events.

Graph systems can increase the distance between those events by generating more intermediate artefacts that appear complete before a human inspects the final substance.

Why separate windows are often the better architecture

The alternative is not “one giant prompt”. That is a false comparison commonly used to justify graphs.

The practical alternative is several independent, narrow contexts controlled directly by the user. One window can perform literature discovery. A second can audit citations. A third can test mathematics. A fourth can attack the argument. A fifth can draft. A sixth can edit against a style specification. The user transfers only the information that needs to cross between them.

This approach preserves the useful part of multi-agent decomposition while removing the artificial manager.

It has several structural advantages. Each context is smaller. Each model receives a clearer objective. There is less accidental contamination from unrelated intermediate reasoning. The user can choose a different model for each task without teaching a supervisor how to route. Failed work can be discarded without polluting shared state. Strong outputs can be copied verbatim rather than paraphrased through another LLM. Verification can target the final claims instead of every intermediate message. Most importantly, the human can see what each window was asked to do and why.

In a graph, one may need to inspect traces to discover that the supervisor assigned the same source search to three workers. In direct windows, one simply does not make the same assignment three times.

There is also a difference in stopping behaviour. Autonomous agents can be difficult to calibrate on how much effort a task deserves. Anthropic explicitly found this problem in production and added effort-scaling rules after agents overinvested in simple queries (Hadfield et al., 2025). A human working with independent windows has an immediate stop rule: once the question is answered to the required standard, do not open another window.

That sounds trivial. It is economically important.

The graph model treats delegation as something to automate because delegation is visible and programmable. But for a single expert user, delegation may already be cheap. The user knows that one task is mathematical, one is bibliographic and one is stylistic. Asking an LLM supervisor to infer those facts can be more expensive than making the decision directly.

This is a recurring mistake in automation. A process is automatable, therefore it is automated, without asking whether the automated layer costs more than the decision it replaces.

Why the graph looks like what people want

Graph orchestration sells an intuitively attractive story. It looks like a team. People understand teams. They understand managers, specialists, critics, researchers and reviewers. A graph turns an opaque model invocation into a visible organisation. That visibility can be genuinely useful for debugging, compliance and product design.

But there is a danger in confusing legibility with capability.

A node labelled “expert reviewer” does not become an independent expert merely because it has a title. If it uses the same model family, receives similar context and is prompted by the same system designer, its errors may be correlated with the other nodes. A “debate” between three instances of closely related models is not equivalent to disagreement among three people with independent training, incentives and information. A “supervisor” is not automatically good at management merely because its system prompt says supervisor.

The graph therefore satisfies a presentational demand before it satisfies an economic one. It gives users a picture of control. It makes a probabilistic system look modular. It provides traceability. It allows vendors to demonstrate activity. It makes demos more compelling because many things happen.

None of those properties establishes that the graph delivers more useful work per token or per hour.

Indeed, the peer-reviewed efficiency literature increasingly assumes the opposite. AgentPrune removes communication. AgentDropout removes agents. S²-MAD sparsifies debate. Adaptive Graph Pruning prunes both nodes and edges. DALA makes agents pay, metaphorically, for the right to communicate. KVCOMM tries to avoid recomputing overlapping context. Each system begins from the observation that unconstrained orchestration is wasteful and then tries to claw back the lost efficiency.

At some point the obvious question must be asked: if the system becomes efficient by deleting most of the system, why was the large graph the default?

The red-team case — when orchestration really does pay

A serious argument has to survive its best counterexample. There are workloads where multi-agent orchestration is clearly rational.

Anthropic’s own research system is one. Breadth-first web research can involve many genuinely independent searches. Sub-agents can explore separate sources in parallel, use separate context windows and then compress findings for a lead agent. Anthropic reports a 90.2% improvement over a single-agent Claude Opus 4 baseline on its internal research evaluation, and parallel tool use cut research time by as much as 90% on complex queries (Hadfield et al., 2025). If the task is valuable enough and the alternatives are slow sequential searches or context overflow, spending fifteen times the chat tokens can be entirely sensible.

A 2026 clinical-scale workload study provides an even stronger counterexample. Klang et al. (2026) compared a single agent processing heterogeneous batches of retrieval, extraction and dosing tasks with a lightweight orchestrator that assigned each task to a dedicated worker. At large batch sizes, the multi-agent design maintained much higher accuracy and used up to 65-fold fewer tokens than the single-agent alternative. That sounds devastating to the anti-graph thesis until one looks at what the comparison actually tests.

The single-agent baseline was being forced to handle increasingly large heterogeneous batches in one context. The orchestrated design isolated tasks and prevented context interference. In other words, it achieved the same basic advantage that I am arguing for with independent windows: narrow contexts assigned to narrow jobs. The efficiency came from partitioning heterogeneous work instead of stuffing everything into one accumulating context. It was lightweight routing across separable tasks, not an elaborate hierarchy of supervisors repeatedly rewriting each other’s prose.

This is an important exception, and it sharpens the rule. Orchestration is justified when it removes a larger inefficiency than it introduces. It can be valuable when tasks are genuinely separable, when different workers need different tools or data, when the total information exceeds a practical single context, when large batches would otherwise contaminate one another, when parallel latency matters, or when the task is valuable enough that substantially higher token expenditure is acceptable.

The exception does not rescue general-purpose graph orchestration. It tells us exactly when not to use it.

If a single user is writing one article, checking one proof, analysing one dataset, reviewing one codebase or researching one coherent question, the system should not begin by assuming that a miniature corporate hierarchy is needed. It should begin with the cheapest coherent architecture and add coordination only when measurement shows that coordination pays.

That is the standard ordinary engineering would apply to any other expensive component.

The graph also increases the surface area for failure

Token waste is not the only problem. Every additional node and edge can introduce a new class of error.

A router can choose the wrong agent. A worker can misunderstand a delegated summary rather than the original question. A supervisor can paraphrase a correct answer incorrectly. A verifier can reject a correct result. A critic can induce an unnecessary rewrite. A shared memory can retain stale state. A summariser can omit the one detail needed later. Parallel agents can duplicate work. Dependent agents can block one another. Retries can repeat expensive failures. A final synthesiser can flatten disagreement into a confident but incorrect conclusion.

Anthropic’s production account describes precisely this compounding behaviour: minor failures can alter later trajectories, stateful systems are difficult to restart cheaply, and debugging requires tracing decision patterns across the system (Hadfield et al., 2025). This is the reliability version of the graph tax.

A direct-window workflow contains failure boundaries. If the citation checker fails, the mathematical analysis is not automatically contaminated. If one research window goes down a bad path, its output can be ignored. If a draft is poor, the source notes still exist independently. Separation is explicit and controlled by the user rather than hidden in a shared state machine.

There is a broader lesson here about probabilistic software. Modularity is valuable when interfaces are stable. LLM outputs are not stable interfaces in the conventional software-engineering sense. They are variable natural-language artefacts whose interpretation may itself require another model. Adding more modules therefore creates more semantic interfaces, not merely more functions.

Each semantic interface can require translation.

Each translation can require inference.

Each inference can be wrong.

A graph does not eliminate uncertainty. It composes uncertainty.

The proper efficiency test

The argument can be reduced to one metric. Define useful efficiency E as:

E = Q / (T + λL + μH + νC)

Q is the quality or value of the final useful output. T is token expenditure. L is latency. H is human oversight time. C is correction and recovery cost. λ, μ and ν convert those costs into a common decision scale appropriate to the user.

A graph is justified only if it raises E.

This definition prevents several common tricks. A system cannot claim success merely because Q rises while token use explodes. It cannot claim productivity because the model works autonomously for twenty minutes if the user then spends an hour correcting it. It cannot claim parallel speed-up if a supervisor serialises the final bottleneck. It cannot claim lower cognitive load if the user has to monitor a fragile orchestration system instead of checking the substance.

It also explains why the correct architecture varies by task. For a high-value open-ended investigation, a large increase in Q may dominate a fifteen-fold token cost. For a compact writing task, Q may barely change while T, L and H rise sharply. In the latter case, the graph is not advanced. It is inefficient.

The current evidence strongly suggests that this latter case is common enough to matter. Anthropic reports a 15× token multiple for multi-agent systems relative to chat (Hadfield et al., 2025). Sanabria (2026) finds roughly 5×–6× token and latency overhead in a compact decision-support benchmark without aggregate performance gains. Leins et al. (2026) find two- to four-fold token expenditure for relatively modest, task-dependent orchestration gains. LangChain finds that the supervisor itself introduces translation errors and extra token use (Fu-Hinthorn, 2025). Multiple peer-reviewed systems improve multi-agent performance by deleting communication or agents (Fan et al., 2026; Zeng et al., 2025; Zhang et al., 2025; Z. Wang et al., 2025). NeurIPS work identifies repeated context processing as a major systems overhead (Ye et al., 2025). The empirical pattern is not ambiguous: coordination is expensive.

What I would use instead

For individual knowledge work, my default is therefore not an LLM graph. It is controlled parallelism.

Use a small number of focused windows. Give each one a sharply bounded task. Keep its context as narrow as the task permits. Let it produce an artefact that can stand on its own: a source table, a proof check, a counterargument, a section draft, a list of factual corrections, a code patch. Transfer only the information required by the next step. Use deterministic tools for deterministic work. Use the strongest model where reasoning quality matters and cheaper or local models where transformation is mechanical. Do not pay an LLM to decide something you already know.

This architecture may look less impressive because the graph exists in the user’s head rather than on a dashboard. Economically, that is often its advantage.

The user is not an inconvenient human loop to be engineered away. In many high-skill workflows, the user can be the cheapest and most accurate router in the system. The user knows which result matters, which source is authoritative, which claim requires another check and when the task is complete. Replacing that judgement with an inference call can be automation for its own sake.

There will be cases where the human genuinely becomes the bottleneck. At that point, automate the bottleneck. But measure it first.

Conclusion

AI agent graphs are inefficient because they convert coordination into inference. They add model calls whose purpose is to administer other model calls. They duplicate and reprocess context. They create translation layers that can corrupt specialist output. They expand latency. They create state-management and debugging burdens. They increase the number of places where probabilistic errors can enter the workflow. And they frequently require specialised pruning, summarisation, forwarding, caching and communication-control mechanisms simply to recover resources consumed by the orchestration layer itself.

The strongest evidence comes from the people building these systems. Anthropic reports about 15× the token use of ordinary chat for multi-agent systems (Hadfield et al., 2025). LangChain reports extra token use and performance loss from supervisor translation (Fu-Hinthorn, 2025). The efficiency literature reports reductions of 28% to 73%, more than 90%, or as much as 94.5% of token use in particular multi-agent settings while preserving most or all performance (Li et al., 2025; Zeng et al., 2025; Zhang et al., 2025). A controlled 2026 single-versus-multi-agent benchmark finds approximately five- to six-fold token and latency overhead for orchestration without a corresponding gain in aggregate task quality (Sanabria, 2026). These are not edge observations. They describe the cost structure of the architecture.

The fashionable graph gives people something they recognise: a team, a hierarchy, a manager, a workflow. What it often delivers is a simulation of organisation in which every meeting is billed by the token.

For tasks that genuinely require broad parallel search, independent tool access, context isolation or batch partitioning, orchestration can be worthwhile. Those cases should be demonstrated, not presumed. Everywhere else, the economically rational starting point is simpler: use the model directly, split genuinely independent work into focused windows, keep context local, transfer only what matters, and let the human perform the high-value routing.

If an architecture needs fifteen times the tokens to look organised, the graph is not intelligence. It is overhead.

References

Amdahl, G. M. (1967). Validity of the single processor approach to achieving large scale computing capabilities. In Proceedings of the April 18–20, 1967, Spring Joint Computer Conference (pp. 483–485). Association for Computing Machinery. https://doi.org/10.1145/1465482.1465560

Becker, J., Rush, N., Barnes, B., & Rein, D. (2025, July 10). Measuring the impact of early-2025 AI on experienced open-source developer productivity. METR. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/

Becker, J., Rush, N., Cunningham, T., Rein, D., & Mahamud, K. (2026, February 24). We are changing our developer productivity experiment design. METR. https://metr.org/blog/2026-02-24-uplift-update/

Chase, H. (2025, March 15). AI agent latency 101: How do I speed up my AI agent? LangChain. https://www.langchain.com/blog/how-do-i-speed-up-my-agent

Coase, R. H. (1937). The nature of the firm. Economica, 4(16), 386–405. https://doi.org/10.1111/j.1468-0335.1937.tb00002.x

Fan, Y., Zhang, J., Cai, K., Yang, J., Tang, C., Wang, J., & Wang, K. (2026). Cost-effective communication: An auction-based method for language agent interaction. Proceedings of the AAAI Conference on Artificial Intelligence, 40(35), 29412–29420. https://doi.org/10.1609/aaai.v40i35.40182

Fu-Hinthorn, W. (2025, June 10). Benchmarking multi-agent architectures. LangChain. https://www.langchain.com/blog/benchmarking-multi-agent-architectures

Hadfield, J., Zhang, B., Lien, K., Scholz, F., Fox, J., & Ford, D. (2025, June 13). How we built our multi-agent research system. Anthropic. https://www.anthropic.com/engineering/multi-agent-research-system

Klang, E., Omar, M., Raut, G., Agbareia, R., Timsina, P., Freeman, R., Gavin, N., Stump, L., Charney, A. W., Glicksberg, B. S., & Nadkarni, G. N. (2026). Orchestrated multi agents sustain accuracy under clinical-scale workloads compared to a single agent. npj Health Systems, 3, Article 23. https://doi.org/10.1038/s44401-026-00077-0

LangChain. (n.d.). create_supervisor. Retrieved August 21, 2026, from https://reference.langchain.com/python/langgraph-supervisor/supervisor/create_supervisor

Leins, N., Pelleriti, N., Gonnermann-Müller, J., & Pokutta, S. (2026). When does LLM orchestration pay off? A controlled evaluation of accuracy, cost, and task difficulty [Preprint]. arXiv. https://arxiv.org/abs/2608.00685

Li, B., Zhao, Z., Lee, D.-H., & Wang, G. (2025). Adaptive graph pruning for multi-agent communication [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2506.02951

Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638

Runkle, S. (2026, January 14). Choosing the right multi-agent architecture. LangChain. https://www.langchain.com/blog/choosing-the-right-multi-agent-architecture

Sanabria, D. (2026). OpenAI single-agent LLM architecture reduces computational overhead relative to multi-agent orchestration in a simulated Mars rover decision-support benchmark. Frontiers in Robotics and AI, 13, 1877762. https://doi.org/10.3389/frobt.2026.1877762

Wang, Q., Tang, Z., Jiang, Z., Chen, N., Wang, T., & He, B. (2025). AgentTaxo: Dissecting and benchmarking token distribution of LLM multi-agent systems. In ICLR 2025 Workshops: FM-Wild. https://mlanthology.org/iclrw/2025/wang2025iclrw-agenttaxo/

Wang, Z., Wang, Y., Liu, X., Ding, L., Zhang, M., Liu, J., & Zhang, M. (2025). AgentDropout: Dynamic agent elimination for token-efficient and high-performance LLM-based multi-agent collaboration. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 24013–24035). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.acl-long.1170

Whitfill, P., Wu, C., Becker, J., & Rush, N. (2026, March 10). Many SWE-bench-passing PRs would not be merged into main. METR. https://metr.org/notes/2026-03-10-many-swe-bench-passing-prs-would-not-be-merged-into-main/

Ye, H., Gao, Z., Ma, M., Wang, Q., Fu, Y., Chung, M.-Y., Lin, Y., Liu, Z., Zhang, J., Zhuo, D., & Chen, Y. (2025). KVCOMM: Online cross-context KV-cache communication for efficient LLM-based multi-agent systems. In Advances in Neural Information Processing Systems 38. https://doi.org/10.52202/085713-0605

Zeng, Y., Huang, W., Jiang, L., Liu, T., Jin, X., Tiana, C. T., Li, J., & Xu, X. (2025). S²-MAD: Breaking the token barrier to enhance multi-agent debate efficiency. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (pp. 9393–9408). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.naacl-long.475

Zhang, G., Yue, Y., Li, Z., Yun, S., Wan, G., Wang, K., Cheng, D., Yu, J. X., & Chen, T. (2025). Cut the crap: An economical communication pipeline for LLM-based multi-agent systems. In International Conference on Learning Representations (ICLR 2025). https://proceedings.iclr.cc/paper_files/paper/2025/hash/bbc461518c59a2a8d64e70e2c38c4a0e-Abstract-Conference.html

---

← Back to Substack Archive