10 PRINT CRAP, 20 GOTO 10
The AI Loop That Eats Your Credits
Self-testing graphs, evaluator loops, agent debates and competing drafts turn inference into recursive bureaucracy. The system can look more sophisticated precisely because it is doing more work. The one quantity guaranteed to improve is the usage meter.
Keywords: AI agents, agent graphs, LangGraph, self-correction, self-reflection, evaluator-optimizer, multi-agent systems, test-time scaling, inference cost, token economics, API pricing, recursive agents, Goodhart’s law, AI productivity
Thesis
The fashionable move from a direct model interaction to graph-based agent systems does not merely add software structure. It creates protocols whose ordinary mode of operation is repetition: generate, inspect, criticise, regenerate, compare, debate, judge and, if the protocol has not yet declared itself satisfied, repeat. That repetition is not free. Every additional model turn consumes context, output tokens, tool calls, latency and eventually either subscription credits or API money. Worse, the repeated work can become self-justifying. A model is asked whether another model’s answer needs improvement; the evaluator finds something to improve; a new answer is generated; another evaluator is then asked whether the new answer needs improvement. The graph has created a machine that can manufacture reasons to continue running.
For general-purpose writing, analysis and research, I regard this as an inefficient architecture and, in many cases, an absurd one. It is the computational equivalent of writing a BASIC program whose principal achievement is that it never stops printing.
10 PRINT "CRAP"
20 GOTO 10
The modern version merely has better branding.
10 GENERATE CANDIDATE
20 EVALUATE CANDIDATE
30 CRITIQUE EVALUATION
40 REVISE CANDIDATE
50 GENERATE COMPETING CANDIDATES
60 JUDGE THE CANDIDATES
70 IF "COULD BE IMPROVED" THEN GOTO 10
The joke is that the second program can be sold as an agentic workflow.
The argument is not that iteration can never improve an answer. It plainly can. Nor is it that parallelism is useless. It plainly is not. The argument is that current agent frameworks make iteration, delegation and internal evaluation unusually easy to add while the cost of those additions is usually externalised to the user. If benchmark scores reward the final answer while failing to penalise token expenditure, latency, coordination traffic and human supervision sufficiently, protocols that consume more inference can be selected simply because they are permitted to spend more. Once those protocols are normalised as “best practice”, the user inherits the bill.
There is now enough empirical evidence to stop treating this as a hypothetical concern. Multi-agent systems have been measured using several times the reasoning turns of single-agent systems under matched compute ceilings; self-correction research has repeatedly found that an LLM’s intrinsic criticism is not a reliable source of new information; pruning studies have removed large fractions of multi-agent communication while retaining comparable performance; and early-stopping methods have eliminated much of repeated sampling without sacrificing measured accuracy. Even companies building these systems publicly describe agents that over-search, duplicate work, spawn excessive subagents and continue after they already possess sufficient information (Hadfield et al., 2025; Kamoi et al., 2024; Y. Kim et al., 2026; Y. Li et al., 2024; Zhang et al., 2025).
That is not an unfortunate edge case. It is what happens when the protocol makes continuation cheap to decide but expensive to execute.
The loop is not a metaphor. It is the architecture
The first point is almost embarrassingly literal. Graph-based agent frameworks are designed to represent cycles.
LangGraph’s own documentation presents an “evaluator-optimizer” workflow in which one LLM creates a response and another evaluates it. If the evaluator decides that the response needs refinement, feedback is returned and the response is recreated. The documentation states that this loop continues until an acceptable response is generated (LangChain, n.d.-c). Anthropic describes the same pattern: one call generates, another evaluates, and the process repeats in a loop when iterative refinement appears useful (Schluntz & Zhang, 2024).
There is nothing technically illegitimate about that pattern. A compiler can compile repeatedly. A numerical optimiser can iterate. A test suite can be rerun. The problem is epistemic and economic. A deterministic test normally has a defined relationship to the object being tested. The program either compiles or it does not. A unit test either passes or fails under specified conditions. A theorem prover validates a proof against formal rules. A database query returns evidence that was not present in the model’s prior text.
An LLM evaluator is different. In the common case it is another probabilistic model call, often using the same model family, related training data, similar priors and the same underlying task description. It is not automatically an independent instrument merely because the software diagram gives it a box labelled “critic”. Calling the same class of system twice does not magically manufacture external evidence.
The graph nevertheless turns the critic’s judgement into control flow. If the evaluator emits “needs improvement”, an edge routes execution backwards. That means a linguistic judgement has become a spending decision.
LangGraph has an explicit recursion-limit error because a state graph can continue stepping without reaching a stop condition. Its documentation notes that reaching the limit is often caused by an infinite loop and separately explains that applications with loops require termination mechanisms; in some applications, termination cannot be guaranteed, so a recursion limit must be imposed (LangChain, n.d.-a, n.d.-b). This is good engineering documentation. It is also an admission of what the abstraction makes possible. The graph is capable of converting uncertain model judgements into repeated inference until some other rule forcibly stops it.
That is a profoundly different failure mode from a bad single answer. A bad single answer costs one answer. A bad stopping policy can cost an arbitrary number of answers.
The important object is therefore not merely model quality. It is the transition rule.
Suppose an evaluator has probability q of requesting another round even after the answer has reached a level that the user would accept. If each unnecessary round contains a generator and an evaluator, then every false continuation decision buys two more model calls. If those calls expose additional imperfections simply because another stochastic evaluator has been asked to look for imperfections, the process can sustain itself. The system has no physical sensation of boredom, no salary clock, no instinct that the afternoon is disappearing and no credit-card statement. The stopping discipline must be imposed from outside.
Humans have this problem too. A committee can keep revising a memo because every reviewer can invent another revision. The difference is that organisations have developed cultural limits for committees: deadlines, budgets, authority, embarrassment and eventually somebody saying that the document is finished. Agent graphs can remove those social brakes while retaining the committee structure.
What remains is bureaucracy at machine speed.
“Self-testing” often means asking the same epistemic source twice
The sales pitch for a self-testing agent is intuitively attractive. The first model may make a mistake, so ask a second model to check it. Better still, ask several. Then ask one to adjudicate the disagreement. Then perhaps let the original model revise its answer. It sounds like peer review compressed into seconds.
The analogy is weak.
Human peer review has value partly because reviewers may possess independent expertise, different evidence, different experiences and different incentives. A software test has value because its rules can be external to the program being tested. A measurement instrument has value because it observes something. Internal LLM self-critique can have none of those properties. It can instead be the same statistical machinery rereading a transformation of its own output.
Huang et al. (2024) tested intrinsic self-correction in reasoning and found that the models they studied struggled to improve their reasoning without external feedback; performance could even deteriorate after self-correction. Their result should not be inflated into the universal proposition that self-correction never works. The authors themselves were careful about scope. The relevant point is narrower and more damaging to the generic graph pattern: an additional self-critique call is not evidence that an error has been detected correctly.
Kamoi et al. (2024) subsequently reviewed the literature in detail. Their conclusion was that, for general tasks, prior work had not shown reliable successful self-correction using feedback from prompted LLMs under fair settings, except in tasks unusually well suited to self-correction. They found that self-correction worked much better where reliable external feedback existed, and that the bottleneck was often feedback generation itself.
That distinction should be the organising principle for agent design.
If the loop imports new information, the next round may be worth buying. If a compiler reports a type error, a test returns a failing assertion, a browser retrieves a primary source, a database exposes a contradictory record or a formal verifier rejects a proof step, the model has learned something it did not possess before the tool call. The feedback channel has information content.
If the loop merely asks an LLM, “Can this be improved?”, the predictable answer is often yes. Almost any prose can be improved. Almost any argument can be extended. Almost any research note can be made more comprehensive. Almost any code can be refactored. “Could be improved” is not a stopping criterion. It is an invitation to spend indefinitely.
This is why a graph can look busy without becoming proportionately more intelligent. It generates internal events. One model produces a candidate. Another produces criticism. A third produces a synthesis. These are all observable artefacts, so the trace looks industrious. Yet the information source may not have changed.
The graph has confused motion with progress.
There is positive evidence for iterative refinement, and it deserves to be taken seriously. Self-Refine, for example, reported roughly 20 percentage points of average improvement across seven evaluated tasks using iterative feedback and refinement from the same LLM (Madaan et al., 2023). That result is important because it rules out the equally silly opposite claim that iteration is never useful. But it does not justify an unlimited or default loop. Kamoi et al.’s later critical survey shows why the experimental conditions, baseline strength, task properties and feedback source matter. The correct question is not “Can a loop ever improve output?” It is “Does this particular additional round add enough expected value to justify its marginal cost compared with spending the same compute another way?”
That is an economic question. Agent frameworks too often answer it with another model call.
The arithmetic becomes ridiculous almost immediately
Strip away the diagrams and the multiplication is simple.
A basic evaluator-optimizer round has one generation and one evaluation. If it runs for r rounds, a simplified call count is:
N = 2r
Now add competing variants. Suppose each round generates k candidates, evaluates each candidate separately and then uses one judge to select among them. The call count becomes:
N = r(2k + 1)
This excludes tool calls, retries, summarisation, memory operations, routing calls, safety checks, citation passes and any context-building calls. It is only the model-call skeleton.
With five candidates and four rounds:
N = 4 × (2 × 5 + 1) = 44 calls
A direct single-pass baseline is one call.
This does not prove that the one-call answer is equally good. It proves something more elementary: the architecture has multiplied the opportunity to consume inference by forty-four before we have counted the rest of the machinery.
Figure 1. Recursive agent protocols multiply inference calls by design. The curves show protocol arithmetic rather than benchmark measurements. A direct pass remains one model call. A generator-evaluator loop uses two calls per round. Competitive protocols that generate k candidates, evaluate each and then call a judge use r(2k + 1) calls before tool calls, retries or context replay are counted.
The deeper issue is that each call can make later calls more expensive. Generated text often re-enters context. Critiques are appended to state. Candidate answers are passed to judges. Tool outputs are returned to the model. Summaries are generated and later re-read. The graph is not only producing output tokens; it is manufacturing future input tokens.
The billing identity for a token-priced service is approximately:
C = pᵢI + pₒO + Σfⱼ
where C is monetary cost, pᵢ and pₒ are input and output prices, I and O are billable input and output tokens, and Σfⱼ represents any separately metered tool charges. Caching can reduce part of the input term, batch processing can alter rates, and provider-specific rules differ. None of that changes the direction of the mechanism. If a graph adds calls, tokens or separately billed tool events without removing at least as much work elsewhere, metered consumption rises.
At the time of writing, OpenAI lists GPT-5.6 Sol at US$5 per million input tokens and US$30 per million output tokens for short-context standard API use, with cached input priced separately; its documentation also notes that some tool-specific models attract per-call fees (OpenAI, n.d.). Google’s Gemini API similarly prices input and output tokens, explicitly includes thinking tokens in output pricing for current models, and separately meters some grounded-search usage. Its code-execution documentation is especially revealing because generated code and execution results can become intermediate material that is billed as the system iterates (Google, n.d.-a, n.d.-b).
This is not criticism of token pricing as such. Computation has a cost. Providers are entitled to charge for it. The point is that the graph’s internal bureaucracy is not economically invisible merely because the user asked for one final answer.
The invoice sees the bureaucracy.
Competing agents can turn sampling into a prestige version of brute force
A particularly appealing graph pattern is competition. Generate several independent answers. Let agents debate. Ask judges to compare them. Perhaps use majority voting. Perhaps promote the winner into another refinement stage.
Again, this can work. The question is what has caused the gain.
J. Li et al. (2024) titled their paper More Agents Is All You Need and showed that a simple sampling-and-voting method could improve performance as more agents were instantiated. That result helped establish the intuition that additional agents can create “collective intelligence”. Yet the intervention also creates additional samples. More agents means more inference.
If compute is allowed to increase with the number of agents, architecture and expenditure are confounded. The system with ten agents has not merely been organised differently; it has been allowed to think, sample or communicate more times.
Tran and Kiela (2026) attacked exactly this issue. They compared single-agent and multi-agent systems on multi-hop reasoning while holding reasoning-token budgets constant across three model families. Under those matched budgets, the single-agent systems consistently matched or outperformed the multi-agent systems in their experiments. Their argument is not that no multi-agent architecture can ever win. It is that some published apparent gains are better explained by extra test-time computation and context effects than by an intrinsic advantage in splitting the model into a society of agents.
That distinction is fatal to sloppy benchmarking.
If I allow system A one attempt and system B ten attempts plus a vote, then announce that B’s “architecture” is superior without pricing the nine additional attempts, I have not measured the thing I claim to have measured. I have partly measured my willingness to spend.
The same point appears in work on self-consistency. Repeated chain-of-thought sampling can improve results, but Y. Li et al. (2024) showed that early stopping could reduce the average number of samples by 33.8% to 84.2% across six benchmarks while retaining comparable performance in their experiments. In other words, a substantial portion of the preset repeated sampling was unnecessary once the protocol learned when it had enough evidence.
This is the pattern I expect to recur across agent systems. First, a brute-force protocol gains attention because it produces a higher score. Then somebody asks whether all the calls were necessary. Often, the answer is no.
The waste was not a mysterious requirement of intelligence. It was a property of the first protocol.
When the benchmark rewards the answer but ignores the bill, waste can be selected
This is where the evolutionary metaphor becomes useful, provided it is not taken literally.
Imagine a population of agent protocols competing on a benchmark. Protocol A uses one strong model call. Protocol B generates four candidates and chooses the best. Protocol C generates four candidates, gives each a critic, runs two debate rounds and then uses a judge. Protocol D does all of that and retries whenever confidence falls below some threshold.
Now define fitness as benchmark accuracy.
There is no penalty for latency. There is no penalty for tokens. There is no penalty for tool calls. There is no penalty for the human who later has to inspect the trace. There is no penalty for a run that costs ten times as much if it gains half a percentage point.
What should we expect to be selected?
We should expect expenditure to expand until its marginal contribution to the measured benchmark becomes negligible or some external resource limit intervenes. The benchmark has made consumption cheap in the objective function even though consumption remains expensive in reality.
This is a Goodhart-style failure (Goodhart, 1975). The true objective is something like useful work per unit of scarce resource. The proxy is often final-task score. Optimising the proxy while omitting resource use rewards protocols that buy performance with invisible compute. Once the benchmark winner becomes a blog post, framework template or recommended architecture, the hidden compute returns as somebody else’s token bill.
The mechanism does not require anybody to be stupid. It does not require fraud. It does not require a secret meeting at an AI company where executives demand more loops. It requires only an evaluation regime that rewards answer quality more strongly than efficiency.
The result can resemble evolutionary ornamentation. The user suggested the peacock’s tail, and the analogy is useful with one scientific qualification. The old “handicap principle”, interpreted as the proposition that wastefulness itself is what makes a signal reliable, has been strongly criticised; Penn and Számadó (2020) argue that this formulation became accepted despite defective theoretical foundations. So the correct analogy is not that waste has some mystical adaptive virtue. It is that a selection environment can preserve expensive traits when the environment rewards what accompanies them and fails to punish the cost sufficiently.
Agent traces can acquire the same ornamental quality. Five agents, three critics, a planner, an arbiter, a verifier and a synthesis node look more sophisticated than one well-instructed model window. The graph is visible. The boxes have names. The execution trace resembles an organisation chart. The user can watch activity flow through it.
None of those visual properties establishes productive efficiency.
A peacock cannot invoice you for the tail. An API can.
Nature has now measured the coordination tax directly
The strongest recent evidence comes from Y. Kim et al. (2026) in Nature Machine Intelligence. The study is important because it was designed to separate architecture from uncontrolled compute. Across 260 configurations, six benchmarks, five architectures and three LLM families, the researchers held task prompts, tools and per-system compute ceilings constant while varying coordination structure and model capability.
The results are devastating to the idea that more coordination is automatically better.
Multi-agent architectures incurred large realised reasoning-turn overheads relative to the single-agent baseline: 58% for the independent architecture, 263% for decentralised coordination, 285% for centralised coordination and 515% for the hybrid system. In the full-cost-tracked subset, the single-agent baseline averaged 7.2 reasoning turns, decentralised systems 26.1, centralised systems 27.7 and hybrid systems 44.3. The hybrid therefore used about 6.2 times the reasoning turns of the single-agent baseline under matched per-system compute ceilings (Y. Kim et al., 2026).
Figure 2. Coordination consumes the reasoning budget. Mean reasoning turns reported by Y. Kim et al. (2026) in the 180-configuration full-cost-tracked subset were 7.2 for the single-agent baseline, 26.1 for decentralised systems, 27.7 for centralised systems and 44.3 for hybrid systems. The study used matched per-system compute ceilings, so the additional turns represent coordination structure consuming the available reasoning budget rather than a larger permitted token ceiling.
More strikingly, the researchers found super-linear growth in reasoning turns as agent count increased. Their fitted relationship had an exponent of 1.724 with R² = .974. They interpreted this as a coordination regime in which communication complexity expands rapidly with team size. Their extrapolation predicted roughly 157 turns at ten agents, around 21.8 times the single-agent baseline, although that larger-team number is an extrapolation rather than a directly observed ten-agent mean and should be read accordingly (Y. Kim et al., 2026).
The additional activity did not guarantee additional success. Hybrid systems recorded 515% coordination overhead and 44.3 turns while achieving 47.7% success, statistically indistinguishable from centralised systems at 285% overhead and 27.7 turns in the relevant comparison. The authors report that messages after consensus often added redundancy rather than novel information (Y. Kim et al., 2026).
Their token-efficiency results are equally awkward for the cult of orchestration. The single-agent configuration achieved 67.7 successes per 1,000 tokens in their metric. Centralised fell to 21.5, decentralised to 23.9 and hybrid to 13.6. The hybrid architecture was therefore about five times worse than the single-agent baseline on that token-efficiency measure (Y. Kim et al., 2026).
This is close to the 10 PRINT CRAP; 20 GOTO 10 problem in empirical form. The system is doing things. The trace is longer. Messages are being created. Agents are talking. Yet after a point the additional communication is consuming the very budget that might otherwise have been used for substantive reasoning.
Coordination has become the workload.
The funniest evidence is that researchers can delete the chatter
There is an especially simple way to test whether multi-agent communication is valuable: remove some of it.
Zhang et al. (2025) did exactly that in a paper whose title is refreshingly appropriate: Cut the Crap: An Economical Communication Pipeline for LLM-Based Multi-Agent Systems. Their AgentPrune method was designed to remove redundant communication from multi-agent message-passing graphs. Across six benchmarks they evaluated the pruning approach; on MMLU, they reported comparable performance to state-of-the-art communication topologies at a cost of US$5.60 compared with US$43.70, while integrations with existing multi-agent frameworks reduced token use by 28.1% to 72.8% in their experiments.
The cost comparison is roughly a 7.8-fold ratio. Put differently, the lower figure is about 87% below the higher figure.
That does not prove that 87% of every agent graph is waste. It proves something more useful: in the systems and benchmarks they studied, a very large amount of communication could be removed while maintaining comparable outcomes. The original dense communication structure was not synonymous with useful intelligence.
This is precisely what one would expect from a protocol selected for performance before it is selected for thrift. The first design discovers that communication can help. It adds communication everywhere. Later work discovers that most edges are not equally useful and starts cutting them.
The graph diagram has an unfortunate psychological effect here. An edge looks like structure. Structure looks intentional. Intentional structure looks necessary. But a communication edge in an LLM system is often an inference event. If that edge carries redundant information, the diagram is not merely cluttered. It is a billable clutter map.
AgentPrune should therefore be read as more than an optimisation paper. It is empirical evidence against the assumption that elaborate communication topologies deserve the benefit of the doubt.
The burden should run the other way. Every recurring edge should have to earn its tokens.
Agents themselves are bad at deciding how much agenting is enough
The next defence is predictable: “Fine. We will let the AI decide how many agents and rounds are necessary.”
That moves the resource-allocation decision to the same class of system whose resource appetite we are trying to constrain.
Anthropic’s account of building its multi-agent research system is unusually candid. Early versions spawned 50 subagents for simple queries, searched endlessly for sources that did not exist, duplicated work, continued after sufficient results had already been gathered and selected inefficient search strategies. Anthropic responded by adding explicit effort-scaling rules, detailed delegation instructions and guardrails to stop agents from spiralling out of control (Hadfield et al., 2025).
This matters because Anthropic is not describing a toy criticism from an opponent of agents. It is describing failure modes observed while building a production multi-agent system.
The same article reports that ordinary agents used about four times as many tokens as chat interactions and multi-agent systems about fifteen times as many as chats in Anthropic’s data. It explicitly states that these architectures burn through tokens quickly and are economically viable only where task value is high enough to pay for the additional performance. It also says token usage alone explained 80% of performance variance in the BrowseComp evaluation analysed by the team (Hadfield et al., 2025).
Read those statements together.
More tokens often improve performance. Multi-agent systems are effective at spending more tokens. They also burn tokens quickly. Agents can overinvest in simple queries unless rules prevent them from doing so.
That is not an incidental implementation detail. That is the economic core of the architecture.
An unconstrained agent does not possess an endogenous concept of your opportunity cost. It does not know that you could have completed three other jobs while it was orchestrating a council of critics. It does not care that a subscription has a weekly credit pool. It does not experience the difference between a US$1 run and a US$20 run unless cost is explicitly represented in its objective or constraints. If the success criterion says “produce the best possible answer”, the rational protocol may consume everything you allow it to consume.
“Use the AI to optimise the AI” is therefore not a resource policy. It is recursion.
Credits disappear because the loop is metered, not because the loop is intelligent
Users encounter this problem through two different interfaces.
On an API, the mechanism is transparent. Tokens, tool usage and sometimes other resources are priced. If an orchestrator makes more requests, gross usage rises. OpenAI describes a Codex bug-fixing workflow as involving dozens of back-and-forth Responses API requests: decide the next action, run a tool, return the tool output, repeat (Yu & Nathan, 2026). That may be justified for a difficult coding task. It is also exactly what repeated agentic execution means operationally.
On a subscription or credit-based product, the same computation may be abstracted behind “credits”, “usage”, rate limits or plan allowances. The user no longer sees a dollar amount for each request, but the resource has not disappeared. A loop can simply empty the bucket faster.
This is why the apparent autonomy of these systems is dangerous to budget control. Manual use has a natural friction. I open a window. I ask a question. I inspect the answer. I decide whether another query is worth my time. Human impatience is a rate limiter.
A graph removes that friction. The output of one paid inference can trigger the next paid inference without the user seeing the intermediate decision. Branches can run in parallel. Competing drafts can be generated automatically. Critics can request revisions automatically. Failed tools can be retried automatically. Supervisors can create subagents automatically.
Automation converts a sequence of conscious spending decisions into a protocol.
This is excellent when the protocol has been engineered around a task whose value exceeds its cost. It is terrible when the protocol is “keep improving until an LLM feels satisfied”.
The worst version is effectively an open cheque written to a stochastic stopping rule.
The provider incentive is real even without a conspiracy theory
This leads to the uncomfortable commercial point.
If a provider charges for tokens, tool calls or usage credits, then greater metered consumption produces greater gross billings, all else equal. That is arithmetic, not an allegation about anybody’s state of mind.
A vendor that recommends an architecture containing more model calls therefore occupies an obvious economic position. Its customer may obtain better output. The vendor may also incur higher infrastructure costs. Caching, batching and competition may reduce margins. There is no basis for claiming that every engineer who recommends an evaluator loop is secretly trying to inflate a customer’s invoice. That stronger psychological claim is not verifiable from public evidence, and I will not pretend that it is.
The systemic incentive is sufficient.
A company selling metered inference is not economically indifferent between an architecture that uses one million tokens and one that uses fifteen million tokens, just as a cloud provider is not economically indifferent between an application that occupies one server-hour and one that occupies fifteen. The fact that the larger workload costs the provider more to serve does not make revenue independent of usage.
That means architectural advice from an inference vendor should be evaluated with exactly the same scepticism applied to any supplier recommending greater consumption of the thing it sells.
This is not unique to AI. A consultant benefits from consulting hours. A cloud platform benefits from workloads. An advertising platform benefits from ad spend. None of that proves dishonest advice. It does mean the buyer should insist on evidence of marginal value.
The peculiar problem in agentic AI is that the fashionable architecture itself can hide consumption behind apparent sophistication. “Use an evaluator.” “Add a reflection step.” “Create specialist agents.” “Let them debate.” “Run multiple candidates.” “Add an orchestrator.” Every sentence sounds like quality assurance. Every sentence can also add inference.
The vendor may be perfectly sincere. The meter remains delighted.
Test-time scaling can become an excuse for refusing to engineer
There is a broader trend underneath agents: test-time scaling. If additional inference improves performance, one can buy more reasoning at run time rather than relying only on a better single forward pass.
Again, the principle is not nonsense. Additional compute can improve answers. The infrastructure research now shows the cost curve is the issue.
J. Kim et al. (2026) studied dynamic reasoning and agentic test-time scaling from an infrastructure perspective. They found that increasing compute can improve accuracy but also produces rapidly diminishing returns, wider latency variance and substantial infrastructure and energy costs. Reflection depth and parallel reasoning are not abstract prompt-design choices; they are system-level resource choices.
The danger is that test-time scaling becomes the LLM equivalent of solving inefficient software with a larger server. Instead of asking why the first pass is poor, why the prompt is ambiguous, why retrieval is weak, why deterministic code is not being used, why the task was decomposed badly or why the benchmark encourages brute force, one simply buys more samples.
That approach is commercially convenient because it turns quality problems into consumption.
The history of computing is full of performance reclaimed by eliminating unnecessary work. Caches, indexes, vectorisation, compiler optimisation, sparse representations, early termination and algorithmic improvements exist because doing less work for the same useful result is an engineering achievement.
Agentic fashion occasionally appears to reverse that instinct. More turns become “reasoning”. More agents become “collaboration”. More messages become “verification”. More retries become “resilience”. A longer trace becomes evidence that the system worked hard.
No. Work is a cost. Useful output is the product.
A self-test is valuable only when failure means something
There is a simple rule that separates useful loops from ceremonial ones.
A loop deserves to exist when its feedback can falsify the current answer in a way that matters.
A compiler can falsify the claim that code compiles. A unit test can falsify a specified behavioural claim. A numerical check can falsify arithmetic. A database query can falsify a factual assertion against a chosen source of record. A retrieval step can introduce documents the model did not previously have. A proof checker can reject an invalid proof object.
These mechanisms add information or impose an external constraint.
By contrast, “criticise this answer” is often not falsification. It is another generation task. The critic may prefer a different style, hallucinate a defect, miss the real defect, reverse a correct answer or ask for unnecessary elaboration. A judge model can share the candidate models’ biases. Three agents sampled from the same model can agree because they share the same failure mode.
This is why competing versions are not equivalent to independent replication. Independence has to be earned. Different random seeds are not necessarily different epistemologies.
The practical consequence is severe. If an agent graph is allowed to self-test using criteria that are subjective, open-ended or generated by the same model, its stop condition can drift. “Good enough” becomes a latent linguistic judgement rather than a measurable state. Every additional round can discover another stylistic imperfection because prose has no terminal optimum.
The only guaranteed convergence may be convergence to the budget limit.
Red-team: when the loop is worth paying for
A serious argument has to survive its strongest counterexamples.
There are tasks where graph orchestration is plainly useful. Anthropic reports a 90.2% improvement of its multi-agent research system over a single Claude Opus 4 agent on its internal broad research evaluation, particularly for breadth-first queries that can be decomposed into genuinely independent searches (Hadfield et al., 2025). Y. Kim et al. (2026) found major gains on some structured and decomposable tasks; centralised multi-agent coordination improved Finance Agent by as much as 80.8% in their analysis. The same study found that coordinating architectures reduced factual error rates by an average of 22.7% in the relevant comparison, with larger reductions in structured finance tasks.
Self-Refine found substantial improvements on its selected tasks (Madaan et al., 2023). Kamoi et al. (2024) concluded that self-correction works well when reliable external feedback exists. Multi-agent debate can expose disagreement. Parallel search can overcome a single context window. Tool-using agents can acquire information that a direct answer simply does not contain.
These are real benefits.
They do not rescue the default graph.
They identify the conditions under which extra inference purchases something identifiable: independent search coverage, external verification, decomposition of genuinely parallel subtasks, exploration beyond one context window, or error correction on outputs whose correctness can be assessed. In those cases, the graph is not valuable because it is a graph. It is valuable because the task structure gives the extra calls independent work to do.
The negative cases are just as informative. Y. Kim et al. (2026) found that all tested multi-agent architectures degraded performance on PlanCraft, a sequential planning benchmark, by roughly 39% to 70% relative to the single-agent approach. On SWE-bench Verified, every tested multi-agent architecture was worse than the single-agent baseline: hybrid by 1.3%, centralised by 2.6%, decentralised by 6.4% and independent by 12.8%. Their general finding was that strong single-agent performance predicts when coordination is less likely to help. Their capability-saturation rule correctly predicted the sign of coordination effects in 94% of validation configurations on SWE-bench Verified and Terminal-Bench.
So the rational rule is not “never use agents”. It is almost the opposite of current fashion:
Assume the graph is guilty until its marginal calls prove their value.
A loop should have a hard reason to continue. A branch should have a distinct information objective. A critic should have a verifiable criterion or independent evidence. A multi-agent team should have work that is genuinely parallel rather than cosmetically divisible. Most importantly, the architecture should be compared against a strong direct baseline at the same or explicitly priced compute budget.
Without those controls, complexity is theatre.
The human-management tax makes the economics worse
Token accounting still understates the waste because the user can become the unpaid manager of the artificial organisation.
A direct model window is cognitively simple. I know the task. I read the answer. I correct it if necessary. The state of the work is visible.
An agent graph creates hidden organisational questions. Did the supervisor delegate correctly? Did two agents duplicate the same task? Did a critic reject something that was already right? Did a branch search an irrelevant direction? Did the synthesis omit the best subagent result? Did the system stop because the work was complete or because a recursion limit was hit? Is the impressive trace evidence of coverage or evidence of confusion?
The user now has to supervise the supervisor.
Anthropic’s production account describes exactly these coordination difficulties: vague delegation caused duplication, agents chose poor tools, effort had to be explicitly scaled to task complexity, and guardrails were needed to prevent runaway behaviour (Hadfield et al., 2025). These are engineering costs at the provider level, but equivalent costs appear for users building their own orchestrations.
This is why a graph can consume five times the model budget and more than five times the human attention. The machine is not merely performing the task. It is creating management state that somebody must understand when it behaves badly.
The great promise of automation is to remove administration from the human. A badly designed agent system automates the creation of administration.
A better protocol begins by forbidding waste
The alternative is not to return to primitive one-shot prompting for everything. It is to make efficiency a first-class objective rather than an afterthought.
For most knowledge work I would start with one strong, well-scoped window. If independent subtasks genuinely exist, I would run a small number of independent windows directly and in parallel. I would keep their scopes explicit. I would use deterministic tools for deterministic checks. I would ask models to retrieve external evidence rather than merely to critique their own prose. I would merge results once rather than repeatedly passing them through layers of supervisory paraphrase.
If a loop is required, it should have a hard maximum number of rounds. The stopping criterion should be defined before execution where possible. “No failing tests”, “all citations verified”, “all required fields present” or “numerical residual below ε” are real stopping conditions. “The evaluator thinks it could be better” is not.
If competing candidates are useful, the value of k should be a budget decision rather than a stylistic flourish. If three candidates are enough, generating eight is not “more robust”; it is five unnecessary candidates unless the expected gain can be shown. If confidence becomes decisive after two samples, stop. Y. Li et al.’s (2024) early-stopping results show how much sampling can disappear when the protocol is permitted to recognise that additional votes are no longer informative.
Every graph should expose a cost ledger containing at least model calls, input tokens, output tokens, tool calls, wall-clock latency and retries. The final quality score should be reported beside those numbers, not separately from them. A benchmark that says architecture B scored 83 and architecture A scored 81 while omitting that B used ten times the inference is incomplete.
The metric that matters is not raw answer quality. It is something closer to:
productive efficiency = verified useful output ÷ total scarce resources consumed
Those scarce resources include money, tokens, latency and human attention.
Once that denominator is visible, many impressive agent traces stop looking impressive.
The protocol should not be allowed to decide that its own existence is necessary
There is a recursive governance problem at the centre of all of this.
The graph is often asked to answer two questions simultaneously: “What should the answer be?” and “Have you done enough work?”
Those are different functions.
A system rewarded for improving the answer has a reason to keep finding possible improvements. A critic rewarded for finding defects has a reason to find defects. A researcher instructed to be comprehensive has a reason to keep searching. An agent instructed to verify everything has a reason to create verification work. If the same protocol also decides when to stop, its mission and its resource constraint are in tension.
Human institutions separate these roles for a reason. Researchers want more data; budgets stop them. Engineers want more testing; release deadlines stop them. Lawyers can always write another submission; procedural limits stop them. Academics can always add another robustness check; journals eventually require a paper rather than an eternal research programme.
The budget is not an inconvenience. The budget is part of rational decision-making.
Agent systems need the same external discipline. The model may decide how to spend a fixed allowance. It should not decide that the allowance is infinite.
Otherwise the protocol is structurally close to:
10 DO SOMETHING THAT LOOKS LIKE QUALITY CONTROL
20 ASK WHETHER MORE QUALITY CONTROL WOULD HELP
30 IF YES, SPEND MORE
40 IF NO, ASK A DIFFERENT CRITIC
50 GOTO 10
There will nearly always be another critic.
The peacock tail of agentic AI
This brings me back to the apparent evolutionary oddity.
Technological systems do not evolve biologically, but design ecosystems do select. Researchers select architectures that score well. Framework companies select patterns that demonstrate their frameworks. vendors select examples that showcase advanced capabilities. Social media selects diagrams that look novel. Developers select abstractions that feel powerful. Benchmarks select outputs that satisfy benchmark metrics.
If none of those selectors prices simplicity adequately, complexity accumulates.
A planner node appears. Then a critic. Then a reflection node. Then memory. Then a router. Then specialist agents. Then an arbiter to resolve the specialists. Then a verification agent to inspect the arbiter. Soon the system needs observability software so that humans can understand the machinery created to avoid doing the task directly.
This is not necessarily a conspiracy. It is an ecosystem capable of selecting ornament.
The ornament persists because it has local functions. The critic sometimes catches an error. The extra candidate sometimes wins. The debate sometimes changes the answer. The graph diagram sometimes helps a developer reason about state. Each addition can be individually defensible while the aggregate system becomes economically ridiculous.
That is how bureaucracies grow as well. Almost no individual meeting is created because somebody wishes to destroy productivity. Each meeting has a reason. The calendar is still full.
Agent graphs are bureaucracies with token meters.
Conclusion
The central mistake in current agentic fashion is treating activity as intelligence and structure as efficiency.
A graph can make an LLM system look organised. It can create specialists, reviewers, judges and supervisors. It can display a trace in which every output appears to have been checked by something else. It can produce the visual language of control.
But every box that contains an LLM can be another inference call. Every edge can carry tokens. Every critic can trigger another generation. Every competing candidate can multiply sampling. Every retry can reproduce context. Every supervisor can become a translation layer. Every self-test can be another opinion from essentially the same epistemic machinery.
The empirical literature is now too strong to ignore. Anthropic reports multi-agent systems consuming about fifteen times the tokens of chat interactions in its data and describes early agents wildly overinvesting effort. LangGraph documents looping evaluator patterns and the need for recursion limits. Huang et al. and Kamoi et al. show why intrinsic self-correction cannot simply be assumed to add reliable information. Zhang et al. demonstrate that substantial portions of multi-agent communication can be pruned while retaining comparable results. Y. Li et al. show that repeated sampling can often be stopped much earlier. Tran and Kiela show that multi-agent advantages can disappear when reasoning-token budgets are matched. Y. Kim et al. provide controlled 2026 evidence of multi-agent coordination consuming up to 6.2 times the reasoning turns of a single-agent baseline in observed configurations, with hybrid coordination reaching 515% overhead and much poorer token efficiency (Hadfield et al., 2025; Huang et al., 2024; Kamoi et al., 2024; Y. Kim et al., 2026; Y. Li et al., 2024; Tran & Kiela, 2026; Zhang et al., 2025).
There are tasks where those costs are justified. Use agents there.
For everything else, the default should be suspicion.
Do not let an AI company, a framework diagram or an impressive execution trace persuade you that eleven model calls are inherently more intelligent than one. Make the extra calls prove that they add information. Price them. Cap them. Compare them against a direct baseline. Stop the loop when the task is done, not when the model finally runs out of things to say.
Because if you give a probabilistic system a graph whose easiest action is to continue, and give that graph access to a meter that somebody else pays, there is no mystery in what happens next.
10 PRINT "CRAP"
20 GOTO 10
Only now the GOTO has an API key.
References
Goodhart, C. A. E. (1975). Problems of monetary management: The U.K. experience. Papers in Monetary Economics, 1, 1–20. Reserve Bank of Australia.
Google. (n.d.-a). Code execution. Google AI for Developers. Retrieved August 22, 2026, from https://ai.google.dev/gemini-api/docs/code-execution
Google. (n.d.-b). Gemini Developer API pricing. Google AI for Developers. Retrieved August 22, 2026, from https://ai.google.dev/gemini-api/docs/pricing
Hadfield, J., Zhang, B., Lien, K., Scholz, F., Fox, J., & Ford, D. (2025, June 13). How we built our multi-agent research system. Anthropic. https://www.anthropic.com/engineering/multi-agent-research-system
Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2024). Large language models cannot self-correct reasoning yet. The Twelfth International Conference on Learning Representations.
https://openreview.net/forum?id=IkmD3fKBPQ
Kamoi, R., Zhang, Y., Zhang, N., Han, J., & Zhang, R. (2024). When can LLMs actually correct their own mistakes? A critical survey of self-correction of LLMs. Transactions of the Association for Computational Linguistics, 12, 1417–1440. https://doi.org/10.1162/tacl_a_00713
Kim, J., Shin, B., Chung, J., & Rhu, M. (2026). The cost of dynamic reasoning: Demystifying AI agents and test-time scaling from an AI infrastructure perspective. In 2026 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE Computer Society. https://doi.org/10.1109/HPCA68181.2026.11408569
Kim, Y., Gu, K., Park, C., Park, C., Schmidgall, S., Heydari, A. A., Yan, Y., Zhang, Z., Zhuang, Y., Liu, Y., Malhotra, M., Liang, P. P., Park, H. W., Yang, Y., Xu, X., Du, Y., Patel, S., Althoff, T., McDuff, D., & Liu, X. (2026). Capable language models can outgrow the benefits of collaboration. Nature Machine Intelligence, 8, 1157–1172. https://doi.org/10.1038/s42256-026-01268-y
LangChain. (n.d.-a). GRAPH_RECURSION_LIMIT. LangGraph documentation. Retrieved August 22, 2026, from https://docs.langchain.com/oss/python/langgraph/errors/GRAPH_RECURSION_LIMIT
LangChain. (n.d.-b). Use the graph API: Create and control loops. LangGraph documentation. Retrieved August 22, 2026, from https://docs.langchain.com/oss/python/langgraph/use-graph-api
LangChain. (n.d.-c). Workflows and agents. LangGraph documentation. Retrieved August 22, 2026, from https://docs.langchain.com/oss/python/langgraph/workflows-agents
Li, J., Zhang, Q., Yu, Y., Fu, Q., & Ye, D. (2024). More agents is all you need. Transactions on Machine Learning Research.
https://openreview.net/forum?id=bgzUSZ8aeg
Li, Y., Yuan, P., Feng, S., Pan, B., Wang, X., Sun, B., Wang, H., & Li, K. (2024). Escape sky-high cost: Early-stopping self-consistency for multi-step reasoning. The Twelfth International Conference on Learning Representations. https://proceedings.iclr.cc/paper_files/paper/2024/hash/3fe2a777282299ecb4f9e7ebb531f0ab-Abstract-Conference.html
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B. P., Hermann, K., Welleck, S., Yazdanbakhsh, A., & Clark, P. (2023). Self-Refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36, 46534–46594. https://doi.org/10.52202/075280-2019
OpenAI. (n.d.). API pricing. Retrieved August 22, 2026, from https://openai.com/api/pricing/
Penn, D. J., & Számadó, S. (2020). The handicap principle: How an erroneous hypothesis became a scientific principle. Biological Reviews, 95(1), 267–290. https://doi.org/10.1111/brv.12563
Schluntz, E., & Zhang, B. (2024, December 19). Building effective agents. Anthropic. https://www.anthropic.com/engineering/building-effective-agents
Tran, D., & Kiela, D. (2026). Single-agent LLMs outperform multi-agent systems on multi-hop reasoning under equal thinking token budgets [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2604.02460
Yu, B., & Nathan, A. (2026, April 22). Speeding up agentic workflows with WebSockets in the Responses API. OpenAI. https://openai.com/index/speeding-up-agentic-workflows-with-websockets/
Zhang, G., Yue, Y., Li, Z., Yun, S., Wan, G., Wang, K., Cheng, D., Yu, J. X., & Chen, T. (2025). Cut the crap: An economical communication pipeline for LLM-based multi-agent systems. The Thirteenth International Conference on Learning Representations. https://proceedings.iclr.cc/paper_files/paper/2025/hash/bbc461518c59a2a8d64e70e2c38c4a0e-Abstract-Conference.html