Multi-agent AI loses in 12 of 13 user-service tests

Ethan Yang and Xiangcheng Shi reported in an arXiv preprint published Oct. 21 that multi-agent AI systems lost to single-agent setups in 12 of 13 valid user-service test cells.

Yang and Shi's Oct. 21 arXiv paper challenges a prior capability-saturation rule that put the crossover near 45% single-agent accuracy, while remaining an early research result not yet peer reviewed.

The paper tests systems on τ²-bench, a dual-control benchmark where agents interact with a simulated user while following written policy. It covers airline, retail and telecom tasks across about 7,950 simulated conversations.

Multi-agent system, or MAS, means several model-based agents coordinate on a task; single-agent system, or SAS, relies on one model agent. The authors compared a stock SAS with a centralized MAS built from the same underlying model, using six models from three model families.

The authors wrote: "For agents that serve people, the SAS is the default; a multi-agent design must show compute-matched evidence from the setting itself."

Under a fixed simulated user, the paper reported no MAS advantage in any valid cell. MAS trailed in 12 cells and tied in one, with a fitted weighted mean effect of minus 28 points and a 95% confidence interval from minus 44 to minus 12.

The result cuts against the earlier rule derived from autonomous benchmarks, where no user participates. In the two test cells where that rule indicated MAS should help, the multi-agent setup lost by 13 points and 22 points, according to the preprint.

Token use also weakened the case for coordination. Completion-token ledgers track model output tokens, a proxy for inference workload and cost; without a budget ceiling, MAS at best matched SAS while using 5.5 times the SAS token spend.

The paper does not settle every question. It reported one validity bar as unmet because the MAS planner did not see the tool schemas, which define available tool calls; the abstract does not quantify how that affected performance.

The authors also said the evidence did not resolve whether MAS gains decline as single-agent capability rises. The estimated slope was minus 0.41, with a 95% confidence interval from minus 1.51 to plus 0.68, because model-by-domain effects dominated.

Safety results were limited as well. The paper said MAS did not detectably reduce wrongful database writes, and instead shifted some failures toward budget exhaustion and inaction.

The authors said they released trajectories, token ledgers, prompts and code alongside the preprint.

Source: arXiv preprint, arXiv cs.AI, Oct. 21, 2026