Where Do Multi-Agent Systems Fail? Evidence-Grounded Diagnosis of Collective Mechanisms

Zhengye Han

cs.MASep 30, 2026

Where Do Multi-Agent Systems Fail? Evidence-Grounded Diagnosis of Collective Mechanisms

Authors: Zhengye Han

Organizations: New York University New York, NY, USA

Abstract

When a multi-agent system answers correctly, it is tempting to conclude that its agents shared, checked, and used information as intended. Yet a system can break one of its collective mechanisms, the rules that govern how agents route, admit, store, and act on shared information, and still return the right answer, while a wrong answer rarely reveals which mechanism failed. We ask what evidence from an execution is sufficient to conclude that a particular mechanism was violated. Our answer is a diagnostic contract, which separates what counts as a violation from which execution records can establish one, and concludes that a violation is supported, ruled out, or unknown; removing records can make this conclusion unknown but never reverse it. We test contracts for four mechanisms by replaying executions from the step where a mechanism acts, once unchanged, once with the mechanism broken, and once with it restored. Broken mechanisms often left the answer correct. An LLM diagnoser detected many more violations from internal records than from public outputs, yet with identical records a generic prompt often claimed certainty the records did not support, which prompts stating the contracts largely avoided. The contracts also applied, in narrow form, to mechanisms in independently developed systems, but a diagnostic behavior that was nearly perfect on our benchmark degraded on an independently developed workflow. A correct outcome is therefore no substitute for records of how collective mechanisms operated, and agreement on one benchmark does not show that a diagnoser transfers to another system.

Figures & tables

Figure 1. Where a multi-agent system fails, and how we diagnose it. (1) Both runs answer correctly, but the second breaks the admission rule by counting two copies of one report as independent; neither the answer nor the public transcript reveals this, and frameworks log different things. (2) A diagnostic contract separates the violation predicate, what a violation is , from the evidence specification, which authoritative records must be seen to settle it, and returns supported , ruled out , or unknown . (3) Three tests, each yielding one finding. Three panels. The first shows two runs: in one the rule that copies count once is kept, in the other it is broken, and both reach the right answer; three boxes name the problems (hidden, unclear, inconsistent) and ask which rule broke and which record proves it. The second shows a public transcript and an authoritative record entering a diagnostic contract made of a violation predicate and an evidence specification, which returns supported, ruled out, or unknown. The third shows three tests (replay from a shared prefix, same evidence packages, independently developed systems), each leading to one finding.

Locus	Violation	Authoritative record	Not a violation
Access/ routing	Required information produced upstream is missing from the receiver’s input	The receiver’s complete input	Missing information the decision does not need
Admission/ dependence	Evidence is accepted or weighted against the verification or common-origin rule	The admission decision and each report’s origin	Suspicious reports that are correctly rejected or collapsed
State maintenance	An accepted, relevant item is lost, stale, or overwritten	Shared state before and after the update	A change to state the task does not use
Representation/ action	A correct decision is not carried out as a valid action	The submitted action and its receipt	An incorrect decision, to which the contract does not apply

Table 1. The four diagnostic loci. For each locus: what counts as a violation, which record can establish it, and a similar-looking case that does not count.

	What changes	What is held fixed	What we measure
Q1	One mechanism: clean, broken, or restored	Everything before the targeted step	Reference verdict and task success on each branch
Q2	The records a diagnoser sees, and the diagnoser	The run being diagnosed	Detection rate; errors on missing or irrelevant records
Q3	The system: benchmark or independently developed	Contracts and diagnoser prompts	Accepted and rejected bindings; the same obligations as in Q2

Table 2. The evaluation at a glance.

	B support	Control activation	Correct under violation	$\Delta_{\mathrm{inst}}$
Routing	72/72	0/144	0/72	1.000
Admission	72/72	0/144	43/72	0.403
State	72/72	0/144	0/72	1.000
Action	72/72	0/144	0/72	1.000
Canonical	288/288	0/576	43/288	0.851
DRAC	137/137	0/274	28/137	0.796

Table 3. Mechanism verdicts and task outcomes. B support: target violation supported on perturbed branches; control activation: target supported on clean or restored branches; correct under violation: supported violations with a correct task outcome.

Panel of Figure 2 — Figure 2. Evidence access by locus. Target-locus recovery of the G0 diagnoser on the 288 perturbed canonical branches (72 per locus) from public outputs and from native execution records. Whiskers: 95% bootstrap intervals over the 18 challenges of each locus. Dot plot with one row per locus. Public outputs recover 0/72 routing, 4/72 admission, 0/72 state, and 72/72 action failures; native records recover 72/72, 27/72, 11/72, and 72/72.

	Valid	Agree with R	Unsupp. cert.	Flips	F $\to$ I invariant
G0	1536/1536	764/1536	576/768	138/1152	259/384
G1	1199/1536	1139/1199	53/533	0/748	287/287
G2	1031/1536	1022/1031	8/435	0/561	227/228

Table 4. Same evidence, different procedures. Rates for G1 and G2 are conditional on valid outputs; both fail the preregistered interpretability gate (Section 5 ). Unsupp. cert.: resolved verdict where R returns unknown . Flips: opposite resolved verdicts along deletion edges.

Native mechanism	Locus	$n$	B / A / C
MetaGPT recipient-local delivery	routing	8	8 / 0 / 0
MetaGPT memory replay suppression	state	8	8 / 0 / 0
Agents SDK tool call $\to$ receipt	action	12	12 / 0 / 0
Co-STORM citation membership †	admission	9	9 / 0 / 0
Agents SDK handoff / session	routing / state	rejected
Agents SDK approval gate	admission	rejected

Table 5. External bindings. Held-out A/B/C triplets forked from native checkpoints; entries give target support on B and activation on A and C. † Both blind reviews judged this mapping plausible but ambiguous; it was accepted on development evidence. Co-STORM yielded 18 qualified triplets from 24 held-out tasks; the table reports the 9 of them that fall in the 12-task diagnostic subset fixed by a hash rule before held-out execution, whose other three tasks had no qualified triplet (one lacked an original pre-operation snapshot and two failed native qualification). Rejected candidates are listed below the rule.

Panel of Figure 3 — Figure 3. The same frozen procedures on canonical and independently developed workflow evidence. Left: share of valid pairs in which deleting the decisive witness yields unknown (canonical: F resolved, S unknown ). Right: share in which deleting records outside a retained witness leaves the verdict unchanged. Whiskers: Wilson 95% intervals. Canonical rates are conditional on valid outputs; the reference R scores 28/28 on both Co-STORM obligations. Two slope panels comparing canonical and Co-STORM rates. Witness deletion stays at 1.0 for G1 (280/280 to 28/28) and G2 (228/228 to 22/22). Irrelevant-evidence invariance falls for G1 from 287/287 to 13/28 and for G2 from 227/228 to 8/21.

Explore similar work

Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems
Jul 8, 2026Yufei Xia, Anjun Gao, Yueyang Quan +2Model-Based Multi-Agent Systems
University of Louisville · University of North Texas
Misinformation Propagation in Benign Multi-Agent Systems
Jun 15, 2026Jonas Becker, Jan Philip Wahle, Terry Ruas +1Multi-Agent Debate Misinformation
University of Göttingen, Germany · LKA NRW, Germany
AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
May 9, 2026Aritra Mazumder, Shubhashis Roy Dipta, Nusrat Jahan Lia +10Multi-Agent Communication Data Leakage
University of Utah · University of Maryland, Baltimore County · University of Dhaka +7

Jul 8, 2026, cs.CRJ/K move · Enter open · S save

Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems

Yufei Xia, Anjun Gao, Yueyang Quan, Zhuqing Liu+1

University of Louisville · University of North Texas

Model-Based Multi-Agent Systems

Open paper AskPDF arXiv

Large language model (LLM) based multi-agent systems enable complex problem solving through coordinated reasoning and action, but their distributed structure also introduces new challenges in diagnosing system-level failures. When an execution fails, identifying which agent is responsible and at what point the trajectory first becomes irreversibly misdirected is difficult due to long-horizon interactions and tightly coupled agent behaviors. In this paper, we study the problem of failure localization in LLM-based multi-agent systems and present AgentLocate, a framework that attributes failures to both a specific agent and the earliest decisive step. AgentLocate combines an LLM-based judging mechanism with multi-perspective verification by independent evaluators, whose assessments are aggregated using a confidence-aware strategy. The resulting feedback is further used to adapt the judge through lightweight fine-tuning, improving attribution quality. We evaluate AgentLocate on two complementary benchmarks covering diverse tasks, agent configurations, and trajectory lengths. Experimental results show that AgentLocate consistently outperforms existing failure localization methods in identifying both responsible agents and failure steps, while remaining efficient in terms of token usage and running time.

Abstract

Explore similar work

Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems

Misinformation Propagation in Benign Multi-Agent Systems

AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators