BLADE: Auditing Information Use for Failure Attribution in LLM Multi-Agent Systems

Xiaoguang Guo; Yaohao Long; Chuxu Zhang

cs.AIOct 13, 2026

BLADE: Auditing Information Use for Failure Attribution in LLM Multi-Agent Systems

Authors: Xiaoguang Guo, Yaohao Long, Chuxu Zhang

Organizations: University of Connecticut

Abstract

Large language model multi-agent systems extend individual agents' capabilities through collaboration across specialized roles, tools, and workflows. Yet collaboration makes failures harder to diagnose: one agent may misinterpret a correct result while others carry the mistake forward. Failure attribution therefore seeks to identify the responsible agent and decisive error step. Recent methods trace dependencies, but knowing where information came from is not the same as knowing where it went wrong. We introduce BLADE (Blame Localization via Audited Dependency Evidence), a training-free framework that moves beyond tracing information to auditing its use. Using Jev for structured probabilistic judgments, BLADE uses dependency probabilities to select relevant source-decision handoffs for auditing. At these handoffs, it examines whether receiving decisions use their sources faithfully and within task requirements, distinguishing error introduction, propagation, and repair. These findings inform a global diagnosis that considers all execution steps. Without reference answers, BLADE achieves the best or tied-best step accuracy among eight strategies under a shared Jev decision interface on 1,228 failed runs across five Who&When and Who&When Pro subsets, outperforming All-at-once by 2.2-11.1 percentage points. Making responsibility evidence explicit offers a path toward more interpretable and reliable agentic systems.

Figures & tables

Figure 1: Failure attribution requires distinguishing where an error is first introduced from where it is merely propagated downstream.

Figure 2: Overview of BLADE. (1) Jev reconstructs dependencies between events. (2) The dependencies select the handoffs to audit, and Jev checks how each agent decision uses its source against the task’s requirement spans $R$ . (3) The diagnosis reads the full trajectory with the blame graph $G=(V,E,R,F)$ ; every event remains a candidate.

	Who&When				Who&When Pro						Macro Avg.
	HC		AG		Magentic-One		DyLAN		MetaGPT
Method	Agent	Step	Agent	Step	Agent	Step	Agent	Step	Agent	Step	Agent	Step
All-at-once ( Zhang et al., 2025 )	50.0	24.1	64.3	50.0	80.4	78.7	59.9	52.6	46.9	54.3	60.3	52.0
Step-by-step ( Zhang et al., 2025 )	55.2	27.6	57.9	39.7	80.5	76.0	50.7	46.1	53.1	53.1	59.5	48.5
Binary search ( Zhang et al., 2025 )	56.9	31.0	59.5	45.2	74.6	68.7	58.6	56.6	37.0	37.0	57.3	47.7
TRIAGE ( Wang, 2026 )	56.9	22.4	60.3	44.4	83.5	79.9	52.6	48.0	46.9	46.9	60.0	48.3

Table 1: Agent and step accuracy (%) without the reference answer. All methods are instantiated under the same Jev decision interface (Appendix B.2 ). HC: Hand-Crafted; AG: Algorithm-Generated. Macro Avg. equally weights the five subsets. Bold marks the best and underline the second best.

Figure 3: Step accuracy under increasing localization tolerance

k

, where predictions within

k

events of the annotated step are considered correct.

k=0

corresponds to the exact step accuracy reported in Table 1 .

	Who&When		Who&When Pro
Evidence	HC	AG	M-One	DyLAN
No evidence	24.1	50.0	78.7	52.6
Dependencies only	27.6	51.6	78.4	53.9
Shuffled evidence	25.9	52.4	76.7	52.0
Without breach findings	29.3	52.4	78.3	53.3
Full BLADE	31.0	54.0	80.9	56.6

Table 2: Step attribution accuracy (%) under different evidence configurations. The diagnosis question and candidate events are held fixed. No evidence corresponds to All-at-once; dependencies only includes dependency edges and task requirement spans; shuffled evidence randomly reassigns edges and audit findings to event pairs. M-One: Magentic-One.

Figure 4: Dependency-guided versus exhaustive auditing. Top: step accuracy gain over exhaustive auditing; middle: handoff audits per run (log scale); bottom: share of findings on the annotated decisive step. AG: Algorithm-Generated; M-One: Magentic-One.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Share of audited handoffs labeled breach by the position of the receiving decision, in agent decisions from the annotated decisive step (0). M-One: the Magentic-One runs of Who&When Pro.

Baseline	Gain (pp)	95% CI
All-at-once	+5.6	[2.8, 8.6]
Step-by-step	+9.1	[5.2, 13.2]
Binary search	+9.9	[5.8, 14.0]
TRIAGE	+9.2	[6.4, 12.2]
FALAT	+17.9	[13.5, 22.4]
ErrorProbe	+16.2	[12.6, 20.1]

Appendix

Table 3: Macro-average step accuracy gain of BLADE over each baseline, weighting the subsets equally, with paired task-cluster bootstrap 95% confidence intervals.

Panel of Figure 6 — Figure 6: Question templates for dependency reconstruction, contract auditing, and diagnosis.

Failure Attribution for LLM Agents

See all 34 papers

DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems
Sep 4, 2026Zehao Wang, Lanjun Wang, Shilong Jin +2Agent Failure Analysis Multi-Agent LLM Systems
College of Intelligence and Computing, Tianjin University, Tianjin, China · School of New Media and Communication, Tianjin University, Tianjin, China · College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China +1
Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems
Oct 8, 2026Jiaqi Liao, Yuanzhao Zhai, Huanxi Liu +5Failure Attribution for LLM Agents Agent Failure Analysis
College of Computer Science and Technology, National University of Defense Technology; State Key Laboratory of Complex & Critical Software Environment, Changsha, Hunan, China · School of Mechanical Engineering, Tianjin University, Tianjin, China · College of Computer Science and Technology, National University of Defense Technology; National Key Laboratory of Parallel and Distributed Computing, Changsha, Hunan, China
Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems
Jul 8, 2026Yufei Xia, Anjun Gao, Yueyang Quan +2Agent Failure Analysis Multi-Agent LLM Systems
University of Louisville · University of North Texas

Sep 4, 2026J/K move · Enter open · S save

DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems

Zehao Wang, Lanjun Wang, Shilong Jin, Junjie Chen+1

College of Intelligence and Computing, Tianjin University, Tianjin, China · School of New Media and Communication, Tianjin University, Tianjin, China · College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China+1

Agent Failure Analysis Multi-Agent LLM Systems Multi-Agent Systems Failure Attribution for LLM Agents

Open paper AskPDF arXiv

Large language model (LLM)-based multi-agent systems have experienced rapid growth in recent years. Despite their promise, such systems remain fragile, frequently exhibiting reasoning and coordination errors that can lead to system-level failures. Failure attribution in such systems relies on tracing natural language interactions among agents to identify the decisive error, which refers to the earliest action whose correction can reverse system failure. There are two key challenges: 1) Shallow attribution: Existing methods often capture only minor deviations, such as incomplete retrievals or formatting errors, which verification mechanisms can correct, while missing the decisive cause of system failure. 2) Contextual degradation: As the length of the system traces increases, the model's reasoning ability rapidly deteriorates. To address these challenges, we propose DCFA, a training-free framework for failure attribution. DCFA integrates a global module that constructs structured causal-inspired dependency graphs from system traces to identify the initial decisive error, and a local module that applies local counterfactual-inspired reasoning to refine causal-inspired attribution. Experiments on the Who&When benchmark across six LLMs show that DCFA improves step-level accuracy by up to 8.27% over state-of-the-art baselines.

Abstract

Failure Attribution for LLM Agents

Explore similar work

DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems

Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems

Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems