AgentTrace: Causal Graph Tracing for Root Cause Analysis in Deployed Multi-Agent Systems

Zhaohui Geoffrey Wang

cs.LGMar 16, 2026

AgentTrace: Causal Graph Tracing for Root Cause Analysis in Deployed Multi-Agent Systems

Authors: Zhaohui Geoffrey Wang

Organizations: USC Viterbi School of Engineering

Abstract

As multi-agent AI systems are increasingly deployed in real-world settings - from automated customer support to DevOps remediation - failures become harder to diagnose due to cascading effects, hidden dependencies, and long execution traces. We present AgentTrace, a lightweight causal tracing framework for post-hoc failure diagnosis in deployed multi-agent workflows. AgentTrace reconstructs causal graphs from execution logs, traces backward from error manifestations, and ranks candidate root causes using interpretable structural and positional signals - without requiring LLM inference at debugging time. Across a diverse benchmark of multi-agent failure scenarios designed to reflect common deployment patterns, AgentTrace localizes root causes with high accuracy and sub-second latency, significantly outperforming both heuristic and LLM-based baselines. Our results suggest that causal tracing provides a practical foundation for improving the reliability and trustworthiness of agentic systems in the wild.

Figures & tables

Figure 1: Overview of the AgentTrace framework. (a) Execution logs from a multi-agent workflow are collected. (b) A causal graph is constructed with sequential, communication, and data dependency edges. (c) Backward BFS from the error node identifies candidate root causes. (d) Candidates are ranked using a weighted combination of five interpretable feature groups, with position features dominating ( $w_{p}{=}0.70$ ).

Group ( $w_{i}$ )	Feature	Formula	Range
Position (0.70)	Normalized Position	$\text{pos}(v)/\|V\|$	$[0,1]$
	Distance to Error	$d(v,v_{\text{error}})/\max_{u}d(u,v_{\text{error}})$	$[0,1]$
	Depth Ratio	$\text{depth}(v)/\max_{u}\text{depth}(u)$	$[0,1]$
	Reverse Position	$1-\text{pos}(v)/\|V\|$	$[0,1]$
Structure (0.20)	Out-degree	$\|\{u:(v,u)\in E\}\|/\max_{w}\|\{u:(w,u)\in E\}\|$	$[0,1]$
	In-degree	$\|\{u:(u,v)\in E\}\|/\max_{w}\|\{u:(u,w)\in E\}\|$	$[0,1]$

Table 1: Complete feature list with computation methods

Domain	N	Domain	N
Software Development	52	Healthcare Coordination	60
Customer Service	51	Legal Document Analysis	60
Research Analysis	51	Educational Tutoring	60
Planning & Scheduling	46	Financial Advisory	60
Financial Trading	50	DevOps Automation	60

Table 2: Benchmark domains and scenario counts (550 total)

Method	Hit@1	Hit@3	MRR	$\chi^{2}$	Cohen’s h
Random	9.1%	27.3%	0.18	459.2	2.41
First Node	3.6%	10.9%	0.07	497.0	2.72
Last Node	12.7%	38.2%	0.25	436.5	2.29
LLM Analysis	68.5%	81.4%	0.74	102.5	0.77
AgentTrace	94.9% [92.9, 96.7]	98.4% [96.9, 99.4]	0.97	–	–

Table 3: Root cause localization accuracy and statistical significance on the 550-scenario benchmark. 95% CIs from bootstrap resampling (

B{=}10{,}000

). All

p

-values

<10^{-10}

(McNemar’s test).

Features	Hit@1	$\Delta$	$w_{p}$	Hit@1
All (P+S+C+F+E)	94.9%	–	0.50	91.3%
P only	87.3%	-7.6	0.60	93.5%
S only	34.5%	-60.4	0.70	94.9%
C only	28.7%	-66.2	0.80	93.8%
F only	15.2%	-79.7	0.90	91.1%
E only	12.1%	-82.8

Table 4: Ablation study. Left : Hit@1 with different feature group combinations (P=Position, S=Structure, C=Content, F=Flow, E=Confidence). Right : Sensitivity of Hit@1 to position weight

w_{p}

Stratification	Category	Hit@1	MRR
Bug Type	Logic Error	95.8%	0.98
	Communication Failure	93.6%	0.96
	Data Corruption	94.5%	0.97
	Missing Validation	94.3%	0.96
	Role Confusion	94.8%	0.97
Trace Length	8–9 steps	95.5%	0.97

Table 5: Robustness analysis: Hit@1 and MRR stratified by bug type, trace length, and bug position.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	Agents	Interaction Pattern	N
Software Dev	Planner, Coder, Reviewer, Executor	Sequential + Review Loop	52
Customer Service	Router, Specialist, Resolver, Logger	Hierarchical Dispatch	51
Research	Searcher, Analyzer, Synthesizer, Writer	Pipeline + Feedback	51
Planning	Scheduler, Optimizer, Validator, Notifier	Iterative Refinement	46
Trading	Analyst, Strategist, RiskManager, Executor	Parallel Analysis	50
Healthcare	Triager, Specialist, Pharmacist, Coordinator	Consultation Chain	60

Appendix

Table 6: Domain-specific agent configurations

Domain	N	Hit@1	Hit@3	Hit@5	MRR
Software Development	52	94.2%	96.2%	100.0%	0.96
Customer Service	51	92.2%	96.1%	100.0%	0.94
Research Analysis	51	96.1%	98.0%	100.0%	0.97
Planning & Scheduling	46	91.3%	95.7%	100.0%	0.94
Financial Trading	50	90.0%	98.0%	100.0%	0.94
Healthcare Coordination	60	95.0%	100.0%	100.0%	0.97

Appendix

Table 7: Detailed performance metrics by domain

Error Type	Count	%
Selected error node	82	47.4
Off-by-one (adjacent)	45	26.0
Intermediate step	31	17.9
Completely incorrect	15	8.7
Total	173	100

Appendix

Table 8: Error analysis for LLM baseline

Table 10

Parameter	Value
Max tracing depth	10
$w_{p}$ / $w_{s}$ / $w_{c}$	0.70 / 0.20 / 0.05
$w_{f}$ / $w_{e}$	0.03 / 0.02
Normalization $\epsilon$	$10^{-8}$
Bootstrap iters	10,000
Confidence level	95%

Appendix

Table 11: Complete hyperparameter settings

Explore similar work

Adaptive Influence Graphs for Failure Attribution in Multi-Agent Systems
Aug 25, 2026Yarden Bakish, Amir Dudai, Roy Ganz +4Agent Failure Analysis Multi-Agent LLM Systems
Tel Aviv University · AWS Agentic AI
From Anomalies to Failures: Constructing Causal Error Graphs for Agentic Trace Diagnosis
Sep 26, 2026Shu-Xun Yang, Yidong Wang, Zhuoer Feng +7Agent Failure Analysis Reasoning Trace Analysis
Beijing Institute of Technology, Beijing, China · Zhipu AI, Beijing, China · Tsinghua University, Beijing, China
Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systems
Aug 30, 2026Jiayi Zhang, Zexin Wang, Degang Sun +4Agent Failure Analysis Multi-Agent LLM Systems
Computer Network Information Center, Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences

Aug 25, 2026J/K move · Enter open · S save

Adaptive Influence Graphs for Failure Attribution in Multi-Agent Systems

Yarden Bakish, Amir Dudai, Roy Ganz, Oren Nuriel+3

Tel Aviv University · AWS Agentic AI

Agent Failure Analysis Multi-Agent LLM Systems

Open paper AskPDF arXiv

Multi-agent LLM systems are increasingly deployed in real-world applications, where failures can be costly and difficult to localize. Despite growing efforts to automate failure attribution, diagnosing failed runs still largely relies on human engineers. Yet engineers rarely debug complex systems by reading raw logs end to end. Instead, observability tools organize traces around components, actions, and dependencies to support targeted navigation. We hypothesize that modern LLMs can benefit from the same paradigm. To test this hypothesis, we introduce Adaptive Influence Graphs (AIGs), a two-stage agentic framework that first transforms a failed trace into a structured graph and then navigates it to identify the critical error. Across multiple models, we show that richer trace representations consistently improve failure attribution, with adaptive graph construction and agent-directed traversal yielding the strongest results. AIGs establish a new state of the art on Who&When, the standard benchmark for multi-agent failure attribution. This affirms our hypothesis that attribution depends not only on the diagnosing model, but also on how the trace is represented and explored.

Abstract

Explore similar work

Adaptive Influence Graphs for Failure Attribution in Multi-Agent Systems

From Anomalies to Failures: Constructing Causal Error Graphs for Agentic Trace Diagnosis

Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systems