Localizing Emergent Failures in Agentic AI: Recovering Minimal Repair Families via Counterfactual Replay

Bingjie Li; Yumeng Song; Zhongming Yao; Tianyi Li

cs.AIAug 29, 2026

Localizing Emergent Failures in Agentic AI: Recovering Minimal Repair Families via Counterfactual Replay

Authors: Bingjie Li, Yumeng Song, Zhongming Yao, Tianyi Li

Organizations: School of Computer Science and Engineering, Northeastern University, Shenyang, China · Department of Computer Science, Aalborg University, Aalborg, Denmark · College of Computer Science and Technology, Zhejiang University, Hangzhou, China

Abstract

Failures in agentic AI systems can arise from interactions among messages exchanged by multiple large language model (LLM) agents. Pointwise attribution cannot distinguish a jointly necessary repair from alternative singleton repairs. We formulate Minimal Repair Family Recovery (MRFR): recovering all inclusion-minimal event sets whose counterfactual replay restores task success within a declared size bound. We propose Graph-Constrained Joint Replay (GCJR), which slices failure-relevant events from an execution dependency graph, constructs graph-feasible singleton and pair candidates, and verifies them by replay with paired clean counterparts. For fixed replay outcomes, GCJR is exact within its declared graph domain. On 90 in-scope cases from a 120-DAG controlled benchmark, GCJR achieves 1.000 Family Exact Match while reducing mean replay calls from 56.3 to 25.3 (55.1%) relative to exhaustive search. On a 24-case, four-agent LLM pilot, it again achieves 1.000 Family Exact Match and reduces mean model calls from 21.0 to 10.0 (52.4%); single-event replay misses jointly necessary repairs.

Figures & tables

Fig. 1: Overview of the proposed framework.

Statistic	Controlled-DAG	DualSolve-MAS
Cases	120	24
Schedules	360	–
Agents	4	4
Candidates	6–14	3
Families	S/J/A/SJ	S/J/A
Replay	Simulator	Qwen ( $r=3$ )

TABLE I: Statistics of the experimental workloads.

Method	Family recovery		Replay efficiency
Method	J FEM $\uparrow$	Macro FEM $\uparrow$	Calls $\downarrow$	Matched saving $\uparrow$
Graph-blind exhaustive	1.000	1.000	56.3	0.0%
Single-event replay	0.000	0.667	10.7	N/A
Execution-window replay	0.100	0.700	14.4	N/A
Exact Shapley top-2	1.000	0.333	3285.3	N/A
Centrality top-2	0.033	0.011	0.0	N/A

TABLE II: Family recovery and replay cost on the 90 in-scope Controlled-DAG cases.

Method	Family recovery		LLM efficiency
Method	J FEM $\uparrow$	Macro FEM $\uparrow$	Calls $\downarrow$	Saving $\uparrow$
Exhaustive	1.000	1.000	21.0	0.0%
Single-event	0.000	0.667	12.0	N/A
Execution-window	0.000	0.667	13.0	N/A
LLM log-only	0.000	0.333	1.0	N/A
GCJR (ours)	1.000	1.000	10.0	52.4%

TABLE III: Family recovery and LLM-call cost on the 24-case DualSolve-MAS pilot.

Agent Failure Analysis

Agent failure analysis studies how and why AI agents make errors, traces causes through execution, and helps detect or prevent recurrence.

See all 154 papers

Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?
Jul 10, 2026Jiale Liu, Huajun Xi, Shaokun Zhang +6Agent Failure Analysis LLM Agent Evaluation
Penn State University · AG2ai, Inc. · Mohamed bin Zayed University of Artificial Intelligence +2
A Mechanistic Study of Automated Failure Attribution in LLM-based Agentic Systems
Oct 20, 2026Wendy Zheng, Liang Wu, Yinhan He +3Failure Attribution for LLM Agents
University of Virginia · Nokia
CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures
May 25, 2026Akash Bonagiri, Devang Borkar, Gerard Janno Anderias +2Agent Failure Analysis Causal Interventions in Language Models
Department of Computer Science University of California, Davis Davis, CA 95616

Jul 10, 2026J/K move · Enter open · S save

Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?

Jiale Liu, Huajun Xi, Shaokun Zhang, Yifan Zeng+5

Penn State University · AG2ai, Inc. · Mohamed bin Zayed University of Artificial Intelligence+2

Agent Failure Analysis LLM Agent Evaluation AI Agent Benchmarks Failure Attribution for LLM Agents

Open paper AskPDF arXiv

Automated failure attribution uses LLMs to identify where and why agentic systems fail. As agents become more capable, their failures become subtler, making automated attribution increasingly important. We introduce Who&When Pro, a large-scale benchmark for automated failure attribution in agentic systems. Using a strictly controlled pipeline that injects a failure only after exactly replaying a successful prefix, we construct 12,326 failed trajectories with golden labels across 3 modalities and 26 benchmarks covering various scenarios. Beyond benchmarking, we conduct extensive experiments and analyses, revealing systematic patterns in how models attribute failures across modalities, protocols, and model families, and providing empirical guidance for future automated failure attribution systems.

Abstract

Agent Failure Analysis

Explore similar work

Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?

A Mechanistic Study of Automated Failure Attribution in LLM-based Agentic Systems

CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures