Agent Failure Analysis

Follow this topic Ask about this topic

Agent failure analysis examines how errors arise and spread in tool-using, coding, GUI, and multi-agent systems, including failures hidden by apparently successful final outcomes. Common approaches annotate execution trajectories, classify failure modes, monitor runtime behavior, and use dependency graphs or counterfactual replay to distinguish root causes from downstream symptoms. Benchmarks use real or human-labeled traces, controlled fault injection, and repeated runs to test diagnosis and recovery. A central challenge is making attributions genuinely causal and robust when logs are incomplete, task conditions vary, and benchmark shortcuts can make performance look stronger than it is.

Latest papers

154 papers27 papers in the last four weeks, up 42% on the four weeks before. 0.2% of all new papers.

DAAF: From Failure Localization to Editable System Assets in LLM Agents
Sep 26, 2026Xiaoyang Yuan, Qi Liu, Yubin Ruan +9Failure Attribution for LLM Agents Agent Failure Analysis
Tencent
Where Cyber Agents Struggle: Bottleneck Analysis of Multi-Stage LLM Agents
Sep 23, 2026Saeedeh Lohrasbi, Mohammad Mamun, Ahmed Yehia +2Agent Failure Analysis LLM Agent Evaluation
Cybersecurity, DTRC, National Research Council, Ottawa, Canada · University of Windsor, Windsor, Ontario, Canada
CausalLoss-Fin: Attributing Financial-Agent Loss to Decisions and Infrastructure Faults
Sep 22, 2026Abhishek SharmaAgent Failure Analysis Financial Services
When Are Aggregate Agent Traces Diagnosable? Traffic-Governed Interpretation and Calibrated Abstention
Sep 22, 2026Peiying Zhu, Sidi ChangAgent Failure Analysis Selective Prediction
Blossom AI San Francisco, CA, USA
Silent Failures in Agent-Tool Interaction: An Audit of ToolUniverse
Sep 21, 2026Devansh Singh, Shreya Gopalan, Sundaraparipurnan NarayananAgent Failure Analysis Tool-Using Agents
AI Tech Ethics
Locating Hidden Failures Makes Long-Horizon Agents More Reliable
Sep 15, 2026Salman Rahman, Yubin Kim, Mihir Parmar +15Agent Failure Analysis AI Agent Reliability
University of California, Los Angeles · New York University · Google Research +1
The Troy Moment: How LLM Agents Adjudicate the Decision Point Under Impossible Tasks, Claimed Authority, and Peer Information
Sep 14, 2026Ivy ZhangAgent Failure Analysis Software Engineering Agents
Apart Research
When Tool Calls Succeed but Workflows Fail: Anomalies at the Agent-Tool Boundary
Sep 14, 2026Artem Trofimov, Boris NovikovAgent Failure Analysis Agentic Workflows
AVIV Group · Independent Researcher
Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures
Sep 11, 2026Harsh Raj, David Lee, Anas Mahmoud +7Agent Failure Analysis Agentic Search
Scale AI · Scale AI Research
Guardrailed Meta-Agent Loops: Stress-Testing Policy Pinning, Budget Bounds, and Crash Recovery
Sep 10, 2026Qinzhen Ma, Jialin WuAgent Failure Analysis AI Agent Reliability
Rice University · University of California, San Diego
The Agent Incident Registry: Toward Preventing Repeated AI Agent Failures
Sep 10, 2026Divyanshu Kumar, Rohith HN, Nitin Aravind Birur +2Agent Failure Analysis AI Agent Evaluation
Anaconda
CIVI: A Framework for Diagnosing Search Agent Failures in Civic Information
Sep 8, 2026Dingying Liu, Yunshun Zhong, Wentao Zhang +1Agent Failure Analysis AI Agent Reliability
University of Sydney · University of Toronto · University of Waterloo +1
Audit Without Verification: When LLM Accountability Layers Relay Rather Than Check
Sep 7, 2026Paul-Peter ArslanAgent Failure Analysis AI Accountability
Institute For Future Technologies
WolfSociety: Understanding Collective Risk from Harmful-Agent Scaling in Financial Agent Societies
Sep 4, 2026Lejun Zhang, Sarah Lu-Liang, Xin Jiang +3Agent Failure Analysis Multi-Agent Systems
Shanghai Jiao Tong University · University of California, Berkeley · University of Toronto
DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems
Sep 4, 2026Zehao Wang, Lanjun Wang, Shilong Jin +2Agent Failure Analysis Multi-Agent LLM Systems
College of Intelligence and Computing, Tianjin University, Tianjin, China · School of New Media and Communication, Tianjin University, Tianjin, China · College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China +1
EDGE: Error Dependency Graph-Guided Multi-Error Attribution in Multi-Agent LLM Systems
Sep 1, 2026Jun Hou, Priya Pitre, Yi Fang +1Agent Failure Analysis Multi-Agent LLM Systems
Virginia Tech
Using Grounded Theory for Agent Behavior Analysis at Scale
Aug 31, 2026Zhuoran Lu, Yangyang Yu, Zhuoyan Li +5Agent Failure Analysis Agent Trajectory Analysis
1Purdue University · 2Stevens Institute of Technology · 3Cornell University +2
Cascade Failure Attribution for Multi-Agent Systems
Aug 30, 2026Jiayi Zhang, Zexin Wang, Degang Sun +4Agent Failure Analysis Multi-Agent LLM Systems
Computer Network Information Center, Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences
Safe to Resume? Breaking Execution Continuity of Agent Execution via Rollback
Aug 29, 2026Guanlong Wu, Dahui Li, Ke Jiang +3AI Agent Security Agent Failure Analysis
Southern University of Science and Technology · City University of Hong Kong
Localizing Emergent Failures in Agentic AI: Recovering Minimal Repair Families via Counterfactual Replay
Aug 29, 2026Bingjie Li, Yumeng Song, Zhongming Yao +1Agent Failure Analysis Multi-Agent LLM Systems
School of Computer Science and Engineering, Northeastern University, Shenyang, China · Department of Computer Science, Aalborg University, Aalborg, Denmark · College of Computer Science and Technology, Zhejiang University, Hangzhou, China
Fidelity Is Not Enough: Dispatch-Level Instrumentation for Agentic Datasheet Extraction
Aug 28, 2026Qing Ye, Meng-Hsuan LinLLM Agent Evaluation Document Information Extraction
Infineon Technologies AG Neubiberg, Germany
Agent Mesh: Reliability Primitives for Non-Idempotent Agent Delegation - Identity Adequacy and Evidence Adequacy
Aug 26, 2026Mazhar Shaikh, Anurag Rajkumar Bombarde, Harshal PathakAgent Reliability Agent Failure Analysis
Repair or Resample? Rethinking Failure Debugging in LLM Multi-Agent Systems
Aug 26, 2026Zhongwen Luan, Xiaoyu Zhang, Ming Hu +3LLM Agent Reliability Agent Failure Analysis
Xi’an University of Architecture and Technology Xi’an, China
Resilient Monitoring of Social Dynamical Systems through Collaborative Multi-Agent Networks under Latency
Aug 25, 2026Mohammadreza Doostmohammadian, Sergio PequitoMulti-Agent Systems Agent Failure Analysis
Adaptive Influence Graphs for Failure Attribution in Multi-Agent Systems
Aug 25, 2026Yarden Bakish, Amir Dudai, Roy Ganz +4Agent Failure Analysis Multi-Agent LLM Systems
Tel Aviv University · AWS Agentic AI
CatchBench: When Can an Agent Failure Be Caught?
Aug 24, 2026Yue Zhao, Mengyuan Li, Ruolin Li +5Agent Failure Analysis Benchmark Auditing
University of Southern California · Carnegie Mellon University · AWS Agentic AI +3
The Illusion of Independent Quorums: Epistemic Fault Domains and Correlated Cognitive Failures in Agentic Quorums
Aug 24, 2026Jun He, Deying YuMulti-Agent Systems Agent Failure Analysis
Robustness Analysis of Agentic AI to Inconsistent and Incomplete Tool Responses
Aug 24, 2026Jiachen Xu, Torben Bach Pedersen, Zhongming Yao +2Agent Failure Analysis AI Agent Reliability
Department of Computer Science Aalborg University Aalborg, Denmark · College of Computer Science and Technology Zhejiang University Hangzhou, China · School of Information Science and Engineering Northeastern University Shenyang, China
When Failures Propagate: Causal Failure Attribution in Agentic Retrieval-Augmented Generation
Aug 20, 2026Lauren PothuruAgent Failure Analysis Agentic RAG
Anote
Beyond End-to-End Success: Diagnosing Failures in Long-Horizon Security LLM Agents
Aug 20, 2026Wei Shao, Chongzhou Fang, Zuxiong Tan +4Agent Failure Analysis Long-Horizon Agent Evaluation
University of California, Davis, USA · Rochester Institute of Technology, USA

Sep 26, 2026J/K move · Enter open · S save

DAAF: From Failure Localization to Editable System Assets in LLM Agents

Xiaoyang Yuan, Qi Liu, Yubin Ruan, Xinyi Mou+8

Tencent

Failure Attribution for LLM Agents Agent Failure Analysis LLM Agent Reliability

Open paper AskPDF arXiv

Deployed LLM agents increasingly rely on persistent, versioned system assets such as routing rules, knowledge segments, prompt instructions, and reusable skills. Failure-localization methods can identify where an error manifests in an agent or execution trace, but repair requires a different decision: which editable system asset should be changed, and is that change expected to improve the task outcome? We study this gap through component-attribute failure attribution, where diagnosis targets versioned, addressable items rather than execution locations. We propose the Detection-Aware Attribution Framework (DAAF), which learns the effects of valid attribute replacements and amortizes this intervention evidence into deployment-time diagnosis. DAAF combines sparse and noisy failure signals to decide whether intervention is warranted, learns component-type-conditioned replacement effects from controlled replays evaluated by executable task outcomes, and shares supervision across requests with compatible intervention responses. At diagnosis time, DAAF uses only the observed execution, registered candidates, and available failure signals; it requires neither counterfactual replay nor task reward and returns no_change, a repair target, or an unresolved decision when evidence is insufficient. On held-out tau^2-bench Telecom tasks, DAAF achieves 80.72% attribute Hit@1, recovers 62.65% of failed executions while limiting clean-task regression to 3.23%, and reaches 71.93% overall task success. These results show that intervention-grounded attribute attribution can connect failure localization to executable system repair.