Agent Failure Analysis

Ask about this topic

Overview

Agent failure analysis examines how errors arise and spread in tool-using, coding, GUI, and multi-agent systems, including failures hidden by apparently successful final outcomes. Common approaches annotate execution trajectories, classify failure modes, monitor runtime behavior, and use dependency graphs or counterfactual replay to distinguish root causes from downstream symptoms. Benchmarks use real or human-labeled traces, controlled fault injection, and repeated runs to test diagnosis and recovery. A central challenge is making attributions genuinely causal and robust when logs are incomplete, task conditions vary, and benchmark shortcuts can make performance look stronger than it is.

Latest papers

CardsList

154 papers27 papers in the last four weeks, up 42% on the four weeks before. 0.2% of all new papers.

  1. Babysitting AI Agents: A User-Centered Taxonomy and Study of Agentic Delegation Failures

    Oct 26, 2026Shravika Mittal, Sunnie S. Y. Kim, Scott Counts +3Agent Failure AnalysisAI Agent Reliability

    Georgia Institute of Technology, Atlanta, Georgia, USA · Microsoft Research, New York, New York, USA · Microsoft, Redmond, Washington, USA +1

  2. CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use Agents

    Oct 20, 2026Xu Hu, Haoming Li, Yuxiang Wang +11Computer-Use AgentsComputer-Use Agent Benchmarks

    University of Texas at Dallas · Independent Researcher · University of Manchester +3

  3. MASIF: Implicit Failures for Process Observability in Multi-Agent Systems

    Oct 19, 2026Yanwei Huang, Arpit Narechania, Atanu SinhaMulti-Agent SystemsAgent Failure Analysis

    HKUST · Adobe Research

  4. Poison Valleys: Diagnosing Path-Dependent Failure in LLM Code Repair

    Oct 15, 2026Veljko SkarichAgent Failure AnalysisLLM Code Generation

  5. OmicsBench: a long-horizon multi-turn benchmark of AI agents on research-study-scale bioinformatics

    Oct 15, 2026Gideon Shaked, Siddharth NagisettyBioinformaticsAI Agent Benchmarks

    Department of Computational Medicine and Bioinformatics, University of Michigan · AfterQuery

  6. How GUI Agents Fail: GraceBench for Execution Stability Diagnosis

    Oct 14, 2026Ridong Zhang, Shuxian Liang, Chaoqun Wan +2Computer-Use Agent BenchmarksGUI Agents

    Institute of AI for Engineering, Tongji University

  7. Why Do Agent Teams Fail Across Boundaries? A Failure Taxonomy for Agent-to-Agent Systems with Private States

    Oct 14, 2026Sarath Shekkizhar, Prathyusha Jwalapuram, Zixuan Ke +12Agent Failure AnalysisMulti-Agent Systems

    Salesforce · UNC Chapel Hill · Singapore University of Technology and Design +2

  8. BLADE: Auditing Information Use for Failure Attribution in LLM Multi-Agent Systems

    Oct 13, 2026Xiaoguang Guo, Yaohao Long, Chuxu ZhangFailure Attribution for LLM AgentsAgent Failure Analysis

    University of Connecticut

  9. Executable Prefix Values for Locating Agent Failures

    Oct 13, 2026Bingyu Shen, Jiaqi Tang, Lan Wei +3Agent Failure AnalysisLLM Agent Evaluation

    Independent Researcher · Department of Computer Science and Technology, Kean University, Union, NJ, USA · Bellini College of Artificial Intelligence, Cybersecurity and Computing, University of South Florida, Tampa, FL, USA

  10. Why Agentic Workflows Fail: A Taxonomy Derived from Literature and Empirical Data

    Oct 12, 2026Xin Li, Rami Bahsoon, Tao ChenAgent Failure AnalysisAgentic Workflows

    School of Computer Science, University of Birmingham, Birmingham, United Kingdom

  11. Structural Baselines Matter for Agent Failure Detection and Attribution

    Oct 12, 2026Ramesh Arvind Naagarajan, Stefan StreifAgent Failure AnalysisFailure Attribution for LLM Agents

    Professorship for Automatic Control and System Dynamics Chemnitz University of Technology, Chemnitz, Germany · Department of Bioresources Fraunhofer Institute for Molecular Biology and Applied Ecology Giessen, Germany

  12. Can Failed Trajectories Reveal Where Agents Go Wrong? Agentic Failure Attribution Without Step-Level Labels

    Oct 9, 2026Do Nguyen-Thanh, Junlin Fang, Bing Li +2Failure Attribution for LLM AgentsAgent Failure Analysis

    College of Computing and Data Science, Nanyang Technological University, Singapore · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Australian Artificial Intelligence Institute, University of Technology Sydney

  13. Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems

    Oct 8, 2026Jiaqi Liao, Yuanzhao Zhai, Huanxi Liu +5Failure Attribution for LLM AgentsAgent Failure Analysis

    College of Computer Science and Technology, National University of Defense Technology; State Key Laboratory of Complex & Critical Software Environment, Changsha, Hunan, China · School of Mechanical Engineering, Tianjin University, Tianjin, China · College of Computer Science and Technology, National University of Defense Technology; National Key Laboratory of Parallel and Distributed Computing, Changsha, Hunan, China

  14. ReCast: Attribution-Oriented Step Representation Learning for LLM-Based Agent Systems

    Oct 8, 2026Weilin Jin, Mingyu Wang, Taiyu Zhu +7Failure Attribution for LLM AgentsAgent Failure Analysis

    Peking University, Beijing, China · Joy Future Academy, Beijing, China · Tsinghua University, Beijing, China

  15. Know the Shape, Find the Fault: Topology-Conditioned Diagnosis of Multi-Agent LLM Failures

    Oct 7, 2026Xinwen Liu, Zhuocheng Pan, Isabella Zhu +3Agent Failure AnalysisMulti-Agent Communication Topology

    Beihang University Beijing, China

  16. When the Governor Becomes the Disturbance: Control-Generated Disturbance and Cost-Aware Backoff in Governed Tool-Using Agents

    Oct 6, 2026Veronique ZieglerTool-Using AgentsRuntime Enforcement for AI Agents

    Independent Researcher, Critical Attention Systems

  17. From Evidence to Action: How Tool-Using Agents Fail

    Oct 6, 2026Hongzhan Lin, Shidong Cao, Ziyang Luo +3Agent Failure AnalysisEvidence-Grounded Reasoning

    National University of Singapore · Hong Kong Baptist University · Amazon Web Services +1

  18. Where Did the Repair First Go Wrong? Localizing the Origins of Silent Failures in Agentic Vulnerability Repair

    Oct 5, 2026Wenji Bai, Muhammad Waseem, Zeeshan Rasheed +2Agent Failure AnalysisAutomated Program Repair

    Tampere University, Tampere, Finland

  19. UndoBench: Separating Task Competence from Recovery Capability in Tool-Using AI Agents

    Oct 4, 2026Dolly Sah, Tanmay Sah, Harshul Jain +1Agent Failure AnalysisAI Agent Reliability

    Independent Researcher

  20. Disentangling Task Difficulty from Run-Level Failure in Agent Failure Prediction

    Oct 4, 2026Mohsen EsfandyariDoulabi, Lawrence Arkoh, Biruk Tadesse +4Agent Failure AnalysisLLM Agent Evaluation

    North Carolina State University, Raleigh, North Carolina, USA

  21. StateWise: Diagnosing and Repairing Persistent Operational State Before Agent Actions

    Oct 4, 2026Yongyuan Peng, Zhou Feng, Tongying Wu +5Agent Failure AnalysisSoftware Engineering Agents

    Xi’an Jiaotong University, China · Zhejiang University, Hangzhou, China

  22. DeFA: Dependency-Guided Failure Attribution for LLM Agents

    Oct 1, 2026Bo Deng, Xinlei Zheng, Yi Wei +6Agent Failure AnalysisLLM Agent Evaluation

    Qwen DianJin Team, Alibaba Cloud Computing · Beihang University

  23. Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training

    Sep 30, 2026Kunlun Zhu, Xuyan Ye, Yibo Li +3Agent Failure AnalysisLLM Agent Training

    Apodex

  24. Where Do Multi-Agent Systems Fail? Evidence-Grounded Diagnosis of Collective Mechanisms

    Sep 30, 2026Zhengye HanAgent Failure AnalysisAI Agent Auditing

    New York University, New York, NY, USA

  25. A Competing-Hazards Systematization of Loss of Control in Autonomous Agents

    Sep 29, 2026Mohamed Aly BoukeAgent Failure AnalysisAI Agent Auditing

    Centre for Intelligent Cloud Computing, CoE for Advanced Cloud, Faculty of Information Science and Technology, Multimedia University, Jalan Ayer Keroh Lama, Bukit Beruang, 75450, Melaka, Malaysia

  26. Failure-Transparent Agents: Benchmarking Post-Failure Reporting in Tool-Using Language Models

    Sep 28, 2026Junru Zhu, Shiming Xie, Aime Lu Fan Chen +4Agent Failure AnalysisAI Agent Benchmarks

    Independent Researcher · Ant Group · Tsinghua University +2

  27. ResonAct: Streaming Metrics for Runtime Diagnosis and Self-Healing in Multi-Agent Systems

    Sep 28, 2026Tarun Chintada, Neelamadhav Gantayat, Ishaan Romil +3Agent Failure AnalysisAI Agent Monitoring

    IBM, India · Indian Institute of Technology Patna, India · International Institute of Information Technology Hyderabad, India

  28. GUITAR: Structured Failure Diagnosis of GUI Agents via State Transitions

    Sep 28, 2026Shaoqing Zhang, Kehai Chen, Xuefeng Bai +4Agent Failure AnalysisGUI Agents

    Harbin Institute of Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China · Shanghai Jiao Tong University, China

  29. Re-derivability Decides What a Staged Agent Pipeline Recovers After an Upstream Fault

    Sep 26, 2026Tianqi Bu, YuXuan Peng, Junteng Tu +1LLM Agent ReliabilityAgent Failure Analysis

    Rutgers University · Nanjing Tech University · Nanjing University of Posts and Telecommunications +1

  30. From Anomalies to Failures: Constructing Causal Error Graphs for Agentic Trace Diagnosis

    Sep 26, 2026Shu-Xun Yang, Yidong Wang, Zhuoer Feng +7Agent Failure AnalysisReasoning Trace Analysis

    Beijing Institute of Technology, Beijing, China · Zhipu AI, Beijing, China · Tsinghua University, Beijing, China