Agent Failure Analysis

Ask about this topic

Overview

Agent failure analysis examines how errors arise and spread in tool-using, coding, GUI, and multi-agent systems, including failures hidden by apparently successful final outcomes. Common approaches annotate execution trajectories, classify failure modes, monitor runtime behavior, and use dependency graphs or counterfactual replay to distinguish root causes from downstream symptoms. Benchmarks use real or human-labeled traces, controlled fault injection, and repeated runs to test diagnosis and recovery. A central challenge is making attributions genuinely causal and robust when logs are incomplete, task conditions vary, and benchmark shortcuts can make performance look stronger than it is.

Latest papers

CardsList

154 papers27 papers in the last four weeks, up 42% on the four weeks before. 0.2% of all new papers.

  1. Risk Architecture for AI-Native Engineering Teams: An Organizational Framework for Agentic System Governance

    Jul 1, 2026Laxmipriya Ganesh IyerAgent Failure AnalysisAI Assurance

    Independent Researcher

  2. Exploring the Semantic Gap in Agentic Data Systems: A Formative Study of Operationalization Failures in Analytical Workflows

    Jul 1, 2026Jalal Mahmud, Eser KandoganAgent Failure AnalysisAgentic Workflows

    Megagon Labs California, USA

  3. One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attribution

    Jun 30, 2026Jie Ma, Binfei Chu, Jie Gao +6Agent Failure AnalysisLLM Self-Correction

    Xiamen University · Hello Inc. · Xiaohongshu Inc.

  4. SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation

    Jun 23, 2026Chenyang Zhu, Jiayu Yao, Kushal Chawla +10Agent Failure AnalysisLong-Horizon Agent Evaluation

    1AI Foundations, Capital One · 2General Intelligence Com · Department of Engineering Sciences and Ap

  5. Explaining Failures of Cyber-Physical Systems with Actual Causality

    Jun 23, 2026Khen Elimelech, Tom Yaacov, David A. Kelly +2Agent Failure AnalysisCyber-Physical Systems

    King’s College London · Rice University

  6. TraceView: Interactive Visualization of Agentic Program Repair Trajectories

    Jun 20, 2026Amirali Sajadi, Tu Nguyen, Kimmie Huynh +2Agent Failure AnalysisSoftware Engineering Agents

    Drexel University, Philadelphia, PA, USA · Belmont University, Nashville, TN, USA

  7. When Web Agents Finish but Still Fail: Reproducible Triggers and Trace Diagnostics for Parallel Web Exploration

    Jun 16, 2026Aagam Sogani, Botao Rui, Swetha Vaidyanathan +3Agent Failure AnalysisWeb Agent Benchmarks

    University of Wisconsin–Madison, Madison, WI, USA.

  8. Intelligence Entropy Principle and the ADE Stability Engineering Framework

    Jun 16, 2026Dexing LiuAgent Failure AnalysisMulti-Agent LLM Systems

    Shanghai Qijing Digital Technology · Shanghai Qijing Digital Technology Co., Ltd.

  9. ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures

    Jun 13, 2026Kenneth Ge, Andre AssisAgent Failure AnalysisAI Coding Agents

    Anthropic Fellows Program · Constellation

  10. When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime

    Jun 12, 2026Wei WuAgent Failure AnalysisAI Agent Monitoring

    Independent researcher

  11. Naive Visual Memory is Not Enough: A Failure-Mode Study of GUI Agents

    Jun 12, 2026Seoyoung Choi, Minseok Ko, Hyunseok Lee +4Agent Failure AnalysisGUI Agents

    1Seoul National University · 2POSTECH · 3KAIST +1

  12. Regimes: An Auditable, Held-Out-Gated Improvement Loop Demonstrated on LongMemEval with ActiveGraph

    Jun 8, 2026Yohei NakajimaAgent Failure AnalysisAI Agent Evaluation

    Untapped Capital activegraph.ai

  13. Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures

    Jun 6, 2026Jaineet ShahAgent Failure AnalysisAI Agent Reliability

  14. Silent Failure in LLM Agent Systems: The Entropy Principle and the Inevitable Disorder of Autonomous Agents

    Jun 6, 2026Dexing LiuAgent Failure AnalysisAI Agent Governance

    Shanghai Qijing Digital Technology Co., Ltd.

  15. From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws

    Jun 4, 2026Mengzhuo Chen, Junjie Wang, Zhe Liu +3Agent Failure AnalysisSelf-Improving Agents

    State Key Laboratory of Complex System Modeling and Simulation Technology, Beijing, China · Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China +1

  16. A Taxonomy of Runtime Faults in Model Context Protocol Servers

    Jun 3, 2026Joshua Owotogbe, Indika Kumara, Willem-Jan van den Heuvel +3Agent Failure AnalysisModel Context Protocol

    Jheronimus Academy of Data Science and Tilburg University, The Netherlands · University of Sannio, Benevento, Italy · University of Luxembourg, Luxembourg +1

  17. Channel Fracture: Three Instances of Cross-Boundary Silent Delivery Reliability Failures in Multi-Agent Systems

    Jun 3, 2026Dexing LiuAgent Failure AnalysisMulti-Agent Systems

    Shanghai Qijing Digital Technology Co., Ltd.

  18. Token Budgets: An Empirical Catalog of 63 LLM-Agent Budget-Overrun Incidents, with an Affine-Typed Rust Mitigation as a Case Study

    Jun 2, 2026Sajjad KhanAgent Failure AnalysisMulti-Agent LLM Systems

    independent researcher; MSc in Data Science, University of the West of England, Bristol, UK

  19. StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems

    Jun 2, 2026Taiyu Zhu, Yifan Wu, Weilin Jin +2Agent Failure AnalysisMulti-Agent LLM Systems

    Peking University Beijing, China

  20. Monitoring Agentic Systems Before They're Reliable

    Jun 1, 2026Marisa Ferrara Boston, Glen Hanson, Effi Georgala +2Agent Failure AnalysisAgent Reliability

    Reins AI USA · Veraitech USA

  21. POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems

    Jun 1, 2026Iñaki Dellibarda Varela, R. Sendra-Arranz, Pablo Romero-Sorozabal +5Agent Failure AnalysisAI Agent Reliability

    Center for Automation and Robotics, Spanish National Research Council (CSIC-UPM), Madrid, Spain. · ETSI Telecomunicación, Universidad Politécnica de Madrid (UPM), Madrid, Spain.

  22. Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories

    Jun 1, 2026Jiaming Wang, Ziteng Feng, Jiangtao Wu +8Agent Failure AnalysisDeep Research Agents

    NJU-LINK Team, Nanjing University · JIUTIAN Research

  23. Agent System Operations: Categorization, Challenges, and Future Directions

    Jun 1, 2026Zexin Wang, Changhua Pei, Yuanhao Liu +10Agent Failure AnalysisAI Agent Monitoring

    CNIC, CAS · Hangzhou Institute for Advanced Study, UCAS · Singapore Management University +1

  24. From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents

    Jun 1, 2026Laksh AdvaniAgent Failure AnalysisAI Agent Reliability

    University of Colorado

  25. FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search

    May 30, 2026Md Nakhla Rafi, Md Ahasanuzzaman, Dong Jae Kim +2Agent Failure AnalysisMulti-Agent LLM Systems

    SPEAR Lab Concordia University Montreal, Canada · DePaul University Chicago, USA

  26. How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions

    May 28, 2026Ningzhi Tang, Chaoran Chen, Gelei Xu +5Agent Failure AnalysisCoding Agents

    University of Notre Dame · 2Vanderbilt University · 3Google

  27. OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectories

    May 28, 2026Yibing Liu, Yangze Liu, Xiaolong Yin +4Agent Failure AnalysisAnomaly Localization

    School of Software, Shandong University, Jinan, China · School of Artificial Intelligence, Nanjing University; State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · Center for Medical Artificial Intelligence; Qingdao Academy of Chinese Medical Sciences; Institute of Marine Traditional Chinese Medicine, Shandong University of Traditional Chinese Medicine,2026 Qingdao 266112, China +1

  28. How VLAs Fail Differently: Black-Box Action Monitoring Reveals Architecture-Specific Failure Signatures

    May 27, 2026Krishnam GuptaAgent Failure AnalysisRobot Failure Detection

    Independent Research San Francisco, CA

  29. Intelligence as Managed Autonomy: Failure, Escalation, and Governance for Agentic AI Systems

    May 26, 2026Srini RamaswamyAgent Failure AnalysisAI Agent Reliability

    DNRS.ai USA

  30. CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures

    May 25, 2026Akash Bonagiri, Devang Borkar, Gerard Janno Anderias +2Agent Failure AnalysisCausal Interventions in Language Models

    Department of Computer Science University of California, Davis Davis, CA 95616