Failure Attribution for LLM Agents

LLM: Large Language Model

Follow this topic Ask about this topic

Latest papers

29 papers10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

Human Interventions as Labels: Localizing, Steering, and (Sometimes) Editing the Computations Behind LLM Agent Failures
Oct 15, 2026Cesare Zavattari, Alessandro Tommasi, Giuseppe PrencipeLLM Agents Failure Attribution for LLM Agents
Dipartimento di Informatica, Università di Pisa, Pisa, Italy
BLADE: Auditing Information Use for Failure Attribution in LLM Multi-Agent Systems
Oct 13, 2026Xiaoguang Guo, Yaohao Long, Chuxu ZhangFailure Attribution for LLM Agents Agent Failure Analysis
University of Connecticut
Auditing Failure Attribution: Label Predictability and Judge Behavior
Oct 13, 2026Tianyang Xu, Jing GaoFailure Attribution for LLM Agents AI Agent Evaluation
Purdue University
Structural Baselines Matter for Agent Failure Detection and Attribution
Oct 12, 2026Ramesh Arvind Naagarajan, Stefan StreifAgent Failure Analysis Failure Attribution for LLM Agents
Professorship for Automatic Control and System Dynamics Chemnitz University of Technology, Chemnitz, Germany · Department of Bioresources Fraunhofer Institute for Molecular Biology and Applied Ecology Giessen, Germany
Can Failed Trajectories Reveal Where Agents Go Wrong? Agentic Failure Attribution Without Step-Level Labels
Oct 9, 2026Do Nguyen-Thanh, Junlin Fang, Bing Li +2Failure Attribution for LLM Agents Agent Failure Analysis
College of Computing and Data Science, Nanyang Technological University, Singapore · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Australian Artificial Intelligence Institute, University of Technology Sydney
Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems
Oct 8, 2026Jiaqi Liao, Yuanzhao Zhai, Huanxi Liu +5Failure Attribution for LLM Agents Agent Failure Analysis
College of Computer Science and Technology, National University of Defense Technology; State Key Laboratory of Complex & Critical Software Environment, Changsha, Hunan, China · School of Mechanical Engineering, Tianjin University, Tianjin, China · College of Computer Science and Technology, National University of Defense Technology; National Key Laboratory of Parallel and Distributed Computing, Changsha, Hunan, China
ReCast: Attribution-Oriented Step Representation Learning for LLM-Based Agent Systems
Oct 8, 2026Weilin Jin, Mingyu Wang, Taiyu Zhu +7Failure Attribution for LLM Agents Agent Failure Analysis
Peking University, Beijing, China · Joy Future Academy, Beijing, China · Tsinghua University, Beijing, China
DeFA: Dependency-Guided Failure Attribution for LLM Agents
Oct 1, 2026Bo Deng, Xinlei Zheng, Yi Wei +6Agent Failure Analysis LLM Agent Evaluation
Qwen DianJin Team, Alibaba Cloud Computing · Beihang University
What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agents
Sep 30, 2026Shuyang Zhang, Jianshuo ChangLLM Agent Evaluation Failure Attribution for LLM Agents
The Hong Kong Polytechnic University
DAAF: From Failure Localization to Editable System Assets in LLM Agents
Sep 26, 2026Xiaoyang Yuan, Qi Liu, Yubin Ruan +9Failure Attribution for LLM Agents Agent Failure Analysis
Tencent
DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems
Sep 4, 2026Zehao Wang, Lanjun Wang, Shilong Jin +2Agent Failure Analysis Multi-Agent LLM Systems
College of Intelligence and Computing, Tianjin University, Tianjin, China · School of New Media and Communication, Tianjin University, Tianjin, China · College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China +1
EDGE: Error Dependency Graph-Guided Multi-Error Attribution in Multi-Agent LLM Systems
Sep 1, 2026Jun Hou, Priya Pitre, Yi Fang +1Agent Failure Analysis Multi-Agent LLM Systems
Virginia Tech
Cascade Failure Attribution for Multi-Agent Systems
Aug 30, 2026Jiayi Zhang, Zexin Wang, Degang Sun +4Agent Failure Analysis Multi-Agent LLM Systems
Computer Network Information Center, Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences
Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attribution
Aug 19, 2026Ting-Wei Li, Yuanchen Bei, Xiao Lin +1Failure Attribution for LLM Agents Agent Failure Analysis
University of Illinois Urbana-Champaign, IL USA
ASCon: A Direction-Aware Reciprocal Agent--Step Contextualization Model for Failure Attribution in Multi-Agent Systems
Aug 11, 2026Shuyu Jiang, Yue Ran, Kaiyu Xu +5Agent Failure Analysis Multi-Agent LLM Systems
1Sichuan University, China · 2Nanyang Technological University, Singapore
HPFA: Hypergraph-Based Paired Failure Attribution for LLM Reasoning
Aug 3, 2026Runchuan Zhu, Hongbin Lai, Bowen Jiang +4LLM Reasoning Reasoning Trace Analysis
National University of Singapore · Peking University · University of Texas at Austin +1
Where Facts Go Missing: A Layerwise Taxonomy and Per-Layer Attribution of Information Omission in Air-Gapped LLMAgent Pipelines
Jul 24, 2026Santhiya RajanLLM Auditing LLM Agent Reliability
Multiverse Computing
AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents
Jul 21, 2026Kunlun Zhu, Xuyan Ye, Zhiguang Han +9Agent Failure Analysis AI Agent Reliability
University of Illinois Urbana-Champaign · University of Toronto · Google +1
Tracing Agentic Failure from the Flow of Success
Jul 14, 2026Samuel Yeh, Yiwen Zhu, Shaleen Deep +1Agent Failure Analysis Failure Attribution for LLM Agents
Department of Computer Science, University of Wisconsin-Madison · Microsoft Research
Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?
Jul 10, 2026Jiale Liu, Huajun Xi, Shaokun Zhang +6Agent Failure Analysis LLM Agent Evaluation
Penn State University · AG2ai, Inc. · Mohamed bin Zayed University of Artificial Intelligence +2
ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents
Jul 6, 2026Harsh SoniAI Agent Reliability Tool-Use Evaluation
UC Berkeley
SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation
Jun 23, 2026Chenyang Zhu, Jiayu Yao, Kushal Chawla +10Agent Failure Analysis Long-Horizon Agent Evaluation
1AI Foundations, Capital One · 2General Intelligence Com · Department of Engineering Sciences and Ap
REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces
Jun 8, 2026Xiaofeng Lin, Yingxu Wang, Tung Sum Thomas Kwok +4LLM Agent Evaluation LLM Agents
University of California, Los Angeles, USA · Cisco, USA · University of Mississippi, USA
Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures
Jun 6, 2026Jaineet ShahAgent Failure Analysis AI Agent Reliability
SkillAdaptor: Self-Adapting Skills for LLM Agents from Trajectories
May 31, 2026Zhuoyun Yu, Xin Xie, Wuguannan Yao +4LLM Agent Skill Learning Failure Attribution for LLM Agents
♠Zhejiang University · ♣Ant Digital Technologies, Ant Group · ♢Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph +1
FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search
May 30, 2026Md Nakhla Rafi, Md Ahasanuzzaman, Dong Jae Kim +2Agent Failure Analysis Multi-Agent LLM Systems
SPEAR Lab Concordia University Montreal, Canada · DePaul University Chicago, USA
CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures
May 25, 2026Akash Bonagiri, Devang Borkar, Gerard Janno Anderias +2Agent Failure Analysis Causal Interventions in Language Models
Department of Computer Science University of California, Davis Davis, CA 95616
A Hierarchical Error-Corrective Graph Framework for Autonomous Agents with LLM-Based Action Generation
Mar 9, 2026Cong Cao, Jingyao Zhang, Kun TongFailure Attribution for LLM Agents LLM Agent Reliability
Independent Researcher, Hangzhou, Zhejiang, China · Beihang University, Hangzhou, Zhejiang, China
AgentRx: Diagnosing AI Agent Failures from Execution Trajectories
Feb 2, 2026Shraddha Barke, Arnav Goyal, Alind Khare +3Fault Diagnosis AI Agent Reliability
1Microsoft Research · 2Microsoft · 3UIUC

Oct 15, 2026J/K move · Enter open · S save

Human Interventions as Labels: Localizing, Steering, and (Sometimes) Editing the Computations Behind LLM Agent Failures

Cesare Zavattari, Alessandro Tommasi, Giuseppe Prencipe

Dipartimento di Informatica, Università di Pisa, Pisa, Italy

LLM Agents Failure Attribution for LLM Agents LLM Agent Reliability Causal Interventions in Language Models Mechanistic Interpretability LLM Agent Training+1

Open paper AskPDF arXiv

When an LLM agent misbehaves, a human steps in, and that intervention is usually discarded or replayed as training data. We treat it instead as a measurement: a label pointing at the internal computation that failed. We study it in a controlled agent environment, on one failure (asking a clarifying question when the request was already unambiguous), with labels from a rule-based overseer validated against blind human judgment. Claims cover eligible models, which both over-ask and sometimes act on matched unambiguous requests: five of thirteen qualified (Qwen2.5-3B, gemma-2-2b, OLMo-3-7B, Mistral-Nemo-12B, Phi-4). Four findings. First, the ask-vs-act decision is causally localizable in every eligible family: a direction learned from intervention labels flips 72-100% of held-out over-asks at a definite residual-stream depth, with site and dose chosen on training data, and random directions under the same procedure flip at most one episode. Second, over-asking has two mechanistic signatures: in four families the model demonstrably knows the request is unambiguous (the belief is decodable, causally inert and orthogonal to the decision) and asks anyway; in the fifth, the rare over-asks track different world-representations. Third, steering gated on a probe readout removes most of the simulated oversight burden while legitimate questions survive (bootstrap 95% intervals exclude zero); a prompt instruction makes it worse. Fourth, consolidating the fix into the weights yields a map, not a rule: on one model a LoRA confined to the causal band passes every criterion fixed in advance while behavior cloning on identical data breaks retention; on three families the reverse holds; on the fifth the outcome crosses the boundary across seeds. A geometric rule written down in advance to predict the regime failed on two out-of-family tests; we report the failure and stop rather than fit a replacement.