cs.AINov 9, 2026

Dependable Long-Horizon LLM Agents: A Trajectory-Centered Empirical Study of Where Errors Enter and Which Controls Reach Them

Authors: Shaohuang Wang, Jinghao Wen, Xulei Zhang, Xun Jiao

Organizations: Villanova University, Department of Electrical and Computer Engineering, Villanova, Pennsylvania, USA

Abstract

When an LLM agent fails part-way through a long task, the usual analysis records a symptom: a failure-mode label, or the agent and step judged responsible. It does not say where the error entered, which state carried it to the symptom, whether anything in the trajectory contradicted it, or which control could have caught it. We define an episode record that ties a candidate fault to its origin, propagation, detection, response and consequence, located by operational phase and state domain, and that names the control families whose observation and reach cover it. We turned the record into a written annotation protocol and applied it to 95 failed runs of six multi-agent systems from the public Who&When and MAD corpora, reading each trajectory in full. Faults enter early (median at 0.18 of the run), mostly as wrong decisions or fabrications over task state, and in 37 of 88 Who&When runs before the step the corpus' human labels mark as decisive. In 72% of failed runs the trajectory already contained evidence against the error before the failure; in 88% of those no agent acted on it. Something flagged a deviation in 31% of runs, but no response removed the deviated state. Independent judges reading the unused evidence, task contracts fixed at admission, and re-checking beliefs against the environment reach most episodes; restoring the agent's own state would have sufficed in 35 of 95 runs. A text scan of the raw corpus also shows that 45% of the hand-crafted Who&When failures ended with a provider content-filter crash after which the harness emitted a final answer no agent produced, a failure class that neither symptom taxonomies nor responsible-agent labels represent. Protocol, records and scripts are included.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. DeFA: Dependency-Guided Failure Attribution for LLM Agents

    Oct 1, 2026Bo Deng, Xinlei Zheng, Yi Wei +6Agent Failure AnalysisLLM Agent Evaluation

  2. Structural Baselines Matter for Agent Failure Detection and Attribution

    Oct 12, 2026Ramesh Arvind Naagarajan, Stefan StreifAgent Failure AnalysisFailure Attribution for LLM Agents

  3. FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search

    May 30, 2026Md Nakhla Rafi, Md Ahasanuzzaman, Dong Jae Kim +2Agent Failure AnalysisMulti-Agent LLM Systems