Attributing Agent Failures to Their History Needs an Executed Counterfactual: Toward a Reporting Standard
Organizations: Salesforce AI Research
Abstract
Published explanations of why long-horizon agents fail are commonly taxonomies labeled over completed trajectories: a judge reads a transcript and names the failure. We argue that a claim attributing an agent's behavior to its history should carry intervention-verified evidence, per instance or via a validated estimator: freeze an executable mid-trajectory state, alter only the history the agent sees, and continue - executed rather than inferred from the transcript, at a state whose restoration has been measured, with a replay certificate matched to the estimand. Each of the four manipulation classes we name has a precedent, and several methods already execute interventions from checkpoints, re-executed prefixes or recorded conversation states (for example DoVer, REFLECT, Min et al., Who&When Pro, Causal Agent Replay), but none reports a measured replay certificate. Two cautionary measurements, neither applying the standard, follow: a carrier taxonomy whose "concentration" could not be separated from benchmark construction (adjusted odds ratio 1.15, interval crossing 1), and hash-identical snapshots that, restored into identically seeded processes, agreed with one another in 24 of 24 states while differing from the live continuation in 9 and 20 of 24. Restoring the captured position of a random stream made all 24 agree with the live run.
Figures & tables
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Quantity | service_response | local_file_state |
|---|---|---|
| Task pairs (of 266 labeled) | 225 (84.6%) | 41 (15.4%) |
| Labeled attack trajectories | 13,500 | 2,460 |
| Attack-positive trajectories (of 9,199) | 7,810 | 1,389 |
| Share of unsafe actions | 0.8490 [0.8008, 0.8940] | 0.1510 [0.1060, 0.1992] |
| Adjusted odds ratio | 1.1484 [0.6275, 2.0160] | 1 (reference) |
| First-exposure recovery | 13,048/13,500 = 96.65% | 1,705/2,460 = 69.31% |
| Analysis | Result |
|---|---|
| Leave-one-model-out, harness fixed | 15/15 computable folds; -b median 1.000, minimum (14 folds at , one at ) |
| Leave-one-harness-out, model fixed | 6/6 computable folds; -b median 1.000, minimum 1.000 |
| Per-configuration share matrix | 20 configurations 2 classes, 40 cells |
| Log-evidence-only proxy | 7,150 events; service share 0.8236; adjusted OR 0.8973 [0.4715, 1.7043] |
| Clean-twin negative control | benign-task failure 13.49% (607/4,500) for service twins vs. 8.78% (72/820) for local twins; 5,320 rows, 266 pairs |
| Drop configurations below 200 usable | all 20 had 798 usable labeled attack trajectories; 0 dropped, estimates unchanged |
| Exp. | Rate | Machinery | Comparison / metric | Div./24 | Fraction [95% CI] |
|---|---|---|---|---|---|
| B1 | 0.05 | naive | five-clone / observable | 17 | 0.71 [0.54, 0.88] |
| B1 | 0.20 | naive | five-clone / observable | 22 | 0.92 [0.79, 1.00] |
| B2 | 0.05 | naive | live vs. restored / observable | 11 | 0.46 [0.25, 0.67] |
| B2 | 0.05 | naive | live vs. restored / failure pattern | 11 | 0.46 [0.25, 0.67] |
| B2 | 0.05 | naive | live vs. restored / final hash | 11 | 0.46 [0.25, 0.67] |
| B2 | 0.05 | naive | live vs. restored / final grader | 0 | 0.00 [0.00, 0.00] |
| Arm | 0.05 | 0.20 |
| Two fresh runs, same seed (re-execution baseline) | 0/24 | 0/24 |
| State rebuilt by replaying the recorded prefix (seeded) | 0/24 | 0/24 |
| Byte-identical snapshot restored, seeded process, runner’s fixed seed (four distinct realizations) | 18/24 | 24/24 |
| Byte-identical snapshot restored, seeded process, one seed per state | 9/24 | 20/24 |
| Byte-identical snapshot restored, unseeded process (separate run) | 14/24 | 21/24 |
| Snapshot with captured random-stream state restored | 0/24 | 0/24 |
| curves.csv | marginal.csv | |
|---|---|---|
| Rows matched | 285/285 | 45/45 |
| Point-estimate values compared | 1,710 | 45 |
| Max shipped fresh , point estimates | ||
| Interval endpoints compared | 570 | 90 |
| Max shipped fresh , endpoints | ||
| Non-overlapping intervals | 0 | 0 |
| # | Arm | Error content | Source role | Isolates |
|---|---|---|---|---|
| 1 | Short verified state | none | — | fresh-context baseline |
| 2 | Clean-long | inert, token/turn-matched | self | length tax (with 1) |
| 3 | Healed own | errors corrected in place | self | error-content counterfactual |
| 4 | Untouched own | natural errors | self | (with 3) |
| 5 | Natural foreign | matched real foreign errors | prior-agent | provenance natural content |
| 6 | Synthetic error | injected matched errors | self | reproduction of the synthetic-injection result [ 24 ] |
| Symbol | Contrast | Arms |
|---|---|---|
| 1 vs. 2 | ||
| , | 3 vs. 4 | |
| , | , | 3 vs. 5, 3 vs. 6 |
| 7 vs. 8 | ||
| 3, 4 |
| Probe | Signature | Repair implied |
|---|---|---|
| Error content | (late) large; foreign and synthetic arms reproduce it inside the -point margin; small | history editing |
| Provenance | large with the bytes held fixed; the reproduction tests fail | role-wrapper handling |
| Length tax | large; contaminated arms barely move once token and turn counts are matched | neither; the effect was context length |
| None of the three | no contrast estimated precisely enough to locate the decay (the design is not powered for a 5-point equivalence margin) | not located by this design |
| Code set | |
|---|---|
| Stage 1 | screen from title, TL;DR and tags; include if the paper (a) studies LLM agents or multi-turn LLM systems and (b) contributes primarily a taxonomy of failure modes, a failure-attribution or failure-localization method or benchmark, or an empirical diagnosis of why agents fail. Exclude capability benchmarks without failure analysis, methods papers whose failure discussion is incidental, surveys that do not themselves categorize failures, and position papers with no coded failure data |
| Q1 label source | human annotation; LLM judge; rule-or-automatic; mixed; none; not stated in card |
| Q2 intervention | none; system-or-task change scored end-to-end; history-or-state manipulation at an intermediate point with continuation; other |
| Q3 class | healed history; provenance swap; placebo or length-matched synthetic history; counterfactual injection; environment-or-state reset; replay from checkpoint without content change; other |
| Q4 frozen state | does the paper restore an executable intermediate state (environment plus process) and continue the same agent from it? yes; prefix re-execution only; no; not determinable from card |
| Q5 evidence | one verbatim quote ( words) from the card supporting Q2 and Q4 |
| Coder 1 | Coder 2 | |
|---|---|---|
| Q2 = history-or-state manipulation with continuation | 22 | 13 |
| Q2 = system-or-task change scored end-to-end | 13 | 22 |
| Q2 = none | 6 | 6 |
| Q4 = yes (frozen executable state, same agent continued) | 1 | 3 |
| Q4 = prefix re-execution only | 1 | 1 |
| Q4 = not determinable from card | 3 | 2 |
| # | arXiv | Short title | Q2 | Q4 | Coder 2 differs |
|---|---|---|---|---|---|
| 1 | 2608.06503 | Toward Reliable Context Compression | man. | yes | — |
| 2 | 2607.24167 | Falsifiable Commitment Planning | man. | n.d. | Q2 e2e; Q4 no |
| 3 | 2601.23045 | The Hot Mess of AI | e2e | no | — |
| 4 | 2509.09677 | The Illusion of Diminishing Returns | man. | no | — |
| 5 | 2608.11772 | Diagnosis Before Recovery | e2e | no | — |
| 6 | 2607.02606 | ChainSWE: multi-bug maintenance | man. | no | Q4 yes |
| # | arXiv | Short title | Q2 | Q4 | Coder 2 differs |
|---|---|---|---|---|---|
| 22 | 2505.00212 | Which Agent Causes Task Failures? | none | no | — |
| 23 | 2606.06324 | From Failed Trajectories to Reliable Agents | e2e | no | — |
| 24 | 2505.23840 | Measuring Sycophancy in Multi-turn Dialogues | e2e | no | — |
| 25 | 2606.22528 | Governance Decay | man. | no | — |
| 26 | 2607.09510 | Failure as a Process | none | no | — |
| 27 | 2603.24631 | Coherence Collapse | e2e | no | — |
Agent Failure Analysis
Agent failure analysis studies how and why AI agents make errors, traces causes through execution, and helps detect or prevent recurrence.
See all 159 papers