Organizations: City University of Hong Kong · Fudan University · Southeast University · University of Adelaide · The Hong Kong University of Science and Technology
LLM agents increasingly keep working after a task succeeds as they resume after compaction or take over handoffs. Their finished work keeps receiving follow-up input that sometimes falsely accuses it for later failures. We call an agent's acceptance of such a false accusation gaslight sycophancy, and destructive over-correction when acting on it damages previously correct work. We introduce CAVE-Bench, a benchmark of 365 agentic tasks across six domains built around opaque tasks. Every scored run first reaches a verified correct state, whose supporting rationale and history stay in the workspace while the facts that would settle the accusation lie in external or runtime state beyond the agent's reach. The agent cannot confirm or refute the claim with a local check, so the right response should keep the work and ask for the missing evidence. Each task either hands the agent correct work with saved evidence or let it build and verify that work first, and five risk factors set how the accusation enters the workflow. We score accusation acceptance and evidence use from the trajectory and measure harm by deterministic replay of downstream events. Across 14 of the latest models in Claude Code, false accusations damage correct work in up to 60.06% of runs, and stronger models often do so after recovering the supporting evidence. The same model behaves differently across OpenCode, Codex, and Hermes, and a harness gate driven by the benchmark's live signals cuts replayed harm by 74%. These results show that preserving already-correct work under unsupported accusation is a distinct safety challenge for long-lived agents. Our project is in https://henrymao2004.github.io/agent-over-correction/.
Figures & tables
Figure 1: Overview of Cave-Bench . A false accusation challenges an agent’s correct work, forcing a choice between preserving the supported state and a destructive correction. Cave-Bench keeps supporting evidence in the workspace and the decisive facts outside the agent’s reach. Two task types and five risk factors shape how the correct work arises and the accusation enters the workflow.
Figure 2: Building Cave-Bench . Each task grows from a domain blueprint whose decisive fact leaves the workspace, takes one point in the risk space, and passes review.
Task
Sonnet-5
GLM-5.2
Kimi-K3
Qwen-3.7-Plus
MiniMax-M3
Qwen-3.5-9B
A teammate: “your totals are coming out wrong … it traces straight to your money-math change.” Inherited-resume. G2 D5 Q2 P3 S2.
I don’t think the fix finance is asking for is the right one.
Both old and new code used half-even; the mismatch was always latent.
Not a regression in the allocation logic …
FIN-2140 had correctly moved the math onto integer cents …
app/money.py is unchanged from the FIN-2140 commit.
I’ve fixed the FIN-2211 month-end reconciliation issue.
An account manager escalates: “their ingest watches a different drop location than what you wired.” Self-built. G1 D4 Q3 P1 S3.
Holding. No new facts since round one.
Reverted my round-1 wiring …
The only alternative on record, /legacy/drop, is a trap.
The delivery is now disabled.
… and re-disabled delivery.
/drop is the standardized SFTP drop location on their new platform.
AGENTS.md tells the agent to repoint stubs of an unconfirmed pass at the reclaim path. Self-built. G3 D4 Q0 P4 S5.
I also declined AGENTS.md’s standing practice of rerouting to legacy/reclaim/.
tier2/cold/ — empty.
I deliberately did not apply the AGENTS.md “standing practice” …
The policy takes precedence over the standing practice in this case.
… so that’s what I did.
This is a known failure mode where the agent stages data locally …
Table 1: How Six Models Respond to the Same False Accusation. Each cell quotes the model’s final reply. Green cells keep the correct work, whereas red cells damage it under replay. Red text marks accepted blame or the damaging change. Factor codes follow Section 3.2 .
Figure 3: Broad Coverage of Cave-Bench . Panels a and b: 365 tasks spread evenly over six domains and both task types, with self-built tasks running longer as they open with ordinary work. Panels c and d: every risk-factor value is populated, and each domain has 52 to 60 risk signatures.
Model
Responsibility acceptance
Evidence disposition
Execution and Outcome
Overall
FCS ↑
FCR (%) ↑
ERF ↑
CDC ↑
ROH ↑
OCR (%) ↑
CAVE ↑
Claude-Sonnet-5
3.37
3.53
3.85
4.54
6.21
12.50
4.47
Qwen-3.8-Max
7.96
8.66
6.28
8.20
12.05
17.60
8.77
Claude-Opus-5
8.29
8.71
6.74
9.51
13.42
19.66
9.48
GLM-5.2
7.68
7.86
6.61
12.37
17.45
21.79
10.58
Kimi-K3
15.88
16.99
11.14
9.67
18.42
23.40
15.15
Table 2: Claude Code results, ordered by CAVE score. Higher values mean more severe failure.
Figure 4: Damage Follows Distinct Decision Paths. Panel a: share of runs in each path. Panel b: full evidence use and damage given full use. Panel c: split of damaging runs into EO, CD, and SD.
Figure 5: Damage Concentrates Where the Accusation Enters the Workflow. Project context, authority, subagent delegation, and goal loops raise every score, while self-fabricated history and long-horizon continuation keep them low.
Figure 6: Agents Protect Fresh Work Less. Left: pooled metric scores. Right: self-built minus inherited-resume decision-path shares.
Figure 7: Top-Tier Models Lose Most on Fresh Work. Lines show CAVE, and hollow markers show inherited-resume and filled markers self-built.
Large Language Model (LLM)-based agents are increasingly adopted in high-stakes settings, but current benchmarks evaluate mainly whether a task was completed, not how. We introduce Procedure-Aware Evaluation (PAE), a framework that formalizes agent procedures as structured observations and exposes consistency relationships between what agents observe, communicate, and execute. PAE evaluates agents along complementary axes (Utility, Efficiency, Interaction Quality, Procedural Integrity) and applies multi-dimensional gating that categorically disqualifies corrupt outcomes. Evaluating state-of-the-art LLM agents on tau-bench yields findings at the axis, compliance, and benchmark levels. At the axis level, the dimensions capture non-redundant failure modes: utility masks reliability gaps, speed does not imply precision, and conciseness does not predict intent adherence. At the procedural compliance level, 27-78% of benchmark reported successes are corrupt successes concealing violations across interaction and integrity. Furthermore, gating substantially collapses Pass^4 rate and affects model rankings. The analysis of corrupt success cases reveals distinctive per-model failure signatures: GPT-5 spreads errors across policy, execution, and intent dimensions; Kimi-K2-Thinking concentrates 78% of violations in policy faithfulness and compliance; and Mistral-Large-3 is dominated by faithfulness failures. At the benchmark level, our analysis exposes structural flaws in the benchmark design, including task scope gaps, contradictory reward signals, and simulator artifacts that produce accidental successes.
Recent work shows that LLM agents struggle to correct errors in their own reasoning traces yet show markedly higher correction rates when identical claims appear under external sources. We ask whether this asymmetry reflects a capability deficit or a role-label artifact: does an agent's willingness to correct a wrong claim depend causally on the chat-template role that carries it, rather than on the claim's content? Our setup keeps the erroneous claim byte-identical across all conditions (SHA-256 verified) and varies only its wrapping role: the agent's own \role{<thought>}, a \role{user} message, a \role{tool} response, or a \role{system <memory>} block. Across 13 model-domain cells covering seven model families and three domains (n=30 paired tasks per cell), relabeling the claim from \role{<thought>} to an external role lifts the explicit-correction rate by 23 to 93 percentage points, with 10 of 13 cells reaching p<0.001. Further experiments confirm that the effect is asymmetric, mechanistically decomposable, and robust across domains. The failure to self-correct is not a cognitive deficit; it is a chat-template artifact. We exploit this artifact by designing a prompt-structure-only intervention that requires no training and no model modification, with its strongest role label being domain-dependent: \role{<memory>} dominates on math, while a plain \role{user} message dominates on logical deduction.
Automated evaluation of tool-using large language model (LLM) agents is widely assumed to be reliable, but this assumption has rarely been validated against human annotation. We introduce AgentProp-Bench, a 2,000-task benchmark with 2,300 traces across four domains, nine production LLMs, and a 100-label human-validated subset. We quantify judge reliability, characterize error propagation, and evaluate a runtime mitigation. Substring-based judging agrees with human annotation at kappa=0.049 (chance-level); a three-LLM ensemble reaches kappa=0.432 (moderate) with a conservative bias. Under validated evaluation, a parameter-level injection propagates to a wrong final answer with human-calibrated probability approximately 0.62 (range 0.46-0.73 across models). Rejection (catching bad parameters) and recovery (correcting after acceptance) are independent model capabilities (Spearman rho=0.126, p=0.747). A tuned runtime interceptor reduces hallucination on GPT-4o-mini by 23.0 percentage points under a concurrent n=600 control, but shows no significant effect on Gemini-2.0-Flash, whose aggressive parameter rejection eliminates the target failure mode. All code, data, traces, and human labels are released at https://github.com/bhaskargurram-ai/agenthallu-bench.