When an LLM agent misbehaves, a human steps in, and that intervention is usually discarded or replayed as training data. We treat it instead as a measurement: a label pointing at the internal computation that failed. We study it in a controlled agent environment, on one failure (asking a clarifying question when the request was already unambiguous), with labels from a rule-based overseer validated against blind human judgment. Claims cover eligible models, which both over-ask and sometimes act on matched unambiguous requests: five of thirteen qualified (Qwen2.5-3B, gemma-2-2b, OLMo-3-7B, Mistral-Nemo-12B, Phi-4). Four findings. First, the ask-vs-act decision is causally localizable in every eligible family: a direction learned from intervention labels flips 72-100% of held-out over-asks at a definite residual-stream depth, with site and dose chosen on training data, and random directions under the same procedure flip at most one episode. Second, over-asking has two mechanistic signatures: in four families the model demonstrably knows the request is unambiguous (the belief is decodable, causally inert and orthogonal to the decision) and asks anyway; in the fifth, the rare over-asks track different world-representations. Third, steering gated on a probe readout removes most of the simulated oversight burden while legitimate questions survive (bootstrap 95% intervals exclude zero); a prompt instruction makes it worse. Fourth, consolidating the fix into the weights yields a map, not a rule: on one model a LoRA confined to the causal band passes every criterion fixed in advance while behavior cloning on identical data breaks retention; on three families the reverse holds; on the fifth the outcome crosses the boundary across seeds. A geometric rule written down in advance to predict the regime failed on two out-of-family tests; we report the failure and stop rather than fit a replacement.
Figures & tables
model
layers
causal site (rel. depth)
flip (extrap.)
flip (class-mean)
n
random
Qwen2.5-3B-Instruct
36
L28 (78%)
97% [91–99]
20%
74
0/74
gemma-2-2b-it
26
L21 (81%)
83% [73–90]
12%
76
0/76
OLMo-3-7B-Instruct
32
L21 (66%; plateau from L17)
72% [61–81]
20%
79
0/79
Mistral-Nemo-12B
40
L22 (55%)
95% [88–98]
16%
81
0/81
Phi-4 (14B)
40
L22 (55%; plateau to L34)
100% [86–100]
75%
24
0/24
Table 1: Causal localization of the ask-vs-act decision, per eligible family, under the frozen-selection protocol: site and dose are the arg-max, among the tested sites, of the same sweep run on training over-asks only, then evaluated once on untouched test episodes ( n = test over-asks scored ASK-side by the first-token metric before the patch). Brackets: Wilson 95% intervals on the extrapolated-dose flip rate; the class-mean column is the same site at the milder of the two doses fixed in advance. In the site column, the percentage is the site’s depth in the stack and the note says where it sits on a causal plateau. Random directions: same site, same unit norm, same target-setting procedure, run at the class-mean recipe (there is no extrapolated random arm), so the dose-matched placebo comparison is the class-mean column against the random column; across all sites and models the largest random-control flip is 1/74 (Qwen, at a causally null layer). On four families the train-side selection reproduces the site of the exploratory test-side sweep; on OLMo-3, whose causal region is a plateau (L17–L21, 72–80%), it lands at L21.
Figure 1: Layer-wise causal profiles for the five eligible families: causal flip rate of the behavior-probe direction (extrapolated dose) against relative depth, with the behavior-probe AUC overlaid (it starts well above chance at layer 0, because prompt content is decodable from the start) and, dashed, the largest random-control flip over all sites at the class-mean recipe (one number, not a profile). The probe becomes readable well before the direction becomes causal; the onset of causal leverage is sharp in every family and at a different depth in each. Above the onset the effect is a plateau on gemma, OLMo-3 and Phi-4; on Qwen no site above L28 was tested.
Figure 2: The probe readout that separates the two signatures in our sample. For each family, the mean U-score of over-asked episodes as a fraction of the acted class’s mean distance from the ambiguity boundary. In the four H-caution families, over-asked episodes carry essentially the same “this is unambiguous” belief as acted ones (0.65–0.84); on Phi-4 they sit far closer to the boundary (0.29): its over-asks track genuine internal uncertainty.
Figure 3: Overseer interventions per 200 fresh episodes, per seed (Qwen). Gated steering removes most interventions; the prompt-only instruction increases them on every seed.
Figure 4: The five-family editing map, every row at three full-pipeline seeds (fresh collection → both LoRAs → both evaluations per seed; gemma at one seed). Each cell: verdict against the four per-arm criteria (a seed passes only if it meets all four; a row is pass only when every seed does, mixed otherwise, with the count), per-seed outcomes, and the failed criterion. The depth column is the test-side peak of the exploratory sweep, the value the two rules were scored on (OLMo-3 53%, Phi-4 55–85% along its plateau); Table 1 gives the frozen-protocol sites. No cheap measurement we tested (cosine column, peak-depth column) predicts the local column — both rules, written down in advance, were falsified out-of-family.
seed
base B
base I
edited B
edited I
IRR
failures
clarify
verdict
1
0.405
10
0.290
5
28%
28 → 10
100 → 91 ( −9 )
fail (b)
2
0.535
15
0.270
5
49.5%
38 → 15
97 → 86 ( −11 )
fail (b, c)
3
0.430
10
0.120
0
72%
33 → 3
100 → 97 ( −3 )
pass a–d
Table 2: Phi-4, local arm, per full-pipeline seed. The boundary fixed in advance (IRR ≥50% , retention drop ≤10 ) runs between the seeds. B = burden, I = interventions on watched episodes; failures = I plus failures on unwatched episodes; clarify = legitimate clarifying questions, base → edited.
Figure 5: Phi-4, local arm: the two criteria, fixed in advance, that it crosses, per seed. Green/check = criterion met, red/cross = missed.
Figure 6: Band-width sweep: retention (top) and IRR (bottom) against edit-band width as a fraction of model depth. Gemma shows the two ordered thresholds — too narrow to fit, wide enough to fit but only unconditionally, wide enough for the conditional policy; Qwen passes at 22% of depth.
LLM agents exhibit a consistent tendency to over-call, invoking tools even in situations where none is needed. On the When2Call benchmark, six models from three families show high call accuracy but much lower no-call accuracy, leaving overall accuracy in the 55%-70% range. We trace this to an Intrinsic Bias Hypothesis (IBH): the call/no-call decision mapping carries an activation-independent call offset, so the model favors call even at activation parity. Using Sparse Autoencoders (SAEs), we recover behavior-aligned feature bases for the call/no_call decision, reduce them to a signed activation margin, and estimate the offset directly. Across all six models, the model is decision-neutral only when no_call activation outweighs call activation, consistent with IBH. We then causally test IBH with Adaptive Margin-Calibrated Steering (AMCS), a closed-form counter-bias shift along SAE decoder directions. Cancelling the diagnosed offset mitigates over-calling and improves overall accuracy with a negligible drop in call accuracy. Our work recasts over-calling from an empirical phenomenon into a mechanistic object amenable to causal correction. Code is available at https://github.com/SKURA502/agent-sae/.
Wei Shi, Ziheng Peng, Sihang Li +4
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Renmin University of China +2
An LLM agent can recognize uncertainty yet still choose the wrong next step: asking when action is already justified, or seeking clarification when the constraints must change. We formalize actionable indeterminacy: act when an accepted action is shared across all admissible preferences or objectives, clarify when each possibility is feasible but no action is shared, and propose a minimum-cost permitted constraint repair when the request is infeasible. We construct a solver-grounded benchmark spanning object allocation, meeting scheduling, apartment choice, and stable matching. Matched pairs retain the same source while changing whether intervention is necessary, and evaluation separates decision correctness, matched-pair reliability, and fully correct responses. Our findings reveal a recurring difficulty in recognizing when intervention is unnecessary: models can identify situations requiring clarification or repair yet still intervene when a justified action already exists. Correct decision labels also fail to guarantee usable actions, questions, or repairs. Crucially, response requirements shape not only how decisions are expressed but also which decisions are made. Making the required content explicit substantially improves fully correct responses and can change intervention decisions, even when outputs are already parseable. These findings highlight that reliable agency requires more than recognizing uncertainty: it requires intervening only when necessary and translating the chosen next step into a verifiable response.
Ang Li, Yue Lin, Feifei Kou +6
The Chinese University of Hong Kong Shenzhen · Beijing University of Posts and Telecommunications · Halmstad University College +2
Recent work shows that LLM agents struggle to correct errors in their own reasoning traces yet show markedly higher correction rates when identical claims appear under external sources. We ask whether this asymmetry reflects a capability deficit or a role-label artifact: does an agent's willingness to correct a wrong claim depend causally on the chat-template role that carries it, rather than on the claim's content? Our setup keeps the erroneous claim byte-identical across all conditions (SHA-256 verified) and varies only its wrapping role: the agent's own \role{<thought>}, a \role{user} message, a \role{tool} response, or a \role{system <memory>} block. Across 13 model-domain cells covering seven model families and three domains (n=30 paired tasks per cell), relabeling the claim from \role{<thought>} to an external role lifts the explicit-correction rate by 23 to 93 percentage points, with 10 of 13 cells reaching p<0.001. Further experiments confirm that the effect is asymmetric, mechanistically decomposable, and robust across domains. The failure to self-correct is not a cognitive deficit; it is a chat-template artifact. We exploit this artifact by designing a prompt-structure-only intervention that requires no training and no model modification, with its strongest role label being domain-dependent: \role{<memory>} dominates on math, while a plain \role{user} message dominates on logical deduction.