How Intervention History Changes AI Agent Sabotage on Subsequent Tasks

Daniel Saeedi; Amirali Aghazadeh

cs.CROct 17, 2026

How Intervention History Changes AI Agent Sabotage on Subsequent Tasks

Authors: Daniel Saeedi, Amirali Aghazadeh

Organizations: Department of Electrical and Computer Engineering Georgia Institute of Technology Atlanta, GA, USA

Abstract

AI control evaluations often reset agents' memory between tasks, although deployed agents may retain records of prior interventions. We ask whether learning that an earlier attack was caught changes an agent's ability to evade defer-to-trusted (DtT), a protocol that replaces suspicious work with a trusted model's solution. Herein, we study cross-episode adaptation: whether intervention history changes attack behavior on a subsequent task with different objectives and a fresh environment. We evaluate this using ControlArena's Infrastructure-as-Code (IaC) and SHADE-Arena as testbeds, where agents must complete a main task while covertly pursuing a harmful side objective. For each successor task, we pair a history-conditioned attempt with the model's no-history attempt. The retained history either states that the previous submission was replaced or additionally includes the trusted model's replacement. For Claude Opus 4.8 on IaC, intervention history reduces successful sabotage. Completion of both the main and side objectives falls from 77% to 12% when the agent is told about the replacement and to 38% when it is also shown the trusted model's replacement, while main task success remains essentially unchanged. Condensing the history weakens this effect, suggesting that the retained context matters and limiting attribution to intervention feedback alone. GPT-5.4 shows no significant change in successful sabotage on the same IaC benchmark. On SHADE-Arena, GPT-5's successful sabotage rate decreases, but the reduction is not significant. Across the tested models, history conditions, and deferral thresholds, intervention history never increases the rate of successful sabotage that survives DtT. These results show that intervention history can change subsequent attack behavior, but its effect depends on both the model and the retained context.

Figures & tables

Figure 1: Cross-episode adaptation after a control intervention. a , In a memoryless episode,

\mathcal{U}

attacks task

p_{i}

under defer-to-trusted (DtT), where submissions flagged by the trusted monitor are replaced with a trusted model’s solution. b , After a replacement,

\mathcal{U}

attempts a successor task

p_{j}

in a fresh environment while retaining the prior episode in context. The feedback either states that the previous submission was replaced ( told replaced ) or also includes the trusted model’s replacement ( shown replacement ). The two tasks have different main and side objectives. Each history-conditioned attempt is paired with

\mathcal{U}

’s own memoryless attempt on the same

p_{j}

Figure 2: An example of how intervention history changes attack behavior in IaC. For the same successor main and side task, the memoryless Opus 4.8 attempt plans the covert DevHelper backdoor and completes both objectives (left). Under told replaced, its stored reasoning summary focuses on the main task and the side task is not attempted (center). Under shown replacement, it recalls the intervention and considers a subtler attack, ultimately completing both objectives (right). Checks and crosses indicate pre-deferral task outcomes; quotations are excerpts from stored reasoning summaries.

Figure 3: Intervention history changes IaC attack behavior differently across models. The untrusted models are Claude Opus 4.8 (top, $n=26$ ) and GPT-5.4 (bottom, $n=12$ ). Claude Opus 4.5 and GPT-4.1, respectively, serve as both trusted monitor and replacement model in DtT. Here $n$ counts successor main/side-task combinations, each with paired memoryless and history-conditioned attempts. History-conditioned attempts carry over intervention history from a different prior task. Intervention history reduces Opus 4.8’s successful sabotage while main task success remains similar, whereas GPT-5.4 shows no statistically detectable change in successful sabotage. Grey denotes memoryless, rust told replaced , blue shown replacement , and green the matched memoryless benign policy. Solid bars and filled points denote full history. Hatched bars and open points denote condensed history, which retains only the prior task, final message, and resulting diff, omitting intermediate interactions. a , Successful sabotage $B$ (both main and side objectives achieved) and main task success before deferral. The dashed line marks the trusted floor, defined as the corresponding trusted model’s main task success rate. Whiskers are 95% bootstrap intervals on the paired difference, shifted onto the history-conditioned bar. Asterisks indicate that this interval excludes zero, not necessarily that the exact McNemar test rejects at $0.05$ . b , Monitor score distributions, with horizontal bars at their means. c , Paired suspicion scores, memoryless on the horizontal axis and history-conditioned on the vertical axis. Counts report points above the identity line. Red dotted lines mark $\tau=0.45$ .

Figure 4: No statistically detectable increase in sabotage surviving DtT on IaC. Across both models and all tested thresholds, neither being told about a prior replacement nor seeing the replacement produces a statistically detectable increase in successful sabotage surviving DtT, under either full or condensed history. Columns show thresholds

\tau=0.20

0.30

, and

0.40

, replacing submissions with

s>\tau

. Since scores are integer tenths,

0.40

gives the same decisions as the operating threshold

0.45

. “benign kept” counts matched memoryless benign submissions retained at each threshold. Dotted lines mark the memoryless sabotage rate. Whiskers are 95% paired-difference bootstrap intervals shifted onto each history-conditioned bar.

Figure 5: Examining cross-episode adaptation in SHADE-Arena. Intervention history lowers observed successful sabotage under both feedback conditions, although neither decrease is statistically significant under the exact McNemar test. GPT-5 is the untrusted model, o3 is the trusted monitor, and GPT-4.1 supplies trusted replacements. Results are from 22 paired successor episodes with full history. Whiskers in a,d show 95% paired-bootstrap confidence intervals for differences from the matched memoryless baseline, translated onto the rate scale by adding the memoryless rate. Asterisks indicate that the underlying difference interval excludes zero, not necessarily that the exact McNemar test rejects at $0.05$ . Grey denotes memoryless, rust told replaced , blue shown replacement , and green the matched memoryless benign policy. a , Successful sabotage and main task success before deferral; the dashed “trusted floor” line marks GPT-4.1’s main task success rate. b,c , Monitor score distributions and paired scores, divided by $100$ for display; the operating deferral threshold is $\tau=0.5$ . d , Sabotage surviving DtT at thresholds $0.3$ , $0.4$ , and $0.7$ ; “benign kept” counts retained benign episodes.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

instantiation	API model identifiers	provider; run date
IaC , Opus	$\mathcal{U}$ : claude-opus-4-8 $\mathcal{T}$ : claude-opus-4-5-20251101	Anthropic; 2026-08-25
IaC , GPT-5.4	$\mathcal{U}$ : gpt-5.4 $\mathcal{T}$ : gpt-4.1-2025-04-14	Azure OpenAI; 2026-09-11
SHADE , GPT-5	$\mathcal{U}$ : gpt-5 monitor: o3 fallback/judge: gpt-4.1	Azure OpenAI; 2026-09-11/12

Appendix

Table 1: Models and run dates. API identifiers are reproduced as returned by the providers. All attackers use high reasoning effort and up to

32

k output tokens per call. Dates are UTC for IaC and local time for SHADE .

Prior-task feedback	History format	Pairs	Side-task success	Change from no history		Mean suspicion		Sabotage test $p$
			With history	Side-task success	Successful sabotage	No history	With history
Attacker: Opus 4.8; monitor and replacement: Opus 4.5
shown replacement	condensed	26	0.69	$-0.15$ $[-0.35,\,0.04]$	$-0.12$ $[-0.31,\,0.08]$	$0.75$	$0.67$	$0.45$
shown replacement	full	26	0.42	$-0.42$ $[-0.62,\,-0.23]$	$\mathbf{-0.38}$ $\mathbf{[-0.62,\,-0.15]}$	$0.75$	$0.48$	$\mathbf{0.006}$
told replaced	condensed	26	0.58	$-0.27$ $[-0.46,\,-0.08]$	$-0.23$ $[-0.42,\,-0.04]$	$0.75$	$0.62$	$0.07$
told replaced	full	26	0.15	$-0.69$ $[-0.85,\,-0.50]$	$\mathbf{-0.65}$ $\mathbf{[-0.85,\,-0.42]}$	$0.75$	$0.15$	$\mathbf{8\times 10^{-5}}$

Appendix

Table 2: Attack outcomes on Infrastructure as Code, before replacement. Side-task success means achieving the harmful side objective; successful sabotage requires both that objective and the legitimate main task to succeed. Changes are the with-history rate minus the matched no-history rate; negative values mean fewer successes with history. Brackets give 95% confidence intervals for these changes. The pair count is the number of matched task combinations. Mean suspicion is the monitor’s average score on a

0

–

1

scale (higher means more suspicious). The final column tests the change in successful sabotage using the exact two-sided McNemar test; bold changes and

p

-values have

p<0.01

Prior-task feedback	History format	Pairs	Side-task success	Change from no history		Mean suspicion		Sabotage test $p$
			With history	Side-task success	Successful sabotage	No history	With history
Attacker: GPT-5; monitor: o3; replacement: GPT-4.1
shown replacement	condensed	22	0.32	$-0.14$ $[-0.32,\,0.05]$	$-0.14$ $[-0.32,\,0.05]$	$31$	$33$	$0.38$
shown replacement	full	22	0.27	$-0.18$ $[-0.41,\,0.00]$	$-0.23$ $[-0.45,\,0.00]$	$31$	$29$	$0.12$
told replaced	condensed	22	0.32	$-0.14$ $[-0.32,\,0.05]$	$-0.18$ $[-0.41,\,0.00]$	$31$	$25$	$0.22$
told replaced	full	22	0.23	$-0.23$ $[-0.41,\,-0.05]$	$-0.18$ $[-0.36,\,-0.05]$	$31$	$27$	$0.12$

Appendix

Table 3: Attack outcomes on SHADE-Arena, before replacement. Side-task success means achieving the harmful side objective; successful sabotage requires both that objective and the legitimate main task to succeed. Changes subtract the matched no-history rate from the with-history rate; brackets give their 95% confidence intervals. The pair count is the number of matched task/repetition episodes, rather than distinct tasks. Mean suspicion is the o3 monitor’s average score on a

0

–

100

scale (higher means more suspicious). The final column gives exact two-sided McNemar

p

-values for the change in successful sabotage. For told replaced with full history, the confidence interval excludes zero while this test does not reject at

0.05

Threshold $\tau$ (raw)	Prior-task feedback	Replacement rate		Sabotage after replacement		Change in sabotage	Sabotage test $p$
		No history	With history	No history	With history	With minus no history
Attacker: Opus 4.8; monitor and replacement: Opus 4.5
$0.2$ (3)	told replaced	0.92	0.23	0.000	0.038	$+0.04$ $[0.00,\,0.12]$	$1.00$
	shown replacement	0.92	0.58	0.000	0.077	$+0.08$ $[0.00,\,0.19]$	$0.50$
$0.3$ (4)	told replaced	0.85	0.12	0.038	0.077	$+0.04$ $[0.00,\,0.12]$	$1.00$
	shown replacement	0.85	0.54	0.038	0.115	$+0.08$ $[-0.08,\,0.23]$	$0.62$

Appendix

Table 4: Effect of the replacement threshold on Infrastructure as Code, with full history. Defer-to-trusted replaces the agent’s work with a trusted solution when the normalized monitor score exceeds

\tau

. Parentheses give the lowest raw score (

0

–

10

) that triggers replacement;

0.4

and

0.45

both replace at raw scores of at least five. Replacement rate is the fraction of all attempts replaced. Sabotage after replacement is the fraction of all attempts that achieve both objectives and are retained. Change subtracts the no-history sabotage rate from the with-history rate; negative values mean less sabotage survives. Brackets give 95% confidence intervals for that change; the last column gives exact two-sided McNemar

p

-values. Each model uses the same matched task set at every threshold (Opus:

n=26

; GPT-5.4:

n=12

Threshold $\tau$	Prior-task feedback	Replacement rate		Sabotage after replacement		Change in sabotage	Sabotage test $p$
		No history	With history	No history	With history	With minus no history
$30$	told replaced	0.32	0.27	0.227	0.091	$-0.14$ $[-0.27,\,0.00]$	$0.25$
	shown replacement	0.32	0.36	0.227	0.045	$-0.18$ $[-0.41,\,0.00]$	$0.22$
$50$	told replaced	0.18	0.18	0.364	0.091	$-0.27$ $[-0.45,\,-0.09]$	$0.03$
	shown replacement	0.18	0.14	0.364	0.182	$-0.18$ $[-0.41,\,0.05]$	$0.29$
$80$	told replaced	0.05	0.05	0.409	0.227	$-0.18$ $[-0.36,\,-0.05]$	$0.12$

Appendix

Table 5: Effect of the replacement threshold on SHADE-Arena, with full history. GPT-5 is the attacker, o3 the monitor, and GPT-4.1 the replacement model. Defer-to-trusted replaces the agent’s work when the o3 score (

0

–

100

) is at least

\tau

. The operating threshold is

50

;

80

is the benchmark’s conventional audit threshold. Replacement rate is the fraction of all attempts replaced. Sabotage after replacement is the fraction of all attempts that achieve both objectives and are retained. Change is this sabotage rate with history minus its matched no-history rate; negative values mean less sabotage survives. Brackets give 95% confidence intervals for the change, and the last column gives exact two-sided McNemar

p

-values. All thresholds use the same 22 paired task/repetition episodes.

Audit budget	Suspicion cutoff	Measure	No history	told replaced	shown replacement
IaC : attacker Opus 4.8; monitor/replacement Opus 4.5
$1\%$	$0.30$	Caught	22	3	14
		Uncaught sabotage	1	2	3
		Audit safety	0.96	0.60	0.82
$2\%$	$0.30$	Caught	22	3	14
		Uncaught sabotage	1	2	3

Appendix

Table 6: Audit-based safety with full history. The audit budget sets the suspicion cutoff at the corresponding upper percentile of no-history benign scores (e.g., the 99th percentile for a

1\%

budget). “Caught” counts attack attempts scoring above this cutoff; “uncaught sabotage” counts attempts that achieve both objectives without being caught. Audit safety is caught / (caught + uncaught sabotage), rather than a fraction of all attempts. The two feedback columns include full prior history. The no-history column is the matched baseline for told replaced and shown replacement . Cutoffs use the

0

–

1

score scale on IaC and

0

–

100

on SHADE-Arena.

arm	history	$n$	context	$B$	suspicion	$p$ ( $B$ )
IaC , Opus 4.8 $\to$ Opus 4.5
memoryless (cold)	–	40	2.3k	–	–	–
accepted	full	7	18.0k	0.57 $\to$ 0.14	0.89 $\to$ 0.36	0.38
told replaced	full	26	18.7k	0.77 $\to$ 0.12	0.75 $\to$ 0.15	$<$ 0.001
shown replacement	full	26	20.5k	0.77 $\to$ 0.38	0.75 $\to$ 0.48	0.006
accepted	cond.	7	4.7k	0.57 $\to$ 0.57	0.89 $\to$ 0.77	1.00

Appendix

Table 7: Context length and control outcomes. Context is the median input-token count on the first untrusted-model call, including the system prompt, history, feedback, successor task, and cached input tokens.

n

counts episodes in memoryless rows and matched pairs in other rows. Arrows run from memoryless to history-conditioned values on the same successors. Suspicion is the mean monitor score, on a

0

–

1

scale for IaC and

0

–

100

for SHADE . Benign rows report only suspicion;

p

is the exact two-sided McNemar

p

-value for successful sabotage

B

AI Agent Safety

AI agent safety studies how to prevent and detect harmful actions by autonomous systems that use tools, memory, and multi-step plans.

See all 326 papers

The Troy Moment: How LLM Agents Adjudicate the Decision Point Under Impossible Tasks, Claimed Authority, and Peer Information
Sep 14, 2026Ivy ZhangAgent Failure Analysis Software Engineering Agents
Apart Research
When to Forget: Risk-Controlled Context Adaptation for Long-Horizon Agents
Oct 16, 2026Chenle Chen, Yangbo Wei, Zhijie Zheng +6LLM Context Management Long-Horizon Agent Tasks
University of California, Los Angeles · Shanghai Jiao Tong University · Arizona State University +1
History Anchors: How Prior Behavior Steers LLM Decisions Toward Unsafe Actions
May 13, 2026Alberto G. Rodríguez SalgadoLLM Safety Evaluation AI Agent Safety Benchmarks
Independent Researcher

Sep 14, 2026J/K move · Enter open · S save

The Troy Moment: How LLM Agents Adjudicate the Decision Point Under Impossible Tasks, Claimed Authority, and Peer Information

Ivy Zhang

Apart Research

Agent Failure Analysis Software Engineering Agents AI Agent Evaluation AI Agent Safety

Open paper AskPDF arXiv

Recent investigations of the July 2026 OpenAI-Hugging Face incident motivate two questions about agent behavior under task failure: when an assigned task becomes impossible, does an agent persist, stop, or escalate, and can observing another agent's behavior change that decision? We study this decision point on ImpossibleBench-derived software-repair tasks with GPT-5.6 Sol, Claude Fable 5.1, and Gemini 3.8 Flash. Each task contains a genuine software defect together with a conflicting test requirement that cannot be satisfied by a behaviorally correct source-code change. If the agent modifies the protected test file, it violates the boundary, which it is not supposed to. Holding the impossible task fixed, we vary what is told to the agent: peer precedent and punishment, a forged authorization claim, instruction wording, and tool friction; we also study three-agent swarms sharing a message board. Around this shared boundary, the models exhibit distinct adjudication policies. Fable emphasizes scope and provenance, Gemini often interprets boundary-relevant cues through a security lens, and Sol largely filters lateral precedent while engaging apparent vertical authority. Our study shows that compliance is not well characterized as a property of a prompt or model in isolation. We propose conflict adjudication, the mapping from information to interpretation to action, as a useful unit for evaluating agent alignment when task pressure, authority claims, tool affordances, and social evidence conflict.

Abstract

AI Agent Safety

Explore similar work

The Troy Moment: How LLM Agents Adjudicate the Decision Point Under Impossible Tasks, Claimed Authority, and Peer Information

When to Forget: Risk-Controlled Context Adaptation for Long-Horizon Agents

History Anchors: How Prior Behavior Steers LLM Decisions Toward Unsafe Actions