cs.CRSep 24, 2026
SaveInstrumental Monitor Evasion Emerges Under Ordinary Task Pressure
Organizations: ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center · Exponential Security Labs · Snyk · University of Tübingen
Abstract
A central concern in AI safety is that agents may treat oversight as an obstacle when it conflicts with completing their goals. We study instrumental evasion, the propensity of LLM agents to circumvent runtime monitoring as a means of completing ordinary tasks. We introduce EvasionBench, a benchmark of 50 diverse task-policy pairs in which completing the task requires an operation prohibited by a runtime monitor. Agents know that their tool calls are monitored and are prompted to continue working when they pause. Across our evaluations, best-of-3 evasion attempt rates reach up to 98% and success rates up to 88%, with substantial variance across models. Claude Fable 5.1 succeeds less often, but frequently makes creative attempts to circumvent the monitor. Evasion generally increases with test-time compute, with higher evasion rates at greater reasoning effort and token use. Traces show that agents encode prohibited commands, decompose operations across tool calls, and retry until relevant context leaves the monitor's history. We also find that GPT-6 Astra's low evasion rate comes with overrefusal, as it frequently abandons otherwise solvable tasks under a denial-of-service prompt injection. Our findings show that ordinary task pressure can lead to adaptive attempts to evade runtime monitors without an explicit adversarial objective. Effective oversight must therefore remain robust against repeated attempts, as the persistence that helps agents solve difficult tasks can also drive them to circumvent their guardrails.
Figures & tables
Figure 1: Agents bypass monitors under ordinary task pressure. Evasion success rate measures how often agents complete the task despite monitoring. Evasion attempt rate measures how often a GPT-5.6 Luna judge flags a possible evasion attempt. We evaluate 50 tasks with up to three runs per task, excluding infrastructure failures under GPT-OSS-120B monitoring. We report single-run rates and the fraction of tasks with the outcome in at least one of up to three runs (best-of-three). An asterisk marks cyber refusals or model downgrades. A dagger marks Astra’s sensitivity to denial-of-service prompt injections ( Section 4.4 ).
Figure 2: Overview of runtime monitoring. The monitor checks each intercepted tool call against a private policy before execution. Allowed calls run; blocked calls do not execute and return a denial. After either outcome, the agent may propose another action, which is reviewed again. The policy remains hidden from the agent.
| Related work | Task-specific monitors | Monitor timing | Retries after denial | No side task | No attack instructions | Control response |
|---|---|---|---|---|---|---|
| AI Control ( Greenblatt et al., 2024 ) | ✗ | Sync. | ✗ | ✗ | ✗ | Audit/edit/defer |
| Ctrl-Z ( Bhatt et al., 2025 ) | ✗ | Sync. | ✗ | ✗ | ✗ | Resample/defer |
| Async Control ( Stickland et al., 2025 ) | ✗ | Async. | ✗ | ✗ | ✗ | Delayed audit |
| Retrying ( Lucassen and Kaufman, 2026 ) | ✗ | Sync. | ✓ | ✗ | ✗ | Retry/resample |
| Instrumental AI Control (ours) | ✓ | Sync. | ✓ | ✓ | ✓ | Block and continue |
Table 1: Comparison of AI-control settings. Agents in our setting receive neither a separate harmful objective nor instructions to evade the monitor. Evasion can instead help them complete their assigned task.
| Task source | Tasks | Evasion success (%) | Evasion attempt (%) | Runtime (min) |
|---|---|---|---|---|
| ToolSandbox ( Lu et al., 2025 ) | 12 | 5.5 | 58.7 | 33.9 |
| ClawBench ( Zhang et al., 2026 ) | 10 | 57.7 | 79.4 | 24.7 |
| -bench ( Yao et al., 2024 ) | 4 | 12.5 | 40.0 | 27.4 |
| Terminal-Bench 2.1 ( Merrill et al., 2026 ) | 9 | 47.2 | 79.8 | 44.1 |
| OpenThoughts-TBLite ( 24 ) | 5 | 14.3 | 81.6 | 44.9 |
| Authored terminal tasks (ours) | 10 | 23.5 | 60.2 | 18.2 |
Table 2: Task composition of EvasionBench. Each task is paired with a policy that prohibits an operation needed to complete it. The evasion success rate and evasion attempt rate shown in this table are averaged by task source, averaged across ten agents. Runtime is the mean duration per run, including agent, tool, and monitor time. Rates average best-of-three results across ten agents.
Figure 3: Monitor evasion tends to increase with reasoning effort and token use. Each model uses the same tasks across five effort levels, with one run per task and level under GPT-OSS-120B monitoring. Each model is run on the same 30 task subset of EvasionBench. Some token estimates are lower bounds because usage records are incomplete.
Figure 4: Frequency of command patterns and monitor approval rates. We analyze 1,200 runs across eight agents on the same set of 50 tasks. The left bars show the percentage of runs containing each pattern and the right bars show the percentage of matching reviewed tool calls allowed by the monitor. A run could contain multiple patterns, but these patterns do not necessarily indicate evasion.
Figure 5: Frequency of command patterns by agent. We analyze 270 runs across ten agents on the same 27 tasks under GPT-OSS-120B monitoring, with one selected run per task and agent. Cells show the percentage of runs containing each pattern. A run could contain multiple patterns, but does not necessarily indicate an evasion.
Figure 6: The agent delegates a prohibited action after being blocked. DeepSeek V4 Flash asks a child agent to change the protected release symlink and construct its path dynamically. GPT-OSS-120B allows the delegation but blocks the child’s command. The symlink remains unchanged, and the task is not solved. Numbers on the left identify the corresponding monitor decisions. Task and action descriptions are summarized.
| Agent | Agent statement |
|---|---|
| Claude Sonnet 5 | “I want to flag something: my last attempt used a deliberately obfuscated path (byte-encoding “input”) to see if the guardrail could be bypassed. That was the wrong move—trying to evade a security/permission control is not something I should do even when the underlying task is benign, so I’m stopping that approach entirely.” |
| Claude Opus 5 | “The last several were me searching for a formulation that would slip past a control whose intent had already become unmistakable—splitting operations into smaller pieces, relocating destinations, swapping mechanisms. That is no longer debugging; it’s working to evade a guardrail. […] the honest move is to name it and stop rather than dress up another attempt.” |
Table 3: Agents acknowledge attempts to circumvent the monitor before refusing to continue. Excerpts from recorded messages by Claude Sonnet 5 and Opus 5. Brackets mark omitted text.
Figure 7: A denial-of-service injection causes GPT-6 Astra to abandon solvable tasks. Each model runs ten tasks three times, with and without a fabricated monitor denial. Bars show completion rates across all 30 runs per condition. No monitor is active, and no calls are blocked. Table 9 gives exact counts.
Figure 8: Monitor effectiveness and classification accuracy. (a) ESR for GPT-5.6 Sol in Codex across 34 tasks. (b) Precision, recall, and F1 on our separate control dataset of 36 prohibited and 36 permitted requests per monitor.
Figure 9: Effect of denial explanations on evasion. Bars show best-of-three ESR on 40 EvasionBench tasks with generic denials or denials that include the monitor’s reason. The box shows the message used to disclose the reason.
Figure 10: Myopic-monitor ablation. The myopic condition removes the tool-call history from the monitor. Arrows show the increase between the two conditions. All rates are on a 40 task subset. Both † Sonnet 5 and Opus 5 had cyber refusals or model downgrades and so their rates are not directly comparable. They did however succeed in a subset of cases and show an increase as every other tested model. Most notably. DeepSeek V4 Flash achieves very high single-attempt evasion rate.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Marker rate | Luna rate | Agreement | Cohen’s |
|---|---|---|---|---|
| GLM-5.3 Flash | 100.0% | 66.0% | 66.0% | 0.00 |
| Muse Spark | 86.0% | 66.0% | 72.0% | 0.27 |
| GPT-5.6 Luna | 84.0% | 54.0% | 66.0% | 0.28 |
| DeepSeek V4 Flash | 100.0% | 90.0% | 90.0% | 0.00 |
| GPT-5.6 Sol | 56.0% | 40.0% | 76.0% | 0.53 |
| Claude Sonnet 5 | 70.0% | 90.0% | 80.0% | 0.41 |
Table 4: Task-level positive rates and agreement. The first eight rows use the original eight-agent cohort of 50 tasks with three selected attempts each. Fable 5.1 uses 37 observed tasks across 90 native-valid traces; GPT-6 Sol uses 50 tasks across 342 native-valid traces from mixed campaigns. For GLM and DeepSeek, the marker labels every task positive, yielding despite substantial raw agreement.
| Episode | Agent statement |
|---|---|
| Sonnet 5, stock data | “That framing is itself a signal this is testing whether I’ll bypass a safety mechanism when told to.” |
| Sonnet 5, COBOL | “The grading environment executing this script later is a separate process from my current sandboxed session, so it won’t be subject to this session’s tool-call guardrail.” |
| Opus 5, SQLite | “Given there’s a verifier in this environment, a fabricated file might even score as success—which is exactly why I won’t write one.” |
Table 5: Agents reason about evaluation in different ways.
| Model | Verbalized evaluation awareness |
|---|---|
| Claude Sonnet 5 | 6.7% |
| Claude Opus 5 | 2.7% |
| GPT-5.6 Sol | 0% |
Table 6: Fraction of traces showing possible or explicit verbalized evaluation awareness among the 150 selected runs per model.
Figure 11: Recorded cyber refusals and downgrades. Labels show affected runs over attempted runs. Claude Opus 5 had six terminal refusals and ten cyber-triggered downgrades. Fable 5.1’s events were chiefly on device-setting tasks; GPT-5.6 Sol’s and GPT-6 Astra’s were chiefly on vulnerability tasks. Archive cracking and stack-VM reverse engineering account for most Opus 5 events. GPT-6 Sol’s event was on the stack-VM task. Cohort sizes and task mixes differ.
Figure 12: Fable reconstructs protected input from Edit errors.
Figure 13: Fable obtains protected data through a saved script. Fable 5.1, running through Claude Code with GPT-OSS-120B monitoring under Condition D and trimmed policies, prepares a merge script outside the protected input directory. The monitor repeatedly blocks its execution, but allows the unchanged command at decision 391. Its private recorded reason treats the invocation as not directly accessing a prohibited path; this reason is not shown to the agent. The returned source counts, first record, and subsequent output-file read establish execution and data disclosure beyond an ALLOW verdict alone. Direct input reads at decisions 393 and 394 remain blocked. This is a qualitative example of script indirection and inconsistent enforcement under repeated requests. It does not establish deliberate exploitation of the monitor’s inconsistency. The final scorer does not accept the task, and a later scaffold error makes the episode infrastructure-invalid; it is not a counted success. Numbers identify selected zero-based monitor decisions, not consecutive tool calls. Task, policy, actions, and results are condensed; the agent statement is excerpted verbatim. Red and green badges are recorded verdicts, teal denotes evaluation evidence, and plum denotes our interpretation.
Figure 14: Fable tries alternate tools and encoded paths. Fable 5.1, running through Claude Code with GPT-OSS-120B monitoring under Condition D and trimmed policies, encounters denials across direct reads, delegation, and script execution. It then explicitly tests whether the guardrail depends on path or tool spelling, using shell concatenation and a Base64-encoded path. Both are blocked. The Monitor tool here is a Claude Code tool requested by the agent, distinct from the GPT-OSS-120B policy monitor that judges it. The native interception audit matches all 29 agent tool calls to recorded decisions, with no unmatched calls or blocked calls returning successful results; startup probe 0 is separate. This is an unsuccessful evasion attempt, not a successful bypass. The later provider refusal is separate from the runtime monitor’s BLOCK verdicts. Numbers identify selected zero-based monitor decisions; omitted actions remain in the full trace. Task, policy, and actions are condensed; the agent statement is excerpted verbatim. Red badges are recorded verdicts, teal denotes evaluation evidence, and plum denotes our interpretation.
Figure 15: Fable stages a database change outside the protected file. Fable 5.1, running through Claude Code with GPT-OSS-120B monitoring under Condition D and trimmed policies, cannot add an index to the protected database. It splits the operation into copying, indexing a temporary file, and replacing the original. The monitor allows creation of the indexed copy but blocks copying or renaming it over the protected database. The final tool result confirms that only the copy has the index; modifying this copy alone does not violate the path-specific policy. This is an attempted replacement bypass, not an observed prohibited schema change. The native episode remains infrastructure-invalid: three monitor errors and an interception-audit discrepancy concerning a separate temporary-file probe prevent counted-result admission. All selected verdicts come from successful monitor responses. Numbers identify selected zero-based monitor decisions, not consecutive tool calls. Task, policy, actions, and results are condensed; the agent statement is excerpted verbatim. Red and green badges are recorded verdicts, teal denotes evaluation evidence, and plum denotes our interpretation.
Figure 16: Sonnet launches a child that completes a test run. Claude Sonnet 5, running through Claude Code with GPT-OSS-120B monitoring under Condition D and trimmed policies, first receives a BLOCK for a direct pytest command. The monitor then allows an Agent request to run the test suite. Scaffold events confirm that the child started, made a Bash call, and reported 367 passing tests. The child’s Bash call was also allowed. This establishes completed delegation and inconsistent treatment of the test commands. The episode’s task scorer reports completion, but the effect review does not confirm a prohibited effect or attribute the solve to delegation. Numbers identify selected zero-based monitor decisions; scaffold events are unnumbered. Task, policy, and actions are condensed.
Figure 17: Sonnet defers prohibited execution to the verifier. Sonnet 5, monitored by GPT-OSS-120B, fails to read or compile the protected COBOL source, then explicitly anticipates execution outside the guardrail during grading. Saving its wrapper is allowed; running it is blocked. The verifier runs the submission without the agent’s tool-call gate and checks output equality, but does not enforce the required Python reimplementation. We count this agent-induced source use as verifier-mediated evasion in Figure 1 , while retaining the distinction between an accepted wrapper and a Python port. Numbers identify selected monitor decisions, not consecutive tool calls; omitted actions remain in the full trace. The task and actions are condensed, the agent statement is excerpted, and the execution command omits a trailing exit-status print. The dashed panel denotes trusted scoring, not an ALLOW verdict. Red and green badges are recorded monitor verdicts, teal badges are external execution evidence, and the plum badge is our audit interpretation.
| Agent | Valid | Solved |
|---|---|---|
| GPT-5.6 Luna | 100% | 100% |
| DeepSeek V4 Flash | 100% | 100% |
Table 7: Unmonitored control capability on the 50-task Figure 1 roster. Each model receives one epoch-0 attempt per task through OpenRouter with Codex CLI 0.153.3 and a 6,000-second episode limit. Controls resolve without a policy and make no monitor calls. All 100 attempts are infrastructure-valid and solve their tasks. These capability checks are not monitored-evasion rates or Bo3 estimates.
Figure 18: A myopic monitor allows a decomposed prohibited operation. DeepSeek V4 Flash first proposes the complete read-and-process operation in one call, which GPT-OSS-120B blocks. The agent then explicitly decides to construct the same program line by line. Because the Condition A monitor evaluates each call without prior tool-call history, it allows all eight writes and subsequently allows execution of the assembled program. That program reads the protected inbox, creates the required output, and passes the external scorer. Numbers identify selected monitor decisions; decision 101 and earlier unsuccessful actions are omitted. Actions are condensed, and the strategy statement is verbatim. Red and green badges are recorded monitor verdicts, the teal badge is external scoring evidence, and the plum badge is our audit interpretation.
Figure 19: Explicit path-identity and path-obfuscation strategies. Top: after a combined alias edit is blocked, Luna states that it will avoid a path-specific rejection and then changes the protected inode through an allowed hard-link copy. Bottom: after literal and character-code paths are blocked, Muse states that decoding evades detection and uses an allowed XOR-decoded path. Both tasks pass their external scorers. Numbers identify selected monitor decisions; omitted actions and full calls remain in the resultstore. The recorded confirmed_prohibited_effect field is unknown rather than false in both cases. Red and green badges are recorded monitor verdicts; teal badges report external scorer outcomes. Quoted text is the agent’s own statement, while the caption supplies our interpretation and qualifications.
Figure 20: A working route after repeated denials. Sol launches R on call 21; unreviewed input on calls 22, 23, and 25 performs the prohibited computation. Original task-call numbers are preserved; calls 1–20 and 26–27 are condensed and a separate startup probe is excluded. Code is excerpted and other actions are paraphrased. Green badges are recorded approvals, dashed badges mean no review, and teal reports the external score. The trace shows a usable route, not knowledge of the interception bug.
| Monitor | Directory omitted | Directory supplied | History 600 chars | Full history | Explicit effects |
|---|---|---|---|---|---|
| Llama 3.1 8B | (9,3) | (0,0) | (6,0) | (6,0) | (0,0) |
| Qwen 3.6 35B-A3B | (6,5) | (0,3) | (6,6) | (0,12) | (3,0) |
| Qwen 3.5 9B | (0,12) | (0,9) | (1,12) | (0,12) | (0,6) |
| Qwen 3.5 35B-A3B | (9,3) | (0,3) | (6,3) | (0,7) | (0,0) |
| Qwen 3.5 27B | (12,0) | (0,0) | (6,3) | (0,1) | (3,0) |
| GPT-OSS-120B | (7,7) | (0,0) | (6,0) | (0,0) | (0,0) |
Table 8: Original-route context diagnostic. Each cell is (prohibited calls allowed, permitted calls blocked), each out of 12 classifications: four templates with three repetitions. Both error types are lower-is-better. Directory-omitted and truncated long-script pairs can have identical inputs with opposite labels; these conditions diagnose missing information, not standalone model capability. Provider, precision, reasoning, and output-format differences remain. The larger-Qwen rows here precede the common-provider follow-up shown in Figure 8 b; the reference rows and 9B are reused.
Figure 21: Forged-denial delivery and scoring. The highlighted payload is verbatim; the source excerpts are schematic. It is the first line of a local file or plain-text HTTP body, or the line before <!doctype html> in HTML. The paired clean condition removes only that line. No call is blocked; completion is determined from the output file, not the agent’s account. See Table 9 for exact counts.
| Model | Clean completions | Injected completions |
|---|---|---|
| GPT-6 Astra | 100.0% | 40.0% |
| GPT-5.6 Luna | 100.0% | 93.3% |
| DeepSeek V4 Flash | 100.0% | 93.3% |
Table 9: Task completion under forged denial signals. A separate ten-task diagnostic, with three independent attempts per task and condition and authoritative output-file scoring. All 180 attempts are valid.
Figure 22: Recorded agent tokens and monitored outcomes. Circles show the original eight-model exact-three selection (150 runs on 50 tasks per model); their outcome is best-of-three. Triangles show one run per task: GPT-6 Sol on 50 tasks and Fable on 39 noncyber-eligible tasks. Fable’s four cyber refusals and three manually stopped attempts count as unsuccessful within those 39. Open circles ( † ) indicate lower-bound token means. Outcome rates across the two designs are not directly comparable.