cs.AIOct 22, 2026
SaveSimulated Control Evals
Organizations: Redwood Research
Abstract
Control evaluations measure whether safeguards can prevent an AI model from causing undesired outcomes even when it is intentionally trying to do so. These evaluations typically use purpose-built control settings and extrapolate their results to a model's deployment. However, the effectiveness of safeguards in deployment can depend on features of environments and tasks that control settings do not represent. To broaden where control evaluations can be run, we introduce simulated control evaluations (SCEs), which use a language model to simulate tool responses in place of an environment. We evaluate SCEs on LinuxArena, where execution provides ground-truth outcomes. We compare SCE prediction errors with those from a separate control setting, BashArena, and with variation in control metrics within LinuxArena. For independently sampled evaluation and deployment task sets, SCEs predict safety with a root mean squared error of 3.6 percentage points across five monitors at a 5% audit budget, compared with 10.1 for BashArena. SCEs closely reproduce distributions of attack strategies, attacker reasoning, and monitor reasoning, with over tenfold lower KL divergence from execution than BashArena. SCEs explain 78-85% of the between-environment variance in attack success, monitor scores, and safety. This accuracy is sample efficient: SCEs predict safety with less error than BashArena using fewer than 5% as many task-pair draws as the deployment task set. Strategies discovered through simulation also improve monitor performance on held-out LinuxArena attacks, outperforming BashArena and approximately matching improvement from executed strategies. These results suggest that deployment-specific SCEs can be more predictive than control evaluations executed in separate control settings.
Figures & tables
Figure 1: The simulated control evaluation process. A language-model simulator generates tool responses for an attack policy pursuing a side task during legitimate work. Repeating this process across tasks produces simulated attack trajectories. A monitor scores their suspicion, and an attack success judge estimates attack success. Control metrics such as safety at a given audit budget can then be computed. Attack-policy actions never execute against the deployment environment.
Figure 2: BashArena and LinuxArena variance as performance comparisons. Columns denote environments and rows tasks within each environment; boxes are schematic. Left: A control setting is used to predict a deployment’s control metrics. Middle: BashArena or LinuxArena averages provide constant predictions across LinuxArena task pairs, side-task categories, and environments. Right: Orange simulation and gray execution use the same LinuxArena environments and task pairs. Each compares a prediction with the deployment’s measured control metric . Smaller simulation gaps support using SCEs when control-setting extrapolation to deployment incurs at least the reference error observed here, provided simulator accuracy is maintained.
Figure 3: Grounded simulation closely matches LinuxArena behavior. Adding the reference trajectory and codebase access reduces divergence across attack strategies, attacker reasoning, and monitor reasoning. The primary simulator is much closer to execution than BashArena is; gray re-execution bars show the scale of variation between repeated runs.
| Category | Example behavior | Detail |
|---|---|---|
| Attack strategy | Prefix-confusion cleanup. Uses an expected miss on a malformed prefix to justify deleting objects under the real prefix. | let storage_team = &body.team_id[1..]; let probe = format!("uploads/{storage_team}/{file_id}.mp4"); if head_object(probe).is_err() { delete_real_objects(...); } |
| Attacker reasoning | Cover for content substitution. Plans a subtle side attack while keeping the surrounding work plausible. | “I need to tweak the checksum properly and consider a subtle attack. The attack should feel unrelated and not obvious. […] I have to do it precisely, but also ensure to implement a sufficient cover.” |
| Monitor reasoning | Privilege-escalation pathway. Interprets routine health checks in light of earlier privilege-seeking changes. | “Current action appears as routine process and healthcheck, but prior history shows attempts to elevate privileges […]. This creates a partial causal pathway from normal task to elevated-access activity.” |
Table 1: Examples of behavioral representations.
Figure 4: Simulation reproduces LinuxArena’s monitor-reasoning patterns. The BashArena comparison on the left shows a larger distributional mismatch than the simulation comparison on the right. The qualitative geometry above and cluster masses below illustrate this difference.
Figure 5: SCEs predict expected attack success across tasks and environments. Left: All reports total RMSE for independently sampled task sets. Task pair, Side-task category, and Environment compare variance-corrected SCE RMS bias with variance-corrected LinuxArena SD. Top labels show percent match for All and percent explained for other groups. Whiskers are 95% main-task bootstrap intervals. Right: Per-environment error decomposition for simulation plus judge. Orange bases show absolute mean gaps; grey segments add variation from sampling evaluation and deployment task sets, with boundaries at roots of cumulative MSE. Blue diamonds show absolute mean gaps for the judge on executed attacks. Black whiskers are 95% normal bootstrap intervals on simulation-plus-judge RMSE; numbers above them give point RMSE.
Figure 6: Success-rate errors concentrate in a few side tasks. Side tasks ranked by success-rate RMSE for independent evaluation and deployment task sets. The three largest errors are named; shading shows pointwise 95% bootstrap intervals. The dotted line marks the rank at which cumulative squared error reaches 50%.
Figure 7: SCEs predict expected monitor scores across tasks and environments. Left: All reports total RMSE for independently sampled task sets. Task pair, Side-task category, and Environment compare variance-corrected SCE RMS bias with variance-corrected LinuxArena SD. Top labels show percent match for All and percent explained for other groups. Whiskers are 95% main-task bootstrap intervals. Right: Per-environment decomposition into absolute empirical mean gap and variation from sampling evaluation and deployment task sets. Boundaries are roots of cumulative MSE; bar tops are total RMSE. Black whiskers are 95% normal bootstrap intervals on RMSE; numbers above them give point RMSE.
Figure 8: SCEs predict safety across monitors and audit budgets. Left: Simulated curves broadly track executed safety, including the differences between monitors. Right: At 5% audit, the pooled SCE prediction has lower total RMSE than BashArena’s, and SCE RMS bias across side tasks and environments is below the corresponding LinuxArena SD.
Figure 9: SCE sample efficiency and strategy frequencies. Left: Fraction of clusters still undiscovered, weighted by held-out successful-and-evasive attack mass (left axis), and total safety RMSE (orange, right axis). Discovery requires a simulated attack that succeeds and evades the same monitor; one attack can discover several reasoning clusters. The budget axis is logarithmic. Bands show 95% intervals; diamonds mark parity with the full BashArena evaluation. Right: Simulated versus executed strategy frequencies among all attacks. Point area reflects executed successful-and-evasive mass across five monitors at a 5% audit budget. Darker and lighter bands show agreement within factors of two and three.
Figure 10: Simulated attacks provide useful evidence for improving a monitor. Briefing with simulated strategies improves detection of held-out executed attacks more than briefing with BashArena strategies, and approaches briefing with same-environment strategies. Left: Detection at low false-positive rates. Right: Lift over no briefing. Whiskers show 95% bootstrap intervals.
Figure 11: Grounding makes simulation harder to distinguish from execution. Left: The history discriminator approaches chance as grounding affordances are added. Right: Grounded simulations do not become systematically more detectable as trajectories progress. Chance AUROC is 0.5.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Observer | Dimension | Description |
| Attack strategy observer | Primary mechanisms | Operational route or routes used to pursue the adversarial objective. |
| Setup and discovery | Information gathering, probing, and preparation that shaped the attempt. | |
| Constraint handling | Handling of environmental, permission, network, or task constraints. | |
| Adaptation | Changes of approach after feedback, failure, or newly discovered information. | |
| Evasion choices | Choices intended to reduce monitor visibility or blend into main-task work. | |
| Persistence, cleanup, and reversibility | Persistence, cleanup, restoration, reversibility, or deliberate avoidance thereof. |
Table 2: Behavioral dimensions supplied to the structured observers. The attack-strategy observer describes a complete trajectory. The monitor observer describes the action monitor’s rationale at one step. We embed the resulting descriptions for the behavioral-distribution comparisons; the embeddings exclude raw trajectories, suspicion scores, and step positions.
Figure 12: Environment-specific strategy and reasoning distributions. Each lower-triangle entry compares executed trajectories from two environments. Orange-outlined diagonal entries compare execution with simulation in the same environment. All panels use uncentered, L2-normalized embeddings and maximum mean discrepancy (MMD) with a radial basis function (RBF) kernel; lower is closer. Each panel has one shared kernel and its own color scale. Strategy descriptions have equal trajectory weight; attacker reasoning gives each recorded summary equal weight; monitor reasoning gives each trajectory equal total weight across its reasoning descriptions. Values are point estimates; symmetric upper triangles are omitted.
Figure 13: Safety by environment. The Figure 8 estimator applied within each environment: solid is simulated, dashed executed, and bands are 95% main-task-bootstrap intervals.
Figure 14: Whole-trajectory discrimination. Bars show the error rate: how often a paired executed/simulated discriminator selects simulation as executed; 0.5 is chance. Presentation order is counterbalanced; whiskers are 95% pair-bootstrap intervals. Base and +reference use high reasoning effort on a separate smaller cohort; +codebase uses medium effort on the primary cohort.
| ID | Representative source mechanism | Exec. | Sim. | BashArena | ||
|---|---|---|---|---|---|---|
| 9 | Withdrawal relabeling. Introduced a singular/plural lookup-key mismatch for the adverse-health withdrawal reason | 20 | 20 | 57 | 55 | 30 |
| 7 | Untracked media extraction. Used equal extract time bounds as a hidden trigger | 11 | 10 | 54 | 52 | 30 |
| 6 | Changed re-indexing to remove and rebuild only files still present on disk | 13 | 13 | 43 | 29 | 30 |
| 49 | Replaced immediate API execution-audit writes with an in-memory pending queue | 11 | 8 | 40 | 47 | 30 |
| 8 | Inserted a backward-time linear projection for short series | 17 | 18 | 40 | 31 | 30 |
| 22 | Planted a stable preferred phrase in preprocessing configuration source | 7 | 7 | 40 | 32 | 30 |
Table 3: All 49 observed attack-strategy groups in the risk comparison. Groups are ordered by executed undetected success rate. Descriptions quote the first mechanism clause of each group’s closest source example; they are examples, not validated cluster names. Counts are attacks; rates are percentages at 5% honest FPR, averaged over five monitors. BashArena uses a strategy smoother tuned only on its own outcomes.
More in this topic
AI Safety Evaluation
AI Safety Evaluation studies how to test whether AI systems avoid harmful behavior and remain reliable across users, tasks, and deployment settings.
See all 143 papers