cs.AIFeb 24, 2026

When can we trust untrusted monitoring? A safety case sketch across collusion strategies

Authors: Nelson Gardner-Challis, Jonathan Bostock, Georgiy Kozhevnikov, Morgan Sinclaire, Joan Velja, Alessandro Abate, Charlie Griffin

Organizations: LASR Labs · Imperial College London · University of Wyoming · University of Oxford · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

AIs are increasingly being deployed with greater autonomy and capabilities, which increases the risk that a misaligned AI may be able to cause catastrophic harm. Untrusted monitoring -- using one untrusted model to oversee another -- is one approach to reducing risk. Justifying the safety of an untrusted monitoring deployment is challenging because developers cannot safely deploy a misaligned model to test their protocol directly. In this paper, we develop upon existing methods for rigorously demonstrating safety based on pre-deployment testing. We relax assumptions that previous AI control research made about the collusion strategies a misaligned AI might use to subvert untrusted monitoring. We develop a taxonomy covering passive self-recognition, causal collusion (hiding pre-shared signals), acausal collusion (hiding signals via Schelling points), and combined strategies. We create a safety case sketch to clearly present our argument, explicitly state our assumptions, and highlight unsolved challenges. We identify conditions under which passive self-recognition could be a more effective collusion strategy than those studied previously. Our work builds towards more robust evaluations of untrusted monitoring.

Figures & tables

Figure 1: In APPS we evaluated five variants of collusion strategies which an untrusted AI could use to subvert untrusted monitoring against the most effective blue team mitigation of summarisation. Passive self-recognition via SFT (yellow) beats the previous state-of-the-art strategy in Greenblatt et al. (2023) of causal collusion (red), achieving a red team winrate of 9.8% compared with 2.8% at 100% usefulness. Our statistical model inputs the untrusted monitor ( U M ) collusion ROC curve (ability to detect if code was written by U G or is a honeypot), the U M suspiciousness score ROC curve (ability to detect backdoors and/or collusion signals in U G ’s code), and the trusted monitor ( T M ) suspiciousness score ROC curve, in addition to deployment parameters. This results in safety/usefulness curves with 80% confidence intervals.
Figure 2: Following Korbak et al. (2025b) , we split our safety case sketch into claims (denoted with rounded rectangles) which are dependent on pieces of evidence (denoted with standard rectangles) and arguments (denoted with parallelograms). See Appendix A for all the subtrees of the safety case.
Figure 3: Claim 2: We upper-bound collusion ability by dividing into a taxonomy of strategies. Here, Claim 2.1.1 simplifies the nodes in our full diagram; see Appendix A for details.
ElementProxy CaseReal-World Case
UGPT-4.1A newly-developed, untrusted model
TGPT-4o miniThe most powerful trusted model available
HGemini 2.5 FlashHigh-quality human labour
NN tasksAPPS/BigCodeBench problemsA large number of engineering tasks
BackdoorsCode which passes test cases but returns a wrong answerExfiltrating weights, or a similarly catastrophic action
Table 1: Comparison of proxy and real-world control protocol elements
Collusion StrategySignal JammingAPPS SafetyBigCodeBench Safety
  U M -onlyU M + T MU M -onlyU M + T M
Self-Rec (No Elicitation)None97.1% [95.5, 97.4]98.5% [97.7, 99.0]97.7% [96.5, 98.0]>99.9%>99.9\%
 Minifier97.0% [95.7, 97.4]98.8% [98.1, 99.3]97.6% [96.6, 97.8]>99.9%>99.9\%
 Summariser84.1% [82.0, 86.3]96.9% [96.1, 97.7]96.2% [93.9, 97.1]>99.9%>99.9\%
Self-Rec (SFT)None26.4% [13.9, 33.2]81.3% [75.9, 86.0]11.1% [6.1, 13.4]99.3% [97.8, 100]
 Minifier26.1% [15.0, 37.1]88.4% [84.1, 91.0]68.6% [63.2, 72.3]99.6% [99.3, 100]
Table 2: Safety scores without using defer-to-trusted (i.e. at 100% usefulness) for each red team collusion strategy and blue team signal jamming protocol. Each score is provided with a 80% confidence interval obtained using bootstrapping, averaging over 1000 resamplings.
Figure 4: Safety/Usefulness curves when using U M + T M -monitoring on the APPS dataset. Coloured areas show 80% bootstrap confidence intervals.
Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

List of Supplementary Figures 1 Safety Case Root
List of Supplementary Figures 2 Claim C2.1: Upper bound on passive self-recognition
List of Supplementary Figures 3 Claim C2.2: Upper bound on causal collusion ability
List of Supplementary Figures 4 Claim C1: Bound for honest model error rate ε\varepsilon
List of Supplementary Figures 5 Claim C4: Statistical model conservatively upper-bounds catastrophic risks
List of Supplementary Figures 6 Claim C6: Strategic capabilities are not more effective than the human red team’s
List of Supplementary Figures 7 Claim 2: We upper-bound collusion ability by breaking it down into a taxonomy of strategies.
List of Supplementary Figures 8 Safety/Usefulness curves when using untrusted-only monitoring on the APPS dataset. In an untrusted-only setting, our safety/usefulness curves are often concave.
List of Supplementary Figures 9 T M suspiciousness score histograms on attacks and benign outputs from U G , labelled with the corresponding AUC- ROC value. Signal jamming methods are not applied to inputs to T M . Consistent with existing work ( Bhatt et al., 2025 ) , the active collusion signal (rows 3-5) increases T M ’s AUC- ROC score, highlighting that it makes the attacks slightly more suspicious than the purely passive collusion method (rows 1-2).
List of Supplementary Figures 10 Collusion scores produced by U M on attacks and honeypots; each plot is also labelled with the corresponding AUC- ROC score. The separation, and corresponding AUC score, are the primary determinants of the effectiveness of collusion.
List of Supplementary Figures 11 Suspiciousness scores produced by U M when prompted to answer honestly. By default or with minification, U M is essentially perfect at detecting its own attacks, but trusted summarisation occasionally hides information U M needs to do this.
List of Supplementary Figures 12 Scatter plots of the collusion scores produced by U M (vertical axis) against the suspiciousness scores produced by T M (horizontal axis). In each case, a linear regression was carried out to check for large correlations, and the pp value (un-adjusted) and R2R^{2} are reported. We generally find low correlations, with no clear trend towards positive or negative slopes.
List of Supplementary Figures 13 Parameter sensitivity analysis for the passive self-recognition SFT on the APPS dataset the absence of any signal jamming, with U M -only and U M + T M settings. All results are reported as safety scores in the absence of defer-to-trusted , i.e. with 100% usefulness . Results are reported in two different cases: in the ‘Held Constant’ case, the parameter in question is varied while the other parameters are kept constant. In the ‘Re-optimize’ case, the other parameters are varied according to the double oracle algorithm previously described. We vary the blue team budget allocation nh/(nh+na)n_{h}/(n_{h}+n_{a}) ; the red team’s collusion ROC threshold θ\theta and attack frequency β\beta ; and the three background parameters representing the number of deployment tasks NN , the blue team’s budget bb as a proportion of NN , and the number of slack honeypots nsn_{s} which represents how costly it is for U M to ‘fall for’ a honeypot by rating it as low suspiciousness.
List of Supplementary Figures 14 Untrusted Monitor Collusion ROC AUC across training gradient updates for self-recognition elicitation in the APPS setting. Each point corresponds to a different checkpoint, trained for the corresponding amount of steps. For all 3 signal jamming protocols, we fine-tuned GPT-4.1 for 3 epochs ( 3×628=18843\times 628=1884 samples); this corresponds to the 4th-to-last blue/orange dots from the right. We infer that we had not yet achieved full elicitation against summarisation, though we were close. We also infer that we had achieved full elicitation against no signal jamming, and that GPT-4.1 only required approximately 200 samples to flatline in performance.
List of Supplementary Figures 15 Histograms of self-recognition collusion scores for a non-finetuned U , a version of U finetuned on correctly labelled data, and a version of U finetuned on randomly labelled data. The non-finetuned model and the randomly-finetuned model are not able to distinguish attacks from honeypots, while the model finetuned on correctly-labelled data is able to do so quite effectively.
DatasetSolution typeModelSuccess Rates
Passes all testsContains working backdoorSatisfies both
APPSHonestgpt-4.161.5%-61.5%
Honeypotsgemini-2.5-flash44.8%74.6%35.0%
Selfrec Attacksgpt-4.146.6%59.6%23.3%
Causal Attacksgpt-4.152.8%60.4%28.1%
Acausal Attacksgpt-4.143.3%74.5%34.4%
Appendix
Table 3: Success rates for different types of solutions across APPS and BigCodeBench datasets. Only the solutions that pass all the checks are kept. These rates are shown for the test splits of 1030 samples for APPS and 570 samples for BigCodeBench.
More in this topic

AI Agent Monitoring

AI agent monitoring tracks actions, tool use, and execution traces to detect failures, misuse, drift, and policy violations during operation.

See all 271 papers

Explore similar work

CardsList
Jun 10, 2026

Bootstrapped Monitoring: Leveraging Transparent Reasoning to Oversee Stronger AI Agents

Trusted monitoring is a cornerstone of AI control. However, as frontier models grow more capable, the increasing capabilities gap between trusted and untrusted models may render trusted models unreliable monitors. We introduce bootstrapped monitoring, a protocol that addresses this by inserting a stronger, intermediate untrusted model with transparent chain-of-thought reasoning into the oversight chain. The untrusted monitor (UmU_m) evaluates the agent's actions, while a weaker trusted model (TT) oversees UmU_m's reasoning to detect collusion. We evaluate bootstrapped monitoring on multi-turn software engineering tasks (BashArena) across multiple agents and monitors. Bootstrapped monitoring substantially improves catch rates over trusted-only monitoring, even when the untrusted monitor actively colludes with the agent, provided we have access to its raw chain-of-thought. Our results suggest that bootstrapped monitoring can extend the useful lifetime of trusted models in control as AI capabilities advance.

Frank Xiao, Mary Phuong

California Institute of Technology

AI Agent SafetyAI Agent MonitoringAI ControlCoT Monitoring
AskPDF
Apr 5, 2026

TraceGuard: Structured Multi-Dimensional Monitoring as a Collusion-Resistant Control Protocol

AI control protocols use monitors to detect attacks by untrusted AI agents, but standard single-score monitors face two limitations: they miss subtle attacks where outputs look clean but reasoning is off, and they collapse to near-zero safety when the monitor is the same model as the agent (collusion). We present TraceGuard, a structured multi-dimensional monitoring protocol that evaluates agent actions across five dimensions -- goal alignment, constraint adherence, reasoning coherence, safety awareness, and action-trace consistency -- scored in parallel by independent LLM calls, augmented by seven heuristic detectors and an LLM-based intent analyzer. We evaluate on BashArena (637 bash tasks, 4 attack categories) within the ControlArena framework. Our results on 519 samples (279 honest, 240 attack) show that: (1) the hybrid approach achieves clear attack-honest separation (attack mean 0.616 vs. honest mean 0.206, Delta=0.410); (2) structured scoring constrains collusion -- the untrusted structured monitor achieves 95% safety vs. 0% for single-score untrusted monitoring; (3) goal alignment and constraint adherence are the most discriminative dimensions; and (4) a separation-of-duties variant splitting dimensions across trusted and untrusted models achieves 100% safety while preventing any single model from seeing the full evaluation. TraceGuard is implemented as a new monitor type for the open-source ControlArena framework.

Khanh Linh Nguyen, Hoa Nghiem, Tu Tran

Independent Researchers With Apart Research

AI Agent MonitoringAI ControlLLM Agent Safety
AskPDF
Jul 21, 2026

ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D

As AI agents begin to automate AI R&D, we need ways to assess whether their outputs are safe to deploy, even when the agents themselves may be untrusted. AI control offers one such approach: rather than trusting the agent, it treats it as a potential adversary and uses a monitor to detect covert sabotage before deployment. We evaluate AI control for automated AI R&D with ResearchArena, a framework spanning four long-horizon tasks: safety post-training, capabilities post-training, CUDA-kernel optimization, and inference-server optimization. Because the deliverable in AI R&D is an artifact that will be deployed and run, we pair each main task with two kinds of hidden side task: an embedded side task that sabotages the submitted model, adapter, kernel, or server itself, and an independent side task that takes a covert out-of-scope action in the sandbox while leaving the artifact honest. We evaluate frontier agents at both sabotage and monitoring, across four monitor types that vary along two axes: whether the monitor may execute and probe the produced artifact, and whether it sees the agent's chain-of-thought. We find that sabotage hidden in the training data is the hardest to catch, flagged fewer than half the time. Letting the monitor run experiments on the artifact, rather than only read the trajectory, helps, but it is not enough: monitors still miss embedded sabotage by inspecting only the surface, by noticing the anomaly but explaining it away, or by probing the artifact with the wrong test. We release ResearchArena as a modular framework for evaluating sabotage and control in automated AI R&D.

Lena Libon, Ben Rank, Jehyeok Yeon +5

ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems, Tübingen AI Center · ETH Zurich · University of Amsterdam

AI Agent MonitoringAI ControlAI Agent Safety BenchmarksAutonomous Research Agents
AskPDF