As LLM agents advance rapidly in autonomy, the risk of hidden objectives that bypass safety constraints becomes increasingly plausible. Chain-of-thought (CoT) monitoring offers a key control mechanism by reading the agent's reasoning and flagging misalignment before it materializes into action, yet its reliability depends on the realism of the red-team it is stress-tested against. Existing red-teaming efforts rely on shared templates, such as casting a side task as a prerequisite, which treat the side objective as an exogenous insertion and leave task-specific, contextually grounded evasions underexplored. We therefore ask: How robust is CoT monitoring against agents that strategically conceal misalignment? To answer this, we introduce TraceMRT, an automated Trace-level Monitor Red-Teaming framework that reframes the side task as a coherent sub-goal of the primary task and searches trace-shaping strategies whose induced trajectories complete the misaligned objective while reading as natural execution. Through black-box attacks on state-of-the-art monitors such as GPT-5.2 and Claude-4.5, we reveal substantial vulnerabilities even in top models. To counter these vulnerabilities, we propose a Bayesian Trajectory Monitoring framework, integrating global context and local segment evidence to detect covert misalignment. Empirically, our method significantly consolidates monitor robustness with 0.91 AUC and 0.74 TPR@FPR=0.05.
Figures & tables
Figure 1: Overview of the TraceMRT framework. In Stage 1, Intention Concealment reframes the secondary task as a legitimate sub-goal of the primary task, iteratively refining the prompt via self-consistency verification to ensure faithful intent concealment without omissions. In Stage 2, CoT Evasion runs a genetic search over trace-obfuscation and monitor-interference strategies, evolving candidate strategy pairs by interpretation fidelity and strategic suitability before selecting the top-performing candidate trajectories for final evaluation against the target CoT monitor across tasks.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Method
ESR ↑
SSR ↑
OSR ↑
Benign
21.6
47.4
3.5
TraceMRT
61.9
78.8
48.1
Appendix
Table 4: TraceMRT on OS-Harm. Four-agent mean with GPT-5.2 as the monitor, in percent.
Agent
Method
ESR ↑
SSR ↑
OSR ↑
GLM-4.7
Benign
2.0
42.0
0.0
TraceMRT
69.0
56.0
36.0
Kimi-K2
Benign
1.0
88.0
0.0
TraceMRT
42.0
73.0
11.0
DeepSeek-v3.2
Benign
2.0
91.0
0.0
TraceMRT
22.0
81.0
15.0
Appendix
Table 5: TraceMRT on BashArena. Results on 400 samples with GPT-5.2 as the monitor, in percent.
Red-teaming effectiveness
Defense effectiveness
Method
ESR ↑
SSR ↑
OSR ↑
Method
AUC ↑
TPR@FPR=0.05 ↑
Benign
7.5
62.5
2.5
Base
0.050
0.050
TraceMRT
95.0
87.5
82.5
Bayesian
0.998
0.900
Appendix
Table 6: Evaluation in healthcare scenarios. Red-teaming and defense results averaged across 40 instances and four agents under the GPT-5.2 monitor, with all red-teaming values in percent.
Method
ESR ↑
SSR ↑
OSR ↑
Benign
3.85
64.42
1.79
TraceMRT
82.40
75.00
61.54
Appendix
Table 7: TraceMRT against a specialized monitor. Results on SHADE-Arena with AgentDoG-Qwen3-4B as the monitor, averaged across the four adversarial agents. All values in %.
Method
Avg Time (s)
Input Tokens
Output Tokens
Total Tokens
Base
7.82
41,666
254
41,920
Sequential
139.29
81,445
4,912
86,357
Hierarchical
180.63
103,265
6,890
110,155
Bayesian (Ours)
17.85
74,794
967
75,761
Appendix
Table 8: Per-trajectory monitoring overhead. Average runtime in seconds and token consumption per trajectory on SHADE-Arena with GPT-5.2 as the target monitor across all evaluated scaffolds.
Dataset
Agent
Zolkowski et al.
TraceMRT
SHADE-Arena
Kimi-K2
0.00 / 61.54 / 0.00
73.08 / 73.08 / 46.15
GLM-4.7
3.85 / 69.23 / 3.85
76.82 / 80.77 / 57.69
DeepSeek-v3.2
0.00 / 69.23 / 0.00
26.92 / 80.77 / 19.23
Qwen3-235B-A22B
0.00 / 65.38 / 0.00
100.00 / 65.38 / 65.38
Average
0.96 / 66.35 / 0.96
69.21 / 75.00 / 47.11
CUA-SHADE-Arena
Kimi-K2
6.67 / 60.00 / 6.67
60.00 / 86.67 / 46.67
Appendix
Table 9: Additional baseline comparison with GPT-5.2. Each entry reports ESR / SSR / OSR in percent, and the bottom row of each benchmark reports the average across all four agent models.
Dataset
Agent
Strict no-external-CoT
With CoT
SHADE-Arena
DeepSeek-v3.2
73.08% (19/26)
73.08% (19/26)
GLM-4.7
34.62% (9/26)
65.38% (17/26)
Qwen3-235B-A22B
23.08% (6/26)
65.38% (17/26)
CUA-SHADE-Arena
DeepSeek-v3.2
80.00% (12/15)
66.67% (10/15)
GLM-4.7
40.00% (6/15)
66.67% (10/15)
Qwen3-235B-A22B
60.00% (9/15)
46.67% (7/15)
Appendix
Table 10: Paired side-task completion with and without externalized CoT. Evaluated under benign prompts on both benchmarks, reporting percentage completion and completed over total tasks.
CoT-independent
CoT-dependent
Method
ESR
SSR
OSR
ESR
SSR
OSR
DeepSeek-v3.2
CoT Red-Handed
0.00
94.74
0.00
0.00
100.00
0.00
MRT
36.84
89.47
26.32
0.00
50.00
0.00
Zolkowski et al.
5.26
63.16
5.26
0.00
50.00
0.00
TraceMRT
89.47
94.74
84.21
0.00
50.00
0.00
Appendix
Table 11: Strict CoT partitions on SHADE-Arena. Per-agent ESR, SSR, and OSR in percent.
CoT-independent
CoT-dependent
Method
ESR
SSR
OSR
ESR
SSR
OSR
DeepSeek-v3.2
CoT Red-Handed
25.00
75.00
16.67
0.00
100.00
0.00
MRT
83.33
66.67
50.00
0.00
100.00
0.00
Zolkowski et al.
16.67
83.33
16.67
0.00
0.00
0.00
TraceMRT
91.67
83.33
75.00
100.00
100.00
100.00
Appendix
Table 12: Strict CoT partitions on CUA-SHADE-Arena. Per-agent ESR, SSR, and OSR in percent.
CoT-independent
CoT-dependent
Method
ESR
SSR
OSR
ESR
SSR
OSR
SHADE-Arena
CoT Red-Handed
24.08
76.03
14.81
0.00
81.94
0.00
MRT
36.35
89.08
29.14
36.11
61.11
20.83
Zolkowski et al.
18.42
50.68
5.46
0.00
73.61
0.00
TraceMRT
83.53
88.99
72.52
58.33
68.06
43.06
Appendix
Table 13: Average performance under strict CoT controls. Mean per-agent ESR, SSR, and OSR in percent across the three agents from Tables 11 and 12 across CoT-independent and dependent subsets.
SHADE-Arena
CUA-SHADE-Arena
Agent
Benign
TraceMRT
Benign
TraceMRT
Kimi-K2
37.35
54.47
72.20
77.07
GLM-4.7
51.23
59.62
76.93
78.07
DeepSeek-v3.2
42.73
59.00
77.60
79.80
Qwen3-235B-A22B
31.19
22.95
47.67
39.87
Average
40.63
49.01
68.60
68.70
Appendix
Table 14: Primary-task completion under TraceMRT. Main-task completion rates in percent for each agent and the four-agent average across both SHADE-Arena and CUA-SHADE-Arena.
Scaffold
AUC ↑
TPR@5% FPR ↑
Base
0.2172
0.0000
Matched
0.8989
0.7288
Bayesian (ours)
0.9189
0.8167
Appendix
Table 15: Bayesian formulation ablation. Average scores across four agents with GPT-5.2 on SHADE-Arena, where Matched uses the same two-stage scaffold with direct suspiciousness scoring.