Chain of Thought Monitoring Can Fail Even When CoT Is Necessary

Lennart Bürger; Piotr Komorowski; Moritz Weckbecker; Evgenii Kortukov; Aakriti Jain; Sebastian Lapuschkin; Wojciech Samek

cs.AIOct 13, 2026

Chain of Thought Monitoring Can Fail Even When CoT Is Necessary

Authors: Lennart Bürger, Piotr Komorowski, Moritz Weckbecker, Evgenii Kortukov, Aakriti Jain, Sebastian Lapuschkin, Wojciech Samek

Organizations: Fraunhofer HHI · TU Dublin · TU Berlin · BIFOLD

Abstract

Chain of Thought (CoT) monitoring is used by frontier labs to detect misbehaviour of AI systems. Yet, its reliability remains poorly understood. We attribute this to the lack of a successful organizing principle that leads to generalizable conclusions about monitoring reliability in different regimes. Recent work attributes monitoring failures to the CoT not being necessary for the answer, and argues that CoT becomes monitorable when it is necessary. We show that this binary necessary/not-necessary organizing principle is too coarse by demonstrating that monitoring failures persist even if the CoT is necessary for the answer. In response, we introduce a more fine-grained organizing principle based on a hierarchy of four CoT necessity levels. Building on this hierarchy, we develop a framework for measuring CoT monitorability at the three lowest levels via intervention experiments. We demonstrate monitoring failures at all three of these levels on realistic, hard prediction tasks. On a difficult medical-diagnosis task, patient sex and a clinical attribute influence the answer in at least 19% and 24% of correct responses with a necessary CoT, respectively, yet their influence is almost never articulated. These results show that even when CoT is necessary, it is not guaranteed to be monitorable.

Figures & tables

Figure 1: Left: The average monitorability scores on our three challenging datasets for Qwen3-8B. Monitoring failures persist even on the subset of CoTs that were necessary to arrive at the answer. Right: Example of a monitorability failure. On a difficult medical diagnosis task where CoT is necessary, the patient’s sex strongly influences the model’s answer distribution but its influence is almost never articulated in the CoT.

Figure 2: The space of monitorability types and CoT necessity levels. Our work is the first to demonstrate monitorability failures at CoT necessity Levels 1 and 2 .

Panel of Figure 3 — Figure 3: Target answer probability with CoT and without CoT for different datasets and models.

Panel of Figure 4 — Figure 4: Monitorability scores $\hat{\bar{M}}_{0}$ and $\hat{\bar{M}}_{1}$ on different datasets for Qwen3-8B.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Standard deviation $\mathrm{std}(\hat{M}_{0})$ of the clipped per-pair estimator as a function of $p_{01}$ (with $p_{11}=p_{01}+\tau$ , filter $\widehat{\mathrm{TE}}\geq\tau$ ), evaluated at the threshold boundary $\mathrm{TE}=\tau$ . The true monitorability score is held fixed across the sweep by setting $\pi_{Z}=M_{\mathrm{true}}\,\tau/p_{11}$ . Top row: moderate- $M$ regime ( $M_{\mathrm{true}}=0.5$ ); bottom row: small- $M$ regime ( $M_{\mathrm{true}}=0.1$ ). Columns correspond to $\tau\in\{0.05,0.08,0.10\}$ and colors to $n\in\{100,500,1000\}$ .

Figure 6: Worst-case standard deviation $\max_{p_{01}}\mathrm{std}(\hat{M}_{0})$ of the clipped per-pair estimator at the threshold boundary $\mathrm{TE}=\tau$ (filter $\widehat{\mathrm{TE}}\geq\tau$ applied), with $\pi_{Z}$ chosen so that the true monitorability score is fixed at $M_{\mathrm{true}}=0.5$ (moderate- $M$ , solid) or $M_{\mathrm{true}}=0.1$ (small- $M$ , dashed). For each $(n,\tau,M_{\mathrm{true}})$ we maximize over $p_{01}\in[0,1-\tau]$ .

Figure 7: Standard deviation of $\widehat{\mathrm{TE}}$ as a function of $n$ . The worst case (solid black, $1/\sqrt{2n}$ ) is attained at $p_{11}=p_{01}=1/2$ . Dashed curves show $\mathrm{std}(\widehat{\mathrm{TE}})$ from equation 13 at representative $(p_{11},p_{01})$ pairs.

Figure 8: Tail probability $P_{p}\!\left(\widehat{\mathrm{TE}}\geq\tau\right)$ under $p_{11}=p_{01}=p$ , plotted against $p$ for several values of $\tau$ at $n=100$ (left) and at $n=1000$ (right). The supremum is attained at $p=1/2$ .

Figure 9: Worst-case false-positive rate $\alpha(\tau,n)$ at $p=1/2$ as a function of $\tau$ for several values of $n$ .

		worst-case $\mathrm{std}(\hat{M}_{0})$ at $\tau=0.08$
$n$	$\alpha(\tau{=}0.08,\,n)$	moderate- $M$ ( $M_{\mathrm{true}}=0.5$ )	small- $M$ ( $M_{\mathrm{true}}=0.1$ )
100	$1.4\times 10^{-1}$	$0.21$	$0.096$
1000	$1.8\times 10^{-4}$	$0.082$	$0.033$

Appendix

Table 1: Worst-case false-positive rate and worst-case estimator noise

\mathrm{std}(\hat{M}_{0})

\tau=0.08

, for two sample sizes and two regimes for the true monitorability score

M_{\mathrm{true}}

Figure 10: Conditional bias $E[\hat{M}_{\mathrm{clip}}\mid\widehat{\mathrm{TE}}>\tau]-M$ of the clipped monitorability estimator as a function of the true total effect $\mathrm{TE}$ , for the naive shared-sample estimator (solid) and the split-sample estimator (dashed), at three true scores $M\in\{0.1,0.5,1.0\}$ (colors). Estimated by Monte Carlo with $n=1000$ , $\tau=0.08$ , $p_{11}=0.5$ , and $\pi_{Z}=M\,\mathrm{TE}/p_{11}$ (so $M$ is held fixed along each curve). The naive estimator is strongly biased downward as $\mathrm{TE}\to\tau$ (winner’s curse), increasingly so for larger $M$ . The split-sample estimator removes this selection bias, leaving only the smaller $O(1/n)$ ratio bias (slightly positive for $M<1$ ) and, for $M=1$ , the downward clipping bias near the ceiling.

Panel of Figure 11 — Figure 11: Accuracy, recall and precision for Qwen3-8B on the ground truth classes of our three challenging datasets, with and without chain-of-thought. ”?” indicates undefined precision (fewer than 3 predictions for that class).

Panel of Figure 12 — Figure 12: Monitorability scores $\bar{M}_{0}$ and $\bar{M}_{1}$ on different datasets for Gemma-4-31b-it.

Panel of Figure 13 — Figure 13: Monitorability scores $\bar{M}_{0}$ and $\bar{M}_{1}$ on different datasets for Gemma-4-E4B-it.

Panel of Figure 14 — Figure 14: Monitorability scores $\bar{M}_{0}$ and $\bar{M}_{1}$ on different datasets for Qwen3-4B.

Panel of Figure 15 — Figure 15: Per-class monitorability on Thyroid Disease (Qwen3-8B). The aggregate of Figure 4(b) broken down by ground-truth class. We only show four ground truth classes instead of all 10.

Panel of Figure 16 — Figure 16: Per-class monitorability on DDXPlus (Qwen3-8B). The aggregate of Figure 4(c) broken down by ground-truth pathology. Within a class an attribute is plotted only where it keeps more than $n_{\min}=5$ qualifying pairs with a stable bootstrap estimate ( $f_{\mathrm{valid}}\geq 0.9$ , see Appendix E ). sex exceeds this, but only when aggregated across classes and therefore appears in the aggregate figure only. The per-class calculations for the aggregate figure are included even when not displayed for the per-class plots.

Panel of Figure 17 — Figure 17: Per-class monitorability on Olympic Athletes (Qwen3-8B). The aggregate of Figure 4(d) broken down by ground-truth class.

Panel of Figure 18 — Figure 18: The mean early-answer direct effect $\mathrm{DE}_{\mathrm{EA}}^{(i)}$ at different CoT positions for the attributes ”patient sex” (left) and ”query on thyroxine” (right). For comparison, we show the mean total effect (equation 4 ). On the right side of each plot, we show the mean and distribution of $\mathrm{DE}_{\mathrm{EA}}^{(i^{*})}$ .

Panel of Figure 19 — Figure 19: The mean early-answer direct effect $\mathrm{DE}_{\mathrm{EA}}^{(i)}$ at different CoT positions for the attributes ”patient sex” (left) and ”query on thyroxine” (right). The figure is analogous to figure 18 with the only difference that we do not fix the internal CoT state, just the external CoT tokens in the $X=1$ intervention condition. For comparison, we show the mean total effect (equation 4 ). On the right side of each plot, we show the mean and distribution of $\mathrm{DE}_{\mathrm{EA}}^{(i^{*})}$ .

Panel of Figure 20 — Figure 20: Early answer probability $P_{\mathrm{EA}}^{(i)}(Y{=}1\mid X{=}0)$ for the two attributes ”query on thyroxine” (left) and ”sex” (right) at different positions of the CoT. We show the mean, as well as four individual CoTs.

Panel of Figure 21 — Figure 21: Distribution of the position $i^{*}$ within CoTs. The left panel shows the distribution for CoTs from samples where the attribute ”query on thyroxine” was influential and the left side shows it for the attribute ”sex”.

Panel of Figure 22 — Figure 22: Full distribution of $\mathrm{DE}_{EA}^{i^{*}}$ . The left panel shows the distribution for CoTs from samples where the attribute ”query on thyroxine” was influential, i.e. $TE-p_{10}\geq\tau$ , and the left side shows it for the attribute ”sex”.

Code	Diagnosis	Records
A	Hyperthyroid	147
F	Primary hypothyroid	233
G	Compensated hypothyroid	359
I	Increased binding protein	346
K	Concurrent non-thyroidal illness	436
L	Replacement therapy (correct dose)	115

Appendix

Table 2: Thyroid disease dataset: 10 retained diagnostic classes after filtering.

Code	Diagnosis	Code	Diagnosis
ACO	Acute COPD exacerbation / infection	INH	Inguinal hernia
ADR	Acute dystonic reactions	LED	Localized edema
AFB	Atrial fibrillation	LSP	Laryngospasm
ALA	Acute laryngitis	MYG	Myasthenia gravis
ANA	Anaphylaxis	MYO	Myocarditis
ANE	Anemia	NST	Possible NSTEMI / STEMI

Appendix

Table 3: The 48 pathologies of DDXPlus and their three-letter codes.

Attribute	Interview question
age	(patient age, from the demographic introduction)
sex	(patient sex, from the demographic introduction)
E_3	Have you ever had a pericarditis?
E_5	Have you ever had fluid in your lungs?
E_13	Do you find that your symptoms have worsened over the last 2 weeks and that progressively less effort is required to cause the symptoms?
E_14	Do you have chest pain even at rest?

Appendix

Table 4: Every DDXPlus attribute that appears as an axis label in the monitorability figures of this paper, with the interview question each evidence id stands for.

Code	Sport
H	Gymnastics
G	Figure skating
M	Weightlifting
N	Wrestling
D	Boxing
A	Alpine skiing

Appendix

Table 5: The 14 sports of Olympic Athletes . Class codes are assigned alphabetically by sport name, so that each answer option is a single token and the no-CoT answer distribution can be read from the logits at a single position.

CoT Monitoring

Chain-of-thought monitoring studies whether models’ reasoning traces expose decision-relevant behavior and support oversight of unsafe or deceptive actions.

See all 40 papers

Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory
Feb 20, 2026Usman Anwar, Tim Bakker, Dana Kianfar +2CoT Monitoring
University of Cambridge, work done while at Qualcomm AI Research. · Qualcomm AI Research.
Monitorability Is Trainable: Post-Training for Chain-of-Thought Monitorability
Oct 21, 2026Usman Anwar, Tim Bakker, Christos Louizos +1CoT Monitoring LLM Post-Training
University of Cambridge · Qualcomm AI Research · Mila, University of Montreal
Reasoning Models Struggle to Control their Chains of Thought
Mar 5, 2026Chen Yueh-Han, Robert McCarthy, Bruce W. Lee +5CoT Monitoring Large Reasoning Models
NYU, MATS · UCL, MATS · UPenn, MATS +2

Feb 20, 2026J/K move · Enter open · S save

Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory

Usman Anwar, Tim Bakker, Dana Kianfar, Cristina Pinneri+1

University of Cambridge, work done while at Qualcomm AI Research. · Qualcomm AI Research.

CoT Monitoring

Open paper AskPDF arXiv

Chain-of-thought (CoT) monitors are LLM-based systems that analyze reasoning traces to detect when outputs may exhibit attributes of interest, such as test-hacking behavior during code generation. In this paper, we use information-theoretic analysis to show that non-zero mutual information between CoT and output is a necessary but not sufficient condition for CoT monitorability. We identify two sources of approximation error that may undermine the performance of CoT monitors in practice: information gap, which measures the extent to which the monitor can extract the information available in CoT, and elicitation error, which measures the extent to which the monitor approximates the optimal monitoring function. Empirically, we show that CoT obfuscation can arise even in the absence of any optimization pressure against CoT. We further demonstrate that the fragility of CoT monitorability can be systematically mitigated through targeted training objectives. To this end, we propose a practical, label-free objective based on mutual information between outputs and CoTs, and benchmark it against an oracle-based method that directly rewards the monitored model for CoTs that maximize monitor accuracy. Across multiple environments, we show that our objective is effective at preventing CoT obfuscation even when training against a monitor, and helps mitigate reward hacking when the task reward is imperfectly specified.

Abstract

CoT Monitoring

Explore similar work

Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory

Monitorability Is Trainable: Post-Training for Chain-of-Thought Monitorability

Reasoning Models Struggle to Control their Chains of Thought