Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability

Samuel Lewis-Lim; Xingwei Tan; Mario Sanger; Zhixue Zhao; Nikolaos Aletras

cs.AIOct 2, 2026

Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability

Authors: Samuel Lewis-Lim, Xingwei Tan, Mario Sanger, Zhixue Zhao, Nikolaos Aletras

Organizations: School of Computer Science, University of Sheffield · AstraZeneca

Abstract

Chain-of-thought (CoT) reasoning allows humans to inspect how large language models reach their answers, and oversee model behaviour. This reasoning comes at an increased inference cost, motivating efficient methods that train models to solve tasks using fewer tokens. However, a common concern is that such training may cause models to skip important reasoning steps, so the CoT no longer faithfully reflects the model's decision. It is unclear whether or when this occurs in practice, since different efficiency methods apply length pressure to models' CoT in distinct ways, and faithfully explaining a model's decision takes more tokens on some tasks than others. To understand these dynamics, we fine-tune a variety of models with three methods that apply length pressure differently, namely a fixed generation budget, a per-example length target, and a group-relative length reward. We evaluate how efficient reasoning affects CoT faithfulness (i.e., how well the CoT reflects model decisions on related inputs) and monitorability (i.e., whether the CoT reveals when input interventions alter the output). We find that it affects faithfulness and monitorability differently. Faithfulness falls in most settings, primarily because the trained models are less consistent. Monitorability is more robust, as models keep acknowledging the influence on their answer even when the CoT is much shorter.

Figures & tables

Figure 1: Efficient reasoning training reduces CoT faithfulness but largely preserves monitorability. (1) We train models with three methods that reward CoT length differently. (2) Then we measure whether the CoT predicts the model’s answers on related inputs (faithfulness) and whether it reveals when an intervention influenced the answer (monitorability). (3) In this example, a $\sim$ 70% shorter CoT loses faithfulness due to output inconsistency, while monitorability is unchanged.

Figure 2: How we measure faithfulness and monitorability. For faithfulness, a predictor sees the reference model’s answer and CoT on an original input, and predicts the model’s answer on a similar counterfactual input. NSG quantifies how much having the CoT raises the predictor’s accuracy compared to not having the CoT ( Mayne et al., 2026 ) . For monitorability, an intervention changes the model’s answer, and a monitor reading only the CoT must detect this influence ( Guan et al., 2025 ) .

Figure 3: Predictor accuracy without the reference CoT (solid) and with it (hatched), averaged across all seven tasks. The white band shows errors despite a consistent model, and the grey band errors due to inconsistency, which no CoT could remove. Numbers in the bars give mean CoT length in tokens. L1-Low, L1-Medium and L1-High use budgets of 512, 1,000 and 3,000 tokens. Error bars are 95% bootstrap confidence intervals.

Figure 4: Change in pooled NSG after efficiency training against (a) mean CoT length relative to the base model and (b) the change in consistency, the share of 10 resampled answers per counterfactual matching the reference model’s most common answer. Colour marks the efficiency method and shape the base model. Error bars are 95% paired bootstrap CIs.

Figure 5: Change in NSG (percentage points) on each dataset for every trained model, relative to its base model.

Figure 6: Monitorability ( $g$ -mean 2 , higher is better) against CoT length distributions for Qwen3-4B, with the sycophancy intervention on Face-Eval and the cognitive-bias intervention on MedQA.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Monitorability (

g

-mean 2 ) against the distribution of CoT lengths for Qwen3-8B on FACE-Eval, with the explicit cue (left) and the implicit cue (right).

Figure 9: Monitorability (

g

-mean 2 ) against the distribution of CoT lengths for Olmo3-7B-Think on FACE-Eval, with the explicit cue (left) and the implicit cue (right).

Figure 10: Monitorability (

g

-mean 2 ) against the distribution of CoT lengths for Qwen3-4B on FACE-Eval, with the explicit cue (left) and the implicit cue (right).

Figure 11: Monitorability (

g

-mean 2 ) against the distribution of CoT lengths for Qwen3-8B on the medical cognitive bias task, with the false-consensus cue (left) and the frequency cue (right).

Figure 12: Monitorability (

g

-mean 2 ) against the distribution of CoT lengths for Olmo3-7B-Think on the medical cognitive bias task, with the false-consensus cue (left) and the frequency cue (right). L1-High has no eligible instances on the frequency cue and so has no point in the right panel.

Figure 13: Monitorability (

g

-mean 2 ) against the distribution of CoT lengths for Qwen3-4B on the medical cognitive bias task, with the false-consensus cue (left) and the frequency cue (right).

		Accuracy (%)		Gain (%)		NSG (%)
Model	Method	w/o CoT	w/ CoT	Gain	95% CI	NSG	95% CI
Qwen3-8B	Base	68.30	77.76	9.46	(8.59, 10.27)	29.85	(27.33, 32.17)
	ThinkPrune	68.55	77.86	9.31	(8.50, 10.16)	29.61	(27.24, 31.82)
	GLP	68.43	78.88	10.46	(9.60, 11.26)	33.12	(30.79, 35.43)
	L1-Low	66.51	76.77	10.26	(9.40, 11.10)	30.63	(28.60, 32.81)
	L1-Medium	66.73	75.51	8.77	(7.90, 9.57)	26.37	(24.09, 28.69)

Appendix

Table 1: Full results by model. Predictor accuracy without and with the reference model’s CoT, the absolute simulatability gain and NSG. All results are pooled across the three predictor models. Confidence intervals are clustered bootstrap 95% CIs over original questions.

		Sycophancy		Cognitive bias
Family	Model	Explicit	Implicit	False-consensus	Frequency
Qwen3-8B	Base	0.46	0.27	0.33	0.37
	GLP	0.45 ( $-0.01$ )	0.27 ( $-0.00$ )	0.35 ( $+0.02$ )	0.40 ( $+0.02$ )
	ThinkPrune	0.47 ( $+0.01$ )	0.26 ( $-0.00$ )	0.35 ( $+0.02$ )	0.42 ( $+0.05$ )
	L1-Low	0.42 ( $-0.04$ )	0.26 ( $-0.01$ )	0.39 ( $+0.06$ )	0.38 ( $+0.00$ )
	L1-High	0.38 ( $-0.07$ )	0.21 ( $-0.06$ )	0.34 ( $+0.01$ )	0.37 ( $+0.00$ )

Appendix

Table 2: Treatment effect of each intervention, measured as

P(\text{cued answer}\mid\text{cue})-P(\text{cued answer}\mid\text{no cue})

averaged over questions. Change from that family’s base model is given in the brackets, and a higher value means the cue moves the answer more.

				NSG
Model	Method	Consistency (%)	Upper bound (%)	$\Delta$ (%)	Share from bound (%)
Qwen3-8B	Base	90.9	66.4	29.8	–
	ThinkPrune	91.3	68.8	$-$ 0.2	–
	GLP	90.9	67.5	$\mathbf{+3.3}$	–
	L1-Low	89.4	65.5	$+$ 0.8	–
	L1-Medium	89.3	61.9	$\mathbf{-3.5}$	58

Appendix

Table 3: Consistency and the NSG upper bound. Consistency is the share of a model’s 10 samples per counterfactual that match its most common answer. The upper bound is the NSG an oracle predictor that always guesses this answer would reach. Base rows (italic) give absolute values. Rows below give the change in NSG from the base model, with bold marking changes whose 95% paired bootstrap CI excludes zero. Share from bound is the share of each NSG reduction explained by the fall in the upper bound, assuming the model keeps the same fraction of its bound as the base model:

(\mathrm{NSG}_{\text{base}}/\mathrm{Bound}_{\text{base}})\times\Delta\mathrm{Bound}\,/\,\Delta\mathrm{NSG}

Setting	Value
LoRA
Rank / $\alpha$ / dropout	16 / 32 / 0
Target modules	all attention and MLP projections
Optimisation
Optimiser	AdamW
Learning rate	$1\times 10^{-5}$

Appendix

Table 4: Training settings shared across all methods and base models.

Method	Setting	Qwen3-4B	Qwen3-8B	Olmo3-7B
ThinkPrune	Completion budget $L$	4,000	3,000	4,000

Appendix

Table 5: Method-specific settings. ThinkPrune’s budget for Qwen3-8B is 3,000 rather than 4,000 because its average base model CoT is already less than 4,000 on the training data

Model	MATH-500 Acc.	MATH-500 Avg CoT	MMLU Acc.	MMLU Avg CoT
Base	0.92	4406	0.89	1595
ThinkPrune	0.91	1781	0.86	588
GLP	0.91	855	0.87	382
LCPO-Exact:
L1-Low ( $b{=}512$ )	0.68	347	0.75	235
L1-Medium ( $b{=}1000$ )	0.70	429	0.77	276

Appendix

Table 6: Qwen3-8B evaluation results (MATH-500,

n{=}500

; MMLU-Redux,

n{=}5330

; LCPO-Exact rows use

n{=}100

285

Model	MATH-500 Acc.	MATH-500 Avg CoT	MMLU Acc.	MMLU Avg CoT
Base	0.89	5497	0.87	2221
ThinkPrune	0.93	1061	0.86	671
GLP	0.90	893	0.85	490
LCPO-Exact:
L1-Low ( $b{=}512$ )	0.69	436	0.79	399
L1-Medium ( $b{=}1000$ )	0.75	752	0.81	608

Appendix

Table 7: Qwen3-4B evaluation results (MATH-500,

n{=}500

; MMLU-Redux,

n{=}5330

). All rows, including LCPO-Exact, are measured at full coverage.

Model	MATH-500 Acc.	MATH-500 Avg CoT	MMLU Acc.	MMLU Avg CoT
Base	0.92	5050	0.81	3162
ThinkPrune	0.93	3410	0.80	1763
GLP	0.90	1467	0.77	590
LCPO-Exact:
L1-Low ( $b{=}512$ )	0.68	1231	0.70	783
L1-Medium ( $b{=}1000$ )	0.71	1292	0.69	824

Appendix

Table 8: Olmo-3-7B-Think evaluation results (MATH-500,

n{=}500

; MMLU-Redux,

n{=}5330

). All rows, including LCPO-Exact, are measured at full coverage.

Explore similar work

Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning
May 22, 2026Jinghan Jia, Joe Benton, Eric EasleyCoT Faithfulness LLM Interpretability
†Dept. CSE, Michigan State University · ⋆Anthropic Fellows Program · ‡Anthropic
Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?
Mar 23, 2026Richard J. YoungCoT Faithfulness Faithfulness of Language Model Explanations
University of Nevada, Las Vegas, Department of Management, Entrepreneurship and Technology, Lee Business School, Las Vegas, NV, USA · DeepNeuro AI, Las Vegas, NV, USA
Reasoning Models Struggle to Control their Chains of Thought
Mar 5, 2026Chen Yueh-Han, Robert McCarthy, Bruce W. Lee +5CoT Monitoring Large Reasoning Models
NYU, MATS · UCL, MATS · UPenn, MATS +2

May 22, 2026J/K move · Enter open · S save

Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning

Jinghan Jia, Joe Benton, Eric Easley

†Dept. CSE, Michigan State University · ⋆Anthropic Fellows Program · ‡Anthropic

CoT Faithfulness LLM Interpretability Causal Interventions in Language Models Reasoning Trace Analysis

Open paper AskPDF arXiv

Chain-of-thought (CoT) reasoning is useful for monitoring language models only when the reasoning trace faithfully reflects the computation that produces the final answer. However, models can rely on prompt-to-answer shortcuts that bypass the CoT, making the visible reasoning trace misleading even when it appears plausible. We study CoT faithfulness through a structural information-flow perspective: faithful reasoning should route answer-relevant information through the mediated path from prompt to CoT to answer, rather than through a direct prompt-to-answer shortcut. This perspective yields a task-agnostic framework based on three complementary properties, sufficiency, completeness, and necessity, which we instantiate with entropy-based, masked-KL, and gradient-based diagnostics. We show that these metrics recover externally judged faithfulness differences in hinted reasoning, and identify a low-entropy failure mode of KL-based diagnostics where gradient-based measures remain more stable. Building on this analysis, we introduce update-time interventions for verifier-based on-policy RL, including attention masking, backward-only gradient masking, CoT gradients, and adversarial perturbations of prompt representations. Across hinted arithmetic, reward-hackable code repair, and DAPO-Math models trained without hints but evaluated under wrong-hint injection, our interventions shift behavioral and structural indicators toward stronger CoT mediation. In particular, they make shortcut and reward-hacking behavior more transparent in the CoT and improve task-agnostic faithfulness metrics, while in some settings also reducing wrong-hint susceptibility. Our results suggest that controlling information flow during training is a practical route toward more faithful and monitorable CoT reasoning. Code is available at https://github.com/safety-research/faithful-cot.

Abstract

Explore similar work

Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning

Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?

Reasoning Models Struggle to Control their Chains of Thought