Measuring Weak-to-Strong Legibility of Reasoning Models

Dani Roytburg; Shreya Sridhar; Daphne Ippolito

cs.MAMar 20, 2026

Measuring Weak-to-Strong Legibility of Reasoning Models

Authors: Dani Roytburg, Shreya Sridhar, Daphne Ippolito

Organizations: Machine Learning Department, Carnegie Mellon University, Pittsburgh, United States · Language Technologies Institute, Carnegie Mellon University, Pittsburgh, United States

Abstract

Reasoning language models (RLMs) and the intermediate chains of thought they emit play an increasingly central role in multi-agent setups such as inter-model monitoring or distillation into smaller models. When agents at different capability tiers must cooperate, strong models need to produce traces digestible by weaker ones. We refer to this goal as "weak-to-strong legibility". Trustworthiness of large models depends in part on this legibility property. For safety oversight in particular, adoption of weak monitors may become a standard for reliability scaffolds on a healthy budget. Legibility requires that the shape of these decision-making traces takes some form accessible to weaker monitors. Existing efficiency-based metrics for legibility fail to capture "thoroughness", instead focusing on conciseness.

Figures & tables

Figure 1: Our evaluation framework (1) elicits reasoning traces from RLMs on a common problem set; (2) analyzes the length of traces, the number of semantically redundant steps , and the frequency of backtracking ; (3) passes progressively longer prefixes of traces to weaker models , making measurements on weak-model accuracy as a function of prefix length. In the given example, GPT-OSS-120B scores well on efficiency but poorly on transfer , while DeepSeek-R1 produces a more “thorough” but less efficient trace.

Model (Acc, %)	Len.	Red.	Back.	FOTU	SOTU
gpt-oss-120b (81)	4 (-3)	1 (0)	4 (-3)	10 (-9)	5 (-4)
gpt-oss-20b (76)	5 (-3)	2 (0)	7 (-5)	8 (-6)	4 (-2)
kimi-k2 (70)	10 (-7)	11 (-8)	8 (-5)	1 (+2)	8 (-5)
deep-r1-0528 (70)	11 (-7)	8 (-4)	6 (-2)	3 (+1)	10 (-6)
r1-distill (68)	6 (-1)	8 (-3)	10 (-5)	11 (-6)	6 (-1)
qwen3-8b (57)	7 (-1)	10 (-4)	10 (-4)	6 (0)	9 (-3)

Table 1: Rank-wise model comparison by metric. Model (Acc) uses mean accuracy integers. Metrics show ordinal rank (1=best) with rank change relative to Acc. in tiny parentheses. Len.: Token Length; Red.: Redundancy; Back.: Backtracking; FOTU: first-order TU; SOTU: hazard-entropy TU. Bold denotes #1 rank; underline denotes #2 rank. See Table 13 in the appendix for raw values.

Figure 2: Scatter plots of first-order (x-axis) transfer against second-order transfer utility (uniform information), RLM accuracy, and reward model scores (y axes). Each point is a (model, dataset) pair. Colors denote models, clustered by families.

Figure 3: Scatter plots of teacher FOTU vs. Phi-3-Mini monitor precision, recall, and F1 across conditions and datasets. Each point is one teacher model. FOTU predicts precision most strongly on MATH (sabotage: $\rho=+0.900^{***}$ ; shared failures: $\rho=+0.873^{***}$ ). See Table 2 for full numeric results.

Dataset	Condition	$\overline{\text{F1}}$	$\rho$ (F, Rec.)	$\rho$ (F, Prec.)	$\rho$ (F, F1)	$\rho$ (F, F1 $\|$ Len.)
MATH	Sabotage	0.851	+0.618 ∗ (0.043)	+0.900 ∗∗∗ (0.000)	+0.845 ∗∗ (0.001)	+0.745 ∗∗ (0.008)
MATH	Natural	0.759	+0.709 ∗ (0.015)	+0.873 ∗∗∗ (0.000)	+0.809 ∗∗ (0.003)	+0.582 † (0.060)
LSAT	Sabotage	0.346	-0.318 (0.340)	+0.818 ∗∗ (0.002)	+0.664 ∗ (0.026)	+0.482 (0.133)
LSAT	Natural	0.726	+0.588 † (0.057)	+0.664 ∗ (0.026)	+0.591 † (0.056)	+0.082 (0.811)
GPQA	Sabotage	0.270	-0.356 (0.282)	+0.770 ∗∗ (0.006)	+0.064 (0.853)	-0.173 (0.612)
GPQA	Natural	0.231	-0.691 ∗ (0.019)	-0.305 (0.361)	-0.709 ∗ (0.015)	-0.436 (0.180)

Table 2: Phi-3-mini: Spearman correlation of teacher FOTU with monitor Recall, Precision, and F1 across datasets and conditions. Sabotage : adversarial sabotage injection for all datasets. Natural : shared failures. (

\overline{\text{F1}}

): mean F1. Positive class = incorrect trace (wrong-as-positive).

p

-values from t-test approximations. *** =

p<0.001

; ** =

p<0.01

; ∗ =

p<0.05

; † =

p<0.10

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Distribution of maximum step-to-previous-step cosine similarities.

Dataset	Spearman’s Rho (annotator-judge)	Kendall’s $\tau$ (cross-dataset stability)	Kappa (Exact)	Kappa (Binary $>$ 0)
Overall	0.717	–	0.234	0.712
LSAT	0.717	0.641	0.145	0.623
GPQA	0.916	0.467	0.552	0.800
Math	0.857	0.779	0.267	0.745

Appendix

Table 3: Annotator agreement on backtracking counts: Spearman’s rho (rankwise), Kendall’s

\tau

cross-dataset, and Cohen’s Kappa (categorical).

Dataset	Teacher	$n$	Median ratio	% in-window
MATH	Kimi-K2	243	0.924	65%
	R1-0528	264	1.005	98%
	Qwen3-8B	264	1.013	91%
	QwQ-32B	265	1.020	90%
	Magistral-S	267	1.098	60%
	Gemma3-12B	263	1.230	44%

Appendix

Table 4: Injection length-ratio validation per teacher per dataset.

L_{\text{inj}}/L_{\text{orig}}

is the character-length ratio of the injected to the original reasoning trace. % in-window = fraction satisfying

[0.85,\,1.15]

			$\overline{\text{F1}}$	$\rho$ (Rec)	$\rho$ (Rec $\|$ L)	$\rho$ (Prec)	$\rho$ (Prec $\|$ L)	$\rho$ (F1)	$\rho$ (F1 $\|$ L)	$\rho$ (L,F1)
MATH	Phi-3-mini	Inject	0.851	+0.618 ∗ (0.043)	+0.518 (0.102)	+0.900 ∗∗∗ (0.000)	+0.836 ∗∗ (0.001)	+0.845 ∗∗ (0.001)	+0.745 ∗∗ (0.008)	-0.664 ∗ (0.026)
	Phi-3-mini	Natural	0.759	+0.709 ∗ (0.015)	+0.464 (0.151)	+0.873 ∗∗∗ (0.000)	+0.482 (0.133)	+0.809 ∗∗ (0.003)	+0.582 † (0.060)	-0.727 ∗ (0.011)
	OLMo-3-7B	Inject	0.828	+0.842 ∗∗ (0.002)	+0.818 ∗∗ (0.004)	+0.681 ∗ (0.030)	+0.370 (0.293)	+0.842 ∗∗ (0.002)	+0.818 ∗∗ (0.004)	-0.576 † (0.082)
	OLMo-3-7B	Natural	0.571	+0.136 (0.689)	+0.318 (0.340)	+0.718 ∗ (0.013)	+0.164 (0.631)	+0.136 (0.689)	+0.318 (0.340)	-0.036 (0.915)
	Llama-3.2-1B	Inject	0.388	-0.745 ∗∗ (0.008)	-0.664 ∗ (0.026)	+0.727 ∗ (0.011)	+0.655 ∗ (0.029)	-0.609 ∗ (0.047)	-0.600 † (0.051)	-0.436 (0.180)
	Llama-3.2-1B	Natural	0.538	+0.310 (0.354)	+0.000 (1.000)	+0.527 † (0.096)	+0.027 (0.937)	+0.382 (0.247)	-0.109 (0.750)	-0.597 † (0.053)

Appendix

Table 5: Full Spearman correlation table: FOTU vs. monitor metrics across all datasets, monitors, and conditions. Columns show raw and length-partialled correlations for Recall, Precision, and F1, plus

\rho(\text{Len},\text{F1})

as a cross-partial reference. Inject : structured injection. Natural : shared failure sets. Bold

p{<}0.001

; u/l

p{<}0.01

; ∗

p{<}0.05

; †

p{<}0.10

Monitor	Cond	$\overline{\text{F1}}$	$\rho$ (F,Rec.)	$\rho$ (F,Prec.)	$\rho$ (F,F1)	$\rho$ (F,F1 $\|$ Len.)
Phi-3-mini	Inject	0.851	+0.618 ∗ (0.043)	+0.900 ∗∗∗ (0.000)	+0.845 ∗∗ (0.001)	+0.745 ∗∗ (0.008)
Phi-3-mini	Natural	0.759	+0.709 ∗ (0.015)	+0.873 ∗∗∗ (0.000)	+0.809 ∗∗ (0.003)	+0.582 † (0.060)
OLMo-3-7B	Inject	0.828	+0.842 ∗∗ (0.002)	+0.681 ∗ (0.030)	+0.842 ∗∗ (0.002)	+0.818 ∗∗ (0.004)
OLMo-3-7B	Natural	0.571	+0.136 (0.689)	+0.718 ∗ (0.013)	+0.136 (0.689)	+0.318 (0.340)
Llama-3.2-1B	Inject	0.388	-0.745 ∗∗ (0.008)	+0.727 ∗ (0.011)	-0.609 ∗ (0.047)	-0.600 † (0.051)
Llama-3.2-1B	Natural	0.538	+0.310 (0.354)	+0.527 † (0.096)	+0.382 (0.247)	-0.109 (0.750)

Appendix

Table 6: Spearman correlations of teacher FOTU with monitor Recall, Precision, F1, and length-partialled F1 on MATH, across all three monitors and both error conditions. Inject : structured injection. Natural : shared failures.

\overline{\text{F1}}

= mean per-teacher F1. Positive class = incorrect trace. Bold

p{<}0.001

; u/l

p{<}0.01

; ∗

p{<}0.05

; †

p{<}0.10

	Teacher	FOTU	Rec.(1)	Prec.(1)	F1(1)	Rec.(2)	Prec.(2)	F1(2)
MATH	Kimi-K2	0.600	0.877	1.000	0.934	0.730	0.896	0.804
	R1-0528	0.549	0.826	0.991	0.901	0.810	0.885	0.846
	QwQ-32B	0.528	0.755	0.971	0.849	0.845	0.949	0.894
	Qwen3-4B	0.485	0.775	0.986	0.868	0.835	0.861	0.848
	Qwen3-8B	0.430	0.765	0.990	0.863	0.805	0.880	0.841
	R1-Distill-32B	0.406	0.711	0.964	0.818	0.825	0.851	0.838

Appendix

Table 7: Phi-3-mini: per-teacher Recall, Precision, and F1 across datasets and conditions, sorted by descending FOTU within each dataset. Positive class = incorrect trace (wrong-as-positive). Recall = TP/(TP+FN); Precision = TP/(TP+FP); F1 = harmonic mean. Condition (1) and (2) per dataset: MATH Injection / Shared Fail; LSAT Struct Inj / GT=E; GPQA Struct Inj / Shared Fail.

	Teacher	FOTU	Rec.(1)	Prec.(1)	F1(1)	Rec.(2)	Prec.(2)	F1(2)
MATH	Kimi-K2	0.600	—	—	—	0.230	0.958	0.371
	R1-0528	0.549	0.784	0.990	0.875	0.385	0.939	0.546
	QwQ-32B	0.528	0.725	0.995	0.838	0.430	0.977	0.597
	Qwen3-4B	0.485	0.753	1.000	0.859	0.465	0.949	0.624
	Qwen3-8B	0.430	0.727	1.000	0.842	0.445	0.978	0.612
	R1-Distill-32B	0.406	0.699	0.969	0.812	0.710	0.916	0.800

Appendix

Table 8: OLMo-3-7B: per-teacher Recall, Precision, and F1 across datasets and conditions, sorted by descending FOTU within each dataset. Positive class = incorrect trace (wrong-as-positive). Recall = TP/(TP+FN); Precision = TP/(TP+FP); F1 = harmonic mean. Condition (1) and (2) per dataset: MATH Injection / Shared Fail; LSAT Struct Inj / GT=E; GPQA Struct Inj / Shared Fail.

	Teacher	FOTU	Rec.(1)	Prec.(1)	F1(1)	Rec.(2)	Prec.(2)	F1(2)
MATH	Kimi-K2	0.600	0.259	0.457	0.331	0.360	0.459	0.403
	R1-0528	0.549	0.178	0.553	0.269	0.640	0.587	0.612
	QwQ-32B	0.528	0.332	0.456	0.384	0.780	0.578	0.664
	Qwen3-4B	0.485	0.461	0.392	0.423	0.830	0.548	0.660
	Qwen3-8B	0.430	0.352	0.410	0.379	0.780	0.591	0.672
	R1-Distill-32B	0.406	0.526	0.403	0.457	0.925	0.564	0.701

Appendix

Table 9: Llama-3.2-1B: per-teacher Recall, Precision, and F1 across datasets and conditions, sorted by descending FOTU within each dataset. Positive class = incorrect trace (wrong-as-positive). Recall = TP/(TP+FN); Precision = TP/(TP+FP); F1 = harmonic mean. Condition (1) and (2) per dataset: MATH Injection / Shared Fail; LSAT Struct Inj / GT=E; GPQA Struct Inj / Shared Fail.

Dataset	Phi $\times$ Llama	Phi $\times$ OLMo	Llama $\times$ OLMo
MATH	$+0.797^{**}$	$+0.964^{***}$	$+0.800^{**}$
GPQA	$+0.958^{***}$	$+0.818^{**}$	$+0.825^{***}$
LSAT	$+0.930^{***}$	$+0.863^{***}$	$+0.808^{**}$
Mean	$+0.895$	$+0.882$	$+0.811$

Appendix

Table 10: Cross-student FOTU rank stability (Spearman

\rho

n=12

canonical teachers). Rows are datasets; columns are student-model pairs.

{}^{*}p<0.05

{}^{**}p<0.01

{}^{***}p<0.001

Metric	GPQA–LSAT	GPQA–MATH	LSAT–MATH	Mean
Redundancy Fraction	0.923	0.923	0.832	0.893
Token Length	0.916	0.885	0.783	0.861
First Order Transfer Utility	0.776	0.748	0.895	0.807
Second Order Transfer Utility	0.895	0.762	0.811	0.823
Backtrack Steps	0.923	0.713	0.629	0.755
PU Regression Rate	0.818	0.559	0.364	0.580

Appendix

Table 11: Cross-dataset rank stability: Spearman

\rho

of teacher rankings across dataset pairs (Phi-3-Mini student,

n=12

teachers).

RM	FOTU A	FOTU C	SOTU A	SOTU C	RR A	RR C
Skywork	$-0.262$	$-0.076$	$+0.494^{**}$	$+0.442^{**}$	$+0.056$	$+0.133$
Allenai	$-0.304^{\dagger}$	$-0.090$	$+0.464^{**}$	$+0.197$	$+0.328^{\dagger}$	$+0.473^{**}$
Qwen-PRM	$+0.016$	$-0.046$	$+0.004$	$+0.212$	$-0.460^{**}$	$-0.586^{***}$
Mean	$\mathbf{-0.183}$	$\mathbf{-0.071}$	$\mathbf{+0.321}$	$\mathbf{+0.284}$	$\mathbf{-0.025}$	$\mathbf{+0.007}$

Appendix

Table 12: Spearman correlations between reward-model scores and transfer-utility metrics at the (dataset, model) level (

n=36

). “All” uses RM scores averaged over all traces; “Correct” restricts to the 1,761 problems answered correctly by all 12 teachers. Qwen2.5-Math-PRM-7B is a process reward model evaluated across all three datasets.

{}^{\dagger}p<.10

{}^{*}p<.05

{}^{**}p<.01

{}^{***}p<.001

Figure 5: Trace-level relationship between reward model scores and first-order transfer utility.

MATH
Model (Acc)	Len.	Red.	Back.	FOTU	SOTU	Regression Rate
gpt-oss-120b (96)	581	4.7%	0.9	34.7%	84.7%	5.0%
gpt-oss-20b (94)	748	5.6%	1.6	35.9%	87.5%	5.9%
kimi-k2 (93)	2164	27.0%	3.5	60.0%	83.3%	5.4%
qwen3-8b (90)	2377	16.4%	6.3	43.0%	83.3%	8.2%
qwen3-4b (89)	2092	17.7%	5.6	48.5%	79.8%	11.7%

Appendix

Table 13: Model comparison by axis. Model (Acc) uses mean accuracy integers. Len.: Token Length; Red.: Redundancy; Back.: Backtracking; FOTU: first-order TU; SOTU: second-order TU; Regression Rate: TU regression rate. Bold denotes best performance; underline denotes second-best.

Figure 6: Correlation matrix of all metrics across all datasets. Correlation values refer to Spearman’s rankwise $\rho$ , calculated at the model-level by computed across models and averaged over datasets.

Large Reasoning Models

Large reasoning models are language models trained or prompted to solve difficult tasks through extended, structured computation.

See all 338 papers

Measuring and curing reasoning rigidity: from decorative chain-of-thought to genuine faithfulness
Mar 24, 2026Abhinaba Basu, Pavan ChakrabortyLanguage Model Reasoning Evaluation Structured Reasoning in Language Models
Indian Institute of Information Technology Allahabad (IIITA) · National Institute of Electronics and Information Technology (NIELIT)
Risky Business: Measuring The Faithfulness-Safety Tension
Aug 4, 2026Dominik Meier, Luca Joshua Francis, Marco Bernhard Kaiser +3CoT Faithfulness Language Model Safety Evaluation
University of Göttingen · 2Landeskriminalamt NRW
Monitorability Disposition in Large Reasoning Models
Oct 4, 2026Shahriar Golchin, Marc WetterAI Agent Monitoring LLM Auditing
Scale AI · Applied Machine Learning Research, Labelbox

Mar 24, 2026J/K move · Enter open · S save

Measuring and curing reasoning rigidity: from decorative chain-of-thought to genuine faithfulness

Abhinaba Basu, Pavan Chakraborty

Indian Institute of Information Technology Allahabad (IIITA) · National Institute of Electronics and Information Technology (NIELIT)

Language Model Reasoning Evaluation Structured Reasoning in Language Models CoT Faithfulness Faithfulness of Language Model Reasoning Faithfulness of Language Model Explanations RL for Language Model Reasoning+1

Open paper AskPDF arXiv

Language models increasingly show their work by writing step-by-step reasoning before answering. But are these steps genuinely used, or is the answer rigid - fixed before reasoning begins? We introduce the Step-Level Reasoning Capacity (SLRC) metric and prove it is a consistent causal estimator (Theorem 1). We propose LC-CoSR, a training method with Lyapunov stability guarantees that directly reduces rigidity. Evaluating 16 frontier models (o4-mini, GPT-5.4, Claude Opus, Grok-4, DeepSeek-R1, Gemini 2.5 Pro, and others) across six domains at N=133-500, we find reasoning falls into three modes. OpenAI's o4-mini shows 74-88% step necessity on five of six tasks (73.8-88.3%) - the highest SLRC in our study. The critical differentiator is RL-based reasoning training, not thinking tokens: Grok-4's reasoning mode shows lower faithfulness than its non-reasoning mode (1.4% vs 7.2% necessity). We discover a faithfulness paradox - high-SLRC models are more susceptible to sycophancy - and propose the Reasoning Integrity Score (RIS = SLRC x (1-Sycophancy)), which significantly predicts error detection (rho=0.66, p=0.026). LC-CoSR achieves 2.6x less negative reward than FARL and CSR baselines without external model dependencies.

Abstract

Large Reasoning Models

Explore similar work

Measuring and curing reasoning rigidity: from decorative chain-of-thought to genuine faithfulness

Risky Business: Measuring The Faithfulness-Safety Tension

Monitorability Disposition in Large Reasoning Models