VERS: Steering Reasoning Models by Editing Their Activation Explanations

Ihor Vitenko

cs.CLNov 2, 2026

VERS: Steering Reasoning Models by Editing Their Activation Explanations

Authors: Ihor Vitenko

Abstract

Reasoning language models write a chain of thought before they answer, and many checks in it do not change the final answer. Natural language autoencoders (NLAs) explain a model's activations in text and rebuild activations from text, and editing an explanation can steer a model at a single token. We study whether such edits can also steer how a model reasons. We build VERS (Verbalize, Edit, Reconstruct, Steer): it turns explanation edits at the doubt points of one reasoning trace into one fixed direction, and adds this direction at paragraph breaks. On the full GSM8K test, VERS cuts generated tokens by 45.8% at an accuracy change of -0.82 percentage points (pp); on MATH-500 it saves 31.3% at -3.05 pp. At half strength, its trade-off between tokens and accuracy lies between those of the published methods SEAL and DEER. With fitted affine maps and a strength tuned for each model, the direction transfers to larger Qwen3 models, up to 235B-A22B, and saves 30-35% of tokens. On competition problems, however, VERS loses 3-7 pp at every model size, and a fixed direction did not separate necessary checks from unneeded ones.

Figures & tables

Figure 1: A natural language autoencoder. The AV turns the layer- $l$ activation $h_{l}$ of the target model into a text description. The AR reads the description and rebuilds the activation as $\hat{h}_{l}$ . [ 4 ]

Panel of Figure 2 — Figure 2: The two parts of VERS. Top: offline, we build the direction $N$ from the donor sites (Section 3.2 ). Bottom: during generation, we add $N$ at eligible paragraph boundaries. Generation calls neither the AV nor the AR.

	Next paragraph starts with
Word in explanation	“Wait”	other doubt marker	no marker
“Wait”	43.6%	12.1%	1.7%
“answer”	61.6%	58.1%	22.5%
“check”	46.7%	53.3%	20.8%
“correct”	41.2%	37.4%	8.7%
“continuation”	10.0%	12.8%	32.9%

Table 1: Share of AV explanations that contain each word (whole word, any case), at three boundaries of similar depth in the same 289 correct traces. Each column is a separate set of 289 explanations. One explanation can contain several words, so the values do not sum to 100%. Paired McNemar tests against the plain boundary give

p<10^{-6}

for every row (for “Wait”, the boundary before “Wait”; for the other words, the boundary before another doubt marker).

Added at the boundary	Before ordinary paragraph	Before “Wait”
Nothing (base)	8.8%	96.5%
Doubt edit	35.1%	96.7%
Continue edit	6.5%	74.2%
Random vector, same length	6.5%	94.1%

Table 2: One edit at one boundary (

\alpha=0.25

). Each value is the share of 60-token continuations whose next paragraph starts with doubt. 200 boundaries of each kind; 4 samples per ordinary boundary and 8 per boundary before “Wait”.

Questions	Base acc.	Tokens saved	$\Delta$ accuracy	Tokens saved	$\Delta$ accuracy
		VERS ( $N$ )		Concision prompt
All 1,319	90.3%	45.8%	$-0.82\ [\mathbf{-1.42}\mathord{\mathchar 59\relax}{-}0.21]$	34.9%	$-0.09\ [-0.83\mathord{\mathchar 59\relax}{+}0.63]$
gsm997 part	99.1%	42.6%	$-1.03\ [-1.50\mathord{\mathchar 59\relax}{-}0.58]$	36.6%	$-0.98\ [-1.55\mathord{\mathchar 59\relax}{-}0.45]$
Other 322	63.3%	51.0%	$-0.16\ [-2.25\mathord{\mathchar 59\relax}{+}1.86]$	32.1%	$+2.64\ [{+}0.08\mathord{\mathchar 59\relax}{+}5.36]$

Table 3: The full GSM8K test (gsm1319, four samples per question, one run). Each method is compared with base from the same run. Accuracy changes are in pp with 95% bootstrap intervals over questions.

Dataset	Method	Mean length	Tokens saved	Detected doubt reduced	$\Delta$ accuracy [CI], pp
gsm997	Base (98.92%)	1,813	reference
	VERS ( $N$ )	1,035	42.9%	90.4%	$-0.78\ [\mathbf{-1.29}\mathord{\mathchar 59\relax}{-}0.27]$
	$D$	1,327	26.8%	66.5%	$-0.30\ [-0.72\mathord{\mathchar 59\relax}{+}0.12]$
	Concision prompt	1,143	37.0%	52%	$-0.55\ [-1.10\mathord{\mathchar 59\relax}{-}0.01]$
	Random (3 vectors)	1,813–2,156	$-18.9$ to 0.0%	$-25$ to 18%	$-0.2$ to $+0.1$
MATH-500	Base (91.15%)	5,602	reference

Table 4: Frozen-policy results. VERS rows are shaded and the main results are in bold. VERS saves the most tokens on both datasets (42.9% and 31.3%) and removes the most detected doubt. On gsm997 its loss stays within the

-2

pp tolerance (lower end

-1.29

, bold). On MATH-500 the saving costs 3.05 pp, more than

D

or the prompt lose. Random vectors with the same strength make outputs longer, not shorter. Mean length counts all generated tokens (thinking and final answer). “Detected doubt reduced”: share of doubt paragraphs (English marker rule, Section 3.2 ) that disappear relative to base. Accuracy changes have 95% intervals for the paired per-question difference: from a normal approximation on gsm997, and from resampling questions (bootstrap) on MATH-500 (Section 4.1 ). Each dataset is compared with its own base. “Random” adds three random unit vectors, one at a time, with the same

\alpha

and schedule as VERS. Its row gives the range over the three vectors, which have no intervals. Negative values mean longer output or more detected doubt.

Figure 5: Tokens saved and accuracy change of each method compared with base, on confirm400 (Qwen3-1.7B, 400 questions, four samples each). Up and to the right is better. Bars are 95% paired bootstrap intervals over questions, in both directions. The dashed line is the

-2

pp tolerance. VERS uses delay 256. DEER’s cost includes its trial-answer tokens. These intervals compare each method with base; the pairwise comparisons in the text use Table 18 .

Direction added in the larger model	8B	30B-A3B	235B-A22B
VERS: 1.7B direction $N$ , mapped	31.9%	30.2%	35.1%
Difference of means $D$ , built in the larger model	26.7%	25.9%	not run ‡
Difference of means $D$ from the 1.7B, mapped	22.0%	30.6% †	not run ‡
Random 1.7B direction, mapped	$-6.6$ %	$-7.2$ %	$-3.5$ % †

Table 6: Does the saving in larger models come from

N

, or from the maps? Only

N

saves this much. A random direction sent through the same maps makes outputs longer, and

D

saves less, whether it is built in the larger model or mapped from the 1.7B. Values are tokens saved relative to each model’s base on confirm400; negative values mean longer output. † On development data, not confirm400. On the same 30B-A3B development questions, mapped

N

saves 42.4% at

-0.2

pp, while the mapped 1.7B

D

saves 30.6% at

-1.9

pp. ‡ The 235B-A22B test covered only

N

and a random control.

Traces judged	Base	VERS	Difference (pp)
All levels (200 pairs)	85.0%	83.5%	$-1.5$ $[-6.5\mathord{\mathchar 59\relax}{+}3.5]$
Level 5 (40 pairs)	75.0%	67.5%	$-7.5$

Table 8: Share of correct MATH-500 traces that a blind judge accepts as a worked example. The 95% interval is given for all levels together.

Test set	Steering	$\Delta$ acc (pp)	Tokens saved
sel400	$N$	$-3.3$ $[-4.9\mathord{\mathchar 59\relax}{-}1.6]$	29.5%
	$N$ built only from redundant doubt	$-3.7$ $[-5.4\mathord{\mathchar 59\relax}{-}2.0]$	30.2%
gate400	Random skipping (control)	reference	29.5%
	Gate with probe B	$-1.0$ $[-2.7\mathord{\mathchar 59\relax}{+}0.6]$	20.1%

Table 9: Two selective constructions on hard MATH questions (sel400 and gate400: 400 level-4 and level-5 MATH test questions each, four samples per question). Tokens saved is relative to base.

\Delta

acc is relative to base on sel400, and relative to random skipping on gate400.

$\alpha$	Longer output	$\Delta$ doubt paragraphs	Longer output	$\Delta$ doubt paragraphs
	Doubt-edit direction $P$		Reversed direction $-N$
0.25	3.4%	$+0.17$ $[-1.75\mathord{\mathchar 59\relax}{+}2.15]$	35.2%	$+24.5$ $[{+}21.1\mathord{\mathchar 59\relax}{+}28.4]$
0.5	10.3%	$+1.26$ $[-0.65\mathord{\mathchar 59\relax}{+}3.22]$	73.2%	$+50.9$ $[{+}45.6\mathord{\mathchar 59\relax}{+}56.9]$
1.0	14.7%	$-1.13$ $[-2.97\mathord{\mathchar 59\relax}{+}0.70]$	229.8%	$+194.6$ $[{+}179.1\mathord{\mathchar 59\relax}{+}212.2]$

Table 10: Steering toward doubt on dev400.

\Delta

doubt paragraphs is the change in detected doubt paragraphs per rollout, with 95% intervals; base has about 27–29. Each direction is compared with the base model from its own run.

P

makes outputs longer, but every interval for the change in doubt includes zero.

-N

adds doubt at every strength.

	fresh1000 (mixed)		gate400 (hard)
Student	$\Delta$ acc (pp)	Tokens saved	$\Delta$ acc (pp)	Tokens saved
Short-SFT †	$-0.52$	0.2%	$-0.88$	2.1%
Steered-SFT	$-0.85$	33.2%	$-1.44$	28.7%
DPO-natural †	$-0.12$	8.5%	$-0.25$	7.7%
DPO-steered †	$-0.40$	9.7%	$-0.50$	9.7%

Table 11: Hook-free 8B students, compared with the base model before student fine-tuning. Shading marks the student with the largest token saving (Steered-SFT). † Passes the

-2

pp accuracy rule on both sets. Intervals and paired comparisons are in Appendix G .

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Direction construction	Mean tokens	Accuracy	Detected doubt paragraphs
No steering (base)	2,158.3	92.5%	13.46
35 sites, one trace, normalized ( $N$ )	1,282.7	96.7%	0.69
40 traces, one random site each, normalized	1,250.8	95.8%	0.44
40 traces, first site of each, normalized	1,288.8	92.5%	0.88
465 sites, 30 traces, normalized	1,344.3	92.5%	1.13
465 sites, 30 traces, raw	1,289.6	92.5%	0.81

Appendix

Table 12: Construction variants on cons120 (one sample per question). “Normalized” averages unit difference vectors and then scales the mean to unit length, as for the deployed

N

(Eq. 2 ). “Raw” averages the unscaled difference vectors. The 297-site row uses dpairs297, the 297 traces from which we also build

D

. Detected doubt paragraphs are the mean count per answer, by the English marker rule of Section 3.2 . With one sample on 120 questions, accuracy differences of this size (for example, 96.7% vs 92.5%) are within noise.

Name	Source	Size	Samples	Purpose	Used in
Test cohorts
gsm997	GSM8K test questions not used in earlier experiments; all solved 4/4 in the census	997	4	Main frozen-policy test	§ 4.3
gsm1319	All GSM8K test questions: gsm997 and the 322 others	1,319	4	Frozen-policy test on natural GSM8K	§ 4.3
MATH-500	MATH-500 (levels 1–5: 43/90/105/128/134)	500	4	Frozen-policy test; levels	§ 4.3 , § 6.1
confirm400	MATH test minus MATH-500; same level mix	400	4	Transfer test; comparators	§ 4.4 , § 5 , App. D , G
hard128	AIME 2024 (30), AIME 2025 (28; the donor-trace problem and one development problem left out), AMC 2023 (40), HMMT Feb. 2025 (30)	128	8	Competition stress test	§ 6.1

Appendix

Table 13: Cohorts used in this paper. “Samples” is the number of generated samples per question and method. “Fixed” marks saved traces that we only read, with no new generation. “4+4” means four base and four steered teacher traces per question. The census is an earlier screening run that sampled every GSM8K question four times; a band is a difficulty band from this census (see below). Token budgets are 16,384 for gsm997, gsm1319, dev200, and dev400; 32,768 for MATH-500, confirm400, gate400, and hard128; 12,288 for cont387 and dom150; and 6,144 for cons120. The test cohorts gsm997, MATH-500, confirm400, gate400, sel400, and fresh1000 share no question text with each other or with dev400. gsm1319 contains gsm997 and the 86 GSM8K test questions of dev400. gate400 is one question set used twice: as the probe-gate test for the 1.7B and as the hard test for the 8B students.

Policy	Mean length	Tokens saved	$\Delta$ accuracy [CI], pp
Base	3,856	reference
$N$ , delay 0	1,981	48.6%	$-4.0\ [-9.9\mathord{\mathchar 59\relax}{+}1.9]$
$N$ , delay 256	1,870	51.5%	$+2.5\ [-2.6\mathord{\mathchar 59\relax}{+}7.6]$
$N$ , delay 512	2,049	46.9%	$-1.0\ [-5.8\mathord{\mathchar 59\relax}{+}3.8]$
$N$ , delay 1,024	2,152	44.2%	$+1.5\ [-2.3\mathord{\mathchar 59\relax}{+}5.3]$

Appendix

Table 14: Delay ablation for

N

\alpha=1

, on dev200 (one sample per question; base accuracy 55.0%). Delay 0 means the first eligible paragraph boundary, not the first generated token. Length counts all generated tokens. Accuracy changes have paired 95% intervals from a normal approximation; positive values favor the intervention.

Setting	Mean length	Tokens saved	Detected doubt reduced	$\Delta$ accuracy [CI], pp
Base	3,716	reference
$N$ , $\alpha=1$ , $d=256$	1,902	48.8%	89%	$-0.4\ [-3.4\mathord{\mathchar 59\relax}{+}2.7]$
$N$ , $\alpha=1$ , $d=1{\mathord{\mathchar 59\relax}}024$	2,174	41.5%	73%	$-2.0\ [-4.3\mathord{\mathchar 59\relax}{+}0.3]$
$N$ , $\alpha=0.25$ , $d=256$	2,825	24.0%	52%	$-0.6\ [-3.3\mathord{\mathchar 59\relax}{+}2.1]$
$N$ , $\alpha=0.25$ , $d=1{\mathord{\mathchar 59\relax}}024$	2,974	20.0%	43%	$-1.1\ [-3.4\mathord{\mathchar 59\relax}{+}1.1]$
$D$ , $\alpha=0.5$ , $d=512$	2,495	32.9%	72%	$-1.9\ [-4.5\mathord{\mathchar 59\relax}{+}0.7]$

Appendix

Table 15: Stage 2 of policy selection on dev400 (two samples per question; base accuracy 55.8%). Chosen settings are shaded. Length counts all generated tokens. “Detected doubt reduced”: share of doubt paragraphs (English marker rule, Section 3.2 ) that disappear relative to base. Accuracy changes have paired 95% intervals.

Paragraph position	Base	Steered	Random vector	Doubt removed
	Paragraphs that start with doubt
1st–5th	16.0%	0.0%	16.2%	100%
6th–10th	41.8%	0.4%	40.2%	99%
11th–15th	39.0%	0.3%	41.5%	99%
16th–20th	40.7%	0.0%	42.9%	100%
21st–30th	41.6%	0.5%	43.0%	99%

Appendix

Table 16: Share of paragraphs that start with a doubt marker (Section 3.2 ), by position in the answer. Each row groups paragraphs by their number in the answer. For example, 41.8% of the 6th to 10th paragraphs of base answers start with doubt, but only 0.4% of steered ones. “Doubt removed” is

1-\text{steered}/\text{base}

, here 99%. If the effect of steering faded over a long answer, the steered values would rise in the lower rows. They stay near 0% down to the 50th paragraph.

Method	Mean length	Tokens saved	Detected doubt reduced	$\Delta$ accuracy [CI], pp
Base (accuracy 91.4%)	5,520	reference
VERS, $\alpha=1$	3,744	32.2%	89.9%	$-3.12\ [-4.62\mathord{\mathchar 59\relax}{-}1.69]$
VERS, $\alpha=0.5$	3,877	29.8%	83.0%	$-1.62\ [-3.00\mathord{\mathchar 59\relax}{-}0.31]$
Matched ban	4,151	24.8%	88.5%	$-2.62\ [-4.00\mathord{\mathchar 59\relax}{-}1.31]$
Global ban	3,836	30.5%	99.1%	$-5.06\ [-7.00\mathord{\mathchar 59\relax}{-}3.25]$
SEAL, default	4,617	16.4%	66.3%	$-1.06\ [-2.38\mathord{\mathchar 59\relax}{+}0.25]$

Appendix

Table 17: Comparator results on confirm400 (four samples per question). Mean length counts all generated tokens, including DEER’s trial answers. “Detected doubt reduced” is the share of doubt paragraphs, detected by the English marker rule of Section 3.2 , that disappear relative to base. Accuracy changes and 95% intervals are relative to the base of this run. The SEAL vector on VERS’s schedule uses

\alpha=0.5

VERS	Other method	Tokens saved (VERS vs other)	Difference in tokens saved [CI]	$\Delta$ accuracy [CI], pp
$\alpha=0.5$	Matched ban	29.8% vs 24.8%	$+4.96\ [+1.66\mathord{\mathchar 59\relax}{+}8.40]$	$+1.00\ [-0.38\mathord{\mathchar 59\relax}{+}2.38]$
$\alpha=1$	Matched ban	32.2% vs 24.8%	$+7.38\ [+3.49\mathord{\mathchar 59\relax}{+}11.52]$	$-0.50\ [-2.01\mathord{\mathchar 59\relax}{+}0.94]$
$\alpha=0.5$	SEAL, tuned	29.8% vs 18.6%	$+11.12\ [+7.41\mathord{\mathchar 59\relax}{+}14.87]$	$+0.12\ [-1.31\mathord{\mathchar 59\relax}{+}1.62]$
$\alpha=0.5$	SEAL vector, VERS schedule	29.8% vs 19.5%	$+10.27\ [+6.99\mathord{\mathchar 59\relax}{+}13.85]$	$+0.62\ [-0.81\mathord{\mathchar 59\relax}{+}2.06]$
$\alpha=0.5$	SEAL, default	29.8% vs 16.4%	$+13.39\ [+10.42\mathord{\mathchar 59\relax}{+}16.40]$	$-0.56\ [-2.06\mathord{\mathchar 59\relax}{+}0.88]$
$\alpha=1$	DEER	32.2% vs 37.6%	$-5.43\ [-10.28\mathord{\mathchar 59\relax}{-}0.75]$	$-0.19\ [-1.88\mathord{\mathchar 59\relax}{+}1.62]$

Appendix

Table 18: Pairwise comparisons on confirm400, VERS minus the other method. The difference in tokens saved is VERS’s saving minus the other method’s saving, in points of base tokens. It is computed before rounding, so it can differ by 0.1 from the difference of the rounded savings. A positive accuracy difference favors VERS.

Method	Tokens per accepted rollout
Base	13,084
SEAL, default	11,084
VERS, $\alpha=0.5$	9,688
DEER	9,083

Appendix

Table 19: Tie-break on confirm400: tokens per accepted rollout in the check-and-retry pipeline (lower is cheaper). Coverage is 96.25% for VERS, SEAL, and DEER.

What the model does after its answer settles	Accuracy	Thinking tokens	$\Delta$ accuracy [CI], pp
(1) Return the settled answer	39.5%	0	$-13.7\ [-17.8\mathord{\mathchar 59\relax}{-}9.6]$
(2) Close thinking and write an answer	49.4%	0	$-3.9\ [-7.3\mathord{\mathchar 59\relax}{-}0.5]$
(3) Keep thinking normally (base)	53.2%	2,388	reference
(4) Keep thinking with VERS ( $N$ )	53.0%	588	$-0.3\ [-3.2\mathord{\mathchar 59\relax}{+}2.7]$
(5) Keep thinking with $D$	54.5%	1,468	$+1.3\ [-1.3\mathord{\mathchar 59\relax}{+}3.8]$

Appendix

Table 20: Continuation value on cont387. All arms start from the same text, at the point where the answer settles. Thinking tokens are counted after this point. Accuracy changes are relative to keeping thinking normally, with 95% intervals clustered by question.

Method	Tokens	Detected doubt paragraphs	What happens
Base	3,682	13	finds $15\pi$ , then checks again and again
VERS, $\alpha=0.5$	2,603	0	keeps one geometric check, with no marker
Matched ban	3,147	0	keeps checking, in Chinese
SEAL, default	2,746	4	keeps some checks
DEER	2,153	1	stops after the area calculation

Appendix

Table 21: One MATH test problem solved by each method (seed 0). Tokens count all generated tokens; for DEER they include its trial answer. Doubt paragraphs are detected by the English marker rule of Section 3.2 .

Doubt score	8B	30B-A3B	8B	30B-A3B
	Spearman $\rho$ [95% CI]		Behavioral AUC, target / 1.7B
Doubt-word cue	$+0.15$ [0.08, 0.23]	$+0.11$ [0.04, 0.18]	0.736 / 0.687	0.685 / 0.686
$\langle\mathrm{AR}(e)\mathord{\mathchar 59\relax}N\rangle$	$+0.19$ [0.12, 0.27]	$+0.12$ [0.04, 0.21]	0.780 / 0.754	0.707 / 0.707
Blind judge	$+0.09$ [0.02, 0.16]	$+0.07$ [0.00, 0.14]	0.755 / 0.729	0.710 / 0.693

Appendix

Table 23: The reading test.

\rho

compares the doubt-score difference of the two explanations with the difference in the two models’ probability of doubting next. The behavioral AUC (see text) compares explanations of the target state with explanations of the 1.7B state on the same text.

Figure 6: VERS at its primary strength on hard128, for each model size. Left: generated tokens saved (%). Right: accuracy change from the same model’s base (pp). Error bars are pointwise 95% intervals from a paired bootstrap over questions. The dashed line marks the

-2

pp non-inferiority margin.

Label	Forced answer, before $\rightarrow$ after	Episodes	Share
Productive	wrong $\rightarrow$ right	769	1.8%
Redundant	right $\rightarrow$ right, same answer	18,026	41.4%
Failed	wrong $\rightarrow$ wrong	24,188	55.6%
Harmful	right $\rightarrow$ wrong	520	1.2%
Other	right $\rightarrow$ right, different form	10	0.02%
Excluded	no usable answer before	26	0.06%

Appendix

Table 24: Labels of the 43,539 doubt episodes in probe1000. Useful doubt is rare: only 1.8% of episodes fix a wrong answer.

	Probe A	Probe B
Read at	the boundary just before a doubt paragraph	every paragraph boundary inside thinking
Predicts	productive or redundant episode	a productive episode starts next
Examples	18,795 episodes (769 productive, 18,026 redundant); failed and harmful left out	150,501 boundaries; 769 positive (about 0.5%)
Evaluation	5-fold cross-validation, grouped by question	held-out questions (80/20 split by question)
AUC [95% CI]	0.75 [0.718, 0.782]	0.84 [0.809, 0.873]
Use	selective directions	the gate

Appendix

Table 25: The two logistic-regression probes on the layer-20 state, both fit on probe1000.

Steering	Productive	Redundant
Base (no steering)	0.735	21.125
VERS ( $N$ )	0.115	1.255
Random skipping	0.24	2.91
Gate with probe B	0.48	7.595

Appendix

Table 26: Doubt episodes per rollout on gate400, counted on a sample of 200 rollouts per arm, with the labels of Table 24 . The gate keeps more productive episodes than random skipping, but it keeps even more redundant ones.

$\alpha$	$\Delta$ accuracy, pp	Longer output	Doubt paragraphs	At token limit
0.25	$-0.62$ $[-3.50\mathord{\mathchar 59\relax}{+}2.25]$	35.2% $[29.2\mathord{\mathchar 59\relax}41.7]$	53.2	1.25%
0.5	$-1.38$ $[-4.25\mathord{\mathchar 59\relax}{+}1.50]$	73.2% $[65.2\mathord{\mathchar 59\relax}82.3]$	79.6	4.00%
1.0	$-5.00$ $[-8.00\mathord{\mathchar 59\relax}{-}2.38]$	229.8% $[208.5\mathord{\mathchar 59\relax}254.2]$	223.2	55.25%

Appendix

Table 27:

-N

on dev400, compared with base from the same run (base: 54.2% accuracy, 3,850 tokens, 28.7 detected doubt paragraphs per rollout, 0.75% of outputs at the token limit). Intervals are 95% bootstrap intervals over questions.

Pair type	Preferred trace	Rejected trace, DPO-natural	Rejected trace, DPO-steered
Length	shortest correct trace of the arm (natural or steered)	longest correct natural trace	longest correct natural trace
Correctness	the length pair’s preferred trace (if there is no length pair, a random correct trace of the arm)	longest wrong natural trace	longest wrong natural trace
Anti-premature	a random correct natural trace (the same in both arms)	shortest wrong natural trace that is not longer	shortest wrong steered trace that is not longer

Appendix

Table 28: The three DPO pair types. “Correct” means usable (correct, finished, and within 16,384 tokens). The arms differ in two places. In length and correctness pairs, DPO-steered prefers a steered trace. In anti-premature pairs, DPO-steered rejects a wrong steered trace. Anti-premature pairs teach the student not to stop too early with a wrong answer. Because of them, the steered arm also contains some natural traces as the preferred trace.

Set	Student	Accuracy	Mean tokens	$\Delta$ acc vs base, pp [95% CI]
fresh1000	Base	97.70%	4,578	reference
	Short-SFT	97.18%	4,568	$-0.52$ [ $-1.03\mathord{\mathchar 59\relax}{-}0.07$ ]
	Steered-SFT	96.85%	3,060	$-0.85$ [ $-1.35\mathord{\mathchar 59\relax}{-}0.35$ ]
	DPO-natural	97.58%	4,188	$-0.12$ [ $-0.50\mathord{\mathchar 59\relax}{+}0.25$ ]
	DPO-steered	97.30%	4,136	$-0.40$ [ $-0.80\mathord{\mathchar 59\relax}{-}0.02$ ]
gate400	Base	96.44%	6,686	reference

Appendix

Table 29: 8B students without a hook. Many evaluation samples do not replace independent training seeds.

Paired comparison	fresh1000 [95% CI]	gate400 [95% CI]
Steered-SFT vs Short-SFT, $\Delta$ acc (pp)	$-0.32$ [ $-0.85\mathord{\mathchar 59\relax}{+}0.23$ ]	$-0.56$ [ $-1.62\mathord{\mathchar 59\relax}{+}0.44$ ]
Steered-SFT vs DPO-natural, $\Delta$ acc (pp)	$-0.72$ [ $-1.22\mathord{\mathchar 59\relax}{-}0.22$ ]	$-1.19$ [ $-2.19\mathord{\mathchar 59\relax}{-}0.25$ ]
DPO-steered vs DPO-natural, $\Delta$ tokens	$-1.24$ % [ $-2.79\mathord{\mathchar 59\relax}{+}0.37$ ]	$-2.21$ % [ $-4.45\mathord{\mathchar 59\relax}{-}0.02$ ]
Hooked teacher vs its base, $\Delta$ acc (pp)	$-0.02$ [ $-0.50\mathord{\mathchar 59\relax}{+}0.45$ ]	$-0.31$ [ $-1.31\mathord{\mathchar 59\relax}{+}0.69$ ]

Appendix

Table 30: Main paired comparisons. The hooked teacher runs with the hook and has its own base run, so its row is a diagnostic comparison only.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 388 papers

Oct 5, 2026

COMPASS: Finding Where Reasoning Lives in Language Models

Explicitly eliciting reasoning substantially improves LLM performance. Existing approaches require a predefined characterization of reasoning, whether through CoT prompt design, contrastive CoT directions, or via SAE derived reasoning features. For mathematical reasoning with verifiable answers, we show that a much simpler signal suffices, which is the correctness of the model's own direct answer attempts. This signal yields a latent direction that elicits reasoning. This direction is decodable within the activations of most attention heads, but only a small subset of them can be effectively intervened. We introduce COMPASS, an inference-time steering method that identifies these heads using a logit-space attribution score and steers their activations along the correctness direction, requiring only per-head activation statistics. Across three model families and multiple math benchmarks, COMPASS outperforms the activation-steering baselines we compare against, improves GSM8K accuracy by 16 percentage points on average, and approaches CoT accuracy with 20-70% fewer generated tokens. Interventions transfer without re-fitting to unseen benchmarks, and ablations show that both the correctness direction and the small set of heads carrying it are necessary, with the effect concentrated in remarkably few heads.

Pratyay Dutta, Kowshik Thopalli, Vivek Narayanaswamy

University of California, Riverside, CA 92521. Work performed during a summer internship at Lawrence Livermore National Laboratory · Lawrence Livermore National Laboratory, Livermore, CA 94550.

Efficient Language Model Reasoning Activation Steering LLM Interpretability Numerical Reasoning in Language Models

Ask PDF

Oct 13, 2026

DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

Reasoning models achieve strong performance through long chains of thought (CoT), but often spend more tokens than needed to solve a problem. Existing ways to shorten the CoT require additional training for each model, add a fixed steering vector to the model's hidden states, or use decoding heuristics such as stopping early or banning words like "wait", which can reduce accuracy. We propose DoubtLess, a training-free decoding method in which a much smaller reasoning model steers the large one towards brevity. The small model reads the large model's CoT so far twice, once prompted to be brief (the brief thinker) and once prompted to be verbose (the verbose thinker), and the difference between its two logit vectors, scaled by a strength, is added to the large model's logits at every step of the CoT. On seven benchmarks of competition mathematics, graduate-level science and competitive programming, DoubtLess shortens the CoT of Qwen3-32B by 31% with unchanged accuracy, outperforming five inference-time baselines. We also show that DoubtLess reduces thinking tokens per turn by 20% in a self-debugging coding agent with unchanged task performance. It also generalizes to two other model families with unchanged accuracy. When combined with training-based compression, it further shortens the CoT of a model already trained for brevity. Ablations and behavioral analysis show that DoubtLess reduces repeated checking, with most token savings coming after the correct answer first appears in the CoT. Our code is available at https://github.com/leoluolol/DoubtLess.

Minghao Luo, Liang Chen

The Chinese University of Hong Kong · EPFL

Efficient Language Model Reasoning CoT Compression Contrastive Decoding

Ask PDF

Oct 14, 2026

To Reason but Not to Stop: Understanding and Recovering Pruned Large Reasoning Models

Weight pruning can degrade the accuracy of large reasoning models (LRMs), yet it remains unclear whether it damages reasoning itself. We find that much of this degradation stems from a failure to stop rather than to reason. As sparsity increases, pruned LRMs increasingly fail to terminate within the token budget, often after the correct answer has appeared in their reasoning, instead falling into repetitive loops, even though they still recognize where the dense model would stop. We further find that the hidden-state difference between terminating and looping continuations is consistent across pruned checkpoints, suggesting a shared direction associated with committing to an answer. Building on this, we propose Termination-Guided Activation Steering (TGAS), a training-free method that adds this direction to the residual stream at inference time. Across six LRMs, three pruning pipelines, and math and code benchmarks, TGAS improves termination and overall accuracy while reducing repetitive reasoning and generation length, showing that this failure is actionable.

Minh-Anh Nguyen, Nguyen T. Ha Lan, Xiyan Fu +2

Nanyang Technological University, Singapore · VinUniversity, Vietnam

Repetition in Language Generation LLM Reasoning Large Reasoning Models LLM Pruning Activation Steering

Ask PDF

Abstract

Efficient Language Model Reasoning

Explore similar work

COMPASS: Finding Where Reasoning Lives in Language Models

DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

To Reason but Not to Stop: Understanding and Recovering Pruned Large Reasoning Models