Concise-9B: Removing the Slack from a Reasoning Model's Thinking

Alexander Boesgaard Lorup

cs.LGNov 1, 2026

Concise-9B: Removing the Slack from a Reasoning Model's Thinking

Authors: Alexander Boesgaard Lorup

Organizations: An Openhagen technical report

Abstract

Reasoning models answer better when they think longer, but a thinking trace keeps going well after its answer has formed. We measure this slack in Qwen3.5-9B and remove most of it with concise self-training: two rounds of on-policy rejection-sampling fine-tuning that keep the shortest correct samples, plus a small share of answer-synthesis targets, giving Concise-9B. On 8,983 held-out problems (competition and hard mathematics, SuperGPQA, a five-area confirmation set and MMLU-Pro), Concise-9B generates 0.66x the tokens of Qwen3.5-9B at a 32K thinking budget, with accuracy equivalent within +/-1 point (-0.20 [-0.77, +0.36] points). Forcing each trace on the hard and confirmation sets to answer every 2,048 thinking tokens, we find that 54-64% of Qwen3.5-9B's thinking tokens come after the point from which every forced answer equals the final one (41-48% for Concise-9B). Stopping there would keep every answer at 0.35-0.44x the base model's tokens, but finding that point takes hindsight: stopping when three consecutive forced answers agree costs the base model 2.4-9.2 points. Concise self-training removes slack without that cost: 86% [83, 90] of the thinking tokens it saves come after the answer has formed. It has a limit: with traces continued to 80K tokens, Concise-9B solves 25 fewer of 1,276 hard problems (-25 [-49, -1]), mostly in competition mathematics. The measurement also yields a cheap commit signal for adaptive sampling that matched two-sample escalation within +/-1 point at 0.67x its tokens in a pre-registered test. Results on Qwen3.5-4B, on which the recipe was developed, and a replication on Olmo-3-7B-Think suggest that the saving depends on how verbose the base model is.

Figures & tables

Figure 1: Most thinking comes after the answer is fixed, and concise training removes mostly that part. (a) Mean thinking tokens per trace at a 32K budget, split at the point after which every forced answer equals the trace’s final answer; labels give the share after it ( Table 4 ). (b) Share of traces whose answer is fixed by a given fraction of their thinking (hard and confirmation sets, 3,551 traces per model). A trace whose answer only settles at its end counts at 1.0.

System	Acc.	Tokens	Acc.	Tokens	$\Delta$ [95% CI]	Ratio	Equiv.
	Qwen3.5-9B		Concise-9B
Single sample (mean of 8 draws)	86.5	16.0K	85.9	9.8K	$-3.0$ [ $-7.6$ , $+1.5$ ]	0.61 $\times$	$\pm$ 2
Single trace, 32K budget	84.8	14.2K	84.5	9.4K	$-1.5$ [ $-10.5$ , $+7.0$ ]	0.66 $\times$	$\pm$ 2
Majority of 2	86.4	31.4K	87.0	19.4K	$+3.0$ [ $-5.5$ , $+11.5$ ]	0.62 $\times$	$\pm$ 3
Majority of 3	87.9	47.8K	88.0	29.3K	$+0.5$ [ $-6.0$ , $+7.0$ ]	0.61 $\times$	$\pm$ 2
Majority of 4	88.3	63.8K	87.9	39.1K	$-2.0$ [ $-9.5$ , $+5.0$ ]	0.61 $\times$	$\pm$ 2

Table 1: Development suite (498 problems, 64K budget per trace, two sampling seeds pooled). Accuracy in %; tokens are mean generated tokens per problem, summed over the traces a system uses.

\Delta

is Concise-9B minus Qwen3.5-9B in problems, with a 95% paired interval; the last column gives the smallest margin, in points, within which the two are equivalent. Escalation (2

\to

4) draws two traces and answers if both finish and agree; otherwise it draws two more and takes the majority of four.

Figure 2: Development suite: accuracy against generated tokens per problem. Lines are majorities of

k=1,\dots,4

traces (numbers mark

k

;

k=1

is the eight-draw single-sample mean); stars are escalation (2

\to

4). Values and paired intervals are in Table 1 .

		Accuracy		Tokens		32K budget			Uncapped
Set	$n$	Qwen	Concise	Qwen	Concise	$\Delta$ [95% CI]	Equiv.	Trunc.	$\Delta$ [95% CI]
Hard set	1,276	73.2	72.9	16.9K	11.7K	$-4$ [ $-28$ , $+20$ ]	$\pm$ 2	213 / 93	$-25$ [ $-49$ , $-1$ ]
Competition mathematics	279	68.1	69.2	26.9K	21.4K	$+3$ [ $-10$ , $+16$ ]	–	124 / 70	$-15$ [ $-28$ , $-2$ ]
Hard MATH	397	98.2	97.0	11.8K	7.6K	$-5$ [ $-10$ , $0$ ]	$\pm$ 3	18 / 7	$-5$ [ $-11$ , $+1$ ]
SuperGPQA	600	59.0	58.7	15.6K	9.8K	$-2$ [ $-21$ , $+18$ ]	–	71 / 16	$-5$ [ $-24$ , $+14$ ]
Confirmation set	2,275	85.5	85.5	11.5K	7.7K	$0$ [ $-20$ , $+20$ ]	$\pm$ 1	117 / 26	$-2$ [ $-22$ , $+18$ ]

Table 2: Held-out sets: one trace per problem at a 32K thinking budget, and uncapped. Accuracy in %; tokens are mean generated tokens per problem.

\Delta

is Concise-9B minus Qwen3.5-9B in problems with a 95% paired interval; “Equiv.” is the smallest equivalence margin in points. “Trunc.” counts traces that reached the budget ( Qwen3.5-9B / Concise-9B ). In the uncapped column every truncated trace is continued from its own prefix to 80K tokens. “–” in the Equiv. column means that no margin up to

\pm

3 points passes.

Set	$n$	System	Base	Trained	$\Delta$ points [95% CI]	Ratio
Qwen3.5-4B $\to$ Concise-4B
Development suite	498	single sample	84.2	84.0	$-0.3$ [ $-1.4$ , $+0.9$ ]	0.57 $\times$
Development suite	498	majority of 4	86.8	86.7	$-0.1$ [ $-1.6$ , $+1.4$ ]	0.57 $\times$
Held-out MATH	484	single sample	98.1	97.7	$-0.4$ [ $-1.1$ , $+0.3$ ]	0.60 $\times$
Held-out MATH	484	majority of 4	99.2	99.0	$-0.2$ [ $-0.6$ , $0.0$ ]	0.60 $\times$
SuperGPQA	600	single sample	54.0	52.7	$-1.3$ [ $-3.2$ , $+0.6$ ]	0.53 $\times$

Table 3: The recipe on Qwen3.5-4B, where it was developed, and on Olmo-3-7B-Think. Accuracy in %;

\Delta

is the trained model minus its base model in points with a 95% paired interval; “Ratio” is the ratio of generated tokens. Development-suite rows pool two seeds and four traces per problem (Qwen3.5-4B, 64K budget) or two seeds (Olmo, 57,344-token budget); held-out MATH and SuperGPQA use one seed. “Single sample” is the mean over a problem’s draws, “single trace” the first trace. Olmo majorities vote over its three further traces, ties broken by trace order; Olmo intervals cluster the two seeds by problem. Concise-4B is the model on which the recipe was developed, so its development-suite rows carry selection.

Figure 3: Accuracy against thinking budget. A trace counts as correct at budget

B

if it finished by

B

with a correct answer, or if its latest readout at or before

B

is correct; open markers continue truncated traces to 80K tokens. Bands are 95% Wilson intervals [ 39 ] ; paired differences are in Table 10 .

Figure 4: Accuracy by thinking length, one trace per problem at a 32K budget. Bins hold traces that finished within the given number of thinking tokens; “cap” holds traces that reached the budget. Bins with fewer than 20 traces are not shown. Bars are 95% Wilson intervals; counts are in Table 11 .

Set	Model	All	Before	After	After (%) [95% CI]	Correct	Wrong	Cap	Median
		Thinking tokens				After, by outcome (%)
Hard set	Qwen3.5-9B	16,057	7,379	8,677	$54.0$ [ $51.9$ , $56.2$ ]	65	59	36	0.35
Hard set	Concise-9B	10,753	6,349	4,403	$41.0$ [ $38.9$ , $43.1$ ]	46	41	28	0.55
Confirmation set	Qwen3.5-9B	10,729	3,880	6,849	$63.8$ [ $62.4$ , $65.2$ ]	67	59	53	0.37
Confirmation set	Concise-9B	6,814	3,521	3,293	$48.3$ [ $46.7$ , $49.9$ ]	50	44	44	0.56
MMLU-Pro	Concise-9B	5,916	3,065	2,851	$48.2$ [ $47.1$ , $49.2$ ]	49	48	27	0.55

Table 4: Thinking before and after the answer is fixed (32K budget, one trace per problem). Mean thinking tokens per trace, split at the first readout from which every readout gives the trace’s final answer (the end of the trace if there is none). “After” is the share of all thinking tokens after that point, with a 95% interval; the next three columns give it for traces that finished correct, finished wrong, or reached the budget. “Median” is the median fraction of a trace’s thinking at which its answer is fixed. MMLU-Pro readouts exist for Concise-9B only.

Set	Model	Acc.	Tokens	Ratio	$\Delta$ points [95% CI]	Ratio	$\Delta$ points [95% CI]	Ratio
		Full trace		Fixed	3 agree		2 agree
Hard set	Qwen3.5-9B	73.2	16.9K	0.44 $\times$	$-9.2$ [ $-11.1$ , $-7.4$ ]	0.45 $\times$	$-13.7$ [ $-15.9$ , $-11.5$ ]	0.30 $\times$
Hard set	Concise-9B	72.9	11.7K	0.56 $\times$	$-6.7$ [ $-8.3$ , $-5.3$ ]	0.62 $\times$	$-12.3$ [ $-14.3$ , $-10.3$ ]	0.43 $\times$
Confirmation set	Qwen3.5-9B	85.5	11.5K	0.35 $\times$	$-2.4$ [ $-3.2$ , $-1.6$ ]	0.56 $\times$	$-4.5$ [ $-5.6$ , $-3.5$ ]	0.40 $\times$
Confirmation set	Concise-9B	85.5	7.7K	0.49 $\times$	$-1.5$ [ $-2.2$ , $-0.8$ ]	0.75 $\times$	$-3.4$ [ $-4.4$ , $-2.4$ ]	0.59 $\times$

Table 5: Stopping rules on single traces (32K budget). “Fixed” stops each trace where its answer is fixed (hindsight; accuracy unchanged by construction). “3 agree” and “2 agree” stop at the first readout that repeats the previous two (one) readouts’ answer key and return that readout (the answer-consistency rule of Liu and Wang [21] , here with

k=3

k=2

on the 2,048-token grid); traces where this never happens run to their end. Token ratios are against the same model’s full traces and include 48 tokens per readout;

\Delta

is in points against the full trace.

Figure 5: Trimming by stopping rules and by training. Accuracy change and generated tokens relative to one full Qwen3.5-9B trace. Squares stop each trace where its answer is fixed, which needs hindsight; open markers stop when three or two consecutive readouts agree. Concise-9B ’s full trace (filled diamond) sits near the hindsight bound in accuracy without any stopping rule.

Set	$n$	Qwen	Concise	Saving	$\Delta$ before [95% CI]	$\Delta$ after [95% CI]	Share after (%)	Robustness
Hard set	1,276	16,057	10,753	5,304	$1{,}030$ [ $686$ , $1{,}366$ ]	$4{,}274$ [ $3{,}926$ , $4{,}624$ ]	$81$ [ $75$ , $87$ ]	90 (694)
Confirmation set	2,275	10,729	6,814	3,915	$358$ [ $197$ , $522$ ]	$3{,}556$ [ $3{,}347$ , $3{,}768$ ]	$91$ [ $87$ , $95$ ]	95 (1,124)
Both	3,551	12,643	8,229	4,414	$600$ [ $439$ , $763$ ]	$3{,}814$ [ $3{,}630$ , $3{,}999$ ]	$86$ [ $83$ , $90$ ]	93 (1,818)

Table 6: Where Concise-9B ’s token saving comes from (32K budget, thinking tokens per trace, paired by problem).

\Delta

before and

\Delta

after are Qwen3.5-9B minus Concise-9B in the thinking before and after the answer is fixed, with 95% intervals; “Share after” is the share of the total saving that comes after the answer is fixed. The last column repeats it on problems where both traces finished naturally and are at least 4,096 thinking tokens long (

n

in parentheses).

Policy	Correct	Tokens	$\Delta$ [90% CI]	Ratio
Concise-9B, confirmation set (2,310 problems, 16K traces); margin $\pm$ 23 problems (1 point)
Single trace	1,967	7.2K	–	–
Majority of 3	1,983	21.6K	–	–
Two-sample escalation (2 $\to$ 3), comparator	1,984	16.6K	–	–
Self-agreement escalation, $m=1$	1,977	10.6K	$-7$ $[-14,\ 0]$	0.64 $\times$
Self-agreement escalation, $m=2$ (primary)	1,979	11.1K	$-5$ $[-12,\ +2]$	0.67 $\times$

Table 7: Self-agreement escalation, pre-registered tests. Correct problems and mean generated tokens per problem;

\Delta

is against two-sample escalation with a 90% paired interval (equivalence when it lies inside the stated margin), and “Ratio” its token ratio (registered limit 0.75). Readouts cost 48 tokens each. Both tests use all problems of their set, as registered, including the 35 confirmation-set and 3 hard-set problems excluded elsewhere for overlap with training; the registered evaluator uses 20,000 resamples for its intervals.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Problems
MATH training set	3,002
MATH training set, hard subset	500
GSM8K training set	1,500
MMLU test questions outside the development suite	3,000
GPQA main set, outside GPQA-Diamond	250
Total (unique problems)	8,252 (7,752)

Appendix

Table 8: Training pool. The hard MATH subset shares identifiers with MATH training problems, hence the smaller number of unique problems.

Round	Entries	Samples	Synthesis	Samples	Synthesis	Target tokens	Steps
		Collected		Kept targets
Round 1	3,000	12,000	6,000	3,833	397	14.7M	133
Round 2 (Concise-9B)	3,000	12,000	6,000	4,273	417	15.1M	147

Appendix

Table 9: The two rounds of concise self-training on Qwen3.5-9B . Each round draws 3,000 pool entries (2,945 and 2,934 distinct problems, since the hard MATH subset repeats MATH identifiers), collects four 8K samples and two synthesis passes per entry and keeps the targets described in Section 3 .

Set	Model	4K	8K	12K	16K	24K	32K	Uncapped
Hard set	Qwen3.5-9B	56.7	61.7	65.3	67.6	70.5	73.2	75.7
Hard set	Concise-9B	55.6	63.8	67.5	69.6	71.3	72.9	73.7
	$\Delta$ (problems)	$-14$ [ $-41$ , $+13$ ]	$+27$ [ $+1$ , $+53$ ]	$+28$ [ $+3$ , $+53$ ]	$+25$ [ $0$ , $+50$ ]	$+11$ [ $-14$ , $+36$ ]	$-4$ [ $-28$ , $+20$ ]	$-25$ [ $-49$ , $-1$ ]
Confirmation set	Qwen3.5-9B	77.8	82.6	83.4	84.4	85.4	85.5	85.7
Confirmation set	Concise-9B	78.1	83.6	84.4	85.3	85.6	85.5	85.6
	$\Delta$ (problems)	$+7$ [ $-23$ , $+37$ ]	$+22$ [ $-1$ , $+46$ ]	$+23$ [ $0$ , $+46$ ]	$+21$ [ $0$ , $+42$ ]	$+4$ [ $-17$ , $+25$ ]	$0$ [ $-20$ , $+20$ ]	$-2$ [ $-22$ , $+18$ ]

Appendix

Table 10: Accuracy (%) against thinking budget, from readouts (one trace per problem).

\Delta

is Concise-9B minus Qwen3.5-9B in problems with a 95% paired interval; the last column continues truncated traces to 80K tokens.

Benchmark	Model	$<$ 2K	2–4K	4–8K	8–16K	16–32K	cap
Competition mathematics	Qwen3.5-9B	–	–	(6)	100 (42)	91 (107)	37 (124)
	Concise-9B	–	(4)	94 (36)	96 (69)	77 (100)	17 (70)
Hard MATH	Qwen3.5-9B	(16)	99 (70)	98 (110)	99 (117)	100 (66)	(18)
	Concise-9B	98 (41)	100 (122)	97 (136)	99 (73)	(18)	(7)
SuperGPQA	Qwen3.5-9B	(13)	55 (38)	59 (143)	61 (188)	65 (147)	41 (71)
	Concise-9B	52 (25)	62 (152)	60 (184)	61 (152)	51 (71)	(16)

Appendix

Table 11: Accuracy (%) by thinking length, with the number of traces in parentheses (one trace per problem at a 32K budget; “cap” are traces that reached the budget).

Set	Model	Stated early (%)	Slack, text (%)	Slack, readouts (%)	Readouts agreeing (%)
Hard set	Qwen3.5-9B	50	8.1	54.0	76 / 95
Hard set	Concise-9B	30	4.5	41.0	65 / 93
Confirmation set	Qwen3.5-9B	59	10.3	63.8	86 / 98
Confirmation set	Concise-9B	39	6.0	48.3	77 / 100
MMLU-Pro	Qwen3.5-9B	61	11.2	–	–
MMLU-Pro	Concise-9B	44	6.5	48.2	80 / 96

Appendix

Table 12: Explicit answer statements in the thinking. “Stated early” is the share of traces with a stable explicit statement of the final answer before the end; “Slack” is the share of thinking tokens after it, next to the readout-based share of Table 4 ; the last column gives the share of readouts that equal the final answer before / after the statement.

Traces	$n$	Actual	Predicted	Residual [95% CI]
Qwen3.5-9B, further sample 1	1,266	922	905	$+17$ [ $-6$ , $+40$ ]
Qwen3.5-9B, further sample 2	1,264	929	897	$+32$ [ $+8$ , $+57$ ]
Qwen3.5-9B, further sample 3	1,270	912	903	$+9$ [ $-16$ , $+34$ ]
Concise-9B	1,276	930	866	$+64$ [ $+36$ , $+92$ ]

Appendix

Table 13: Calibration of the pace comparison (hard set). Predicted correct counts use Qwen3.5-9B ’s readouts at each trace’s stopping length; the three further Qwen3.5-9B traces are its 64K traces scored at 32K (a few problems without a readout at 32,768 tokens are left out). The residual is actual minus predicted, with a 95% paired interval.

Comparison	Qwen	Concise	$\Delta$ [95% CI]	90% CI	Ratio
SuperGPQA (600)
Escalation vs. escalation	370	363	$-7$ [ $-23$ , $+9$ ]	$[-20,\ +6]$	0.61 $\times$
Escalation vs. majority of 4	372	363	$-9$ [ $-24$ , $+7$ ]	$[-22,\ +4]$	0.39 $\times$
Single trace vs. single trace	359	354	$-5$ [ $-25$ , $+14$ ]	$[-21,\ +11]$	0.60 $\times$
Mathematics (679)
Escalation vs. escalation	624	611	$-13$ [ $-24$ , $-2$ ]	$[-22,\ -4]$	0.69 $\times$

Appendix

Table 14: Four traces per model and problem (hard set). Qwen3.5-9B : its native trace continued to 80K where truncated, plus three further traces capped at 64K (128 of them still truncated over the hard set, 9 on SuperGPQA); Concise-9B : its native trace and three 32K traces, each continued to 80K where truncated. Escalation as in Table 1 . The SuperGPQA comparison of escalations was pre-registered with a

\pm

2-point (12-problem) margin and a token-ratio limit of 0.60; the other rows are descriptive. All 1,279 hard-set problems are included, as registered, including the 3 excluded elsewhere for overlap with training.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 382 papers

Oct 13, 2026

DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

Reasoning models achieve strong performance through long chains of thought (CoT), but often spend more tokens than needed to solve a problem. Existing ways to shorten the CoT require additional training for each model, add a fixed steering vector to the model's hidden states, or use decoding heuristics such as stopping early or banning words like "wait", which can reduce accuracy. We propose DoubtLess, a training-free decoding method in which a much smaller reasoning model steers the large one towards brevity. The small model reads the large model's CoT so far twice, once prompted to be brief (the brief thinker) and once prompted to be verbose (the verbose thinker), and the difference between its two logit vectors, scaled by a strength, is added to the large model's logits at every step of the CoT. On seven benchmarks of competition mathematics, graduate-level science and competitive programming, DoubtLess shortens the CoT of Qwen3-32B by 31% with unchanged accuracy, outperforming five inference-time baselines. We also show that DoubtLess reduces thinking tokens per turn by 20% in a self-debugging coding agent with unchanged task performance. It also generalizes to two other model families with unchanged accuracy. When combined with training-based compression, it further shortens the CoT of a model already trained for brevity. Ablations and behavioral analysis show that DoubtLess reduces repeated checking, with most token savings coming after the correct answer first appears in the CoT. Our code is available at https://github.com/leoluolol/DoubtLess.

Minghao Luo, Liang Chen

The Chinese University of Hong Kong · EPFL

Efficient Language Model Reasoning CoT Compression Contrastive Decoding

Ask PDF

May 29, 2026

Quantized Reasoning Models Think They Need to Think Longer, but They Do Not

Post-training quantization (PTQ) is widely used to deploy large language models efficiently, but its effect on reasoning models is not well understood. Across math, coding, and science QA, we find that aggressive PTQ reduces accuracy while increasing chain-of-thought (CoT) length. Surprisingly, we show that in up to 52% of the quantized models' failures, models reach the right answer in intermediate reasoning steps but do not output it as a final answer. To understand why quantization leads to this increase in overthinking errors, we measure the token-level KL divergence between quantized and full-precision output distributions. Positions with high KL divergence correlate strongly with high next-token entropy, and at these positions quantized models disproportionately sample overthinking markers such as "wait", "but", and "alternatively". We show that simply introducing a training-free logit penalty on a curated set of overthinking markers can reduce CoT length by 12--23% while preserving or improving accuracy across 5 models (1.5B-32B parameters), 3 quantization methods, and 5 benchmarks, yielding a favorable Pareto frontier of accuracy against reasoning cost compared to penalizing other token sets. Overthinking errors produced by quantized models are particularly reduced by up to 58%.

Sanae Lotfi, Polina Kirichenko, Steven Li +1

1FAIR at Meta · 2Meta AI

Logit Adjustment Overthinking in Language Models CoT Reasoning Efficient Language Model Reasoning Post-Training Quantization LLM Reasoning+1

Ask PDF

Mar 5, 2026

CRISP: Compressed Reasoning via Iterative Self-Policy Distillation

Reasoning models often generate far more tokens than a task requires, which raises inference cost and can compound errors. We introduce CRISP (Compressed Reasoning via Iterative Self-Policy Distillation), an on-policy self-distillation method that teaches a model to reason more concisely by distilling its own concise behavior back into itself. The method uses a single idea: condition the same model on a "be concise" instruction to obtain teacher logits, then minimize the per-token reverse KL divergence between the student and this teacher on the student's own rollouts. It requires no ground-truth answers, no token budgets, and no difficulty estimators. The reverse-KL objective is naturally difficulty-adaptive: it compresses easy problems aggressively while preserving the reasoning steps that hard problems require. On Qwen3-14B, CRISP cuts reasoning length by up to 56% on MATH-500 and 38% on the harder AIME 2024, while improving MATH-500 accuracy by up to 3.3 points over the base model and holding AIME 2024 accuracy within about one point. This behavior generalizes across model sizes and families: Qwen3-8B shows the same compression with accuracy preserved, and DeepSeek-R1-Distill-Llama-8B improves accuracy on all five benchmarks while shortening its responses. General capabilities are preserved across all three models. Code is available at https://github.com/HJSang/OPSD_Reasoning_Compression.

Hejian Sang, Yuanda Xu, Zhengze Zhou +3

CoT Distillation Efficient Language Model Reasoning On-Policy Distillation CoT Compression LLM Reasoning

Ask PDF

Abstract

Efficient Language Model Reasoning

Explore similar work

DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

Quantized Reasoning Models Think They Need to Think Longer, but They Do Not

CRISP: Compressed Reasoning via Iterative Self-Policy Distillation