To Think or Not to Think: Allocating Reasoning Where It Helps

Zhengdong He; Yunfan Zhou; Jianguo Yao; Haibing Guan; Xijun Li

cs.AIAug 30, 2026

To Think or Not to Think: Allocating Reasoning Where It Helps

Authors: Zhengdong He, Yunfan Zhou, Jianguo Yao, Haibing Guan, Xijun Li

Organizations: Shanghai Key Laboratory of Scalable Computing and Systems · School of Computer Science, Shanghai Jiao Tong University

Abstract

Reinforcement learning (RL) has proven effective in enhancing the reasoning performance of large language models (LLMs), particularly in complex mathematical and programming tasks. However, this capability comes with systematic length misallocation, in which models devote excessive reasoning to simple questions while terminating prematurely on harder ones, degrading inference efficiency with negligible accuracy improvement. Many length-adaptive methods mitigate this issue by allocating token budgets according to question difficulty, under the implicit assumption that harder questions benefit monotonically from extended reasoning. In contrast, we find that the effect of reasoning length on accuracy is concentrated on partially solvable questions. Our further analysis reveals that explicit length rewards can produce unintended training dynamics. Motivated by these findings, we propose CARE---Contrastive Accuracy Reward Estimation---which compares the beneficial length adjustment per question from online sampled responses and applies adaptive length rewards within Group Relative Policy Optimization, with no extra hyperparameters or additional inference cost. Experiments across multiple reasoning benchmarks demonstrate that our method improves Pass@1 by up to $4\%$ while simultaneously reducing reasoning length by $37\%$ , achieving higher token efficiency. Code will be available upon the acceptance of this paper.

Figures & tables

Figure 1: Relationship between reasoning length and accuracy across difficulty groups on GSM8K (top) and MATH500 (bottom) for Qwen3-1.7B, 4B, and 8B. Reasoning lengths are z-score normalized per question using correct-response statistics. Partially solvable questions (Blue/Orange curves) show a clear peak near zero z-score, whereas easy and hard questions show little sensitivity to length.

Figure 2: Density heatmaps of $\Delta$ length vs. $\Delta$ accuracy for short-reward Qwen3-1.7B across four difficulty subsets. A notable fraction of questions in the $(0.25,0.5]$ and $(0.5,0.75]$ subsets show improved accuracy, contradicting the assumption that harder questions benefit from additional tokens.

Figure 3: Training trajectory of Qwen3-4B under the short reward.

Method	AIME 2025		AIME 2026		HMMT 2026		MATH500		Average
Method	Pass@1	Token	Pass@1	Token	Pass@1	Token	Pass@1	Token	Pass@1	Token
Qwen3-1.7B
Qwen3-1.7B	26.88	13320.41	30.21	13716.90	16.67	14062.21	86.47	5246.36	40.06	11586.47
GRPO	30.42	10826.55	28.96	10245.45	17.61	11296.01	87.74	3829.58	41.18	9049.40
ALP	31.87	11623.52	32.08	11789.75	19.41	12170.17	85.76	3863.09	42.28	9861.63
DAST	26.04	14052.29	29.48	14312.01	15.81	14639.91	83.03	4746.27	38.59	11937.62

Table 1: Results on reasoning benchmarks under different methods. For each benchmark, we report Pass@1 (%) and length (Token).

Model	$g$	Early Exact	Middle Exact	Late Exact	Overall Exact	Overall Non-Zero
Qwen3-1.7B	4	85.39%	87.19%	87.52%	86.69%	50.90%
	8	91.95%	92.81%	92.81%	92.52%	73.73%
	12	96.27%	96.23%	96.40%	96.30%	87.91%
Qwen3-4B	4	89.85%	90.67%	91.62%	90.69%	50.94%
	8	94.63%	94.73%	94.99%	94.78%	73.82%
	12	97.39%	97.44%	97.40%	97.41%	87.92%

Table 2: Stability of CARE’s length preference signal under different rollout group sizes. Exact agreement measures consistency between the subsampled signal

d_{g}(q)

and the full-group reference

d_{16}(q)

. Non-zero agreement is computed only when

d_{16}(q)\neq 0

Model	Method	AIME25	AIME26	HMMT26	MATH500	Average
Qwen3-1.7B	CARE-Magnitude	31.35/10464.91	30.73/10496.70	20.83/11380.66	85.08/3314.91	42.00/8914.30
Qwen3-1.7B	CARE	33.96/9352.41	32.29/10073.94	21.78/10176.61	87.25/3012.70	43.82/8153.92
Qwen3-4B	CARE-Magnitude	53.02/8843.30	51.98/8117.22	26.99/9450.59	90.59/2718.12	55.65/7282.31
Qwen3-4B	CARE	52.92/8455.09	55.62/7817.14	28.60/8735.88	92.39/2618.05	57.38/6906.54
Qwen3-8B	CARE-Magnitude	54.17/9583.34	59.06/8874.49	32.01/10569.64	90.65/2892.74	58.97/7980.05
Qwen3-8B	CARE	53.85/9166.40	58.85/8548.70	32.67/10310.60	93.10/2803.10	59.62/7707.20

Table 3: Ablation on accuracy-gap magnitude weighting. Each entry reports Pass@1 (%) / average response length (Token). CARE-Magnitude weights the length term by

\left|\bar{y}_{\mathcal{L}}-\bar{y}_{\mathcal{S}}\right|

Model	Method	AIME25	AIME26	HMMT26	MATH500	Average
Qwen3-1.7B	CARE ( $\lambda=0.5$ )	32.81/10627.63	30.31/10700.56	20.55/11500.73	85.25/3397.02	42.23/9056.49
Qwen3-1.7B	CARE	33.96/9352.41	32.29/10073.94	21.78/10176.61	87.25/3012.70	43.82/8153.92
Qwen3-4B	CARE ( $\lambda=0.5$ )	53.02/8555.19	51.25/8218.75	27.27/9179.88	90.84/2742.92	55.60/7174.18
Qwen3-4B	CARE	52.92/8455.09	55.62/7817.14	28.60/8735.88	92.39/2618.05	57.38/6906.54
Qwen3-8B	CARE ( $\lambda=0.5$ )	54.06/9674.28	56.67/8993.24	32.95/10813.47	90.63/3004.62	58.58/8121.40
Qwen3-8B	CARE	53.85/9166.40	58.85/8548.70	32.67/10310.60	93.10/2803.10	59.62/7707.20

Table 4: Ablation on the length-term coefficient. Each entry reports Pass@1 (%) / average reasoning length (Token). We compare the default CARE setting

\lambda=1

with

\lambda=0.5

Appendix figures & tables35 assets

Supplementary material from the paper’s appendix.

Appendix

Inference Engine	Samples per Question	Temperature	Top-p	Max Reasoning Length	Bin Width
vLLM	64	1.0	1.0	16,384	0.5

Appendix

Table 5: Sampling configuration for difficulty estimation on GSM8K and MATH500.

	Qwen3-1.7B	Qwen3-4B	Qwen3-8B
GSM8K	1286/1319 (97.50%)	1277/1319 (96.82%)	1292/1319 (97.95%)
MATH500	475/500 (95.00%)	474/500 (94.80%)	479/500 (95.80%)

Appendix

Table 6: Number of questions retained after z-score filtering on GSM8K and MATH500.

Model	Dataset	Hard $[0,0.25]$		Easy $(0.75,1]$
Model	Dataset	$p=0$	$0<p\leq 0.25$	$0.75<p<1$	$p=1$
Qwen3-1.7B	GSM8K	33 (30.56%)	75 ( 69.44% )	179 ( 15.85% )	950 (84.15%)
Qwen3-1.7B	MATH500	25 (36.76%)	43 ( 63.24% )	112 ( 28.28% )	284 (71.72%)
Qwen3-4B	GSM8K	42 (41.18%)	60 ( 58.82% )	184 ( 15.62% )	994 (84.38%)
Qwen3-4B	MATH500	26 (40.62%)	38 ( 59.38% )	107 ( 25.72% )	309 (74.28%)
Qwen3-8B	GSM8K	23 (30.26%)	53 ( 69.74% )	70 ( 5.75% )	1148 (94.25%)

Appendix

Table 7: Fine-grained composition of the hard and easy difficulty subsets. Each entry reports the number of questions (fraction within the corresponding interval).

Figure 4: Relationship between reasoning length and accuracy across difficulty groups on GSM8K (top) and MATH500 (bottom) for Gemma-4-E2B-it, Gemma-4-E4B-it, and DeepSeek-R1-Distill-Llama-8B.

Advantage Estimator	Training Temperature	Global Batch Size	Mini Batch Size	Rollout Number	Regularization	Actor Clip Ratio	Max Response Length	Learning Rate
GRPO	1.0	256	256	16	KL/Entropy	0.2	6144	1e-6

Appendix

Table 8: Hyperparameters for training.

Samples per Question	Temperature	Top-p	Max Reasoning Length	Aggregation
32	0.6	0.95	16,384	Mean over samples

Appendix

Table 9: Evaluation configuration.

Panel of Figure 5 — Figure 5: Density heatmaps of $\Delta$ length vs. $\Delta$ accuracy for Qwen3-1.7B under Short-Reward.

Panel of Figure 6 — Figure 6: Density heatmaps of $\Delta$ length vs. $\Delta$ accuracy for Qwen3-1.7B under Long-Reward.

Panel of Figure 7 — Figure 7: Density heatmaps of $\Delta$ length vs. $\Delta$ accuracy for Qwen3-4B under Short-Reward.

Panel of Figure 8 — Figure 8: Density heatmaps of $\Delta$ length vs. $\Delta$ accuracy for Qwen3-4B under Long-Reward.

Panel of Figure 9 — Figure 9: Density heatmaps of $\Delta$ length vs. $\Delta$ accuracy for Qwen3-8B under Short-Reward.

Panel of Figure 10 — Figure 10: Density heatmaps of $\Delta$ length vs. $\Delta$ accuracy for Qwen3-8B under Long-Reward.

Panel of Figure 11 — Figure 11: Training trajectories under Short-Reward for Qwen3-1.7B .

Panel of Figure 12 — Figure 12: Training trajectories under Long-Reward for Qwen3-1.7B .

Panel of Figure 13 — Figure 13: Training trajectories under Short-Reward for Qwen3-4B .

Panel of Figure 14 — Figure 14: Training trajectories under Long-Reward for Qwen3-4B .

Panel of Figure 15 — Figure 15: Training trajectories under Short-Reward for Qwen3-8B .

Panel of Figure 16 — Figure 16: Training trajectories under Long-Reward for Qwen3-8B .

Model	Short-reward		Long-reward
Model	Stable Interval	Unstable Interval	Stable Interval	Unstable Interval
Qwen3-1.7B	N/A	N/A	Steps 1–15	Steps 31–45
Qwen3-4B	Steps 1–10	Steps 60–70	Steps 1–15	Steps 31–45
Qwen3-8B	Steps 1–10	Steps 60–70	Steps 1–15	Steps 31–45

Appendix

Table 10: Step intervals selected as the Stable Interval and Unstable Interval under short-reward and long-reward training for each model.

Panel of Figure 17 — Figure 17: CRR comparison for Qwen3-1.7B under Long-Reward training.

Panel of Figure 18 — Figure 18: CRR comparison for Qwen3-4B under Short-Reward training.

Panel of Figure 19 — Figure 19: CRR comparison for Qwen3-4B under Long-Reward training.

Panel of Figure 20 — Figure 20: CRR comparison for Qwen3-8B under Short-Reward training.

Panel of Figure 21 — Figure 21: CRR comparison for Qwen3-8B under Long-Reward training.

Panel of Figure 22 — Figure 22: Accuracy gap $\Delta(q)$ at each difficulty group for Qwen3-1.7B .

Panel of Figure 23 — Figure 23: Accuracy gap $\Delta(q)$ at each difficulty group for Qwen3-4B .

Panel of Figure 24 — Figure 24: Accuracy gap $\Delta(q)$ at each difficulty group for Qwen3-8B .

Difficulty Subset	AIME 2025				AIME 2026
	GRPO		CARE		GRPO		CARE
	Pass@1	$D(q)$	Pass@1	$D(q)$	Pass@1	$D(q)$	Pass@1	$D(q)$
$[0,\,0.25]$	$8.04$	$14.48$	$11.61$	$12.27$	$2.78$	$4.85$	$4.86$	$9.81$
$(0.25,\,0.5]$	$37.50$	$8.95$	$37.50$	$10.81$	$40.62$	$15.63$	$37.50$	$4.65$
$(0.5,\,0.75]$	$77.08$	$14.49$	$91.67$	$16.30$	$76.56$	$29.15$	$73.44$	$29.05$
$(0.75,\,1.0]$	$95.00$	$13.10$	$92.50$	$12.48$	$80.21$	$17.48$	$85.42$	$15.21$

Appendix

Table 11: Pass@1 (%) and

D(q)

(%) for Qwen3-1.7B across four benchmarks.

Difficulty Subset	AIME 2025				AIME 2026
	GRPO		CARE		GRPO		CARE
	Pass@1	$D(q)$	Pass@1	$D(q)$	Pass@1	$D(q)$	Pass@1	$D(q)$
$[0,\,0.25]$	$12.02$	$12.53$	$14.90$	$9.83$	$16.35$	$12.58$	$18.27$	$13.17$
$(0.25,\,0.5]$	$46.88$	$12.35$	$53.12$	$10.96$	$47.92$	$24.12$	$56.25$	$15.63$
$(0.5,\,0.75]$	$65.00$	$31.16$	$67.50$	$21.36$	–	–	–	–
$(0.75,\,1.0]$	$90.62$	$11.25$	$90.62$	$10.06$	$87.50$	$12.47$	$88.84$	$9.84$

Appendix

Table 12: Pass@1 (%) and

D(q)

(%) for Qwen3-4B across four benchmarks.

Difficulty Subset	AIME 2025				AIME 2026
	GRPO		CARE		GRPO		CARE
	Pass@1	$D(q)$	Pass@1	$D(q)$	Pass@1	$D(q)$	Pass@1	$D(q)$
$[0,\,0.25]$	$16.74$	$14.57$	$14.73$	$12.33$	$14.49$	$18.03$	$15.91$	$13.21$
$(0.25,\,0.5]$	$62.50$	$20.27$	$65.62$	$16.31$	$60.94$	$20.92$	$63.54$	$17.84$
$(0.5,\,0.75]$	$70.62$	$12.80$	$72.50$	$12.72$	–	–	–	–
$(0.75,\,1.0]$	$96.56$	$13.41$	$92.81$	$13.39$	$95.43$	$13.60$	$92.79$	$13.10$

Appendix

Table 13: Pass@1 (%) and

D(q)

(%) for Qwen3-8B across four benchmarks.

Panel of Figure 25 — Figure 25: Pass@1 under varying token budgets for Qwen3-1.7B .

Panel of Figure 26 — Figure 26: Pass@1 under varying token budgets for Qwen3-4B .

Panel of Figure 27 — Figure 27: Pass@1 under varying token budgets for Qwen3-8B .

Model Size	Method	AIME 2025	AIME 2026	HMMT 2026	MATH500
Qwen3-1.7B	Base	$28.68\pm 1.65\,[24.59,32.77]$	$31.63\pm 1.29\,[28.42,34.85]$	$16.41\pm 0.88\,[14.23,18.60]$	$85.42\pm 2.02\,[80.41,90.44]$
	GRPO	$29.48\pm 0.81\,[27.46,31.50]$	$30.21\pm 0.81\,[28.19,32.23]$	$19.16\pm 1.38\,[15.73,22.59]$	$\mathbf{87.15\pm 1.03\,[84.60,89.70]}$
	ALP	$\mathbf{32.29\pm 0.72\,[30.50,34.08]}$	$30.21\pm 1.83\,[25.66,34.76]$	$20.14\pm 0.67\,[18.47,21.81]$	$85.90\pm 0.16\,[85.51,86.29]$
	DAST	$26.22\pm 0.22\,[25.68,26.75]$	$28.96\pm 1.09\,[26.26,31.66]$	$16.22\pm 0.36\,[15.33,17.12]$	$83.14\pm 0.13\,[82.82,83.46]$
	CARE	$32.26\pm 1.50\,[28.54,35.97]$	$\mathbf{32.64\pm 0.87\,[30.47,34.81]}$	$\mathbf{21.28\pm 0.44\,[20.19,22.36]}$	$86.66\pm 1.09\,[83.96,89.37]$
Qwen3-4B	Base	$46.88\pm 0.63\,[45.30,48.45]$	$51.49\pm 0.59\,[50.02,52.96]$	$24.27\pm 0.85\,[22.17,26.38]$	$89.57\pm 1.11\,[86.80,92.33]$

Appendix

Table 14: Pass@1 (%) across three random seeds for Qwen3 family. Each entry reports mean

\pm

sample SD [95% Student-

t

CI]. The best mean result is shown in bold.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 341 papers

LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models
May 10, 2026Songtao Wei, Yi Li, Zhikai Li +7RL for Language Model Reasoning Efficient Language Model Reasoning
University of Texas at Dallas · Emory University · University of Texas at Arlington +2
SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
Mar 9, 2026Chenzhi Hu, Qinzhe Hu, Yuhang Xu +6Efficient Language Model Reasoning Large Reasoning Models
Shanghai Jiao Tong University · Beihang University
Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation
Aug 20, 2026Gijs Kassenaar, Zhao Yang, Vincent François-LavetEfficient Language Model Reasoning RL for Language Model Reasoning
Vrije Universiteit Amsterdam

May 10, 2026J/K move · Enter open · S save

LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models

Songtao Wei, Yi Li, Zhikai Li, Xu Hu+6

University of Texas at Dallas · Emory University · University of Texas at Arlington+2

RL for Language Model Reasoning Efficient Language Model Reasoning Large Reasoning Models

Open paper AskPDF arXiv

Large reasoning models, such as OpenAI o1 and DeepSeek-R1, tend to become increasingly verbose as their reasoning capabilities improve. These inflated Chain-of-Thought (CoT) trajectories often exceed what the underlying problems require, wasting compute, latency, and context budgets. While introducing length-based efficiency rewards during reinforcement learning offers a natural remedy, existing methods struggle with two fundamental challenges: the optimal balance between correctness and efficiency is non-stationary throughout training, and intrinsic reasoning budgets vary drastically across problems. Relying on static reward weights and global length constraints inevitably forces a compromise between degraded accuracy and unrealized compression. To overcome these limitations, we propose LEAD (Length-Efficient Adaptive and Dynamic reasoning), a method that replaces static heuristics with online, self-adaptive mechanisms. LEAD dynamically calibrates the correctness-efficiency trade-off at each step using a Potential-Scaled Instability, directing optimization capacity to the most informative learning signal. Furthermore, it estimates an adaptive per-problem target length online based on the model's own correct rollouts, applying a symmetric efficiency reward that penalizes both overthinking and over-compression. Evaluated on five mathematical reasoning benchmarks, LEAD achieves the highest accuracy and Accuracy-Efficiency Score among RL-trained efficient-reasoning methods while producing substantially shorter outputs than the base model.

Abstract

Efficient Language Model Reasoning

Explore similar work

LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models

SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning

Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation