CODA: Difficulty-Aware Compute Allocation for Adaptive Reasoning

Siye Wu; Jian Xie; Yikai Zhang; Yanghua Xiao

cs.CLMar 9, 2026

CODA: Difficulty-Aware Compute Allocation for Adaptive Reasoning

Authors: Siye Wu, Jian Xie, Yikai Zhang, Yanghua Xiao

Organizations: ♠Fudan University

Abstract

The emergence of large reasoning models demonstrates that scaling inference-time compute significantly enhances performance on complex tasks. However, it often falls into another trap: overthinking simple problems, where repetitive rationales yield minimal accuracy gains at a disproportionately high cost. This motivates adaptive reasoning: dynamically aligning reasoning depth with instance difficulty. In this paper, we study adaptive reasoning from an optimality perspective, formalizing it as a utility maximization problem where tokens are allocated until the marginal accuracy gain falls below the incremental cost. Based on this, we propose CODA (Compute Allocation by Difficulty Awareness), a method that operationalizes this principle by allocating tokens via a policy-internal difficulty signal. Specifically, CODA estimates difficulty via group-based rollouts and maps it to two non-negative gates that modulate a length-dependent shaping term on top of the binary base reward. The easy-side gate penalizes verbosity on simple instances, whereas the hard-side gate encourages more deliberative rollouts on challenging ones. Across model scales and benchmarks, CODA achieves adaptive reasoning without external annotations or user-provided budgets: on easy tasks, CODA reduces token costs by over 60% while maintaining strong accuracy, whereas on hard tasks it incentivizes more deliberative rollouts to maximize performance.

Figures & tables

Panel of Figure 1 — Figure 1: Adaptive compute allocation across difficulty levels on Qwen3-8B-Base. (a) Compared to GRPO, Coda dynamically allocates reasoning tokens by question difficulty, consuming substantially fewer tokens on easier problems while increasing compute for harder ones. (b) On easy tasks (GSM8K, MATH), extra tokens yield marginal gains, and Coda achieves optimal accuracy at minimal cost by avoiding unnecessary reasoning. On hard tasks (AIME24&25), additional tokens substantially improve performance, and Coda encourages deeper reasoning to maximize accuracy.

Models	Accuracy (%) ( $\uparrow$ )									#Tokens ( $\downarrow$ )
	Math †						General ‡		Avg.	Math †						General ‡		Avg.
	SVAMP	GSM8K	MATH	AMC23	AIME24	AIME25	CSQA	GPQA	Avg.	SVAMP	GSM8K	MATH	AMC23	AIME24	AIME25	CSQA	GPQA	Avg.
Qwen3-4B-Base	34.8	38.3	55.2	35.9	9.2	4.2	58.4	31.2	33.4	2805	1712	1280	1091	2311	2008	2193	1910	1913
GRPO	92.5	90.9	85.3	67.2	25.8	19.6	77.7	42.6	62.7	803	722	2774	6042	9618	9391	492	3182	4128
Vlp	92.5	91.3	84.5	67.5	22.1	17.5	78.3	42.5	62.0	245	405	1716	4254	6715	6298	118	1983	2717
Asrr	93.1	92.0	84.8	66.3	22.5	17.9	78.0	39.5	61.7	350	289	1464	4022	6784	6315	185	1984	2674

Table 1: Accuracy (%) and average token cost of different methods across evaluation benchmarks. † denotes in-domain mathematical benchmarks, and ‡ denotes out-of-domain general benchmarks. “#Tokens” reports the average number of generated tokens. To reduce single-sample variance, we report

\mathrm{mean@8}

results by sampling

8

outputs per prompt. “

\Delta

” compares Coda against GRPO, reporting the accuracy change in percentage points and the relative token reduction.

Figure 4: Training dynamics under different easy-penalty strengths

\alpha

. Moderate

\alpha

effectively suppresses unnecessary reasoning while preserving base reward. However, excessively large

\alpha

over-penalizes long responses, limiting length scaling and leading to observable gaps in base reward.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Processing	Size
Easy Training
GSM8K [ Cobbe et al., 2021 ]	full	7.4K
MATH [ Hendrycks et al., 2021 ]	level-1	1.0K
		8.4K
Hard Training
DeepScaleR [ Luo et al., 2025b ]	sp $<10\%$	13.1K

Appendix

Table 3: Training data construction for difficulty-shift experiments.

Figure 6: Dynamics of difficulty-gated weights under different training difficulty distributions, showing that Coda automatically adapts its compute allocation to the observed difficulty.

$\alpha$	Accuracy (%) ( $\uparrow$ )									#Tokens ( $\downarrow$ )
	Math †						General ‡		Avg.	Math †						General ‡		Avg.
	SVAMP	GSM8K	MATH	AMC23	AIME24	AIME25	CSQA	GPQA	Avg.	SVAMP	GSM8K	MATH	AMC23	AIME24	AIME25	CSQA	GPQA	Avg.
0	93.0	92.5	87.9	73.1	30.0	22.9	79.7	47.2	65.8	604	655	2420	5244	8635	7870	412	2822	3583
0.1	93.3	92.6	87.8	72.8	29.6	22.5	79.0	46.0	65.4	558	573	2003	4605	8321	7328	372	2832	3324
0.2	92.9	92.5	87.6	72.5	29.6	22.9	79.9	47.2	65.6	203	311	1700	4159	8023	7275	139	2047	2982
0.3	92.6	92.1	86.4	71.9	28.8	21.3	79.9	45.4	64.8	186	231	1464	3773	6939	6528	86	1543	2594

Appendix

Table 4: Ablation on easy-side penalty strength

\alpha

for Coda (8B models).

$\beta$	Accuracy (%) ( $\uparrow$ )									#Tokens ( $\downarrow$ )
	Math †						General ‡		Avg.	Math †						General ‡		Avg.
	SVAMP	GSM8K	MATH	AMC23	AIME24	AIME25	CSQA	GPQA	Avg.	SVAMP	GSM8K	MATH	AMC23	AIME24	AIME25	CSQA	GPQA	Avg.
0	92.7	92.5	87.3	72.2	27.5	21.3	79.7	46.1	64.9	220	320	1639	3522	6548	5930	209	1852	2530
0.1	92.6	92.4	87.2	71.9	28.3	21.7	79.6	46.1	65.0	216	313	1672	3812	7581	6635	198	1792	2777
0.2	92.9	92.5	87.6	72.5	29.6	22.9	79.9	47.2	65.6	203	311	1700	4159	8023	7275	139	2047	2982
0.3	92.7	92.4	87.7	72.5	29.2	22.5	79.5	46.8	65.4	232	352	1765	4157	7505	6928	237	1938	2889

Appendix

Table 5: Ablation on hard-side bonus strength

\beta

for Coda (8B models).

$(\tau_{\text{easy}},\tau_{\text{hard}})$	Accuracy (%) ( $\uparrow$ )									#Tokens ( $\downarrow$ )
	Math †						General ‡		Avg.	Math †						General ‡		Avg.
	SVAMP	GSM8K	MATH	AMC23	AIME24	AIME25	CSQA	GPQA	Avg.	SVAMP	GSM8K	MATH	AMC23	AIME24	AIME25	CSQA	GPQA	Avg.
(0.6, 0.25)	92.8	92.1	87.3	72.2	29.2	22.5	79.8	46.0	65.2	183	285	1621	4100	7664	6956	145	1797	2844
(0.75, 0.25)	92.9	92.5	87.6	72.5	29.6	22.9	79.9	47.2	65.6	203	311	1700	4159	8023	7275	139	2047	2982
(0.9, 0.25)	92.8	92.6	88.1	72.5	30.0	22.9	79.5	47.0	65.7	343	424	1876	4372	8176	7208	307	2688	3174
(0.75, 0.1)	92.9	92.5	87.7	72.8	29.2	22.5	79.5	46.5	65.4	216	312	1795	4262	7787	7078	248	1980	2960

Appendix

Table 6: Ablation on difficulty-gating thresholds

(\tau_{\text{easy}},\tau_{\text{hard}})

for Coda (8B models).

Adaptive Reasoning in Language Models

See all 21 papers

Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation
Aug 20, 2026Gijs Kassenaar, Zhao Yang, Vincent François-LavetEfficient Language Model Reasoning RL for Language Model Reasoning
Vrije Universiteit Amsterdam
Nice Fold or Hero Call: Learning Budget-Efficient Thinking for Adaptive Reasoning
May 12, 2026Zhaomeng Zhou, Lan Zhang, Junyang Wang +2RL for Language Model Reasoning Efficient Language Model Reasoning
University of Science and Technology of China · The Chinese University of Hong Kong
ADaPT: Token-Level Decoupling for Efficient Large Reasoning Models
Jun 18, 2026Tingyun Li, Zishang Jiang, Jinyi Han +8Inference-Time Optimization RL for Language Model Reasoning
School of Data Science, Fudan University · Shanghai Institute of Artificial Intelligence for Education, East China Normal University · College of Computer Science and Artificial Intelligence, Fudan University +1

Aug 20, 2026J/K move · Enter open · S save

Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation

Gijs Kassenaar, Zhao Yang, Vincent François-Lavet

Vrije Universiteit Amsterdam

Efficient Language Model Reasoning RL for Language Model Reasoning LLM Mathematical Reasoning

Open paper AskPDF arXiv

Reasoning language models trained with reinforcement learning typically operate under a fixed token budget rather than an explicitly adaptive one, which can lead to over-computation on easy problems and insufficient computation on difficult ones. We study whether a model can learn to allocate its own reasoning effort by choosing, as the first token of its response, one of three modes: NoThink (answer as quickly as possible), Short (brief reasoning), or Long (extended reasoning). The choice is learned inside Group Relative Policy Optimization (GRPO) with no separate router, through a shaped reward that makes each mode worthwhile at a different response length, together with hard per-mode token caps that keep the modes distinct. On a 1.5B distilled model trained on MATH, the three modes emerge without collapsing to a single choice, and the brief modes end up more accurate than Long, which shows that the router sorts problems by difficulty rather than at random. Averaged over three seeds, the resulting policy stays close to the base model's accuracy on the held-out MATH500 (

0.782

vs.\

0.796

) while cutting the mean response length from

4{,}796

2{,}811

tokens (a

41\%

reduction). Interestingly, it also transfers to other benchmarks without retraining, with the largest savings where problems are easier, with for instance 76% token reduction on GSM8K and at higher accuracy than the baselines at similar response length. In short, we build a reasoning model that adaptively chooses how much to reason for each problem.

Abstract

Adaptive Reasoning in Language Models

Explore similar work

Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation

Nice Fold or Hero Call: Learning Budget-Efficient Thinking for Adaptive Reasoning

ADaPT: Token-Level Decoupling for Efficient Large Reasoning Models