Increasing the thinking budget of AI models can significantly improve accuracy, but not all questions warrant the same amount of reasoning. Users may prefer to allocate different amounts of reasoning effort depending on how they value output quality versus latency and cost. To leverage this tradeoff effectively, users need fine-grained control over the amount of thinking used for a particular query, but few approaches enable such control. Existing methods require users to specify the absolute number of desired tokens, but this requires knowing the difficulty of the problem beforehand to appropriately set the token budget for a query. To address these issues, we propose Adaptive Effort Control, a self-adaptive reinforcement learning method that trains models to use a user-specified fraction of tokens relative to the current average chain-of-thought length for each query. This approach eliminates dataset- and phase-specific hyperparameter tuning. Users can dynamically adjust the cost-accuracy trade-off through a continuous effort parameter specified at inference time. We observe that the model automatically learns to allocate resources proportionally to the task difficulty. Across model scales ranging from 1.5B to 32B parameters, our approach reduces chain-of-thought length on math tasks by 1.9-3.7x at matched accuracy relative to the base model used for RL training.
Figures & tables
Figure 1: Variable Effort Control and Improved Reasoning Efficiency. (Left) After training, increasing r in the prompt leads to an overall increase in both accuracy and the number of generated tokens across datasets of varying difficulty. Note that in addition to allowing control, this training approach also allows us to significantly reduce the number of generated tokens compared to the baseline (in crosses) and with regular RL training (squares). (Right) We apply our approach across model scales ranging from 1.5B to 32B and show results averaged across math datasets (see Appendix Fig. 11 for additional dataset-specific results), with our approach enabling more efficient reasoning and improved performance relative to the base models. We observe superior performance vs. number of generated token curves with increasing model size.
Figure 2: (Left) Comparison against other length-control baselines. Our controllable model ( e1 ) obtains a superior accuracy-token tradeoff compared to S1 across all effort levels. When the effort level is above 60%, it also outperforms L1-Exact, and slightly outperforms L1-Max (which does not allow precise control over the generation length). See Appendix Fig. 10 for performance on individual datasets. (Right) Adaptation to query difficulty. In addition to providing control over the number of tokens, e1 is automatically able to adapt the length of the reasoning chain to the difficulty of the input task. We see this by ordering the problems from easiest to hardest (based on fraction of times the model solves a problem correctly) and plotting the cumulative percentage of total tokens used for solving those problems. A diagonal line indicates equal tokens allocation to each problem, and increasingly convex curves indicate fewer token allocation to easier problems. Our approach (shown for r=1 , purple) allocates a smaller percentage of tokens to easier problems than other length-control baselines and the base model used for RL training. In particular our model allocates only ∼ 20% of tokens to the easiest 50% of problems. See Appendix Fig. 13 for analysis of the 7B model’s adaptation to query difficulty.
Figure 3: Calibration for interpretable control over percentage effort. After training, we can reparameterize the effort parameter r to linearly control either the (Left) relative accuracy or (Right) relative number of tokens and enable more intuitive user control. Error bars denote standard deviation.
Figure 4: Learning dynamics of AEC training. We plot the accuracy and number of tokens as a function of training step for the three math evaluation datasets. Early in training, the model has similar accuracy and number of generated tokens regardless of the effort level, which means that the model needed to learn to become sensitive to the effort level. We find that the number generated tokens roughly decreases as a power law with exponent β across training steps. We find that shorter problems (MATH500) decrease faster (larger β ) than longer problems (AIME). Despite token lengths decreasing through training, we find that accuracy increases when the effort level is at least 70% .
Figure 5: (Left) Transferability of effort control across different domains. Despite training on math questions, we find that after AEC training, we can modulate the number of generated tokens and performance on different domains by varying the effort parameter r . Across all of the datasets we find that the number of generated tokens and accuracy increase overall with the effort parameter supplied in the prompt. (Right) Effect of reward function and robustness to prompt . We compare our AEC reward function to a modified reward function (Eq. 3 ) that penalizes the length of the generation regardless of problem difficulty. For this experiment we use a modified prompt that specifies the effort through a “points” parameter p . After training with our AEC reward function (circles), we observe the ability to increase accuracy and the number of generated tokens by increasing p , highlighting that the dependence on the effort parameter does not depend on the specific prompt used. Training with a small length penalty ( λ=1e−4 ) does not allow modulating the number of generated tokens or accuracy by varying the effort parameter p (squares). Training with a large length penalty ( λ=1e−3 , triangles) leads to the ability to modulate the number of tokens and accuracy (within a narrow range), but results in significantly fewer generated tokens (less than 2000 for all datasets).
Figure 6: Conceptual Depth and task-specific effort selection. A user or agent should select the number of chain of thought tokens to balance the performance on the task and the cost incurred for achieving that performance. In math exams such as AMC (red) and AIME (green), the correct answer rendered after the time limit is worthless, and different exams have different time limits. When time and loss are compounded through the cost of time, the minimum value measures what we call the conceptual depth of the model. An “intelligent” agent should be able to choose this operating point, shown as black stars, depending on the specific task and query at hand. The error rate (where λ=0 ) is shown in a lighter shade. The plotted effort levels range from 40% to 100%. The optimal operating point can be specified by either the number of tokens or the effort level. For a given λ , the optimal effort level is similar across the different exams, even though the optimal number of tokens differs significantly depending on the particular exam and its time limit.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Example model response with effort level 50% for a problem from MATH500. 224 tokens were generated.
Figure 8: Example model response with effort level 100% for a problem from MATH500. 1183 tokens were generated.
Figure 9: Example model response with effort level 50% for a problem from AIME. 3503 tokens were generated.
Figure 10: Comparison against other token-based control approaches separated per dataset. Same setup as Fig. 2 (Left).
Figure 11: Same as Fig. 1 (Left) for various model sizes. Together with Fig. 1 (Left), these were aggregated to form Fig. 1 (Right).
Token reduction at matched accuracy ( × )
Model
MATH500
AMC
AIME
Avg.
1.5B
3.7
3.5
2.9
3.4
7B
3.4
3.2
2.5
3.0
14B
1.9
2.1
2.4
2.1
32B
—
2.0
2.1
2.0
Appendix
Table 1: Per-dataset reduction in generated tokens at matched accuracy relative to the base model, computed from Fig. 1 and Fig. 11 . Token reduction is computed by comparing the number of generated tokens at the lowest effort level that matches the target accuracy, with linear interpolation between adjacent effort levels. Averages are computed over the datasets in which the base model’s accuracy is reached (on MATH500 for the 32B it is not reached; our model reaches 3 percentage points below it at maximum effort while using 3.7 × fewer tokens).
Figure 12: Calibration of percentage effort across datasets. We plot the average error (and standard deviation) across effort values from Fig. 3 (Right) for each dataset.
Figure 13: Adaptation to query difficulty for 7B model, using R1 Distilled Qwen as Base Model. In addition to providing control over the number of tokens, we show that training with our approach leads to smaller token allocation to easier problems, and larger token allocation to more difficult problems. We show results for various values of r . For values of r≥0.7 , we observe fewer token allocation to easier problems, but not for low percentage effort when the total number of generated tokens and accuracies were significantly lower.
Figure 14: Relative accuracy and relative token length through training. We evaluate the relative accuracy and relative number of tokens through training by comparing relative to 100% effort. Despite problems being of varying difficulty and taking varying length, we find that the relative accuracy across datasets remains roughly constant over training for a fixed effort level, as the curves are overlapping. We observe a similar phenomenon when analyzing the ratio of number of tokens.
Figure 15: Learning dynamics of AEC training for 32B model. We plot the accuracy and number of tokens as a function of training step for the three math evaluation datasets. Dashed lines indicate a power-law fit with exponent β .
Figure 16: Learning dynamics of AEC training for 14B model. We plot the accuracy and number of tokens as a function of training step for the three math evaluation datasets. Dashed lines indicate a power-law fit with exponent β .
Figure 17: Learning dynamics of AEC training for 1.5B model. We first plot the accuracy and number of tokens as a function of training step for the three math evaluation datasets. Dashed lines indicate a power-law fit with exponent β .
Figure 18: (Left) We plot the error rate across generated tokens for effort levels 40% to 100% . (Center) We compute the change in error over the change in tokens across the different effort levels (dashed lines). (Right) When rescaling these curves by the time allocated to problems from these datasets (12 minutes for AIME, 3 minutes for AMC, and we approximate 1 minute for MATH500), we observe the rescaled curves are approximately overlapping (solid lines). This means that the change in error rate dTdED(T) for different datasets D is inversely proportional to the real time allocated per question at different effort levels.
Figure 19: To evaluate whether AEC depends on an R1-distilled initialization, we trained Qwen 2.5-7B-Instruct using AEC (with a maximum response length of 4K tokens). After training, we observe that accuracy and token usage increase overall with effort level, analogous to our main results with R1-distilled models. Results are shown at step 720.
Figure 20: We compare training with the AEC reward function against using a maximum token-based reward function, where the maximum number of tokens is also specified in the prompt (Eq. 4 ). We find the ability to control the number of tokens emerges earlier in training using our AEC objective (by step 180, left plot). Results are shown at steps 180 (Left), 420 (Center) and 1000 (Right).
Reasoning language models trained with reinforcement learning typically operate under a fixed token budget rather than an explicitly adaptive one, which can lead to over-computation on easy problems and insufficient computation on difficult ones. We study whether a model can learn to allocate its own reasoning effort by choosing, as the first token of its response, one of three modes: NoThink (answer as quickly as possible), Short (brief reasoning), or Long (extended reasoning). The choice is learned inside Group Relative Policy Optimization (GRPO) with no separate router, through a shaped reward that makes each mode worthwhile at a different response length, together with hard per-mode token caps that keep the modes distinct. On a 1.5B distilled model trained on MATH, the three modes emerge without collapsing to a single choice, and the brief modes end up more accurate than Long, which shows that the router sorts problems by difficulty rather than at random. Averaged over three seeds, the resulting policy stays close to the base model's accuracy on the held-out MATH500 (0.782 vs.\ 0.796) while cutting the mean response length from 4,796 to 2,811 tokens (a 41% reduction). Interestingly, it also transfers to other benchmarks without retraining, with the largest savings where problems are easier, with for instance 76% token reduction on GSM8K and at higher accuracy than the baselines at similar response length. In short, we build a reasoning model that adaptively chooses how much to reason for each problem.
Large Reasoning Models (LRMs) often overthink easy problems and underthink hard ones, leading to inefficient computation allocation. Existing methods regulate generated computation or select between direct answering and explicit reasoning, but do not jointly control whether}to reason and how much computation to allocate within reasoning. We call the resulting difficulty-dependent loss in accuracy under computation reduction the efficiency tax. We propose When2Think, an RLVR-based post-training framework for instance-adaptive computation allocation. Its core mechanism, Instance-level Difficulty-Aware Control (IDAC), uses cached reference statistics of success and token cost to modulate a correctness-gated efficiency bonus based on generated token count. Importance sampling supports exploration of Think and NoThink, while Batch-Wise Standardization constructs standardized advantages for critic-free optimization. The framework requires neither a learned reward model nor a learned critic, and offline reference caching avoids online reference-model queries during policy updates. On AIME24, When2Think improves Pass@3 by 10.0 percentage points while reducing token usage by 27.9% relative to the backbone.
Large reasoning models improve performance on challenging problems by allocating additional computation before answering, but longer reasoning does not always lead to better results and can introduce substantial redundant reasoning on simple problems. Conversely, aggressively shortening reasoning can degrade performance on difficult ones. Effective reasoning therefore requires dynamically deciding when additional computation is useful based on the reasoner's capabilities and evolving solution state. Existing approaches often rely on predefined budgets or intervention rules, retrain the target reasoner, or require additional supervision. We introduce MetaCtrl, a lightweight controller that adaptively regulates a frozen reasoner without predefined token budgets or reasoner retraining. We formulate reasoning regulation as a sequential metacognitive control problem: MetaCtrl observes the evolving reasoning trace and decides whether to continue, simplify, skip redundant steps, or conclude reasoning. It is trained directly with reinforcement learning using a reward that prioritizes correctness while favoring shorter trajectories among correct solutions, requiring neither supervised intervention trajectories nor problem-specific budgets. Across seven benchmarks spanning mathematics, science, and code, MetaCtrl consistently improves the accuracy of LRMs while reducing their reasoning length. On DeepSeek-R1-Distill-Qwen-7B, it improves average accuracy by 4.7 points while reducing generation length by 53.3%. Without further training, the same controller transfers to an unseen reasoner (e.g., Qwen3-14B), improving average accuracy by 2.9 points and reducing generation length by 50.3%. These results establish MetaCtrl as a plug-and-play controller for improving reasoning accuracy while substantially reducing inference-time generation. The code is available at https://github.com/binbin2xs/MetaCtrl.
Zhibin Wen, Tao Han, Lei Bai +2
Southern University of Science and Technology · The Hong Kong University of Science and Technology · Shanghai Artificial Intelligence Laboratory +1