cs.AIOct 30, 2025
Savee1: Learning Adaptive Control of Reasoning Effort
Organizations: AWS Agentic AI
Abstract
Increasing the thinking budget of AI models can significantly improve accuracy, but not all questions warrant the same amount of reasoning. Users may prefer to allocate different amounts of reasoning effort depending on how they value output quality versus latency and cost. To leverage this tradeoff effectively, users need fine-grained control over the amount of thinking used for a particular query, but few approaches enable such control. Existing methods require users to specify the absolute number of desired tokens, but this requires knowing the difficulty of the problem beforehand to appropriately set the token budget for a query. To address these issues, we propose Adaptive Effort Control, a self-adaptive reinforcement learning method that trains models to use a user-specified fraction of tokens relative to the current average chain-of-thought length for each query. This approach eliminates dataset- and phase-specific hyperparameter tuning. Users can dynamically adjust the cost-accuracy trade-off through a continuous effort parameter specified at inference time. We observe that the model automatically learns to allocate resources proportionally to the task difficulty. Across model scales ranging from 1.5B to 32B parameters, our approach reduces chain-of-thought length on math tasks by 1.9-3.7x at matched accuracy relative to the base model used for RL training.
Figures & tables
Figure 1: Variable Effort Control and Improved Reasoning Efficiency. (Left) After training, increasing in the prompt leads to an overall increase in both accuracy and the number of generated tokens across datasets of varying difficulty. Note that in addition to allowing control, this training approach also allows us to significantly reduce the number of generated tokens compared to the baseline (in crosses) and with regular RL training (squares). (Right) We apply our approach across model scales ranging from 1.5B to 32B and show results averaged across math datasets (see Appendix Fig. 11 for additional dataset-specific results), with our approach enabling more efficient reasoning and improved performance relative to the base models. We observe superior performance vs. number of generated token curves with increasing model size.
Figure 2: (Left) Comparison against other length-control baselines. Our controllable model ( e1 ) obtains a superior accuracy-token tradeoff compared to S1 across all effort levels. When the effort level is above 60%, it also outperforms L1-Exact, and slightly outperforms L1-Max (which does not allow precise control over the generation length). See Appendix Fig. 10 for performance on individual datasets. (Right) Adaptation to query difficulty. In addition to providing control over the number of tokens, e1 is automatically able to adapt the length of the reasoning chain to the difficulty of the input task. We see this by ordering the problems from easiest to hardest (based on fraction of times the model solves a problem correctly) and plotting the cumulative percentage of total tokens used for solving those problems. A diagonal line indicates equal tokens allocation to each problem, and increasingly convex curves indicate fewer token allocation to easier problems. Our approach (shown for , purple) allocates a smaller percentage of tokens to easier problems than other length-control baselines and the base model used for RL training. In particular our model allocates only 20% of tokens to the easiest 50% of problems. See Appendix Fig. 13 for analysis of the 7B model’s adaptation to query difficulty.
Figure 3: Calibration for interpretable control over percentage effort. After training, we can reparameterize the effort parameter to linearly control either the (Left) relative accuracy or (Right) relative number of tokens and enable more intuitive user control. Error bars denote standard deviation.
Figure 4: Learning dynamics of AEC training. We plot the accuracy and number of tokens as a function of training step for the three math evaluation datasets. Early in training, the model has similar accuracy and number of generated tokens regardless of the effort level, which means that the model needed to learn to become sensitive to the effort level. We find that the number generated tokens roughly decreases as a power law with exponent across training steps. We find that shorter problems (MATH500) decrease faster (larger ) than longer problems (AIME). Despite token lengths decreasing through training, we find that accuracy increases when the effort level is at least .
Figure 5: (Left) Transferability of effort control across different domains. Despite training on math questions, we find that after AEC training, we can modulate the number of generated tokens and performance on different domains by varying the effort parameter . Across all of the datasets we find that the number of generated tokens and accuracy increase overall with the effort parameter supplied in the prompt. (Right) Effect of reward function and robustness to prompt . We compare our AEC reward function to a modified reward function (Eq. 3 ) that penalizes the length of the generation regardless of problem difficulty. For this experiment we use a modified prompt that specifies the effort through a “points” parameter . After training with our AEC reward function (circles), we observe the ability to increase accuracy and the number of generated tokens by increasing , highlighting that the dependence on the effort parameter does not depend on the specific prompt used. Training with a small length penalty ( ) does not allow modulating the number of generated tokens or accuracy by varying the effort parameter (squares). Training with a large length penalty ( , triangles) leads to the ability to modulate the number of tokens and accuracy (within a narrow range), but results in significantly fewer generated tokens (less than 2000 for all datasets).
Figure 6: Conceptual Depth and task-specific effort selection. A user or agent should select the number of chain of thought tokens to balance the performance on the task and the cost incurred for achieving that performance. In math exams such as AMC (red) and AIME (green), the correct answer rendered after the time limit is worthless, and different exams have different time limits. When time and loss are compounded through the cost of time, the minimum value measures what we call the conceptual depth of the model. An “intelligent” agent should be able to choose this operating point, shown as black stars, depending on the specific task and query at hand. The error rate (where ) is shown in a lighter shade. The plotted effort levels range from 40% to 100%. The optimal operating point can be specified by either the number of tokens or the effort level. For a given , the optimal effort level is similar across the different exams, even though the optimal number of tokens differs significantly depending on the particular exam and its time limit.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Example model response with effort level 50% for a problem from MATH500. 224 tokens were generated.
Figure 8: Example model response with effort level 100% for a problem from MATH500. 1183 tokens were generated.
Figure 9: Example model response with effort level 50% for a problem from AIME. 3503 tokens were generated.
Figure 10: Comparison against other token-based control approaches separated per dataset. Same setup as Fig. 2 (Left).
Figure 11: Same as Fig. 1 (Left) for various model sizes. Together with Fig. 1 (Left), these were aggregated to form Fig. 1 (Right).
| Token reduction at matched accuracy ( ) | ||||
| Model | MATH500 | AMC | AIME | Avg. |
| 1.5B | 3.7 | 3.5 | 2.9 | 3.4 |
| 7B | 3.4 | 3.2 | 2.5 | 3.0 |
| 14B | 1.9 | 2.1 | 2.4 | 2.1 |
| 32B | — | 2.0 | 2.1 | 2.0 |
Table 1: Per-dataset reduction in generated tokens at matched accuracy relative to the base model, computed from Fig. 1 and Fig. 11 . Token reduction is computed by comparing the number of generated tokens at the lowest effort level that matches the target accuracy, with linear interpolation between adjacent effort levels. Averages are computed over the datasets in which the base model’s accuracy is reached (on MATH500 for the 32B it is not reached; our model reaches percentage points below it at maximum effort while using 3.7 fewer tokens).
Figure 12: Calibration of percentage effort across datasets. We plot the average error (and standard deviation) across effort values from Fig. 3 (Right) for each dataset.
Figure 13: Adaptation to query difficulty for 7B model, using R1 Distilled Qwen as Base Model. In addition to providing control over the number of tokens, we show that training with our approach leads to smaller token allocation to easier problems, and larger token allocation to more difficult problems. We show results for various values of . For values of , we observe fewer token allocation to easier problems, but not for low percentage effort when the total number of generated tokens and accuracies were significantly lower.
Figure 14: Relative accuracy and relative token length through training. We evaluate the relative accuracy and relative number of tokens through training by comparing relative to effort. Despite problems being of varying difficulty and taking varying length, we find that the relative accuracy across datasets remains roughly constant over training for a fixed effort level, as the curves are overlapping. We observe a similar phenomenon when analyzing the ratio of number of tokens.
Figure 15: Learning dynamics of AEC training for 32B model. We plot the accuracy and number of tokens as a function of training step for the three math evaluation datasets. Dashed lines indicate a power-law fit with exponent .
Figure 16: Learning dynamics of AEC training for 14B model. We plot the accuracy and number of tokens as a function of training step for the three math evaluation datasets. Dashed lines indicate a power-law fit with exponent .
Figure 17: Learning dynamics of AEC training for 1.5B model. We first plot the accuracy and number of tokens as a function of training step for the three math evaluation datasets. Dashed lines indicate a power-law fit with exponent .
Figure 18: (Left) We plot the error rate across generated tokens for effort levels to . (Center) We compute the change in error over the change in tokens across the different effort levels (dashed lines). (Right) When rescaling these curves by the time allocated to problems from these datasets (12 minutes for AIME, 3 minutes for AMC, and we approximate 1 minute for MATH500), we observe the rescaled curves are approximately overlapping (solid lines). This means that the change in error rate for different datasets is inversely proportional to the real time allocated per question at different effort levels.
Figure 19: To evaluate whether AEC depends on an R1-distilled initialization, we trained Qwen 2.5-7B-Instruct using AEC (with a maximum response length of 4K tokens). After training, we observe that accuracy and token usage increase overall with effort level, analogous to our main results with R1-distilled models. Results are shown at step 720.
Figure 20: We compare training with the AEC reward function against using a maximum token-based reward function, where the maximum number of tokens is also specified in the prompt (Eq. 4 ). We find the ability to control the number of tokens emerges earlier in training using our AEC objective (by step 180, left plot). Results are shown at steps 180 (Left), 420 (Center) and 1000 (Right).
More in this topic
Efficient Language Model Reasoning
Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.
See all 370 papers