AERA: Adaptive Evidence Residual Allocation for Efficient Test-Time Reasoning
Authors: Ziming Wang, Ivor Tsang, Hangwei Qian
Organizations: National University of Singapore, Singapore · Agency for Science, Technology and Research (A*STAR), Centre for Frontier AI Research (CFAR), Singapore · Nanyang Technological University, Singapore
Test-time scaling improves language-model reasoning by generating additional candidate solutions, but allocating the same inference budget to every problem is computationally wasteful. Existing adaptive stopping methods commonly rely on confidence, agreement, or answer stability, implicitly assuming that stronger current evidence indicates that further computation is unnecessary. We show that this assumption can fail: checkpoint-level correctness evolves non-monotonically, and observable evidence may strengthen before an answer collapses or weaken before it recovers. Motivated by this mismatch, we introduce Adaptive Evidence Residual Allocation (AERA), a sequential controller that learns whether additional computation is likely to recover a better answer from checkpoint-observable evidence. AERA characterizes cumulative response prefixes using answer-distribution, temporal, re-solving, semantic, and compute features, and repeatedly decides whether to stop or allocate the next response block. Future checkpoint correctness is used only to construct offline supervision and is never available to the controller at inference time. Across GSM8K and GPQA Diamond, AERA identifies question-specific residual opportunities while substantially reducing inference computation. In a frozen-threshold incremental-generation evaluation on 300 untouched GSM8K questions, AERA achieves 92.61% accuracy versus 93.01% with 128 responses while reducing completion tokens by 95.99%. These results suggest that adaptive reasoning should estimate the future value of computation rather than equating present confidence with correctness.
Figures & tables
Figure 1: Motivation for AERA. A representative GPQA trajectory first recovers and then collapses, contradicting the assumption that stronger observable evidence always makes stopping safer. This mismatch motivates sequential residual-utility control.
Figure 2: Empirical motivation and overview of AERA. Observable evidence can move opposite to correctness: evidence sometimes strengthens during collapse and weakens during recovery. AERA therefore characterizes each observed prefix and sequentially estimates whether the next checkpoint remains beneficial.
Transition
N
Δ majority
Δ entropy
Δ semantic
0→0
486
+0.050
+0.044
−0.005
0→1
42
+0.755
+0.073
−0.003
1→0
14
−0.022
+0.118
−0.003
1→1
250
−0.029
+0.069
−0.004
Table 1: Adjacent GPQA correctness transitions over aligned AEC states (4 → 8 through 32 → 64). Entries report transition counts and mean feature changes; entropy denotes normalized entropy.
Figure 3: Changes in observable evidence grouped by adjacent GPQA correctness transition. Boxes summarize the distributions and the horizontal line marks zero. Evidence coordinates overlap substantially across correctness outcomes.
Dataset
Method
Resp./q
Acc. (%)
GSM8K
Fixed-32
32.00
93.99
Fixed-128
128.00
94.12
ESC-aligned ( w=5 )
20.30
94.31
ASC-aligned ( C=.95 )
9.40
94.43
AERA ( τ=.3 )
8.67
94.35
GPQA
Fixed-32
32.00
37.75
Table 2: Representative points on the descriptive, question-disjoint offline accuracy–response frontiers. AERA thresholds are selected retrospectively from the complete sweep and are not confirmatory test settings.
Figure 4: Complete offline accuracy–response frontiers for fixed budgets, checkpoint-aligned ESC and ASC, and AERA.
Method
Resp./q
Acc. (%)
Saving (%)
Fixed-4
4.00
92.47
96.88
Fixed-8
8.00
93.07
93.75
Fixed-16
16.00
92.59
87.50
Fixed-32
32.00
92.52
75.00
Fixed-64
64.00
92.87
50.00
Fixed-128
128.00
93.01
0.00
Table 3: Frozen-threshold online GSM8K test on 300 untouched questions. The threshold is selected on a separate 50-question calibration set. AERA’s separately measured completion-token saving is 95.99%.
Reasoning language models trained with reinforcement learning typically operate under a fixed token budget rather than an explicitly adaptive one, which can lead to over-computation on easy problems and insufficient computation on difficult ones. We study whether a model can learn to allocate its own reasoning effort by choosing, as the first token of its response, one of three modes: NoThink (answer as quickly as possible), Short (brief reasoning), or Long (extended reasoning). The choice is learned inside Group Relative Policy Optimization (GRPO) with no separate router, through a shaped reward that makes each mode worthwhile at a different response length, together with hard per-mode token caps that keep the modes distinct. On a 1.5B distilled model trained on MATH, the three modes emerge without collapsing to a single choice, and the brief modes end up more accurate than Long, which shows that the router sorts problems by difficulty rather than at random. Averaged over three seeds, the resulting policy stays close to the base model's accuracy on the held-out MATH500 (0.782 vs.\ 0.796) while cutting the mean response length from 4,796 to 2,811 tokens (a 41% reduction). Interestingly, it also transfers to other benchmarks without retraining, with the largest savings where problems are easier, with for instance 76% token reduction on GSM8K and at higher accuracy than the baselines at similar response length. In short, we build a reasoning model that adaptively chooses how much to reason for each problem.
Sampling multiple responses improves language model reasoning, but uniform compute allocation is inefficient because easy questions are over-sampled while hard questions remain under-explored. We propose Uncertainty-Aware Budget Allocation (UAB), a concave integer optimization framework that reallocates a fixed sampling budget using uncertainty estimated from the initial samples themselves. In Phase-1, every question receives a small fixed number of generations. Their answer disagreement, measured by vote entropy, provides a difficulty signal while these generations contribute to the final vote. In Phase-2, the remaining budget is allocated by a marginal-greedy algorithm that optimally solves a concave coverage-maximization surrogate, concentrating samples on questions whose initial answers disagree. Across five open-weight models (1.5B--27B parameters) and five reasoning benchmarks of varying difficulty, UAB improves average accuracy by +2.3% over uniform allocation, and achieves gains of up to +5.5% on individual benchmarks, with the largest gains in low-resource settings. Moreover, UAB meets the target budget exactly and requires no auxiliary model or additional LLM calls. Code is publicly available at https://github.com/manhitv/UAB.
Manh Nguyen, Sunil Gupta, Hung Le
Applied Artificial Intelligence Initiative Deakin University, Australia
The emergence of large reasoning models demonstrates that scaling inference-time compute significantly enhances performance on complex tasks. However, it often falls into another trap: overthinking simple problems, where repetitive rationales yield minimal accuracy gains at a disproportionately high cost. This motivates adaptive reasoning: dynamically aligning reasoning depth with instance difficulty. In this paper, we study adaptive reasoning from an optimality perspective, formalizing it as a utility maximization problem where tokens are allocated until the marginal accuracy gain falls below the incremental cost. Based on this, we propose CODA (Compute Allocation by Difficulty Awareness), a method that operationalizes this principle by allocating tokens via a policy-internal difficulty signal. Specifically, CODA estimates difficulty via group-based rollouts and maps it to two non-negative gates that modulate a length-dependent shaping term on top of the binary base reward. The easy-side gate penalizes verbosity on simple instances, whereas the hard-side gate encourages more deliberative rollouts on challenging ones. Across model scales and benchmarks, CODA achieves adaptive reasoning without external annotations or user-provided budgets: on easy tasks, CODA reduces token costs by over 60% while maintaining strong accuracy, whereas on hard tasks it incentivizes more deliberative rollouts to maximize performance.