LLM agents perform tasks over multiple steps and revise their actions based on feedback. As the total reasoning budget is often limited and must support future steps, the agent must choose between short and long reasoning before each action. Longer reasoning may improve the final task outcome by preventing later errors, but it also uses budget capacity that may be needed for future decisions. This creates two challenges: estimating the downstream value of long reasoning and deciding when that value is worth using the remaining budget. We propose Paired Outcome-guided Reasoning Allocation (PORA), which learns this decision by comparing short and long reasoning from the same interaction state. During training, \method{} completes the task twice from each selected state, once with a short reasoning allowance and once with a long allowance, and compares their final quality and total processing to estimate when longer reasoning is valuable. During inference, a controller weighs this estimate against learned prices on remaining-budget pressure and chooses either short or long reasoning without running both alternatives. We evaluate PORA across application, memory, code, retail, and mathematics tasks, achieving strong results across all settings. On BFCL memory, it exceeds the strongest existing method by 17.4--32.9 percentage points. Within the same agent, it matches or exceeds always using long reasoning on code and mathematics with 50.6% fewer actor tokens and 31.2% fewer model requests, respectively.
Figures & tables
Figure 1: Deciding how long to reason at one step (illustrative). ❶ From the same state, short reasoning leads to a wrong write and two repairs, while long reasoning writes correctly; which is better is known only after the task ends. ❷ Long reasoning uses budget that later steps also need, so the same choice can help with ample budget and hurt with a tight one.
Figure 2: Overview of PORA . \raisebox{-0.9pt}{\footnotesize1}⃝ From one checkpoint (si,bi,Ni) , a Preserve branch and a Spend branch continue under the same policy ρk ; their differences in terminal quality and complete processing give the paired targets (qi,gi) . \raisebox{-0.9pt}{\footnotesize2}⃝ The controller learns to predict these targets from the visible history and learns the prices on the pressure terms (Oi,Hi) through the direction label di . \raisebox{-0.9pt}{\footnotesize3}⃝ At inference, the predicted gain is weighed against the priced pressure, and native rules map the proposal to a feasible grant κt . The dashed arrow repeats collection and training under the updated policy. The calendar case is illustrative.
AppWorld
BFCL memory
LCB
τ -bench
MATH500
KV
Vector
Recursive
retail
# tasks
417
155
155
155
126
40
500
Reasoning-budget methods
Plan and Budget
27.3
18.1
16.1
58.7
65.1
62.5
94.8
Certaindex
4.1
41.9
15.5 †
25.8 †
39.7
72.5
73.4
Search and rollout methods
Table 1: Task success rate (%) on seven settings from five benchmarks. Completed cells report percentages over the full task counts shown above. A dash marks a setting without a complete run. Fixed Preserve and fixed Spend run the PORA agent with every allowance decision set to short or long; the LCB entries come from the shared-initial-program study, in which PORA also solves 93 of 126 problems. Bold marks the highest completed rate in each column, including ties. The last row gives PORA minus the best completed external method in percentage points. † Action-adapted configuration (Appendix B ).
Figure 3: Task success against total tokens. Each marker is one method with a recorded token total; hollow markers are lower bounds. The dotted line marks PORA , and each subtitle gives its difference from the best external method in Table 1 . The logarithmic horizontal axis differs across panels.
Variant
Succ.
Δ
Tokens
Allocation policy
BFCL vector ( PORA : 81.3, 3.39M tokens)
Fixed Preserve
71.0
10.3
3.14
Fixed Spend
74.8
6.5
3.84
LCB, shared initial programs ( PORA : 73.8, 0.51M)
Fixed Preserve
68.3
5.6
0.26
Table 2: Controlled comparisons. Each variant changes one factor of the PORA run in its block header. Δ is PORA minus the variant in points. Tokens are actor tokens in millions.
Figure 4: Fixed and learned allocation in the same agent where all three token totals are recorded. Shading marks the region cheaper and more accurate than fixed Spend ; hollow markers are lower bounds.
Coefficient
Success
Repairs
Requests
0.5×
475/500
207
712
1.0×
476/500
187
689
2.0×
474/500
153
655
Table 3: Native MATH500 sensitivity to the processing coefficient with a frozen controller.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Contrast
Gain
95% interval
Adj. p
Bank 16,384 vs 8,192
−5.81
[−14.55,0.80]
0.750
Bank 4,096 vs 8,192
−1.94
[−5.00,0.00]
0.750
Price 0.5× vs 2×
−3.23
[−11.11,3.00]
0.750
Price 0.5× vs 1×
−8.39
[−14.29,−4.67]
0.188
Price 1× vs 2×
5.16
[−0.80,9.50]
0.750
Appendix
Table 4: Additional vector-memory interventions. Differences are percentage points; adjusted p uses the original price or budget family.
Memory mode
Actor positions
Request starts
Encoder positions
Service (s)
Task time (s)
Key–value
6,760,203
1,673
326,149
4,699.93
4,891.14
Vector
3,387,257
1,028
1,056,653
3,117.61
3,866.24
Recursive summary
5,483,736
1,431
409,287
15,604.28
15,917.76
Appendix
Table 5: PORA memory resources, including prerequisite preparation and retained attempts. Actor and encoder positions use their respective tokenizers. Service time sums request durations; task time sums retained task-attempt durations.
Policy
Correct /126
Tokens
Starts
Paired terminal
93
506,398
173
Shuffled terminal
83
286,347
133
Absolute utility
92
≥ 525,529
178
Fixed Preserve
86
264,591
126
Feasible fixed Spend
92
1,025,266
371
Appendix
Table 6: Code policies with shared initial programs and execution. Actor tokens include inherited initial generation and every new attempt. The absolute-utility token total is a lower bound because one timeout has unresolved usage.
Controller input
Correct /126
Rate (%)
Full public-obligation fields
90
71.43
Six numerical fields set to zero
82
65.08
Appendix
Table 7: Public-obligation input study on all 126 code problems. Both rows are separately refitted, development-selected controllers. The full-input row belongs to this control family; the primary paired-terminal deployment in Table 6 completes 93/126.
Resource
PORA
FoT
Reduction 95% CI
Model tokens
3,331,003
98,364,608
[96.31,96.86]%
Requests
722
21,434
[96.40,96.82]%
Service (s)
10,421.51
151,898.33
[92.08,93.95]%
Task time (s)
10,748.02
152,362.67
[91.86,93.78]%
Appendix
Table 8: Retail resources over forty tasks. Reduction intervals resample the same 32 dependence groups.
PORA
Fixed Preserve
Fixed Spend
Actor
Memory
Fit/select
Rate
Tokens
Rate
Tokens
Rate
Tokens
Mistral Small 3.2 24B
Key–value
48 / 24; refit 72
40.6
5.84M
38.1
5.90M
36.8
5.78M
Mistral Small 3.2 24B
Vector
160 / 40; refit 200
52.3
3.26M
45.2
2.01M
50.3
2.11M
Mistral Small 3.2 24B
Recursive
160 / 40; refit 200
57.4
2.53M
57.4
2.50M
60.0
2.44M
Phi-4 14B
Key–value
160 / 40; refit 200
32.3
12.40M
33.5
12.79M
34.2
14.91M
Appendix
Table 9: Additional actors on BFCL memory. Success rates (%) use all 155 questions. Fit/select gives the paired fitting and selection examples. Actor tokens include native memory preparation and evaluation. Within each actor, the fixed policies share its memory interface with PORA .
Subject
Tasks
PORA
Spend
Saved (%)
Algebra
124
144
248
41.94
Counting & Probability
38
62
76
18.42
Geometry
41
60
82
26.83
Intermediate Algebra
97
157
194
19.07
Number Theory
62
78
124
37.10
Prealgebra
82
106
165
35.76
Appendix
Table 10: Physical model requests by MATH500 subject. PORA and fixed Spend share each initial solution; counts include every retained initial and revision attempt. Savings are 100(1−CPORA/CSpend) .
Comparator
/500
Gain
95% interval
Holm p
Fixed Preserve
459
3.60
[2.00,5.40]
0.000031
Fixed Spend
475
0.40
[−0.40,1.20]
1.0
Plan and Budget
474
0.60
[−0.60,1.80]
1.0
Forest-of-Thought
459
3.60
[1.60,5.60]
0.00160
Appendix
Table 11: MATH500 quality relative to PORA ’s 477/500. Intervals and differences are percentage points; adjusted probabilities use the complete four-comparison family.
Enhancing the reasoning capabilities of large language model (LLM) agents requires effective orchestration of diverse expert models and tools. However, existing frameworks typically call APIs, based on coarse-grained matching between tasks and the functions of expert models or tools, while overlooking critical factors such as performance variability and cost efficiency among functionally similar alternatives. To address this, we propose Agora, a framework that uses a confidence-calibrated auction to dynamically allocate tasks to expert models and tools. By treating reasoning steps as tradeable items, Agora bases allocation on calibrated competence rather than raw confidence. Across five main benchmarks, Agora improves or remains competitive with single-model, routing, and cascade baselines under matched candidate pools.
Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computation a task requires. We study budgeted inference-time reasoning for multiple problems under a strict global token constraint and formulate it as an Ordered Stochastic Multiple-Choice Knapsack Problem (OS-MCKP). This perspective highlights a meta-cognitive requirement---anticipating problem difficulty and allocating computation strategically. We propose ROI-Reasoning, a two-stage framework that endows LLMs with intrinsic, budget-aware rationality. First, Meta-Cognitive Fine-Tuning teaches models to predict reasoning cost levels and decide whether a problem is worth attempting before generation. Next, Rationality-Aware Reinforcement Learning optimizes sequential decision making under a hard token budget, allowing models to learn long-horizon allocation strategies. Experiments across mathematical reasoning, multiple-choice question answering, and code generation show that ROI-Reasoning improves scores and generally reduces reference-order regret under tight budgets, with up to a 417% relative score improvement over the strongest same-backbone baseline in mathematics.
Muyang Zhao, Chunhao Tian, Qi Qi +1
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
Agents powered by large language models (LLMs) have demonstrated remarkable progress in solving complex reasoning tasks. However, LLM agents often falter on long-horizon tasks due to cognitive overload, as their working memory becomes cluttered with expanding and irrelevant information, which dilutes their attention and hinders effective planning and reasoning. To mitigate this challenge, we introduce Cognitive Resource Self-Allocation (CORAL), a novel reasoning paradigm that empowers agents to proactively optimize their context. Implemented as an agent-callable working memory management toolset, CORAL allows an agent to checkpoint its task progress and verified facts, and adaptively initiate a new problem-solving episode by resetting cluttered context while preserving only the checkpointed information, enabling it to resume reasoning from a clean state. We further enhance the agent's checkpointing capabilities using a Multi-episode Agentic Reinforced Policy Optimization algorithm. On the GAIA benchmark, CORAL with an 8B model achieves 42.8% average accuracy, with particularly strong gains on the most challenging long-horizon tasks (Level 2 and Level 3). Crucially, these gains come at a substantially lower inference cost: CORAL consumes 6-7× fewer input tokens than a ReAct agent. It also generalizes to multi-hop Knowledge QA benchmarks and to the considerably harder BrowseComp, where it improves a strong proprietary agent by 8.0 points.
Zheng Zhu, Xiaozhuan Liang, Xi Chen +2
The Chinese University of Hong Kong · Tencent · The Hong Kong University of Science and Technology