LLM-based agents show strong potential for long-horizon reasoning, yet their context size is limited by deployment factors (e.g., memory, latency, and cost), yielding a constrained context budget. As interaction histories grow, this induces a trade-off between retaining past information and staying within the context limit. To address this challenge, we propose Budget-Aware Context Management (BACM), which formulates context management as a sequential decision problem with a context budget constraint. It enables agents to assess the available budget before incorporating new observations and decide when and how much of the interaction history to compress. We further develop BACM-RL, an end-to-end curriculum-based reinforcement learning approach that learns compression strategies under varying context budgets. Experiments on compositional multi-objective QA and long-horizon web browsing benchmarks show that BACM-RL consistently outperforms prior methods across model scales and task complexities, achieving over 1.6× gains over strong baselines in high-complexity settings, while maintaining strong advantages as budgets shrink, where most methods exhibit a downward performance trend.
Figures & tables
Figure 1: Overview of the proposed framework. (a) The agent first observes the budget-conditioned state bt=(st,rt,∣ot∣) before loading the pending observation. (b) Conditioned on bt , the policy selects a refinement action ut to perform Null , Partial , or Full commit-block aggregation, yielding an updated context Ct′ . (c) The policy is trained with multi-turn GRPO under a progressively tightened budget curriculum, where only trajectories satisfying the context budget contribute reward and optimization.
Method
Context
BrowseComp-Plus
Multi-objective QA
Easy
Mid
Hard
Avg
2-Obj
8-Obj
16-Obj
32-Obj
Qwen3-235B-A22B-Inst-2507
ReACT
128k
0.380
0.164
0.034
0.136
0.948
1.873
1.315
1.412
ReACT
8k
0.420
0.143
0.011
0.118
0.886
1.782
1.233
0.374
Qwen2.5-7B-Inst
ReACT
8k
0.320
0.098
0.027
0.089
0.395
0.458
0.376
0.090
Table 1: Performance comparison across QA settings. BrowseComp-Plus is evaluated using LLM-as-Judge , while Multi-object QA reports F1 scores summed across objectives . † Reproduced using officially released open-source checkpoints.; ‡ Re-implemented based on released code; \lx@sectionsign Re-implemented based on the original paper due to no public implementation.
Figure 2: Performance under different maximum context window sizes (16k-4k tokens) with varying numbers of objectives.
Figure 3: Cumulative F1 and average compression calls under a fixed 8k context budget.
Variant
Comp.
Budget
Search-R1 (Base)
−
−
Search-R1 (w/ B)
−
✓
Ours (w/o B)
✓
−
Ours (Full)
✓
✓
Table 2: Ablation configurations.
Figure 4: Ablation of key components across context budgets (16k–4k tokens), measured by summed F1 across objectives. Removing budget metadata (B) degrades performance ( Ours w/o B ), while adding B alone to a baseline without context management does not improve performance ( Search-R1 w/ B ). Our full model ( Ours Full ), which conditions compression on B, achieves the best performance across all objectives and budgets.
Variant
2-Obj
8-Obj
16-Obj
32-Obj
w/o Def.
0.833
2.706
3.972
2.818
w/ Def.
0.909
2.790
4.011
2.938
Table 3: Cumulative F1 of deferred vs. non-deferred loading across different objectives at 8k
Figure 5: Ablation of progressive budget curricula under a common 8k evaluation budget summed F1 over objectives
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Case study of budget-aware context management. The agent adaptively selects [NONE] , [Selective] , or [ALL] based on remaining budget, transitioning from full retention to partial and full aggregation as budget tightens, while preserving task-relevant information.
Method
Ctx
2 Obj
8 Obj
16 Obj
32 Obj
D
P
S
D
P
S
D
P
S
D
P
S
Summary
16k
664.469
1517.570
0.661
1451.613
3871.188
2.127
1658.180
4938.746
2.447
2894.227
6306.773
0.656
Search-R1
16k
306.648
792.320
0.768
1481.602
3769.055
2.016
2280.371
5729.789
1.643
3669.387
6810.332
0.372
Mem1
16k
304.439
632.912
0.838
652.492
914.818
2.345
876.083
1096.275
2.391
1046.315
1361.961
1.210
Ours
16k
445.133
2032.777
0.925
1227.461
2986.828
2.756
1919.848
3537.504
4.114
7600.406
8795.160
3.249
Search-R1
8k
304.152
793.641
0.762
1332.250
3445.043
1.805
1920.008
4952.941
1.612
2556.023
5065.641
0.461
Appendix
Table 4: mean dependent cost (D), mean peak tokens (P), and summed F1 (S) across context budgets and objective counts.
Figure 7: Answer rate comparison across context lengths and objectives.
Stage 1
Stage 2
Stage 3
Stage 4
Stage 5
Step Range
1–60
61–120
121–180
181–240
241–300
Max Model Length (tokens)
8,192
7,168
6,144
5,120
4,096
Appendix
Table 5: Curriculum Schedule for max_model_len
Hyperparameter
Value
Optimizer
AdamW
Actor Learning Rate
1×10−6
LR Schedule
Cosine
Warmup Steps
100
Rollout Group Size ( G )
5
KL Coefficient ( β )
0.001
Appendix
Table 6: Global Hyperparameters for RL Training
Figure 8: Training dynamics of Qwen2.5-7B-Instruct across Static (8k), Random (4k/8k), and Curriculum-Ours (8 → 4): Reward, KL Loss, Entropy, and Summary Tool Call Average Times.
LLM agents increasingly face long-horizon tasks such as web search and deep research in real-world applications, where accumulated context can cause long-context degradation and reasoning failures. Prior work mitigates this through context management with agent-side context control or fixed strategies such as summarization, which require training the agent itself for adaptation - making it impractical for closed-source agents and ignoring that different agents may require different strategies. We introduce Adaptive Context Management (AdaCoM), which trains an external LLM to manage the context of a frozen agent through flexible modification actions and end-to-end reinforcement learning. Across diverse agents on web search and deep research benchmarks, AdaCoM substantially improves performance by preserving task constraints and progress while pruning stale content. The learned strategies reveal a Fidelity-Reliability Trade-off: agents with higher vanilla ReAct performance benefit from higher-fidelity context preservation, whereas lower-performing agents require more aggressive compression to stay within a reliable reasoning regime. Transfer experiments show that AdaCoM generalizes most effectively across agents with similar capability (measured by vanilla ReAct performance), suggesting a practical path toward reusable context managers for agent systems.
Lu Yi, Runlin Lei, Liuyi Yao +6
1Renmin University of China · Work done during internship at Tongyi Lab, Alibaba Group · 2Tongyi Lab, Alibaba Group +2
Long-horizon search agents must manage a rapidly growing working context as they reason, call tools, and observe information. Naively accumulating all intermediate content can overwhelm the agent, increasing costs and the risk of errors. We propose that effective context management should be adaptive: parts of the agent's trajectory are maintained at different levels of detail depending on their current relevance to the task. To operationalize this principle, we introduce Context-ReAct, a general agentic paradigm for elastic context orchestration that integrates reasoning, context management, and tool use in a unified loop. Context-ReAct provides five atomic operations: Skip, Compress, Rollback, Snippet and Delete, which allow the agent to dynamically reshape its working context, preserving important evidence, summarizing resolved information, discarding unhelpful branches, and controlling context size. We prove that the Compress operator is expressively complete, while the other specialized operators provide efficiency and fidelity guarantees that reduce generation cost and hallucination risk. Building on this paradigm, we develop LongSeeker, a long-horizon search agent fine-tuned from Qwen3-30B-A3B on 10k synthesized trajectories. Across four representative search benchmarks, LongSeeker achieves 61.5% on BrowseComp and 62.5% on BrowseComp-ZH, substantially outperforming Tongyi DeepResearch (43.2% and 46.7%) and AgentFold (36.2% and 47.3%). These results highlight the potential of adaptive context management, showing that agents can achieve more reliable and efficient long-horizon reasoning by actively shaping their working memory.
Long-horizon agentic tasks require large language models (LLMs) to iteratively retrieve, integrate, and maintain dispersed information across multi-turn interactions, but preserving all interaction histories leads to a continuously growing working context. Recent proactive context management methods allow models to edit their own working context with specialized tools, yet they still face three key limitations: (1) a limited toolset restricted to search, deletion, and summarization, with no support for global planning, long-term memory, and adaptive compression; (2) inefficient exploration that treats context management actions uniformly despite their heterogeneous impacts on final outcomes; and (3) coarse-grained credit assignment that assigns the final trajectory-level reward to all intermediate context editing actions during RL. To bridge these gaps, we introduce ContextPilot, a proactive context management framework for long-horizon agentic reasoning. Our approach systematically augments the toolset with planning, long-term memory, and soft context offloading tools. We further propose an RL method tailored for context management, which uses context and entropy variation to identify critical editing decisions for branch sampling and estimates action-level advantages from all branched trajectories that pass through the corresponding context editing action. Experiments on long-context QA and deep search tasks show that ContextPilot achieves stronger performance with a more compact working context, consistently outperforming existing baselines across various base models and benchmarks. Code is available at https://github.com/Tencent/ContextPilot.
Zhuoshi Pan, Qizhi Pei, Junru Lu +4
Tsinghua University · Tencent Youtu Lab · Shanghai AI Lab