ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents

Yong Wu; YanZhao Zheng; TianZe Xu; ZhenTao Zhang; YuanQiang Yu; JiHuai Zhu; Chao Ma; BinBin Lin

cs.AIApr 2, 2026

ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents

Authors: Yong Wu, YanZhao Zheng, TianZe Xu, ZhenTao Zhang, YuanQiang Yu, JiHuai Zhu, Chao Ma, BinBin Lin,

Organizations: Zhejiang University, Hangzhou, China · Alibaba Group, Hangzhou, China

Abstract

LLM-based agents show strong potential for long-horizon reasoning, yet their context size is limited by deployment factors (e.g., memory, latency, and cost), yielding a constrained context budget. As interaction histories grow, this induces a trade-off between retaining past information and staying within the context limit. To address this challenge, we propose Budget-Aware Context Management (BACM), which formulates context management as a sequential decision problem with a context budget constraint. It enables agents to assess the available budget before incorporating new observations and decide when and how much of the interaction history to compress. We further develop BACM-RL, an end-to-end curriculum-based reinforcement learning approach that learns compression strategies under varying context budgets. Experiments on compositional multi-objective QA and long-horizon web browsing benchmarks show that BACM-RL consistently outperforms prior methods across model scales and task complexities, achieving over $1.6\times$ gains over strong baselines in high-complexity settings, while maintaining strong advantages as budgets shrink, where most methods exhibit a downward performance trend.

Figures & tables

Figure 1: Overview of the proposed framework. (a) The agent first observes the budget-conditioned state $b_{t}=(s_{t},r_{t},|o_{t}|)$ before loading the pending observation. (b) Conditioned on $b_{t}$ , the policy selects a refinement action $u_{t}$ to perform Null , Partial , or Full commit-block aggregation, yielding an updated context $\mathcal{C}^{\prime}_{t}$ . (c) The policy is trained with multi-turn GRPO under a progressively tightened budget curriculum, where only trajectories satisfying the context budget contribute reward and optimization.

Method	Context	BrowseComp-Plus				Multi-objective QA
Method	Context	Easy	Mid	Hard	Avg	2-Obj	8-Obj	16-Obj	32-Obj
Qwen3-235B-A22B-Inst-2507
ReACT	128k	0.380	0.164	0.034	0.136	0.948	1.873	1.315	1.412
ReACT	8k	0.420	0.143	0.011	0.118	0.886	1.782	1.233	0.374
Qwen2.5-7B-Inst
ReACT	8k	0.320	0.098	0.027	0.089	0.395	0.458	0.376	0.090

Table 1: Performance comparison across QA settings. BrowseComp-Plus is evaluated using LLM-as-Judge , while Multi-object QA reports F1 scores summed across objectives .

\dagger

Reproduced using officially released open-source checkpoints.;

\ddagger

Re-implemented based on released code;

\lx@sectionsign

Re-implemented based on the original paper due to no public implementation.

Figure 2: Performance under different maximum context window sizes (16k-4k tokens) with varying numbers of objectives.

Figure 3: Cumulative F1 and average compression calls under a fixed 8k context budget.

Variant	Comp.	Budget
Search-R1 (Base)	$-$	$-$
Search-R1 (w/ B)	$-$	$\checkmark$
Ours (w/o B)	$\checkmark$	$-$
Ours (Full)	$\checkmark$	$\checkmark$

Table 2: Ablation configurations.

Figure 4: Ablation of key components across context budgets (16k–4k tokens), measured by summed F1 across objectives. Removing budget metadata (B) degrades performance ( Ours w/o B ), while adding B alone to a baseline without context management does not improve performance ( Search-R1 w/ B ). Our full model ( Ours Full ), which conditions compression on B, achieves the best performance across all objectives and budgets.

Variant	2-Obj	8-Obj	16-Obj	32-Obj
w/o Def.	0.833	2.706	3.972	2.818
w/ Def.	0.909	2.790	4.011	2.938

Table 3: Cumulative F1 of deferred vs. non-deferred loading across different objectives at

8\text{k}

Figure 5: Ablation of progressive budget curricula under a common 8k evaluation budget summed F1 over objectives

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Case study of budget-aware context management. The agent adaptively selects [NONE] , [Selective] , or [ALL] based on remaining budget, transitioning from full retention to partial and full aggregation as budget tightens, while preserving task-relevant information.

Method	Ctx	2 Obj			8 Obj			16 Obj			32 Obj
Method	Ctx	D	P	S	D	P	S	D	P	S	D	P	S
Summary	16k	664.469	1517.570	0.661	1451.613	3871.188	2.127	1658.180	4938.746	2.447	2894.227	6306.773	0.656
Search-R1	16k	306.648	792.320	0.768	1481.602	3769.055	2.016	2280.371	5729.789	1.643	3669.387	6810.332	0.372
Mem1	16k	304.439	632.912	0.838	652.492	914.818	2.345	876.083	1096.275	2.391	1046.315	1361.961	1.210
Ours	16k	445.133	2032.777	0.925	1227.461	2986.828	2.756	1919.848	3537.504	4.114	7600.406	8795.160	3.249
Search-R1	8k	304.152	793.641	0.762	1332.250	3445.043	1.805	1920.008	4952.941	1.612	2556.023	5065.641	0.461

Appendix

Table 4: mean dependent cost (D), mean peak tokens (P), and summed F1 (S) across context budgets and objective counts.

Figure 7: Answer rate comparison across context lengths and objectives.

	Stage 1	Stage 2	Stage 3	Stage 4	Stage 5
Step Range	1–60	61–120	121–180	181–240	241–300
Max Model Length (tokens)	8,192	7,168	6,144	5,120	4,096

Appendix

Table 5: Curriculum Schedule for max_model_len

Hyperparameter	Value
Optimizer	AdamW
Actor Learning Rate	$1\times 10^{-6}$
LR Schedule	Cosine
Warmup Steps	100
Rollout Group Size ( $G$ )	5
KL Coefficient ( $\beta$ )	0.001

Appendix

Table 6: Global Hyperparameters for RL Training

Figure 8: Training dynamics of Qwen2.5-7B-Instruct across Static (8k), Random (4k/8k), and Curriculum-Ours (8

\rightarrow

4): Reward, KL Loss, Entropy, and Summary Tool Call Average Times.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 248 papers

Learning Agent-Compatible Context Management for Long-Horizon Tasks
May 29, 2026Lu Yi, Runlin Lei, Liuyi Yao +6Long-Horizon Agent Tasks LLM Context Management
1Renmin University of China · Work done during internship at Tongyi Lab, Alibaba Group · 2Tongyi Lab, Alibaba Group +2
LongSeeker: Elastic Context Orchestration for Long-Horizon Search Agents
May 6, 2026Yijun Lu, Rui Ye, Yuwen Du +3Web Search Agents Tool-Using Agents
Shanghai Jiao Tong University
ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL
Aug 28, 2026Zhuoshi Pan, Qizhi Pei, Junru Lu +4LLM Context Management Long-Horizon LLM Agents
Tsinghua University · Tencent Youtu Lab · Shanghai AI Lab

May 29, 2026J/K move · Enter open · S save

Learning Agent-Compatible Context Management for Long-Horizon Tasks

Lu Yi, Runlin Lei, Liuyi Yao, Yuexiang Xie+5

1Renmin University of China · Work done during internship at Tongyi Lab, Alibaba Group · 2Tongyi Lab, Alibaba Group+2

Long-Horizon Agent Tasks LLM Context Management RL for LLM Agents

Open paper AskPDF arXiv

LLM agents increasingly face long-horizon tasks such as web search and deep research in real-world applications, where accumulated context can cause long-context degradation and reasoning failures. Prior work mitigates this through context management with agent-side context control or fixed strategies such as summarization, which require training the agent itself for adaptation - making it impractical for closed-source agents and ignoring that different agents may require different strategies. We introduce Adaptive Context Management (AdaCoM), which trains an external LLM to manage the context of a frozen agent through flexible modification actions and end-to-end reinforcement learning. Across diverse agents on web search and deep research benchmarks, AdaCoM substantially improves performance by preserving task constraints and progress while pruning stale content. The learned strategies reveal a Fidelity-Reliability Trade-off: agents with higher vanilla ReAct performance benefit from higher-fidelity context preservation, whereas lower-performing agents require more aggressive compression to stay within a reliable reasoning regime. Transfer experiments show that AdaCoM generalizes most effectively across agents with similar capability (measured by vanilla ReAct performance), suggesting a practical path toward reusable context managers for agent systems.

Abstract

LLM Context Management

Explore similar work

Learning Agent-Compatible Context Management for Long-Horizon Tasks

LongSeeker: Elastic Context Orchestration for Long-Horizon Search Agents

ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL