Long-horizon LLM agents can coordinate tools and applications to complete complex, multi-step tasks. As their interaction histories grow, effective context management becomes essential for reliable and efficient execution. Existing methods reduce context through summarization, pruning, or history selection. However, shorter contexts do not necessarily support better decisions: summaries can omit exact details needed for later actions, while retaining entire interactions can carry substantial irrelevant information. We propose CarryOn, a lightweight context manager that pairs compact execution state with fine-grained, source-linked evidence. A learned selector and adaptive retention policy keep relevant evidence available without additional LLM calls for context management. We introduce two complementary evaluation metrics: Sα prioritizes task success while accounting for resource use, and normalized cost per success (NCS) measures the resources spent per successful task. Across AppWorld, OfficeBench, and Tau2 Telecom with three LLM Actors, CarryOn achieves 75.7% macro-averaged task success, exceeding the strongest baseline by 6.7 percentage points. It also achieves the best average scores on both metrics.
Figures & tables
Figure 1: CarryOn: carry what matters. Left: Context-management challenges and CarryOn’s separation of compact task state from fine-grained evidence. Right: Success–efficiency comparison (top) and task-success gains over the strongest baseline, averaged by benchmark or Actor (bottom).
Figure 2: CarryOn overview. (1) History: The archive supplies fine-grained observation units. (2) Evidence selection: The selector ranks units using the task and latest observation. (3) Acting: The Actor combines selected evidence, recent history, and compact state to act and update its state. Selector training (bottom): Annotation citations and later evidence use supervise ranking of action–observation blocks. The frozen selector scores finer evidence units at test time.
Dataset
Method
Claude Sonnet 5
Qwen3.8-27B
Gemma-4-31B
Success (%) ↑
S0.8↑
NCS ↓
Success (%) ↑
S0.8↑
NCS ↓
Success (%) ↑
S0.8↑
NCS ↓
Full History
43.5
39.11
1.000
65.5
58.93
1.000
53.6
48.21
1.000
FIFO-5
42.9
37.62
1.592
53.6
47.83
1.410
48.8
43.96
1.084
Retrieval
11.9
10.24
8.549
44.0
38.50
2.525
28.6
24.80
3.636
AppWorld
LLMLingua-2
21.4
19.20
2.195
44.6
39.90
1.658
41.7
37.77
1.127
ACON
70.2
62.84
0.688
53.0
47.08
1.553
48.8
44.01
1.063
Table 1: Main results across three benchmarks and three Actors. Bold marks the best result for each benchmark and Actor, including ties.
Figure 3: Mean calls per task, averaged equally across benchmarks and Actors. Failed tasks are included.
Figure 4: Age of evidence selected by CarryOn, excluding automatic recent retention. Percentages pool selection events across Actors within each benchmark.
Method
Success (%) ↑
S0.8↑
NCS ↓
w/o MEMORY
66.1
58.72
0.824
Ranking only
76.8
69.09
0.568
Fine-grained training
73.2
65.68
0.629
Recency packing
76.2
68.36
0.603
CarryOn
79.8
71.71
0.556
Table 2: Component and training variants on AppWorld with Claude Sonnet 5. Bold marks the best result.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: MEMORY capacity sensitivity on AppWorld with Claude. Dashed lines mark the default capacity of 128 proxy tokens.
Capacity
Success (%) ↑
S0.8↑
NCS ↓
Calls
Tokens (M)
Truncations
64
77.4
69.48
0.585
3,163
25.507
284
128
79.8
71.71
0.556
3,098
24.906
48
256
78.0
70.24
0.548
3,001
23.920
8
Appendix
Table 3: MEMORY capacity sensitivity on AppWorld with Claude. Capacity is measured in proxy tokens.
Figure 6: Sensitivity to metric weights. Scores are averaged equally across benchmarks and Actors. Dashed lines mark the default weights.
Method
TGC ↑
Requirements
Calls
Input (M)
Output (M)
Total (M)
Full History
73/168
774/1,077
2,727
27.253
0.314
27.567
FIFO-5
72/168
498/1,077
5,369
31.839
0.456
32.296
Retrieval
20/168
232/1,077
7,284
47.792
0.569
55.497
LLMLingua-2
36/168
499/1,077
3,572
23.199
0.363
23.562
ACON
118/168
950/1,077
3,608
22.485
0.369
24.857
AGORA
93/168
723/1,077
6,012
35.196
0.684
35.881
Appendix
Table 4: Detailed AppWorld results with Claude Sonnet 5.
Method
TGC ↑
Requirements
Calls
Input (M)
Output (M)
Total (M)
Full History
110/168
790/1,077
1,361
9.386
3.295
12.681
FIFO-5
90/168
669/1,077
1,674
9.674
3.997
13.672
Retrieval
74/168
577/1,077
2,279
14.017
4.830
21.854
LLMLingua-2
75/168
574/1,077
1,630
8.491
4.996
13.487
ACON
89/168
666/1,077
1,797
8.818
4.737
15.127
AGORA
69/168
552/1,077
1,664
7.867
6.244
14.111
Appendix
Table 5: Detailed AppWorld results with Qwen3.8-27B.
Method
TGC ↑
Requirements
Calls
Input (M)
Output (M)
Total (M)
Full History
90/168
767/1,077
2,401
16.740
1.386
18.126
FIFO-5
82/168
735/1,077
2,820
13.088
1.434
14.522
Retrieval
48/168
454/1,077
5,231
24.511
1.858
30.800
LLMLingua-2
70/168
692/1,077
2,428
11.984
1.459
13.443
ACON
82/168
772/1,077
2,650
12.721
1.311
15.109
AGORA
66/168
666/1,077
4,103
17.054
2.459
19.513
Appendix
Table 6: Detailed AppWorld results with Gemma-4-31B.
Method
TGC ↑
Conditions
Calls
Input (M)
Output (M)
Total (M)
Full History
60/95
113/148
1,484
29.454
0.410
29.864
FIFO-5
55/95
101/141
1,744
5.680
0.336
6.016
Retrieval
47/95
69/117
2,384
6.793
0.313
8.483
LLMLingua-2
61/95
124/158
1,569
6.067
0.286
6.353
ACON
64/95
131/162
1,574
7.385
0.272
8.410
AGORA
47/95
93/141
2,381
10.735
0.461
11.196
Appendix
Table 7: Detailed OfficeBench results with Claude Sonnet 5.
Method
TGC ↑
Conditions
Calls
Input (M)
Output (M)
Total (M)
Full History
63/95
128/160
1,531
10.807
1.668
12.475
FIFO-5
51/95
93/137
1,970
6.638
1.375
8.013
Retrieval
49/95
85/131
2,230
8.411
1.422
12.729
LLMLingua-2
57/95
104/142
1,716
5.729
1.554
7.282
ACON
60/95
116/151
1,611
4.387
1.856
6.845
AGORA
44/95
83/134
2,221
9.631
1.589
11.220
Appendix
Table 8: Detailed OfficeBench results with Qwen3.8-27B.
Method
TGC ↑
Conditions
Calls
Input (M)
Output (M)
Total (M)
Full History
49/95
93/139
1,518
29.123
0.977
30.100
FIFO-5
52/95
105/148
2,110
3.442
1.242
4.684
Retrieval
33/95
60/122
2,938
4.264
1.285
6.338
LLMLingua-2
51/95
118/162
1,575
3.849
1.067
4.915
ACON
55/95
111/151
1,669
3.926
0.950
5.114
AGORA
37/95
69/127
2,986
5.408
1.447
6.855
Appendix
Table 9: Detailed OfficeBench results with Gemma-4-31B.
Method
Success
Pass 1 / Avg. Reward
Calls A/U/C
Input (M)
Output (M)
Total (M)
Full History
30/40
0.750
665/853/0
13.287
0.136
13.423
FIFO-5
29/40
0.725
816/963/0
15.385
0.171
15.556
Retrieval
17/40
0.425
1,621/1,301/0
28.615
0.276
28.891
LLMLingua-2
32/40
0.800
694/878/7
13.977
0.145
14.122
ACON
29/40
0.725
641/799/2
12.647
0.135
12.782
AGORA
25/40
0.625
1,087/1,075/0
19.423
0.234
19.657
Appendix
Table 10: Detailed Tau2 Telecom results with Claude Sonnet 5. A/U/C denote Actor, User, and compressor calls.
Method
Success
Avg. Reward
Calls A/U/C
Input (M)
Output (M)
Total (M)
Full History
32/40
0.800
762/890/0
12.619
0.305
12.924
FIFO-5
35/40
0.875
818/981/0
13.385
0.340
13.725
Retrieval
29/40
0.725
995/975/0
14.615
0.386
15.002
LLMLingua-2
32/40
0.800
778/885/5
12.361
0.260
12.621
ACON
35/40
0.875
730/859/3
11.607
0.276
11.883
AGORA
20/40
0.500
2,042/1,623/0
27.498
0.805
28.303
Appendix
Table 11: Detailed Tau2 Telecom results with Qwen3.8-27B. A/U/C denote Actor, User, and compressor calls.
Method
Success
Pass 1 / Avg. Reward
Calls A/U/C
Input (M)
Output (M)
Total (M)
Full History
25/40
0.625
561/647/0
8.658
0.224
8.882
FIFO-5
24/40
0.600
601/649/0
8.231
0.256
8.487
Retrieval
14/40
0.350
1,187/657/0
14.266
0.513
14.779
LLMLingua-2
19/40
0.475
622/659/5
9.075
0.239
9.314
ACON
22/40
0.550
719/765/5
10.747
0.263
11.010
AGORA
13/40
0.325
1,087/960/0
13.921
0.430
14.351
Appendix
Table 12: Detailed Tau2 Telecom results with Gemma-4-31B. A/U/C denote Actor, User, and compressor calls.
Figure 7: Claude Sonnet 5: Success and resource use across three benchmarks, averaged over fixed-order task prefixes.
Figure 8: Qwen3.8-27B: Success and resource use across three benchmarks, with the same metrics and task order as Figure 7 .
Figure 9: Gemma-4-31B: Success and resource use across three benchmarks, with the same metrics and task order as Figure 7 .
Benchmark
Actor
Task runs
Selected evidence ≥10 steps old (%)
Runs with evidence reselection
AppWorld
Claude
168
31.4
96/168
AppWorld
Qwen
168
32.5
94/168
AppWorld
Gemma
168
27.6
105/168
OfficeBench
Claude
95
40.5
22/95
OfficeBench
Qwen
95
19.2
22/95
OfficeBench
Gemma
95
28.0
13/95
Appendix
Table 13: Long-term evidence retention and reselection. Evidence age is measured in Actor steps, with repeated selections counted separately. The last column reports runs with evidence selected again after leaving the pack, out of all runs, including failures. “–” indicates no learned-selection events.
Figure 11: A payment-card record returns to the evidence pack and is used to add a card, refill the balance, and complete the payments.
Figure 12: Case 1 (AppWorld, Claude Sonnet 5). ACON’s summary omits two qualifying requests, shown with dashed outlines, while CarryOn retains the complete list.
Figure 13: Case 2 (AppWorld, Gemma-4-31B). Dashed outlines mark fields filtered by SWE-Pruner and preserved by CarryOn. Fractions report passed task requirements.
Figure 14: Success by task group, averaged across Actors. The strongest baseline is selected separately for each group. Δ denotes the difference in percentage points.
Actor
Method
Easy
Medium
Hard
Overall
Claude Sonnet 5
Full History
34/57
21/48
18/63
73/168
Claude Sonnet 5
FIFO-5
49/57
17/48
6/63
72/168
Claude Sonnet 5
Retrieval
16/57
1/48
3/63
20/168
Claude Sonnet 5
LLMLingua-2
24/57
7/48
5/63
36/168
Claude Sonnet 5
ACON
48/57
36/48
34/63
118/168
Claude Sonnet 5
AGORA
48/57
23/48
22/63
93/168
Appendix
Table 14: Successful tasks by AppWorld difficulty.
Actor
Method
L1 TGC
L2 TGC
L3 TGC
Passed/checked
Unchecked
Claude Sonnet 5
Full History
31/42
12/22
17/31
113/148
61
Claude Sonnet 5
FIFO-5
31/42
13/22
11/31
101/141
68
Claude Sonnet 5
Retrieval
32/42
10/22
5/31
69/117
92
Claude Sonnet 5
LLMLingua-2
33/42
14/22
14/31
124/158
51
Claude Sonnet 5
ACON
30/42
17/22
17/31
131/162
47
Claude Sonnet 5
AGORA
26/42
10/22
11/31
93/141
68
Appendix
Table 15: Successful tasks and evaluator coverage by OfficeBench level. Unchecked conditions were not evaluated.
Actor
Method
Easy
Hard
None
MMS
Mobile data
Service
Claude Sonnet 5
Full History
11/14
10/13
9/13
6/16
9/9
15/15
Claude Sonnet 5
FIFO-5
9/14
9/13
11/13
7/16
7/9
15/15
Claude Sonnet 5
Retrieval
5/14
6/13
6/13
5/16
4/9
8/15
Claude Sonnet 5
LLMLingua-2
13/14
10/13
9/13
8/16
9/9
15/15
Claude Sonnet 5
ACON
11/14
8/13
10/13
5/16
9/9
15/15
Claude Sonnet 5
AGORA
9/14
9/13
7/13
6/16
6/9
13/15
Appendix
Table 16: Successful Tau2 tasks by user persona and issue type.
Overall results
Dataset
Success (%)
Completed/total
Actor calls
Tokens (M)
AppWorld
47.0
79/168
4,961
30.982
OfficeBench
46.3
44/95
1,219
7.311
Tau2 Telecom
67.5
27/40
998
19.473
Appendix
Table 17: CarryOn-E (9B) results across benchmarks and task groups.
LLM agents increasingly face long-horizon tasks such as web search and deep research in real-world applications, where accumulated context can cause long-context degradation and reasoning failures. Prior work mitigates this through context management with agent-side context control or fixed strategies such as summarization, which require training the agent itself for adaptation - making it impractical for closed-source agents and ignoring that different agents may require different strategies. We introduce Adaptive Context Management (AdaCoM), which trains an external LLM to manage the context of a frozen agent through flexible modification actions and end-to-end reinforcement learning. Across diverse agents on web search and deep research benchmarks, AdaCoM substantially improves performance by preserving task constraints and progress while pruning stale content. The learned strategies reveal a Fidelity-Reliability Trade-off: agents with higher vanilla ReAct performance benefit from higher-fidelity context preservation, whereas lower-performing agents require more aggressive compression to stay within a reliable reasoning regime. Transfer experiments show that AdaCoM generalizes most effectively across agents with similar capability (measured by vanilla ReAct performance), suggesting a practical path toward reusable context managers for agent systems.
Lu Yi, Runlin Lei, Liuyi Yao +6
1Renmin University of China · Work done during internship at Tongyi Lab, Alibaba Group · 2Tongyi Lab, Alibaba Group +2
Long-horizon agentic tasks require large language models (LLMs) to iteratively retrieve, integrate, and maintain dispersed information across multi-turn interactions, but preserving all interaction histories leads to a continuously growing working context. Recent proactive context management methods allow models to edit their own working context with specialized tools, yet they still face three key limitations: (1) a limited toolset restricted to search, deletion, and summarization, with no support for global planning, long-term memory, and adaptive compression; (2) inefficient exploration that treats context management actions uniformly despite their heterogeneous impacts on final outcomes; and (3) coarse-grained credit assignment that assigns the final trajectory-level reward to all intermediate context editing actions during RL. To bridge these gaps, we introduce ContextPilot, a proactive context management framework for long-horizon agentic reasoning. Our approach systematically augments the toolset with planning, long-term memory, and soft context offloading tools. We further propose an RL method tailored for context management, which uses context and entropy variation to identify critical editing decisions for branch sampling and estimates action-level advantages from all branched trajectories that pass through the corresponding context editing action. Experiments on long-context QA and deep search tasks show that ContextPilot achieves stronger performance with a more compact working context, consistently outperforming existing baselines across various base models and benchmarks. Code is available at https://github.com/Tencent/ContextPilot.
Zhuoshi Pan, Qizhi Pei, Junru Lu +4
Tsinghua University · Tencent Youtu Lab · Shanghai AI Lab
LLM agents increasingly take on long-running tasks whose history grows far beyond a single model context window. Existing approaches compress earlier interactions or extract selected information into fixed memory representations, committing to what to preserve before future needs are known. We present Scroll, a context manager that treats each agent session as an executable Session Environment. The environment is backed by an append-only Event Log and a sandboxed, persistent Python kernel. The kernel maintains a typed namespace across model calls, allowing tool outputs, retrieved history, and derived state to be bound to variables rather than serialized into the prompt at each call. Model-written code searches, materializes, and transforms session state through exec; only explicitly printed projections enter the model's working view for the next call. Context management thus becomes a programming task that inherits the improving coding abilities of LLMs, while the Event Log preserves lossless historical ground truth. As the working view approaches its budget, stale spans are evicted but remain recoverable: an eviction index keeps compact landmarks tied to exact Event Log addresses, so that the agent navigates directly to evicted regions instead of searching the full log. With Qwen3.8-Max as the backbone, Scroll achieves 94.8% on LongMemEval_S; 73.1% on BEAM_10M, surpassing the best published memory system by 5.1 points; and 86.7% on LOCA_256K, exceeding the best published long-horizon agent by 37.4 points.