Beyond Skill Evolution: Self-Evolving Context Management Policies for Long-Horizon Agent Harnesses

Weiyuan Li; Jinghan Xu; Aili Chen; Xintao Wang; Shuang Liang; Jiaqing Liang; Deqing Yang

cs.AISep 28, 2026

Beyond Skill Evolution: Self-Evolving Context Management Policies for Long-Horizon Agent Harnesses

Authors: Weiyuan Li, Jinghan Xu, Aili Chen, Xintao Wang, Shuang Liang, Jiaqing Liang, Deqing Yang

Organizations: School of Data Science, Fudan University · Shanghai Key Laboratory of Data Science · College of Computer Science and Artificial Intelligence, Fudan University

Abstract

Harness evolution improves LLM agents by learning from execution trajectories, but existing experience- and skill-based methods are less effective on long-horizon tasks. As interactions grow, useful evidence can be buried by redundant or outdated context, making context management itself a key bottleneck. We introduce ContextEvo, a framework that learns a context policy from long-horizon trajectories. ContextEvo reconstructs the model-visible context at key decision points, identifies context-related failures, and applies targeted policy updates. Starting from the open-source Pi-agent harness, ContextEvo improves performance across three long-horizon task benchmarks, achieving results comparable to or better than several prominent agent harnesses, including Codex, OpenCode, and OpenClaw. Additional analyses show that fixed or locally evolved context strategies can fall short under long-horizon information pressure, while our methods adapt to the information demands of each environment.

Figures & tables

Figure 1: From skill evolution to context-policy evolution. As interaction histories grow, skill evolution may fail at both stages: extracting low-quality skills during evolution and invoking them at the wrong time or following them unreliably during evaluation. In contrast, our ContextEvo evolves an efficient context policy, achieving higher scores with fewer tokens.

Dimension	Role	Common realizations
Input Assembly	Context entry preparation	Tool-result formatting; observation compression
History Maintenance	Historical context management	Compaction; summarization; persistent state
Context Orchestration	Context access and sharing across agents and subtasks	Instructions and tools for context access; subagent instructions and feedback design

Table 1: Context-policy dimensions and common realizations in agent harnesses.

Figure 2: The overview of ContextEvo . The context policy governs how information is presented, organized, and shared through the agent harness while the base model remains fixed. ContextEvo reconstructs trajectory evidence, attributes context-management failures to policy dimensions, and applies constrained updates for the next evolution round.

Harness (DeepSeek-V4-Flash)
	Long-Horizon TB			DeepSWE			BrowseComp-Plus
Harness / policy	Held-in (%)	Held-out (%)	All (%)	Held-in (%)	Held-out (%)	All (%)	Held-in (%)	Held-out (%)	All (%)
OpenCode	36.1	40.3	38.5	62.5	66.2	64.6	92.1	77.6	83.9
Codex	41.8	40.1	40.8	45.8	63.1	55.8	84.2	84.5	84.4
OpenClaw	36.9	45.9	42.0	45.8	60.0	54.0	89.5	88.8	89.1
Pi-agent (base)	43.3	40.4	41.7	54.2	72.3	64.6	81.6	82.7	82.2

Table 2: Performance of baseline harnesses and ContextEvo over two evolution iterations. Scores are reported on the evolution (Held-in), evaluation (Held-out), and full task sets. Values in parentheses indicate percentage-point changes from the preceding baseline (Pi-agent for iter-1 and iter-1 for iter-2). Bold and underline indicate the best and second-best scores, respectively.

Agent	Reward $\uparrow$	Tokens (M) $\downarrow$
Baseline (Pi)	0.417	8.57
Skill-only	0.370	9.61
ContextEvo	0.448	7.52
ContextEvo + skill	0.419	9.69

Table 3: Context-policy vs. skill evolution on LHTB-46. Reward and mean token usage per run are reported for different settings.

Static context management
Method	DeepSWE	LHTB	BrowseComp-Plus
Pi-agent base	0.133	0.303	0.379
ReSum	0.062	0.266	0.385
ACON	0.124	0.262	0.374
Dynamic context management
TACO	0.115	0.281	0.374

Table 4: Performance comparison across context-management methods.

Figure 3: Environment-specific context problems and policy transfer. Bars show the share of audited cases with each multi-select context problem; labels report counts. The heatmap shows signed score changes (percentage points) from applying the LHTB-evolved policy under DeepSeek and Luna.

Variant	Held-in	Held-out	All	$\Delta$ vs. Full
Native Pi-agent	43.30%	40.39%	41.66%	$-3.13$ pp
Full ContextEvo	46.95%	43.12%	44.78%	–
w/o Paginated Trajectory Review	41.01%	26.19%	32.64%	$-12.15$ pp
w/o Cross-Case Aggregation	40.46%	44.05%	42.49%	$-2.29$ pp
w/o Counterfactual Replay	48.57%	35.29%	41.06%	$-3.72$ pp
w/o Context-Policy Update	32.57%	42.78%	38.34%	$-6.44$ pp

Table 5: LHTB-46 component ablations with DeepSeek-V4-Flash-0731 under the matched Pi-agent protocol;

\Delta

denotes the all-task difference from Full.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Scenario	All	Held-in	Held-out
LHTB	terminal workflows	46	20	26
DeepSWE	repository software engineering	113	48	65
BrowseComp-Plus	multi-hop deep research	174	76	98

Appendix

Table 6: Benchmark inventories and fixed evolution splits. Held-in tasks provide trajectories for evidence reconstruction, attribution, replay, and policy updating; held-out tasks are reserved for evaluation.

Benchmark	Trajectories	Tool calls	Agent turns	Estimated trace tokens
LHTB	44	$75.0\;[52.5,\,96.5]$	$69.0\;[45.0,\,87.3]$	$80{,}712\;[58{,}862,\,111{,}707]$
DeepSWE	112	$120.5\;[85.8,\,158.0]$	$103.5\;[65.8,\,138.5]$	$124{,}205\;[92{,}726,\,157{,}711]$
BrowseComp-Plus	174	$32.5\;[17.3,\,52.8]$	$27.5\;[14.0,\,45.8]$	$70{,}866\;[38{,}626,\,128{,}303]$

Appendix

Table 7: Workload statistics from frozen Pi/DeepSeek-V4-Flash baseline trajectories. Entries are median

[\mathrm{p25},\mathrm{p75}]

; trace tokens estimate analyst-visible text and are not billing tokens.

Selection stage	Criterion
Reference difficulty	o3 and GPT-5 reference answers are both judged incorrect
Long-horizon filter	o3 issues at least 30 searches
Resulting subset	174 questions from the 830-question release
Evolution split	seed 15; 76 held-in / 98 held-out

Appendix

Table 8: BrowseComp-Plus hard-subset construction.

Setting	Configuration
DeepSeek-V4-Flash-0731	openai/deepseek-v4-flash ; reasoning enabled; context 262,144; max output 32,768
GPT-5.6-Luna	openai/gpt-5.6-luna ; reasoning enabled; context 262,144; max output 32,768
Generation parameters	Temperature and top- $p$ use the provider defaults; no additional sampling override

Appendix

Table 9: Model configurations used in the experiments.

Harness	Execution configuration
OpenCode	v1.15.13-20260715; OpenAI-compatible protocol; context/output 262,144/32,768; task timeout 5,400 s; 46/32/64 concurrent trials
Pi-agent	v0.80.6; openai-responses ; context/output 262,144/32,768; task timeout 5,400 s; 24/32/64 concurrent trials
Codex	Frozen comparison release; openai-chat ; task timeout 5,400 s; 46/32/64 concurrent trials
OpenClaw	v2026.6.1; openai-completions ; context/output 262,144/32,768; task timeout 5,400 s; 46/32/64 concurrent trials

Appendix

Table 10: Harness execution settings. Concurrent trial counts are LHTB/DeepSWE/BCP.

	Policy interface	Lifecycle intervention point	Current realization	Examples of possible policy modifications
1	Input Assembly	Observation entry	observation_ [-1pt] projector	Change whether a newly received observation is retained in its original form or represented by a shorter, source-linked form.
2	History Maintenance	Compaction planning	compaction_ [-1pt] planner	Change when historical material becomes eligible for transformation and which complete dependency groups may be selected.
3	History Maintenance	Historical representation update	summary_writer	Change which facts, constraints, validation results, failed approaches, and source references are preserved in a compact representation.
4	History Maintenance	Evidence memory update	memory_writer	Change how durable evidence records are added, replaced, or removed and how their provenance is maintained.
5	History Maintenance	Evidence recovery	memory_ [-1pt] retriever	Change when prior evidence is requested and which narrowly relevant records are recovered.
6	Input Assembly	Request assembly	context_ [-1pt] assembler	Change how active history, summaries, recovered evidence, and inherited context are selected and ordered.

Appendix

Table 11: Context-policy intervention points. Current realizations and representative policy modifications.

Harness	Evolved	Native baseline	Full-task change
OpenCode	71/174 (40.80%)	61/174 (35.06%)	+10 (+5.74 pp)
OpenClaw	79/174 (45.40%)	33/174 (18.97%)	+46 (+26.44 pp)

Appendix

Table 12: Cross-harness validation on BrowseComp-Plus hard174 with GPT-5.6-Luna and Qwen3.8-27B. The evolved and native columns use the same 174-question denominator; all failed, timed-out, MCP, and upstream-error tasks count as incorrect.

Transfer setting	Target	DeepSeek	Luna
Same environment, cross-model	LHTB	$+3.12$ pp	$+2.56$ pp
Cross-environment, frozen policy	DeepSWE	$-3.54$ pp	$-7.96$ pp
Cross-environment, frozen policy	BCP	$+1.73$ pp	$-2.87$ pp

Appendix

Table 13: Cross-model and cross-environment transfer of the LHTB-evolved context policy. The corresponding matrix is shown in Figure 3 .

Control surface	Evidence	Role
Observation rendering	Observed in LHTB; related changes elsewhere	Select actionable context
History maintenance	Observed across long trajectories	Compact history while protecting active work
Durable evidence state	Present in inspected realizations	Preserve facts, open items, and recovery paths
Grounding and verification guidance	Loaded in LHTB and DeepSWE	Guide context use and verification
Addressable recovery	Registered; reward effect not isolated	Recover displaced evidence by reference
Bounded delegation	Exercised in four BrowseComp-Plus calls	Isolate a branch and return checkable evidence

Appendix

Table 14: Control surfaces in the analyzed policy realizations.

Explore similar work

EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
Aug 5, 2026Xuying Ning, Dongqi Fu, Tianxin Wei +13Long-Horizon Agents Agent Harness
University of Illinois Urbana–Champaign, 2Meta AI
GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)
Apr 18, 2026Jiaqing Liang, Jinyi Han, Weijia Li +15Large Language Model Agents Self-Evolving Agents
Advantage AI Agent Lab (A3 Lab)
Learning Agent-Compatible Context Management for Long-Horizon Tasks
May 29, 2026Lu Yi, Runlin Lei, Liuyi Yao +6Large Language Model Agents Long-Horizon Task Planning
1Renmin University of China · Work done during internship at Tongyi Lab, Alibaba Group · 2Tongyi Lab, Alibaba Group +2

Aug 5, 2026, cs.LGJ/K move · Enter open · S save

EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents

Xuying Ning, Dongqi Fu, Tianxin Wei, Hanqing Zeng+12

University of Illinois Urbana–Champaign, 2Meta AI

Long-Horizon Agents Agent Harness

Open paper AskPDF arXiv

Long-horizon LLM agents increasingly rely on external execution support to maintain state, track progress, invoke tools, verify outcomes, and reuse experience across interactions. However, effective harness use raises two coupled challenges: state formation from noisy interaction traces and runtime control over external-state access. Existing agents usually handle both through prompts, heuristics, or domain-specific conventions, leaving the external workspace and its usage policy manually engineered. To address this, we study the problem of harness policy learning, where agents learn harness policies offline and deploy them to construct and update external harness state online during runtime task execution. We introduce EvoHarness-RL, which exposes Belief, Progress, and Experience (BPE) as policy-facing harness state. Supervised harness fine-tuning teaches the base agent the harness action space and how to construct useful external state, while cost-aware GRPO explores coordination policies to selectively read, update, and consolidate that state during long-horizon interaction. Instantiated on ALFWorld with a Qwen3-8B LLM, EvoHarness-RL reaches 96.9% success and reveals two key dynamics: harness annealing, where training internalizes recurring harness-use patterns into the model policy and shifts the agent from frequent harness calls toward selective external-state access, and harness evolution, where progress updates and experience consolidation refine the harness into a compact, task-adaptive state substrate. These results suggest that long-horizon agents benefit from trainable policies for constructing and coordinating with external harness workspaces, beyond simply adding stronger tools or larger memories.

Abstract

Explore similar work

EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents

GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)

Learning Agent-Compatible Context Management for Long-Horizon Tasks