Learning from Research: Toward Lifelong Agent Harness Evolution

Jingbo Yang; Kwei-Herng Lai; Xiaowen Wang; Yaar Harari; Evgeniy Gabrilovich; Shiyu Chang

cs.AISep 30, 2026

Learning from Research: Toward Lifelong Agent Harness Evolution

Authors: Jingbo Yang, Kwei-Herng Lai, Xiaowen Wang, Yaar Harari, Evgeniy Gabrilovich, Shiyu Chang

Organizations: University of California, Santa Barbara · Microsoft

Abstract

Language agents are expected to solve increasingly complex tasks, creating a growing need for continual improvement. One promising approach is to evolve the agent harness, the software that governs tool use, memory management, and task execution, while keeping the underlying language model fixed. Recent methods automate this process by using a meta coding agent to modify the harness based on execution feedback. However, relying on that agent's existing knowledge and observed failures can restrict exploration and make adaptation reactive. Inspired by how human experts learn from the research literature for new solutions, we introduce ScholarEvolve, a framework that automatically draws on state-of-the-art research to guide harness evolution. ScholarEvolve organizes the harness evolution directions into functional modules and uses topic modeling to identify distinct improvement strategies for each module. It implements these strategies and evaluates their combinations to improve task performance. Moreover, the framework is designed to incorporate new publications over time, allowing research advances to drive proactive lifelong evolution. Experiments demonstrate improvements on AppWorld and Tau2-Bench. ScholarEvolve raises Qwen3.5-27B task goal completion from 49.6% to 63.6% on AppWorld Challenge, and raises GPT-5.4-mini pass@1 from 72.7% to 81.9% on Tau2-Bench Telecom.

Figures & tables

Figure 1: Overview of ScholarEvolve . Left: workflow focused exploration. Middle: research guided module mutation, recombination, and lifelong evolution. Right: Qwen3.5-27B task (TGC) and scenario (SGC) goal completion on AppWorld Challenge.

Figure 2: ScholarEvolve : (a) research guided module mutation, (b) selection and crossover across generations, and (c) evolved modules operating around a fixed backbone.

Panel of Table 1 — Table 1: Main results on AppWorld and $\tau^{2}$ -Bench as mean $\pm$ standard deviation over 3 runs. $\Delta$ is the absolute change from the matched backbone ( orange : improvement, blue : degradation). Bold and underline mark the two largest gains per column.

Figure 3: Evolution over time and across held-out tasks. Left: best harness found in each generation on AppWorld Normal, with bands showing one standard deviation. Right: task-level success-count transitions pooled over Normal and Challenge (

n=585

per backbone).

	Qwen3.5-27B		GPT-5.4-mini
Variant	TGC $\uparrow$	SGC $\uparrow$	TGC $\uparrow$	SGC $\uparrow$
ScholarEvolve	81.4 $\pm$ 1.19	69.0 $\pm$ 4.10	72.4 $\pm$ 1.83	55.4 $\pm$ 3.55
$-$ topic-guided selection	80.0 $\pm$ 0.91	66.7 $\pm$ 4.12	69.8 $\pm$ 0.91	50.0 $\pm$ 4.72
$-$ module-wise mutation	78.6 $\pm$ 1.79	64.9 $\pm$ 4.49	69.3 $\pm$ 1.24	48.8 $\pm$ 3.72
$-$ research guidance	76.8 $\pm$ 1.19	62.5 $\pm$ 3.57	67.5 $\pm$ 2.44	45.8 $\pm$ 2.70

Table 2: Cumulative ablations on AppWorld Normal (%). Each row additionally removes the named component. Scores are mean

\pm

SD.

	Qwen3.5-27B		GPT-5.4-mini
Configuration	TGC $\uparrow$	SGC $\uparrow$	TGC $\uparrow$	SGC $\uparrow$
Initial harness	67.8 $\pm$ 1.01	43.9 $\pm$ 3.04	66.1 $\pm$ 4.43	47.3 $\pm$ 9.12
$+$ Tool only	72.5 $\pm$ 5.36	50.9 $\pm$ 13.25	N/A
$+$ Context only	74.9 $\pm$ 2.68	57.9 $\pm$ 5.26	N/A
$+$ Skills only	73.7 $\pm$ 7.02	47.4 $\pm$ 13.93	69.0 $\pm$ 3.65	52.6 $\pm$ 5.25
$+$ Memory only	81.3 $\pm$ 1.01	59.6 $\pm$ 3.04	72.5 $\pm$ 2.02	45.6 $\pm$ 3.06

Table 3: Module composition on AppWorld DEV (57 tasks, three runs). Scores are mean

\pm

SD (%). N/A: module absent from the combination.

	Reference APIs
Backbone	$\leq 9$ ( $n$ =168)	10–12 ( $n$ =147)	$>12$ ( $n$ =102)
Qwen3.5-27B	13.1	18.4	9.5
GPT-5.4-mini	5.4	15.9	7.2

Table 4: Challenge TGC gains (points) by reference API count.

Figure 4: Residual Qwen3.5-27B failure taxonomy with five families and 16 subtypes.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Metric $\uparrow$	Method	$t_{0}$	$t_{1}$	$t_{2}$	$t_{3}$
TGC	ScholarEvolve	$69.00\pm 2.35$	$73.41\pm 2.09$	$79.76\pm 2.15$	$\mathbf{81.55}\pm 1.79$
	Meta Harness	$69.00\pm 2.35$	$71.03\pm 2.81$	$70.44\pm 2.09$	$69.64\pm 2.73$
SGC	ScholarEvolve	$48.80\pm 2.75$	$54.17\pm 2.72$	$61.91\pm 2.73$	$\mathbf{66.67}\pm 3.72$
	Meta Harness	$48.80\pm 2.75$	$47.62\pm 2.73$	$51.19\pm 2.73$	$51.19\pm 2.06$

Appendix

Table 5: Lifelong evolution on AppWorld Normal with Qwen3.5-27B. Scores are percentages. Here,

t_{0}

is the shared initial harness, and

t_{1}

–

t_{3}

report the best harness found in each generation.

Figure 5: Reference API distributions across AppWorld splits. Bars show the percentage of tasks at each integer API count, with shared axes across the complete splits. The shaded regions mark at most nine APIs and more than 12 APIs, using the training third quartile and maximum. Dashed lines separate these ranges at integer-bin boundaries.

Figure 6: Reliability transitions by AppWorld test split. Columns and rows give initial and evolved success counts out of three. Cells report task counts. Teal denotes more successes, slate the same count, and rose fewer successes. Darker shades indicate larger fractions of each split on a shared square-root scale.

Figure 7: A hierarchy of observable agent failures. Five families organize evidence-linked deviations found in Qwen3.5-27B AppWorld trajectories. Each subtype identifies a distinct behavior or execution outcome. Labels can coexist within an episode. Recovery status is recorded separately.

	Normal		Challenge		Recovered
Subtype	Initial	Evolved	Initial	Evolved	(pooled)
Retrieval & grounding
Incomplete retrieval	12	8	65	62	0
Entity confusion	7	11	8	12	0
Unsupported inference	1	1	3	5	0
Goal & reasoning

Appendix

Table 6: Supported unresolved deviations in the full Qwen AppWorld cohorts. Counts are multi-label. Each harness has 504 Normal and 1,251 Challenge episodes. The final column counts recovered occurrences within the failed-episode cohort, pooled across splits and harnesses.

Figure 8: Completion and answer diagnostics on the full Qwen cohorts. Points show fractions of all episodes, and counts give initial

\rightarrow

evolved totals. The four endpoint categories follow the diagnostic order from top to bottom. Behavior labels in Figure 7 provide a separate, overlapping description of the trajectories.

Figure 9: Implemented mechanisms span the agent’s decision process. Schematic organization of alternatives from the Qwen AppWorld candidate pool. Each module lists two research-derived mechanisms. Arrows show information and action flow through the harness. The alternatives are candidates for selection and recombination.

Module	Topic	Source paper	$\Delta$	90% interval	W/L
Tool interface	Action Space Control	Budget-constrained tool learning ( Zheng et al., 2024 )	+2.9	[ $-$ 3.5, +9.4]	15/11
	Abstract Strategy Actions	CheMatAgent ( Wu et al., 2025 )	+1.2	[ $-$ 4.1, +6.4]	12/10
	Structured API Actions	ToolACE-R ( Zeng et al., 2026b )	+4.7	[ $-$ 0.6, +9.9]	14/8	$\star$
	Constructed Composite Actions	RefTool ( Liu et al., 2026b )	+1.8	[ $-$ 4.7, +8.2]	15/10
Context	Lossy Context Rewriting	Compression without MLPs ( Honig et al., 2025 )	$-$ 45.6	[ $-$ 53.8, $-$ 36.8]	1/40
	Independent Unit Scoring	ICPC ( Yu & Liu, 2025 )	$-$ 43.3	[ $-$ 52.0, $-$ 34.5]	2/40

Appendix

Table 7: All candidates for Qwen3.5-27B on AppWorld. Each candidate changes one module of the initial harness.

\Delta

is the standalone TGC gain in points on the 57 development tasks over three runs, with a 90% paired task-bootstrap interval. W/L counts tasks whose mean reward is higher/lower than under the initial harness.

\star

marks the modules in the final harness.

	AppWorld		$\tau^{2}$ -Bench Telecom
Module	Qwen3.5-27B	GPT-5.4-mini	Qwen3.5-27B	GPT-5.4-mini
Tool interface	ToolACE-R ( Zeng et al., 2026b ) Structured API Actions	–	Lower Privileges Suffice ( Yang et al., 2026b ) Action-Centric Post-Training	Interactive Semantic Parsing ( Yao et al., 2019 ) Action Validation and Repair
Context	Prompt compression limits ( Nagle et al., 2024 ) Budget and Scope Routing	–	Diable ( Lesci et al., 2023 ) Structured State Memory	Measure Before You Manage ( Chen et al., 2026 ) Context Buffer Management
Skills	Skill-as-Pseudocode ( Li et al., 2026 ) Skill Compilation	Skill Drift Is Contract Violation ( Fan et al., 2026 ) Skill Library Maintenance	Procedural KG Extraction ( Carriero et al., 2024 ) Skill Induction	Procedural Knowledge at Scale ( Wu et al., 2026a ) Procedural Memory and Retrieval
Memories	Oracle Agent Memory ( Alake et al., 2026 ) Structured Memory Architecture	CoEvo-Mem ( Ye et al., 2026 ) Adaptive Retrieval Control	CAST ( Ma et al., 2026 ) Structured Relational Memory	MemRL ( Zhang et al., 2026d ) Adaptive Retrieval and Retention
Workflow	TMAS ( Wu et al., 2026b ) Multi-Agent Deliberation	Plan-and-Solve ( Wang et al., 2023 ) Plan-and-Execute	Textual-to-Visual Self-Verification ( Xu et al., 2025 ) Hierarchical Decomposition	Same-Model Self-Verification ( Phalod, 2026 ) Adaptive Routing

Appendix

Table 8: Module implementations in the main-experiment harnesses. Each entry gives the source paper and, in italics, the assigned topic. A dash retains the initial module.

Explore similar work

HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
Aug 3, 2026Luan Zhang, Ruochen Zhou, Dandan Song +9Evolving Harness Agent Harness
Beijing Institute of Technology, China · City University of Hong Kong, China · Independent, China
Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses
Aug 9, 2026Tailin ZhouEvolving Harness Large Language Model Agents
HKUST
Evo-Bench: Can Language Models Improve Agent Harness?
Aug 10, 2026Lisheng Huang, Chen Yang, Hao Zhou +6Evolving Harness Agent Harness
1Gaoling School of Artificial Intelligence, Renmin University of China · 2BOSS Zhipin, Beijing, China

Aug 3, 2026, cs.LGJ/K move · Enter open · S save

HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses

Luan Zhang, Ruochen Zhou, Dandan Song, Zhengyu Chen+8

Beijing Institute of Technology, China · City University of Hong Kong, China · Independent, China

Evolving Harness Agent Harness Evolutionary Optimization Methods

Open paper AskPDF arXiv

Harness design plays a critical role in agent performance by shaping how large language models (LLMs) perceive, reason over, and act within executable environments. Recent work has proposed automatic harness evolution, which iteratively improves the harness from agent--environment interactions. However, existing methods often overfit to the evolution tasks, rely exclusively on trajectory-derived signals, and optimize harness components jointly, causing interference across components. We propose HarnessCompass, a novel automatic harness evolution framework built around constrained evolution, proactive feedback, and component-wise optimization. HarnessCompass first enforces global constraints on evolution, restricting modifications to task-agnostic harness changes that generalize beyond the evolution tasks. It then augments trajectory-derived evidence with proactive first-person feedback from the agent about harness usage, yielding richer signals for evolution. Finally, it decouples the optimization of different harness components before consolidating them into a unified harness, reducing cross-component interference while preserving component synergy. On SWE-bench Verified with GPT-5.4, HarnessCompass improves Pass@1 from 54% to 66% in only 5 evolution iterations, outperforming AHE in both effectiveness and evolution efficiency. In addition, the evolved harness transfers effectively to held-out tasks and other models, demonstrating substantially stronger generalization than prior automatic harness evolution methods.

Abstract

Explore similar work

HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses

Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses

Evo-Bench: Can Language Models Improve Agent Harness?