Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design

Gaotang Li; Bhavana Dalvi Mishra; Yifei Ming; Yanfei Chen; Zhongying CuiZhu; Lesly Miculicich; Long T. Le; Rui Meng

cs.CLOct 20, 2026

Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design

Authors: Gaotang Li, Bhavana Dalvi Mishra, Yifei Ming, Yanfei Chen, Zhongying CuiZhu, Lesly Miculicich, Long T. Le, Rui Meng,

Organizations: University of Illinois Urbana-Champaign · Google Cloud AI Research

Abstract

Agent harnesses are executable code that specifies the prompts, tools, and control flow around a language model. A single global harness can struggle with heterogeneous tasks that require different execution strategies. In this work, we introduce Object-Oriented Harness (OO-Harness), a harness optimization framework that evolves a hierarchy of executable harness classes using object-oriented design. During training, an evolution agent uses execution records to create a family of specialized harnesses, each implemented as a subclass of the base or an existing specialized harness via inheritance. At test time, a harness router uses routing lessons accumulated during training to select a specific harness for each task. Across three long-horizon benchmarks spanning business workflows, workspace tasks, and tool use, OO-Harness achieves the best performance among the compared methods, with an absolute improvement of up to 15% over the base. Extensive ablation studies examine the contributions of its core components.

Figures & tables

Figure 1: Three ways to organize harness adaptation. (a) A single global harness serves all tasks. (b) A router selects among independent harness implementations. (c) OO-Harness selects among specialized harnesses that share code through inheritance.

Figure 2: Problem setting. We use a train/test split: harnesses evolve on discovery tasks and are evaluated on disjoint held-out tasks.

Figure 3: OO-Harness training and test-time execution. (1) Base execution records initialize task clusters. (2) The evolution agent builds and evaluates subclasses, updating lessons from execution feedback. (3) The resulting harness hierarchy, harness descriptions, and routing lessons are frozen for testing. (4) The harness router uses the task, observable base trajectory, and frozen routing lessons to select a specialized harness or reuse the base result.

Test-time scaling
	AutomationBench			Workspace-Bench			Toolathlon-GYM
Method	Mean $\uparrow$	PASS $\uparrow$	Steps $\downarrow$	Mean $\uparrow$	PASS $\uparrow$	Steps $\downarrow$	Mean $\uparrow$	PASS $\uparrow$	Steps $\downarrow$
base	$61.62_{\pm 0.15}$	$25.44_{\pm 0.51}$	$31.27_{\pm 0.13}$	$25.45_{\pm 2.98}$	$3.67_{\pm 0.58}$	$34.41_{\pm 1.34}$	$71.87_{\pm 0.65}$	$31.15_{\pm 0.28}$	$40.50_{\pm 0.99}$
Parallel ( $K{=}2$ ) ‡	$61.46_{\pm 2.38}$	$24.67_{\pm 2.83}$	$31.64_{\pm 0.23}$	$32.54_{\pm 4.30}$	$5.67_{\pm 1.15}$	$41.62_{\pm 2.56}$	$74.85_{\pm 1.45}$	$31.55_{\pm 1.96}$	$39.94_{\pm 0.32}$
Parallel ( $K{=}5$ ) ‡	$60.24_{\pm 1.07}$	$24.00_{\pm 0.47}$	$31.29_{\pm 0.05}$	$37.71_{\pm 0.45}$	$7.67_{\pm 0.58}$	$48.63_{\pm 1.92}$	$75.69_{\pm 0.79}$	$32.14_{\pm 0.56}$	$38.90_{\pm 0.00}$
Sequential ( $K{=}2$ ) ‡	$59.29_{\pm 0.25}$	$25.00_{\pm 1.41}$	$31.86_{\pm 0.12}$	$27.93_{\pm 2.13}$	$3.33_{\pm 2.52}$	$32.91_{\pm 2.07}$	$73.53_{\pm 1.67}$	$33.33_{\pm 2.81}$	$40.97_{\pm 1.32}$

Table 1: Held-out results with a frozen executor. Mean : mean partial credit on AutomationBench, micro-averaged rubric pass rate on Workspace-Bench, and mean reward on Toolathlon-GYM. PASS : strict task success (reward

\geq 0.999

on AutomationBench and Toolathlon-GYM; all rubrics satisfied on Workspace-Bench). The primary metric is Mean for AutomationBench and Workspace-Bench, and PASS for Toolathlon-GYM. Steps counts only the finally selected trajectory, excluding other rollouts and auxiliary selection calls. Values are mean

\pm

standard deviation over three runs; scores are percentages. Bold marks each column’s maximum score (

\uparrow

) or minimum steps (

\downarrow

). † : supervised harness evolution with discovery-task feedback. ‡ : unsupervised test-time scaling of the base harness. Held-out grader feedback is unavailable to all methods.

Figure 4: Global versus hierarchical harness optimization. Compared with Meta-Harness , OO-Harness shows (a) larger early training gains, (b) shorter recorded search time, and (c) higher held-out scores. Curves align runs by their fraction of recorded updates.

		OO-Harness routing evidence
Benchmark / metric	Base	Task only	Task + trajectory
Automation / Mean	$61.30$	$64.88$	$\mathbf{77.52}$
Workspace / Mean	$24.57$	${40.03}$	$\mathbf{40.56}$
Toolathlon / PASS	$31.35$	$33.33$	$\mathbf{37.70}$

Table 2: Routing ablation on a frozen OO-Harness catalogue. Task only uses the task description and available task metadata; Task + trajectory also observes the base execution. Base executes without routing. Entries report primary scores (%) from single ablation runs, distinct from the repeated runs in Table 1 .

Figure 5: Inheritance ablation on Workspace-Bench. We find that learning through inheritance outperforms Flat and Base-reset.

		Original Base performance
Routing destination	Tasks	Mean reward	Strict PASS
Reuse Base	134	77.15	36.57
Specialized harness	166	44.40	14.25

Table 3: Base performance by routing destination on AutomationBench. We find that tasks routed to specialized harnesses have substantially weaker Base performance on average, while OO-Harness preserves stronger Base executions through reuse.

Figure 6: Learned clusters and an example router lesson. Panels (a,b) compare learned clusters (rows) with domains or task types (columns); cells count shared discovery tasks. AutomationBench clusters largely reflect execution needs, while Workspace-Bench clusters largely follow task types. Panel (c) schematically shows how repeated query-parsing errors in a Base trajectory match a saved routing lesson, leading to selection of an API error-normalizer harness. Blank cells are zero; Workspace-Bench column IDs match the numbered rows.

Figure 7: Base reuse on the selected OOD test tasks. After evolution on the disjoint training split, OO-Harness reuses Base on every test task, preserving its performance, while applying the learned Meta-Harness harness reduces PASS.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Generating a Word document through the file backend. For the same illustrative write_file call, the base backend saves text under a .docx filename; the specialized backend renders the text into a binary document.

Figure 9: Adding execution checks through inheritance. Successive nodes add a missing-write completion check, a near-budget reminder, and tool-error guidance. Arrows show inheritance. The filled node s29 is active; s9 and s12 are inactive ancestors that supply inherited behavior.

Base execution	Routing lesson	Router choice
A lead-scoring execution makes no writes. HubSpot and Intercom return 401 errors indicating no connected account; a Lead lookup returns 404. The final response is empty.	Repeated connection errors and failed resource lookups suggest that information about connected services and available resources may help.	Resource-manifest harness.
Another execution completes three writes, has no errors, and finishes cleanly.	Successful writes and a clean finish support keeping the Base result.	Reuse Base.

Appendix

Table 4: Execution evidence behind two routing choices. Examples from an earlier AutomationBench workflow pair observed Base behavior with the applicable routing lesson and the recorded choice.

Figure 10: Learned task clusters on Toolathlon-GYM. Rows are learned clusters and columns are evaluated AAH app groups. Cells count shared discovery tasks, showing how a cluster can span several applications. Blank cells are zero; Base and unclustered tasks are omitted.

Agent Harness Optimization

Agent harness optimization improves the prompts, tools, memory, workflows, and control code that shape how AI agents act.

See all 183 papers

HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
Aug 3, 2026Luan Zhang, Ruochen Zhou, Dandan Song +9Agent Harness Optimization LLM Agents
Beijing Institute of Technology, China · City University of Hong Kong, China · Independent, China
HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?
Sep 1, 2026Yuhao Wu, Jingyuan Zhang, Jiajun Shi +16LLM Agent Evaluation Agent Harness Optimization
HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry
Jun 12, 2026Tingyang Chen, Shuo Lu, Kang Zhao +11LLM Agent Orchestration Agent Harness Optimization
Darwin Agent Team

Aug 3, 2026J/K move · Enter open · S save

HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses

Luan Zhang, Ruochen Zhou, Dandan Song, Zhengyu Chen+8

Beijing Institute of Technology, China · City University of Hong Kong, China · Independent, China

Agent Harness Optimization LLM Agents LLM Agent Harnesses

Open paper AskPDF arXiv

Harness design plays a critical role in agent performance by shaping how large language models (LLMs) perceive, reason over, and act within executable environments. Recent work has proposed automatic harness evolution, which iteratively improves the harness from agent--environment interactions. However, existing methods often overfit to the evolution tasks, rely exclusively on trajectory-derived signals, and optimize harness components jointly, causing interference across components. We propose HarnessCompass, a novel automatic harness evolution framework built around constrained evolution, proactive feedback, and component-wise optimization. HarnessCompass first enforces global constraints on evolution, restricting modifications to task-agnostic harness changes that generalize beyond the evolution tasks. It then augments trajectory-derived evidence with proactive first-person feedback from the agent about harness usage, yielding richer signals for evolution. Finally, it decouples the optimization of different harness components before consolidating them into a unified harness, reducing cross-component interference while preserving component synergy. On SWE-bench Verified with GPT-5.4, HarnessCompass improves Pass@1 from 54% to 66% in only 5 evolution iterations, outperforming AHE in both effectiveness and evolution efficiency. In addition, the evolved harness transfers effectively to held-out tasks and other models, demonstrating substantially stronger generalization than prior automatic harness evolution methods.

Abstract

Agent Harness Optimization

Explore similar work

HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses

HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?

HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry