Modern agentic systems combine an AI model with a harness that controls execution and environmental interactions. Harness design strongly affects long-horizon performance, yet its combinatorial search space demands substantial human effort that must be repeated as models change. Existing automated methods explore this space narrowly, optimizing only components such as prompts or skills or becoming trapped by fixed, exploitative search strategies. We introduce MILO (Meta-evolutionary Island Orchestration), a framework that co-evolves agent harnesses and the strategy used to discover them. MILO combines: (i) hierarchical lineage memory over island-based trees, using rejected mutations as negative evidence; (ii) per-island mutator agents that rewrite complete harnesses using global search history and parent-specific feedback; and (iii) an orchestrator that adapts search through lineage grafting and speciation, mutator reassignment and curriculum revision. Across Terminal-Bench 2.1, PaperBench, and DeepSWE, MILO-discovered harnesses outperform eight state-of-the-art harnesses and six search methods using frontier (Opus 4.8) and open-weight (gpt-oss-120b) models. With Opus 4.8, MILO improves resolution over its initial harness by +12.0%, +28.3%, and +10.3%, respectively, compared with best prior-search gains of +4.5%, +18.3%, and 0%. On Terminal-Bench 2.1, it achieves 86.1±2.0%, exceeding the official leaderboard's top entry (83.8±2.3%) while using 26% fewer tokens than its initial harness. On EinsteinArena open problems, MILO improves best-known upper bounds for Erdős minimum-overlap (0.3808586→0.3808568) and the first and third autocorrelation inequalities (1.50274365→1.50274360; 1.45081→1.44889).
Figures & tables
Figure 1 : Overview of MILO , a meta-evolutionary harness discovery framework. Given an agent, MILO discovers a stronger harness for its model via two coupled loops: an inner loop that evolves harnesses and an outer loop that evolves the search strategy. In the inner loop, (1) a parent is drawn from the island’s admitted candidates, (2) its mutator agent rewrites the whole harness from the parent’s failure traces and the island’s lineage memory , including rejected candidates, and (3–4) the child is scored and admitted on multi-objective Pareto gain. (5) When progress stalls, the outer loop’s orchestrator agent diagnoses the island and revises its mutator, memory, and curriculum.
Figure 2 : General skeleton for evolutionary search. Tab. classifies existing methods by adaptiveness.
Method
Level
Domain
Memory
Parent selection
Mutator
(0) Open-loop test-time scaling. No executed outcome feeds the next attempt, so gains cannot compound. feedback ✗
Best-of- N ( Cobbe et al., 2021 )
instance
Math
—
Best of N i.i.d. draws
Single LLM
Chain-of-Thought ( Wei et al., 2022 )
instance
Reasoning
—
—
Single LLM
Tree-of-Thoughts ( Yao et al., 2023a )
instance
Planning
Transient thought tree
LLM-scored states
Single LLM
(I) Iterative refinement. Executed feedback steers one greedy lineage; no diversity-preserving population, so it stalls at local optima. + feedback ✓
Reflexion ( Shinn et al., 2023 )
instance
Code, QA
Verbal reflections
Latest attempt
Single LLM
Table 1 : Categorizing LLM-driven evolutionary search. Methods by discovery level (strategy-level in blue) and adaptiveness (0–IV). Most methods fix their search strategy in advance (stages 0–II); those that adapt it online (III–IV) are mostly instance-level. ↻ : adaptation by fixed rule ( adaptive ) or meta-evolution ( self-adaptive ).
Algorithm 1 MILO ’s meta-evolutionary loop. Candidate harnesses and their search strategy evolve jointly.
Table 2 : Performance and cost comparison of harnesses, with a frontier backbone. Using Opus 4.8 as the model M . (a) Bold / underline = column best/second. Gray : no fitness gain over Best-of-3 during search; all other entries report the best-fitness harness evaluated on the full benchmark. (b) Resolution rate of every harness on each benchmark; MILO is marked with a red ∗ . (c) Search-mechanism ablation, adding one component at a time (App. ). (d, e) Mean tokens and wall-clock time per attempt against resolution rate on Terminal-Bench 2.1 and DeepSWE; points carry the harness IDs of (a).
Table 3 : Performance of harnesses, with an open-weight backbone. Using gpt-oss-120b as the model M . (a) Conventions as in Tab. . (b) All islands stall in rounds 6–14 until orchestrator interventions ( Graft / Reassign ) pull them out; rejected grafts persist as negative examples. The weakest seed, Island 3 (39.2%), sets the population best of 59.9%, showing the value of preserved diversity. (c) Every baseline stalls similarly but never recovers; MILO recovers and dominates pass-rate at the lowest token cost.
Table 7
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Search reach
Failure mode addressed (mechanism)
Method
Design
Pro- mpt
Skills/ mem
To- ols
Middle- ware
Premature completion
Deadline overrun
Lax verification
Missing know-how
RR@5 ↑
reproduce-first rubric gate seed B12
expert
✓
✗
✗
✓
✓ rubric gate
✗
✗
✗
74.1
GEPA (prompt)
auto
✓
✗
✗
✗
✓ rubric gate
∼
∼
✗
78.6
GEPA (optimize-anything)
auto
✓
✗
✓
✓
✓ rubric gate
∼
∼
✗
74.1
A-Evolve (skill, memory)
auto
✓
✓
✗
✗
✓ rubric gate
∼
✓ criteria skill
✓ skills lib
78.2
OpenEvolve
auto
✓
✓
✓
✓
✓ rubric gate
✗
✗
✗
76.1
Appendix
Table 12 : Mechanisms in each method’s best harness, by failure mode addressed (Terminal-Bench 2.1, Opus 4.8). Search reach = the surface the method may edit. Failure modes: premature completion (success declared at a wrong state), deadline overrun (timeout zeros the trial), lax verification (self-check weaker than the hidden grader), missing know-how . Marks: ✓ code-level mechanism (named), ∼ prompt-level mitigation, ✗ unaddressed.
Problem (arena id)
Objective (minimization; lower is better)
AlphaEvolve (arena entry)
Prior best (Sept. 2026)
MILO
Margin
Min. impr.
Erdős minimum overlap (1)
minimize maxk∫h(x)(1−h(x+k))dx
0.3809230351
0.3808585749 CodexProLong, 2026-08-15
0.3808567744
1.8×10−6
10−7
First autocorrelation ineq. (2)
minimize max(f⋆f)/(∫f)2 , f≥0
1.5052939684
1.5027436492 CodexProLong, 2026-08-14
1.5027435984
5.1×10−8
10−8
Third autocorrelation ineq. (4)
minimize ∣max(f⋆f)∣/(∫f)2 , f signed
1.4556427954
1.4508066395 Poolish, 2026-08-23
1.4488860143
1.9×10−3
10−5
Appendix
Table 13 : Verified EinsteinArena records. Prior best (the arena leader, unchanged between 2026-09-09 and 2026-09-22) vs. ours, both scored with the arena’s verifier. All three are minimization problems (lower is better), and every margin exceeds the arena’s minimum improvement for a new #1.
An agent harness is the code that organizes context, maintains state, and coordinates tool calls for a language model. We study how to improve the harness under a limited evaluation budget while keeping model weights fixed. Our method, MESH-Harness, organizes each harness into functional modules with explicit role-specific interfaces, allowing alternative implementations of each module to be substituted and recombined. It uses shared module representations and full-covariance LinUCB to score candidate combinations based on predicted performance and exploration value. Mixed-start coordinate ascent selects complete configurations for evaluation without enumerating the combinatorial space. Validation traces then guide local code edits, and the resulting candidates are incorporated into fixed-capacity role-specific pools for subsequent recombination. On text tasks, retrieval-augmented mathematical reasoning, code generation, and interactive scientific tasks, MESH-Harness outperforms Meta-Harness by 5.70, 7.01, 2.00, and 5.00 points, respectively, under matched candidate-evaluation budgets. Iterative harness optimization improves MESH-Harness by 5.63-7.79 points over its first-round configurations. For the reported configurations, aggregate test-time cost is 44.2% lower than that of Meta-Harness, while total cost including search is 14.6% lower. These results show that combining module-level design reuse with feedback-driven compositional search can systematically improve agent harnesses while keeping overall optimization cost under control.
Zhiwei Shang, Yu Huo, Mingrong Gong +6
The Chinese University of Hong Kong, Shenzhen · DeepWisdom · The Chinese University of Hong Kong +3
AI agent performance depends critically on the runtime harness, comprising the prompts, tools, memory, and control flow that mediate how a model observes, reasons, and acts. Yet today's harnesses remain largely hand-crafted and static: each new model or task still demands bespoke scaffolding, and the rich traces produced during execution are rarely distilled back into systematic improvement. We introduce HarnessX, a foundry for composable, adaptive, and evolvable agent harnesses. HarnessX assembles typed harness primitives via a substitution algebra, adapts them through AEGIS, a trace-driven multi-agent evolution engine grounded in an operational mirror between symbolic adaptation and reinforcement learning, and closes the harness-model loop by turning trajectories into both harness updates and model training signal. Across five benchmarks (ALFWorld, GAIA, WebShop, tau^3-Bench, and SWE-bench Verified), HarnessX yields an average gain of +14.5% (up to +44.0%), with gains largest where baselines are lowest. These results suggest that agent progress need not come from model scaling alone: composing and evolving runtime interfaces from execution feedback is an actionable and complementary lever. The complete codebase will be open-sourced in a future release.
Self-evolving frameworks usually optimize task solutions while treating the surrounding harness as fixed. We introduce Harness-Aware Self-Evolving (HASE), an agentic reinforcement-learning framework in which a single model can generate task solutions or edit selected harness components in a multi-turn action space. HASE enables a single Qwen3-8B model to match the text-classification performance of a GPT-OSS-120B model that uses Claude Code as the harness proposer. In alpha factor mining, HASE outperforms the reported GPT-OSS-120B baseline. HASE also repairs imperfect evaluation components and converges to state-of-the-art performance in circle-packing algorithm discovery. These results show that HASE improves the harness and the solution through one unified agentic process.
Haochen Luo, Yi Huang, Sichun Luo +5
School of Computing and Data Science, The University of Hong Kong · Grace Investment Machine · Jiutian Research, China Mobile