Embodied agents built on foundation models rely on an execution harness to organize observations, guide actions, and incorporate execution feedback into subsequent decisions. Failures can arise not only from limited model capabilities, but also from how the harness presents evidence and guides its use. These model-external mechanisms provide an adaptation target without updating model parameters, but task-specific interaction experience can be costly to collect. Task-level outcomes alone do not reveal whether relevant evidence was missing from the model input or was available but inadequately used. Moreover, these two aspects are coupled: effective action guidance may require evidence that the current context omits, while exposing that evidence alone does not ensure an appropriate response. We introduce EnactOpt, which uses limited execution feedback to jointly evolve reusable textual skills and context-construction code while keeping the underlying models and runtime fixed. It diagnoses adjacent decisions by relating action feedback to the actual model input and subsequent action. Structured episode summaries guide joint revisions through prompt-level edit operators, and candidate harnesses undergo compatibility checks and validation-based retention. Across VLABench, ALFWorld, and Embodied Agent Interface (EAI), EnactOpt improves task success by approximately 10-27 percentage points over the initial harnesses and outperforms the evaluated GEPA, TextGrad, and Trace2Skill baselines. These gains extend to task conditions not used for harness evolution or candidate selection. The results support a complementary route to embodied adaptation: turning limited execution experience into reusable behavioral improvements by jointly refining how evidence is presented and acted upon.
Figures & tables
Figure 1: Execution harness for embodied agents. The editable harness H=(S,C) comprises textual skills S for action guidance and context-construction code C for organizing observed evidence. The fixed runtime assembles the complete model input xt from constructed context ct and any additional runtime-provided inputs; model parameters and runtime interfaces remain unchanged during harness evolution.
Figure 2: Overview of EnactOpt. (a) A source harness H=(S,C) executes training batch Br , recording actual model inputs, actions, observations, and episode outcomes. (b) Adjacent-decision diagnoses and episode summaries form its feedback Fr . (c) A joint optimizer uses the same H and Fr to propose candidates Cr through prompt-guided structured edit operators. Compatible candidates are compared with the previous pool Pr−1 on validation tasks to form Pr . Scenes are illustrative, not experimental results.
Method
VLABench
ALFWorld
EAI
Reference Systems
Naive
40.87
51.46
68.09
H0
47.39 +6.52
66.06 +14.60
72.34 +4.26
Adaptation Methods
TextGrad †
36.09 -4.78
86.50 +35.04
71.63 +3.55
Trace2Skill †
40.54 -0.33
78.83 +27.37
72.70 +4.61
Table 1: Task success rates (%, higher is better) across three benchmarks. Bold indicates the best result in each column. Colored annotations show percentage-point differences from Naive (green: positive; red: negative). † denotes our skill-only implementations of TextGrad and Trace2Skill.
Figure 3: Generalization on VLABench T3/T4/T6 and ALFWorld Unseen. Bars show task success rates (%); horizontal dashed lines denote the Naive baseline. Adaptation is performed separately for each benchmark. The plotted results come from the same evaluation runs as Table 1 , which reports overall success rates on the full test sets.
Variant
In-Distribution
Generalization
Optimization cost
H0
58.33
43.53
–
w/o Adjacent-Decision Diagnosis
62.08 +3.75
41.62 -1.91
1×
w/o Structured Edit Operators
56.25 -2.08
41.03 -2.50
1×
w/o Joint Revision
63.33 +5.00
50.88 +7.35
2×
EnactOpt
63.75 +5.42
55.74 +12.21
1×
Table 2: VLABench ablations on T1 (In-Distribution) and T3/T4/T6 (Generalization). Rates (%); subscripts: percentage-point changes from H0 . Cost: optimization passes relative to EnactOpt.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Use
Track
Indices
Episodes
Harness evolution (Train30)
T1
0–4
30
Candidate selection (Val30)
T1
5–9
30
Test
T1, T3, T4, T6
10–49
920
Appendix
Table 3: Fixed VLABench episode splits. Indices are zero-based and inclusive within each official task–track list. Counts include only valid selected task–track pairs.
Figure 4: VLABench track examples for book and drink selection. Each row compares initial forward views and task instructions across T1, T3, T4, and T6. T3/T4 illustrate changes in the instruction, while T6 illustrates changes in the visual setting. The images show initial states, not execution outcomes.
Figure 5: VLABench track examples for condiment addition. The columns follow the same T1/T3/T4/T6 order as Figure 4 , showing initial forward views and task instructions for two further episodes.
Task family
Train30
Val30
Seen
Unseen
Pick & Place
5
5
35
24
Pick Two & Place
5
5
24
17
Examine in Light
5
5
13
18
Clean & Place
5
5
27
31
Heat & Place
5
5
16
23
Cool & Place
5
5
25
21
Appendix
Table 4: Task-family counts for the fixed ALFWorld Train30/Val30 split and the complete Seen/Unseen evaluation sets.
Partition
Instances
Categories
Use
Train
30
23
Harness evolution
Val
30
23
Candidate selection
Test
282
26
Evaluation
Total
342
26
All scene_1 instances
Appendix
Table 5: Frozen custom EAI split for VirtualHome Action Sequencing in scene_1 . Category counts are distinct official activity labels within each partition.
Improving embodied agents often focuses on optimizing the underlying model through training, while the surrounding agent harness that controls planning, context, and tool use is typically engineered. We ask whether this harness can instead improve itself directly from experience traces under sparse environmental feedback. We propose EMbodied Agent Harness Optimization (EMHO), a self-evolving framework that keeps the embodied model frozen and iteratively revises its harness by analyzing execution trajectories and prior harness history. EMHO optimizes beyond skills or recovery prompts, modifying how the agent monitors progress, uses vision tools, grounds observations, and responds to failures. To support multiple subtasks with a single harness, we introduce EMHO-Merge, which addresses trade-offs in jointly optimizing a single shared harness across subtasks by using episode-level gains and losses to guide evidence-supported refinement of when and how revised behaviors are applied. We evaluate EMHO on EmbodiedBench across navigation and manipulation tasks, and EMHO consistently improves task success for both Qwen 9B and 27B models. Qualitative analysis shows that EMHO goes beyond recovering from failures and unproductive actions to reshape how the embodied agent interprets and interacts with its environment.
Hyun Jung Lee, Jungtaek Kim, Jongwon Jeong +3
Korea University · University of Arkansas · University of Wisconsin–Madison
Embodied agents are increasingly built as systems around foundation models, where performance depends not only on model weights but also on the skills, context, action interfaces, and execution harness surrounding the model. While supervised fine-tuning and reinforcement learning can adapt agents to new environments, they require additional data, rewards, and training runs; meanwhile, many train-free code-centric approaches rely on programmable robot APIs that may be unavailable in fixed-interface settings. We propose SHAPER, a self-evolving framework for train-free embodied adaptation that keeps model parameters frozen and improves the non-parametric agent system by evolving reusable skills and a context-code harness through target-environment rollouts. In SHAPER, the same frozen model can serve as both planner and optimizer, refining its external skills and context-code harness without parameter updates. We evaluate SHAPER on VLABench and ESI-Bench, covering embodied agents with different low-level action interfaces, and compare against pure execution, supervised fine-tuning, and test-time-scaling baselines such as verifier-free selection and voting. Our results suggest that skill-and-harness optimization is a practical route to self-evolving embodied agents when model training is expensive, unavailable, or undesirable.
AI agent performance depends critically on the runtime harness, comprising the prompts, tools, memory, and control flow that mediate how a model observes, reasons, and acts. Yet today's harnesses remain largely hand-crafted and static: each new model or task still demands bespoke scaffolding, and the rich traces produced during execution are rarely distilled back into systematic improvement. We introduce HarnessX, a foundry for composable, adaptive, and evolvable agent harnesses. HarnessX assembles typed harness primitives via a substitution algebra, adapts them through AEGIS, a trace-driven multi-agent evolution engine grounded in an operational mirror between symbolic adaptation and reinforcement learning, and closes the harness-model loop by turning trajectories into both harness updates and model training signal. Across five benchmarks (ALFWorld, GAIA, WebShop, tau^3-Bench, and SWE-bench Verified), HarnessX yields an average gain of +14.5% (up to +44.0%), with gains largest where baselines are lowest. These results suggest that agent progress need not come from model scaling alone: composing and evolving runtime interfaces from execution feedback is an actionable and complementary lever. The complete codebase will be open-sourced in a future release.