Embodied agents built on foundation models rely on an execution harness to organize observations, guide actions, and incorporate execution feedback into subsequent decisions. Failures can arise not only from limited model capabilities, but also from how the harness presents evidence and guides its use. These model-external mechanisms provide an adaptation target without updating model parameters, but task-specific interaction experience can be costly to collect. Task-level outcomes alone do not reveal whether relevant evidence was missing from the model input or was available but inadequately used. Moreover, these two aspects are coupled: effective action guidance may require evidence that the current context omits, while exposing that evidence alone does not ensure an appropriate response. We introduce EnactOpt, which uses limited execution feedback to jointly evolve reusable textual skills and context-construction code while keeping the underlying models and runtime fixed. It diagnoses adjacent decisions by relating action feedback to the actual model input and subsequent action. Structured episode summaries guide joint revisions through prompt-level edit operators, and candidate harnesses undergo compatibility checks and validation-based retention. Across VLABench, ALFWorld, and Embodied Agent Interface (EAI), EnactOpt improves task success by approximately 10-27 percentage points over the initial harnesses and outperforms the evaluated GEPA, TextGrad, and Trace2Skill baselines. These gains extend to task conditions not used for harness evolution or candidate selection. The results support a complementary route to embodied adaptation: turning limited execution experience into reusable behavioral improvements by jointly refining how evidence is presented and acted upon.
Figures & tables
Figure 1: Execution harness for embodied agents. The editable harness H=(S,C) comprises textual skills S for action guidance and context-construction code C for organizing observed evidence. The fixed runtime assembles the complete model input xt from constructed context ct and any additional runtime-provided inputs; model parameters and runtime interfaces remain unchanged during harness evolution.
Figure 2: Overview of EnactOpt. (a) A source harness H=(S,C) executes training batch Br , recording actual model inputs, actions, observations, and episode outcomes. (b) Adjacent-decision diagnoses and episode summaries form its feedback Fr . (c) A joint optimizer uses the same H and Fr to propose candidates Cr through prompt-guided structured edit operators. Compatible candidates are compared with the previous pool Pr−1 on validation tasks to form Pr . Scenes are illustrative, not experimental results.
Method
VLABench
ALFWorld
EAI
Reference Systems
Naive
40.87
51.46
68.09
H0
47.39 +6.52
66.06 +14.60
72.34 +4.26
Adaptation Methods
TextGrad †
36.09 -4.78
86.50 +35.04
71.63 +3.55
Trace2Skill †
40.54 -0.33
78.83 +27.37
72.70 +4.61
Table 1: Task success rates (%, higher is better) across three benchmarks. Bold indicates the best result in each column. Colored annotations show percentage-point differences from Naive (green: positive; red: negative). † denotes our skill-only implementations of TextGrad and Trace2Skill.
Figure 3: Generalization on VLABench T3/T4/T6 and ALFWorld Unseen. Bars show task success rates (%); horizontal dashed lines denote the Naive baseline. Adaptation is performed separately for each benchmark. The plotted results come from the same evaluation runs as Table 1 , which reports overall success rates on the full test sets.
Variant
In-Distribution
Generalization
Optimization cost
H0
58.33
43.53
–
w/o Adjacent-Decision Diagnosis
62.08 +3.75
41.62 -1.91
1×
w/o Structured Edit Operators
56.25 -2.08
41.03 -2.50
1×
w/o Joint Revision
63.33 +5.00
50.88 +7.35
2×
EnactOpt
63.75 +5.42
55.74 +12.21
1×
Table 2: VLABench ablations on T1 (In-Distribution) and T3/T4/T6 (Generalization). Rates (%); subscripts: percentage-point changes from H0 . Cost: optimization passes relative to EnactOpt.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Use
Track
Indices
Episodes
Harness evolution (Train30)
T1
0–4
30
Candidate selection (Val30)
T1
5–9
30
Test
T1, T3, T4, T6
10–49
920
Appendix
Table 3: Fixed VLABench episode splits. Indices are zero-based and inclusive within each official task–track list. Counts include only valid selected task–track pairs.
Figure 4: VLABench track examples for book and drink selection. Each row compares initial forward views and task instructions across T1, T3, T4, and T6. T3/T4 illustrate changes in the instruction, while T6 illustrates changes in the visual setting. The images show initial states, not execution outcomes.
Figure 5: VLABench track examples for condiment addition. The columns follow the same T1/T3/T4/T6 order as Figure 4 , showing initial forward views and task instructions for two further episodes.
Task family
Train30
Val30
Seen
Unseen
Pick & Place
5
5
35
24
Pick Two & Place
5
5
24
17
Examine in Light
5
5
13
18
Clean & Place
5
5
27
31
Heat & Place
5
5
16
23
Cool & Place
5
5
25
21
Appendix
Table 4: Task-family counts for the fixed ALFWorld Train30/Val30 split and the complete Seen/Unseen evaluation sets.
Partition
Instances
Categories
Use
Train
30
23
Harness evolution
Val
30
23
Candidate selection
Test
282
26
Evaluation
Total
342
26
All scene_1 instances
Appendix
Table 5: Frozen custom EAI split for VirtualHome Action Sequencing in scene_1 . Category counts are distinct official activity labels within each partition.