Independently pretrained language models and vision-language-action (VLA) policies provide complementary building blocks for embodied agents, but connecting them does not automatically produce coordinated behavior. The VLA may generate actions that diverge from the LLM's intended subtask and, upon regaining control after a correction, undo the promising state restored by the LLM. We introduce ESI (Embodied Self-Improvement), an automated flywheel that uses environmental interaction to align reasoning and action. ESI treats the modular agent as the unit of post-training, repeatedly refining the prompts that guide the LLM, the shared skill interface that expresses its intent, and the VLA parameters that realize that intent, then redeploying these components together. Our key insight is that collaboration itself can supply supervision for improving this interface. The LLM's subtask decisions provide semantic supervision for what the policy should execute, while its effective corrections and subsequent student continuations provide action-level supervision for recovery and handoff. The student also learns from its own effective execution, allowing locally useful experience to contribute even when the overall episode fails. On ten selected RoboDojo tasks, ESI achieves a mean Score of 62.03 and a mean success rate of 49.41% using an LLM-VLA agent without generating task-specific control code. On both metrics, it outperforms the published reference results of all hybrid embodied agents, including the Agent+WAM system GO-3 Nano Preview (47.27 Score and 36.60% success), as well as all compared code-based methods.
Figures & tables
Figure 1: The ESI collect–learn flywheel. Within each episode, the LLM plans and reviews, the VLA proposes actions, and teacher corrections create states from which the VLA continues. The shared experience record links local intent, control source, and local execution outcomes. After each collection round, experience informs prompt and skill refinement, while retained effective actions provide VLA training data. The three updated components are deployed together in the next round; LLM weights remain fixed.
Figure 2: ESI architecture and round-level updates. The agent combines a fixed LLM with decision guidance Pr , a shared skill catalog Sr , and a VLA policy θr . Interaction across tasks produces records for three update paths: reviewing difficulties to refine guidance, consolidating recurring custom operations into reusable skills, and constructing policy supervision from locally effective execution. Prompt and skill review uses the interaction records, while VLA training uses the retained effective actions. Training windows preserve continuity across teacher–student handoffs and retain useful behavior from unsuccessful episodes. Updated components form the next-round agent.
All matched cases
Shared successes
Agent
SR (%) ↑
Score ↑
Interventions ↓
Teacher steps ↓
Step share (%) ↓
Interventions ↓
Teacher steps ↓
Baseline (task-only)
29.68
45.67
27.08
187.97
19.26
10.85
79.41
ESI
49.41
62.03
15.92
100.28
11.12
4.71
29.76
Change
+19.74
+16.36
−41.20%
−46.65%
−8.13
−56.63%
−62.53%
Table 2: Matched baseline–ESI comparison. Main evaluation: ten tasks with twenty scheduled cases each. The last two columns compare the 41 cases completed by both agents. Baseline uses task-only conditioning. Interventions count teacher intervention decisions, and steps count teacher-executed actions per episode; step share pools executed steps. Changes in SR and share are percentage points; control-count changes are relative to baseline.
Figure 3: Task-level changes relative to the matched baseline. Left: change in Score. Right: change in success rate, in percentage points. Blue denotes improvement, orange denotes decline, and zero indicates no aggregate change. Task order matches the main comparison table.
Adapted components
Task performance
Configuration
Prompt
Skill
VLA
SR (%) ↑
Score ↑
ESI
52.00
64.60
Baseline guidance
—
41.00
53.90
Free-form subtasks
—
50.00
62.75
Initial VLA weights
—
29.00
50.49
Table 3: Rollback of the three ESI adaptation components. Ten tasks with ten cases each. Checks retain the adapted components; dashes mark rollback. SR and Score are task means.
Figure 4: Teacher control under component rollback. Top: all configurations on zero-based axes. Bottom: enlarged views of ESI, Baseline guidance, and Free-form subtasks; dashed lines mark ESI. Values are mean teacher intervention decisions or teacher-executed steps per episode; parentheses give relative differences from the matched ESI reference in Table 3 .
Experiment
Agent
Successes
Interventions
Teacher steps
Teacher step share (%)
Solve equation
Baseline
2/5
15.00
99.00
42.45
ESI
4/5
7.40
43.60
21.52
Three orders
Baseline
2/3
36.00
294.33
33.16
ESI
3/3
8.00
71.33
11.74
Table 4: Task transfer and alternative-order execution. Five paired cases of Solve equation and three requested orders per agent in one fixed scene. Both agents use subtask conditioning and action review. Baseline is subtask-conditioned. Interventions count teacher intervention decisions, and teacher steps count executed teacher actions, averaged per episode; teacher step share pools executed steps. Figures 5 and 6 visualize the two studies.
Figure 5: Executing a new task with less direct teacher control. Both agents solve the same equation. Curves trace the projected gripper reference point; blue and orange denote VLA and teacher control. Strips show executed control on a shared step axis. GPT counts report teacher intervention decisions and teacher-executed steps; Baseline is subtask-conditioned.
Figure 6: Language-conditioned intent following in a fixed scene. Rows show three requested pad-filling orders toward the same target, 8720 . Columns show the first placement on each successive pad, numbered left to right; labels identify the digit and destination. All panels use the same scene crop. Final outcomes include subsequent pose adjustments.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Field
Meaning
skill_type
Standard operation name or custom .
arm
Acting arm: left , right , or both .
object_ref
Object or entity involved in the operation.
destination_ref
Target entity or region for operations that require one.
target_relation
none , to , on , in , or near .
target_orientation_ref
Desired orientation of the object.
Appendix
Table 5: Fields of the shared subtask representation.
Figure 7: From a recorded interaction to policy supervision. Three frames from a third-round episode align with the controller, instruction, and eligibility tracks. Control and instruction change at 332; eligibility ends at 347. The loader gives window A (start 319) the alignment subtask and window B (start 328) task-only conditioning, using the midpoint cutoff of decision [324,332) . Both 50-step masks stop at 347, even when eligibility resumes at 373. Step intervals are zero-based and half-open.
Setting
Configured value
Student initialization
Round-two checkpoint at step 6000
Language mode
Task plus current subtask, with transition fallback
Action horizon / minimum retained tail
50 / 15 steps
Round-one / two / three sampling mass
0.16 / 0.24 / 0.60
Task and standard/random balancing
Uniform task mass; 0.5 / 0.5 variant mass
Successful / unsuccessful case weights
2 / 1
Appendix
Table 6: Third-round configuration snapshot.
Task
ESI reference
Baseline guidance
Free-form subtasks
Initial VLA weights
Pack objects
64.50 / 50%
35.00 / 10%
50.00 / 30%
30.83 / 10%
Sweep blocks
10.00 / 10%
10.00 / 10%
10.00 / 10%
25.00 / 20%
Largest number
94.00 / 90%
66.50 / 50%
71.50 / 60%
67.14 / 40%
Sort nesting dolls
30.00 / 30%
10.00 / 10%
60.00 / 60%
37.50 / 30%
Classify objects
85.50 / 80%
94.00 / 90%
88.00 / 80%
86.67 / 70%
Fill pen holder
32.50 / 0%
46.00 / 20%
34.50 / 10%
52.22 / 20%
Appendix
Table 7: Task-level component results. Cells show Score / SR (%). Each task has ten planned cases. The reference and outcome definitions follow Table 3 .
Order
Agent
Complete
Score
Interventions
Teacher steps
1234
Baseline
No
30.00
41
366
1234
ESI
Yes
100.00
11
138
2314
Baseline
Yes
100.00
43
368
2314
ESI
Yes
100.00
7
43
4321
Baseline
Yes
100.00
24
149
4321
ESI
Yes
100.00
6
33
Appendix
Table 8: Order-conditioned case results. A fixed initial layout is evaluated under three requested filling sequences. Video inspection confirms order adherence; completion is measured by the native task evaluator. Baseline is subtask-conditioned; interventions and teacher steps follow the definitions in Section 6 .
Token category
Baseline
ESI
Reduction
Uncached input (K)
435.62
317.73
27.06%
Cached input (M)
12.243
7.322
40.19%
Output (K)
59.36
29.61
50.13%
Total (M)
12.738
7.669
39.79%
Appendix
Table 9: Token usage in the order case study. Baseline is subtask-conditioned. Means per selected episode under the shared subtask-conditioning and action-review flow. K and M denote thousands and millions. Total includes cached input; output already includes reasoning tokens.