Organizations: Mohamed bin Zayed University of Artificial Intelligence · University of Sydney · University of Oxford · AI2 Robotics · University of Waterloo · University of Melbourne
Recent vision-language-action (VLA) policies demonstrate promising generalization across diverse short-horizon tasks. However, they remain unreliable on long-horizon tasks, partly because the large-scale training data is biased toward single-stage manipulation tasks that are cheaper to demonstrate. A single weak atomic skill can cause failures across multiple multi-stage tasks. To address such failures, existing methods often require experts to identify the bottleneck and provide additional demonstrations, making the improvement costly and potentially impractical after deployment. To this end, we present a Self-Evolving Embodied System (SEES) that learns from failures and improves the VLA policy without additional expert demonstrations. SEES decomposes long-horizon tasks into atomic tasks and routes them to corresponding family policies. Each family consists of related atomic skills that share one VLA adapter. During execution, the system automatically monitors atomic-task outcomes to identify the most frequently failing atomic skills as the current bottlenecks. To overcome these bottlenecks, SEES constructs tailored RL tasks in simulation by restoring previously encountered states and generating task-specific success criteria with an LLM. Online RL updates the shared family adapters to promote positive transfer among related atomic skills and cumulative improvement across evolution rounds. Extensive experiments show that SEES can be integrated with different VLA backbones to progressively improve their long-horizon performance. We also observe continued improvement on unseen tasks, providing evidence of transfer beyond the evolution settings.
Figures & tables
Figure 1: Human-supervised adaptation versus SEES self-evolution. (a) Related atomic failures can block progress across different long-horizon tasks. (b) Human-supervised adaptation relies on expert diagnosis and additional demonstrations. (c) SEES automatically identifies recurring bottlenecks, constructs targeted RL tasks by restoring encountered states and generating success criteria, and updates shared family adapters.
Figure 2: The SEES execution–evolution loop. In the top row, SEES decomposes long-horizon instructions, executes atomic tasks with family policies, and identifies bottlenecks using learned termination and outcome statistics. In the bottom row (right to left), restored states and LLM-generated success criteria define targeted simulation RL tasks. Online RL jointly trains each family adapter on its selected bottleneck tasks. The checkpoint with the largest positive gain on bottleneck-task validation is registered and returned to execution.
Figure 3: Representative SEES evolution case. SEES identifies an atomic bottleneck from composite-task failures, constructs a targeted RL task from restored pre-task states, and updates the corresponding family adapter. The resulting update improves the bottleneck task, with gains extending to sibling task and composite task execution.
Table 1: RoboCasa365 results. Success rates (%) across two VLA backbones. Starred results are taken from the original RoboCasa365 paper. We evaluate all other results ourselves. Bold indicates the best result within each backbone.
Suite
Task group
Group success
Overall success
Initial SFT
After evolution
Object
Bottleneck tasks
59.24
82.86
66.13 → 86.00
Sibling tasks
82.22
93.33
Spatial
Bottleneck tasks
40.44
49.78
41.33 → 50.53
Sibling tasks
49.33
57.33
Goal
Bottleneck tasks
39.11
56.15
43.73 → 59.47
Table 2: LIBERO results after three evolution rounds. Success rates (%) compare the initial SFT policy with the policy retained after evolution. Bottleneck tasks are selected for PPO in at least one evolution round. Sibling tasks share their family adapter but never participate in PPO. Overall success covers all ten tasks in each suite. Arrows indicate initial-to-evolved performance.
Method
Bottleneck tasks
Sibling tasks
Overall
Before RL
62.7
75.0
67.6
Atomic-specific LoRA
70.7
72.0
71.2
Family-level LoRA
74.0
80.0
76.4
Table 3: Atomic-specific versus family-level RL adaptation. Success rates (%) are reported for the three selected bottleneck tasks and two sibling tasks excluded from RL. Overall success averages all five tasks with equal weight.
Selection strategy
Round 1 target
Round 2 target
Held-out full-task success
Base policy
–
–
9/50 (18%)
Random
Release
Navigate
14/50 (28%)
Highest-success stage
Pick up
Release
17/50 (34%)
Bottleneck (ours)
Scrub
Navigate
32/50 (64%)
Table 4: Ablation of adaptation-target selection strategies on ScrubCuttingBoard . Results report full-task success on 50 held-out scenes after two evolution rounds. All strategies use the same base policy and 40-step RL budget in each round.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: VLM task decomposition. The generated seven-step plan for SetUpCuttingStation .
Decision rule
Outcome Macro-F1 ↑
False success ↓
Termination-time error (steps) ↓
Failure attrib. accuracy ↑
Simulator oracle
100.0
0.0
0.0
100.0
Terminate
94.0
3.3
10.8
92.0
VLM
59.9
17.7
112.6
45.0
Appendix
Table 5: Atomic termination and failure-attribution evaluation. Both rules are evaluated on shared trajectories with oracle-controlled composite execution. Values are percentages except mean absolute termination-time error (control steps).
Frame accuracy
Balanced accuracy
Recall
Episode agreement
LLM-generated predicate
97.8
98.2
96.4
95.9
Appendix
Table 6: Agreement between LLM-generated and official RoboCasa365 success predicates on Atomic-Seen. Values are task-macro percentages, evaluated on identical GR00T N1.5 checkpoint-120000 trajectories.
Task
Initial SFT
After evolution
Gain (pp)
First selected
Alphabet soup
35/75 (46.67%)
56/75 (74.67%)
+28.00
R2
Cream cheese
64/75 (85.33%)
72/75 (96.00%)
+10.67
R2
Salad dressing
27/75 (36.00%)
56/75 (74.67%)
+38.67
R3
Barbecue sauce
50/75 (66.67%)
64/75 (85.33%)
+18.67
R3
Ketchup
50/75 (66.67%)
65/75 (86.67%)
+20.00
Not selected
Tomato sauce
31/75 (41.33%)
57/75 (76.00%)
+34.67
R1
Appendix
Table 7: Per-task sealed LIBERO-Object results. Each entry contains 75 trials using 15 sealed states and five inference seeds. Tasks marked “not selected” never enter PPO training in the accepted chain.
Figure 5: Bottleneck recovery and continued execution. In this paired ScrubCuttingBoard replay, the GR00T baseline loses the sponge and times out at the scrubbing stage (top). The full validated round-3 candidate routing completes scrubbing and the remaining stages (bottom). The evolved row uses the complete family routing, not a single-adapter replacement. The final upper panel marks stopped execution and is not a recorded downstream frame.
Figure 6: Transfer with one shared sink-control adapter. The baseline is compared with the same round-1 sink-control LoRA in both cases: the RL-trained RinseSinkBasin/turn_on_sink subtask (upper pair) and the native TurnOnSinkFaucet atomic task (lower pair). The base fails and the shared adapter succeeds in each displayed pair.