Organizations: Mohamed bin Zayed University of Artificial Intelligence · University of Sydney · University of Oxford · AI2 Robotics · University of Waterloo · University of Melbourne
Recent vision-language-action (VLA) policies demonstrate promising generalization across diverse short-horizon tasks. However, they remain unreliable on long-horizon tasks, partly because the large-scale training data is biased toward single-stage manipulation tasks that are cheaper to demonstrate. A single weak atomic skill can cause failures across multiple multi-stage tasks. To address such failures, existing methods often require experts to identify the bottleneck and provide additional demonstrations, making the improvement costly and potentially impractical after deployment. To this end, we present a Self-Evolving Embodied System (SEES) that learns from failures and improves the VLA policy without additional expert demonstrations. SEES decomposes long-horizon tasks into atomic tasks and routes them to corresponding family policies. Each family consists of related atomic skills that share one VLA adapter. During execution, the system automatically monitors atomic-task outcomes to identify the most frequently failing atomic skills as the current bottlenecks. To overcome these bottlenecks, SEES constructs tailored RL tasks in simulation by restoring previously encountered states and generating task-specific success criteria with an LLM. Online RL updates the shared family adapters to promote positive transfer among related atomic skills and cumulative improvement across evolution rounds. Extensive experiments show that SEES can be integrated with different VLA backbones to progressively improve their long-horizon performance. We also observe continued improvement on unseen tasks, providing evidence of transfer beyond the evolution settings.
Figures & tables
Figure 1: Human-supervised adaptation versus SEES self-evolution. (a) Related atomic failures can block progress across different long-horizon tasks. (b) Human-supervised adaptation relies on expert diagnosis and additional demonstrations. (c) SEES automatically identifies recurring bottlenecks, constructs targeted RL tasks by restoring encountered states and generating success criteria, and updates shared family adapters.
Figure 2: The SEES execution–evolution loop. In the top row, SEES decomposes long-horizon instructions, executes atomic tasks with family policies, and identifies bottlenecks using learned termination and outcome statistics. In the bottom row (right to left), restored states and LLM-generated success criteria define targeted simulation RL tasks. Online RL jointly trains each family adapter on its selected bottleneck tasks. The checkpoint with the largest positive gain on bottleneck-task validation is registered and returned to execution.
Figure 3: Representative SEES evolution case. SEES identifies an atomic bottleneck from composite-task failures, constructs a targeted RL task from restored pre-task states, and updates the corresponding family adapter. The resulting update improves the bottleneck task, with gains extending to sibling task and composite task execution.
Table 1: RoboCasa365 results. Success rates (%) across two VLA backbones. Starred results are taken from the original RoboCasa365 paper. We evaluate all other results ourselves. Bold indicates the best result within each backbone.
Suite
Task group
Group success
Overall success
Initial SFT
After evolution
Object
Bottleneck tasks
59.24
82.86
66.13 → 86.00
Sibling tasks
82.22
93.33
Spatial
Bottleneck tasks
40.44
49.78
41.33 → 50.53
Sibling tasks
49.33
57.33
Goal
Bottleneck tasks
39.11
56.15
43.73 → 59.47
Table 2: LIBERO results after three evolution rounds. Success rates (%) compare the initial SFT policy with the policy retained after evolution. Bottleneck tasks are selected for PPO in at least one evolution round. Sibling tasks share their family adapter but never participate in PPO. Overall success covers all ten tasks in each suite. Arrows indicate initial-to-evolved performance.
Method
Bottleneck tasks
Sibling tasks
Overall
Before RL
62.7
75.0
67.6
Atomic-specific LoRA
70.7
72.0
71.2
Family-level LoRA
74.0
80.0
76.4
Table 3: Atomic-specific versus family-level RL adaptation. Success rates (%) are reported for the three selected bottleneck tasks and two sibling tasks excluded from RL. Overall success averages all five tasks with equal weight.
Selection strategy
Round 1 target
Round 2 target
Held-out full-task success
Base policy
–
–
9/50 (18%)
Random
Release
Navigate
14/50 (28%)
Highest-success stage
Pick up
Release
17/50 (34%)
Bottleneck (ours)
Scrub
Navigate
32/50 (64%)
Table 4: Ablation of adaptation-target selection strategies on ScrubCuttingBoard . Results report full-task success on 50 held-out scenes after two evolution rounds. All strategies use the same base policy and 40-step RL budget in each round.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: VLM task decomposition. The generated seven-step plan for SetUpCuttingStation .
Decision rule
Outcome Macro-F1 ↑
False success ↓
Termination-time error (steps) ↓
Failure attrib. accuracy ↑
Simulator oracle
100.0
0.0
0.0
100.0
Terminate
94.0
3.3
10.8
92.0
VLM
59.9
17.7
112.6
45.0
Appendix
Table 5: Atomic termination and failure-attribution evaluation. Both rules are evaluated on shared trajectories with oracle-controlled composite execution. Values are percentages except mean absolute termination-time error (control steps).
Frame accuracy
Balanced accuracy
Recall
Episode agreement
LLM-generated predicate
97.8
98.2
96.4
95.9
Appendix
Table 6: Agreement between LLM-generated and official RoboCasa365 success predicates on Atomic-Seen. Values are task-macro percentages, evaluated on identical GR00T N1.5 checkpoint-120000 trajectories.
Task
Initial SFT
After evolution
Gain (pp)
First selected
Alphabet soup
35/75 (46.67%)
56/75 (74.67%)
+28.00
R2
Cream cheese
64/75 (85.33%)
72/75 (96.00%)
+10.67
R2
Salad dressing
27/75 (36.00%)
56/75 (74.67%)
+38.67
R3
Barbecue sauce
50/75 (66.67%)
64/75 (85.33%)
+18.67
R3
Ketchup
50/75 (66.67%)
65/75 (86.67%)
+20.00
Not selected
Tomato sauce
31/75 (41.33%)
57/75 (76.00%)
+34.67
R1
Appendix
Table 7: Per-task sealed LIBERO-Object results. Each entry contains 75 trials using 15 sealed states and five inference seeds. Tasks marked “not selected” never enter PPO training in the accepted chain.
Figure 5: Bottleneck recovery and continued execution. In this paired ScrubCuttingBoard replay, the GR00T baseline loses the sponge and times out at the scrubbing stage (top). The full validated round-3 candidate routing completes scrubbing and the remaining stages (bottom). The evolved row uses the complete family routing, not a single-adapter replacement. The final upper panel marks stopped execution and is not a recorded downstream frame.
Figure 6: Transfer with one shared sink-control adapter. The baseline is compared with the same round-1 sink-control LoRA in both cases: the RL-trained RinseSinkBasin/turn_on_sink subtask (upper pair) and the native TurnOnSinkFaucet atomic task (lower pair). The base fails and the shared adapter succeeds in each displayed pair.
Vision-Language-Action (VLA) policies are typically adapted using successful demonstrations, which provide direct action supervision but rarely cover failure-prone states. Deployment failures expose these states, yet lack the corrective actions needed for conventional supervised learning. We propose FailPatch, a failure-driven residual patching framework that decouples action supervision from execution-reliability supervision. Successful demonstrations ground how the policy should act, while deployment trajectories indicate when its behavior becomes unreliable. We further observe that action hidden representations exhibit clear linear separability between reliable and failure-associated states while directly conditioning action generation. Building on these insights, FailPatch introduces a Null-gated Residual Expert Bank into the action hidden space of a frozen VLA policy. A unified Preserve--Redirect--Trust objective retains the original policy in reliable states, selects residual experts in failure-associated states and redirects representations from failure regions toward success-associated regions under bounded intervention. With only 0.52% trainable parameters, FailPatch improves success rates by 11.0 percentage points on four long-horizon RoboTwin tasks under clean evaluation, 9.5 percentage points under clean-to-random generalization, and 16.7 percentage points over the baseline across three real-world tasks. Project and code: https://github.com/yupeng-2003/FailPatch.
Peng Yu, Jiacheng Wang, Ziheng Zhang +6
Xi’an Jiaotong University · Dexmal · Nanjing University
Vision-language-action (VLA) policies provide strong priors for language-conditioned manipulation, but remain brittle in off-nominal states requiring targeted recovery. We propose ReCoVLA -- a failure-conditioned residual recovery framework that keeps a pretrained VLA policy frozen, uses an external vision-language model (VLM) to infer the failure mode and recovery stage, and compiles a structured reward from task-relevant components. Rather than using the VLM to generate actions or rewards directly, ReCoVLA uses it as a semantic reward selector: it predicts a recovery descriptor and reward mask for in-simulation residual-policy training, followed by zero-shot sim-to-real deployment of the trained recovery policies. This decouples high-level failure understanding from low-level corrective control to support different VLAs. Experiments across short-horizon, long-horizon, and contact-rich manipulation tasks show that ReCoVLA outperforms the tested baselines on average. In simulation, our reward compiler improves average success from 36.7% for the fine-tuned π0.5 baseline to 66.7%. In physical zero-shot sim-to-real experiments, ReCoVLA achieves the best average performance, with 61.7% success.
Haodi Hu, Chung-Ta Huang, Jing Liu +4
University of Southern California, USA · Mitsubishi Electric Research Laboratories (MERL), USA · Harvard University, USA
Vision-language-action (VLA) policies generate executable robot actions, but plausible proposals can have different consequences for the task. We present EmbodiedSkills, a skill-based agent framework that augments frozen VLA policies with world action models for consequence-aware action selection. The framework defines reusable skills for scene observation, context encoding, candidate generation, consequence prediction, outcome evaluation, candidate comparison, and action execution. Five stages organize their input-output dependencies in a shared feedback loop, accommodating both continuous action chunks and pick-and-place primitives. At each decision point, independently trained world action models predict the latent consequences of a common set of policy proposals. Selection combines their outcome scores and prediction disagreement with a lightweight comparator that estimates the benefit of an alternative from agreement in its predicted effects. The predictive and comparison modules learn from matched-state candidate branches, while the action policy and pretrained observation encoders remain frozen. On MetaWorld MT50 and CLIPort, EmbodiedSkills improves success rates from 76.28% to 78.64% and from 33.94% to 36.83%, respectively. An additional CALVIN evaluation with the frozen FLOWER policy increases five-task-chain success from 77.10% to 77.90%. These results demonstrate the applicability of consequence-aware selection across different policy backends and action representations.
Wei Wang, Wenqiao Zhang, Yutong Lin +14
College of Computer Science and Technology, Zhejiang University · Nanjing University of Aeronautics and Astronautics · Cornell University +3