Organizations: University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · Zhongguancun Academy · Striding AI · Beijing Institute of Technology · Tsinghua University
Vision-language-action (VLA) policies can handle noncritical stages of long-horizon manipulation yet fail during short precision-critical phases. Lightweight reinforcement learning (RL) specialists can address these failures without directly applying RL to the full VLA, but retain auxiliary controllers and switching mechanisms at deployment. We study how to transfer their capabilities into a single flow-matching VLA. Direct imitation of specialist rollouts can reproduce action jitter and leave the VLA unprepared for states encountered during its own execution. We introduce REPAIR, a value-guided distillation framework that combines offline initialization with on-policy refinement. At each critical-phase observation, REPAIR queries the corresponding specialist for multiple action candidates and weights them using its critic to construct a supervision distribution. A shared weighted flow-matching objective trains the VLA first on observations from specialist-assisted rollouts, then on observations collected through its own execution. Specialists provide supervision without taking control during on-policy collection and are removed at deployment. Across six real-robot tasks with 30 evaluation trials per task, REPAIR achieves 100% success on all six tasks, with lower mean end-effector jerk than specialist-assisted execution on every task.
Figures & tables
Figure 1: Overview of REPAIR. Local RL expertise provides value-guided supervision for offline initialization and on-policy repair. The VLA acts without specialist takeover during online distillation and is deployed as a single policy.
Figure 2: Specialist deployment across task horizons. (a) Peg-insertion settings. (b) Success rates for full and critical-phase takeover. (c) Position traces with local magnification.
Figure 3: State coverage after offline distillation. PCA of RL-token observations. Shading marks 95% KDE regions.
Figure 4: Training framework of REPAIR. (a) Human demonstrations initialize the base VLA π0.5 . (b) With this VLA frozen, critical-phase RL trains multiple specialist actors and their critics. (c) Offline observations from DE and on-policy observations from Dπ feed a shared value-refined supervision mechanism. For each critical observation, the selected phase specialist generates the action candidates and its critic determines their relative weights. The VLA learns from these targets in critical phases and replayed VLA actions in noncritical phases. Only the VLA is deployed.
Figure 5: The tasks in our experiments.
Stick P&P
Single peg
Dual peg
Bagging
Table clearing
Drawer
Method
SR ↑
Jerk ↓
SR ↑
Jerk ↓
SR ↑
Jerk ↓
SR ↑
Jerk ↓
SR ↑
Jerk ↓
SR ↑
Jerk ↓
Base SFT ( π0.5 )
30.0
4.2±1.2
33.3
3.3±0.5
0.0
–
13.3
4.7±0.9
20.0
4.7±0.8
0.0
–
RLT
100.0
15.7±2.0
100.0
4.7±0.8
100.0
10.3±0.9
100.0
7.2±0.8
100.0
10.4±2.5
100.0
16.7±5.1
EXPO-FT
80.0
13.5±0.8
93.3
4.2±1.0
33.3
9.0±1.3
0.0
–
0.0
–
0.0
–
ConRFT
93.3
12.8±2.3
100.0
3.9±0.6
20.0
7.1±2.4
0.0
–
0.0
–
0.0
–
HG-DAgger
100.0
5.9±1.2
60.0
3.1±0.6
63.3
6.3±1.1
90.0
4.4±0.8
63.3
4.4±0.6
80.0
10.1±1.8
Table 1: Main results on six real-robot tasks (30 trials per method per task). SR: success rate (%, higher is better). Jerk: mean ± standard deviation over successful trials ( ms−3 , lower is better), with identical sampling and filtering across methods. Dashes denote no successful trials.
Table 4: Distillation hyperparameters for REPAIR. Batch size counts observations before candidate expansion.
Setting
Value
RL-token
2048
proprioceptive dimensions
20
Action dimension
20
chunk horizon
10
Actor / critic hidden widths
(512,512,512) / (512,512,512)
Reference dropout probability
0.5
Appendix
Table 5: RLT training hyperparameters.
Figure 7: Single-Arm Peg Insertion failure. Misalignment prevents the peg from entering the opening and completing insertion within the allotted time.
Figure 8: Stick Pick-and-Place. Left: the red boundaries mark the object initialization region. Right: the gripper fails to establish a secure grasp. Dimensions and region annotations are retained from the supplied task illustration.
Figure 9: Dual-Arm Peg Insertion failure. Misalignment between the held components prevents the peg from entering the socket despite repeated attempts.
Figure 10: Toy Stowing in a Drawer failures. Left: the toy is deposited on top of the drawer housing. Right: the drawer remains partially open after the closing attempt.
Figure 11: Grocery Bagging setup and failures. Top left: the red region defines randomized object initialization. Top right: the right gripper fails to secure the chip container. Bottom left: inaccurate alignment prevents placing the container through the bag opening. Bottom right: a chip packet presses against the right rim of the bag and fails to enter. The bottom-right panel is an AI-enhanced illustration based on the supplied experimental frame.
Figure 12: Bimanual Table Clearing setup and failures. Top: the red region defines bowl initialization. Bottom left: the right gripper misses the blue bowl’s rim. Bottom right: the left gripper fails to secure the bowl during handover, causing it to fall when released.
Human trajectories
Dual peg SR ↑
Table clearing SR ↑
50
0.0
–
100
0.0
20.0
150
–
26.7
200
0.0
–
300
0.0
30.0
Appendix
Table 6: Human-only SFT scaling on two long-horizon tasks (30 evaluation trials per setting). Trajectory counts refer to human teleoperation demonstrations. The main experiments use 100 demonstrations for each of these tasks; other buffer sizes are supplementary controls. SR is reported in %; dashes denote untested settings.
Vision-Language-Action (VLA) models remain brittle in long-horizon, contact-rich manipulation because success-only imitation provides little supervision for execution drift, while failed rollouts are often discarded. We introduce RePO-VLA, a recovery-driven policy optimization framework that assigns distinct roles to success, recovery, and failure trajectories. RePO-VLA first applies Recovery-Aware Initialization (RAI), slicing each corrective suffix into a new episode so corrective actions depend on the current adverse state rather than the preceding failure. It then learns a Progress-Aware Semantic Value Function (PAS-VF), aligning spatiotemporal trajectory features with instructions and successful references. The resulting labels salvage useful failure prefixes via reliability decay, while low-value labels mark drift and terminal breakdowns, teaching differences among nominal, failed, and corrective actions. The data engine turns adverse states into planner-generated or human-collected corrective rollouts, teaching recovery to the success manifold. Value-Conditioned Refinement (VCR) trains the policy to prefer high-progress actions. At deployment, a fixed high-value token biases actions toward the learned success manifold without online failure detectors or heuristic retries. We introduce the Failure-Recovery Benchmark (FRBench), with standardized error injection and recovery-focused evaluation. Across simulated and real-world bimanual tasks, RePO-VLA improves robustness, raising adversarial success from 20% to 75% on average and up to 80% in scaled real-world trials. Code and video results are available at https://1018weijia.github.io/RePO-VLA-Web/.
Weijia Liufu, Xiaoyu Guo, Ruiyi Chen +16
Sun Yat-sen University · South China University of Technology · Peng Cheng Laboratory +3
Vision-language-action (VLA) models often struggle in the precision-critical phases of multi-stage manipulation tasks. To mitigate this issue, VLA models can be used in conjunction with reinforcement learning (RL) specialists that are specifically trained to handle the precision-critical phases. However, the coordination between the base VLA model and the RL specialists, which dictates when a specialist should take over from the base VLA and vice-versa, remains an open research question. In this paper, we address this gap by introducing RouteRLT, a modular framework that coordinates a generalist VLA, used as the default controller, with designated precision-critical RL specialists. At a high level, our framework trains a phase-aware coordination mechanism that handles handoffs between the generalist and the specialists. We evaluate RouteRLT on the LIBERO and LIBERO-Plus benchmarks, as well as on a physical connector pickup and insertion task. Overall, we find that RouteRLT improves success on LIBERO, and retains net gains on LIBERO-Plus. On the physical task, RouteRLT completes 65.7% of trials, compared with 8.6% for the baseline. Altogether, these results demonstrate that learned coordination builds on generalist VLA capabilities to improve task completion in precision-critical manipulation.
Chongyu Zhu, Hyegang Kim, Jaden Hinds +3
Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON, Canada.
Although pre-trained Vision-Language-Action (VLA) models exhibit impressive generalization in robotic manipulation, post-training remains crucial to ensure reliable performance during deployment. However, standard offline Supervised Fine-Tuning (SFT) suffers from distribution shifts and catastrophic forgetting of pre-trained capabilities, while online Reinforcement Learning (RL) struggles with sparse rewards and poor sample efficiency. In this paper, we propose On-Policy VLA Distillation (VLA-OPD), a framework bridging the efficiency of SFT with the robustness of RL. Instead of relying on sparse environmental rewards, VLA-OPD leverages an expert teacher to provide dense, token-level supervision on the student's self-generated trajectories. This enables active error correction on policy-induced states while preserving pre-trained general capabilities through gentle alignment. Crucially, we formulate VLA-OPD via a Reverse-KL objective. Unlike standard Forward-KL that induces mode-covering entropy explosion, or Hard-CE that causes premature entropy collapse, our bounded mode-seeking objective ensures stable policy learning by filtering out the teacher's epistemic uncertainty while maintaining action diversity. Experiments on LIBERO and RoboTwin2.0 benchmarks demonstrate that VLA-OPD significantly improves sample efficiency over RL and robustness over SFT, while effectively mitigating catastrophic forgetting during post-training.