Physical intelligence requires embodied agents to ground language in observations and execute actions while retaining useful skills. Vision-language-action (VLA) policies offer a shared interface, but task-specific post-training often produces separate specialists. Consolidation is difficult: mixed-task reinforcement learning provides sparse trajectory-level credit, while offline distillation may miss student-visited observations and endpoint distillation does not supervise intermediate action-generation states. We introduce VLA Multi-teacher On-Policy Distillation (VLA-MOPD), a protocol for consolidating flow-matching VLA specialists. After acquiring task or suite experts, a shared student collects robot rollouts and queries the instruction-routed frozen teacher at each visited observation and intermediate ordinary differential equation (ODE) state. Detached per-flow-step transition targets supervise valid action chunks through a masked local matching loss; an optional reference-policy term supports retention on other tasks. Starting from one trajectory per LIBERO task, VLA-MOPD raises mean success-once from 75.4% to 98.9%, compared with 98.15% for endpoint action distillation. It improves mean RoboTwin 2.0 success from 73.0% to 89.3%; on a Piper robot, two consolidation rounds improve average success from 70.0% to 90.0%. In a near-matched Spatial-adaptation study, reference matching limits the Object suite decrease to 3.0 points, versus 13.2 without the reference term.
Figures & tables
Figure 1: VLA-MOPD consolidation. Student rollouts provide observations and ODE states; the task-routed specialist labels transitions. The student matches these targets and refreshes rollouts. Velocity and ODE step-mean regression are Euler-equivalent; the transition-KL interpretation assumes shared Gaussian covariance (Eq. 4 ).
Method
Spatial
Object
Goal
LIBERO-10
Avg.
Full dataset (50-traj)
Octo [ 35 ]
78.9
85.7
84.6
51.1
75.1
OpenVLA [ 18 ]
84.7
88.4
79.2
53.7
76.5
NORA [ 14 ]
92.2
95.4
89.4
74.6
87.9
π0 + FAST [ 2 , 31 ]
96.4
96.8
88.6
60.2
85.5
π0.5 [ 32 ]
98.8
98.2
98.0
92.4
96.9
Table 1: Main results on LIBERO (success-once rate, %). VLA-MOPD is trained from a 1-traj-per-task student initialization. Published references use their reported backbones and data. The routed teacher is the RLinf multi-task PPO policy for Spatial, Object, and Goal, and its LIBERO-10-adapted checkpoint for LIBERO-10.
Method
Place Empty Cup
Pick Dual Bottles
Stack Bowls Two
Avg.
Student initialization (50-traj SFT)
69.0
75.0
75.0
73.0
Routed PPO teachers
93.0
97.0
92.0
94.0
VLA-MOPD (ours)
87.0
93.0
88.0
89.3
Table 2: Main results on RoboTwin 2.0 (success-once rate, %). Each entry is evaluated over 100 episodes.
Figure 2: Convergence over the first 500 training steps. The five panels show success-once rates for the four LIBERO suites and their unweighted average. Points mark available evaluations. All suite scores use 500 evaluation episodes. MixRL and VLA-MOPD use matched per-step rollout budgets and are plotted at 50-step checkpoints without smoothing.
Figure 3: Retention and rollout-budget analyses. (a) Object retention at model snapshots selected for similar Spatial adaptation from checkpoints saved at fixed step intervals. The selected Spatial scores differ by at most 1.4 percentage points; Object success measures retention. (b) Average success during rollout-budget sensitivity with 64, 32, and 16 parallel environments.
Figure 4: Real-world experiment scenes. Rows show Pick Banana (a–c), Stack Cup (d–f), and Sort Number (g–i).
Vision-language-action (VLA) policies can handle noncritical stages of long-horizon manipulation yet fail during short precision-critical phases. Lightweight reinforcement learning (RL) specialists can address these failures without directly applying RL to the full VLA, but retain auxiliary controllers and switching mechanisms at deployment. We study how to transfer their capabilities into a single flow-matching VLA. Direct imitation of specialist rollouts can reproduce action jitter and leave the VLA unprepared for states encountered during its own execution. We introduce REPAIR, a value-guided distillation framework that combines offline initialization with on-policy refinement. At each critical-phase observation, REPAIR queries the corresponding specialist for multiple action candidates and weights them using its critic to construct a supervision distribution. A shared weighted flow-matching objective trains the VLA first on observations from specialist-assisted rollouts, then on observations collected through its own execution. Specialists provide supervision without taking control during on-policy collection and are removed at deployment. Across six real-robot tasks with 30 evaluation trials per task, REPAIR achieves 100% success on all six tasks, with lower mean end-effector jerk than specialist-assisted execution on every task.
Zhennan Jiang, Tingying Xie, Shilong Ji +11
University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · Zhongguancun Academy +3
Specialized vision-language-action (VLA) policies acquire strong domain-specific skills, yet consolidating these skills into one shared policy remains challenging. On-policy distillation (OPD) transfers teacher knowledge on inputs induced by the student, addressing the distribution mismatch encountered during deployment. For flow-based VLAs, this mismatch arises at two levels: environment observations and intermediate action-generation states. We propose a two-level on-policy distillation framework that couples student-driven environment interaction with teacher velocity supervision along student-generated action flow trajectories. Frozen domain specialists jointly supervise one student. After each update, the student collects new environment experience and generates fresh action-flow trajectories. Starting from a joint supervised fine-tuning (SFT) initialization, Velocity Loss OPD yields absolute success-rate gains of 22.55% on LIBERO-40, 33.4% on CALVIN five-instruction sequences, and 22.6% on Meta-World MT50. Compared with Action Loss OPD and Teacher-rollout SFT, it achieves absolute success-rate gains of 6.4-9.5% on LIBERO-40, 9.7-18.3% on CALVIN, and 3.2-4.2% on Meta-World MT50. Analysis by sequence length further shows larger success-rate gains on longer CALVIN sequences. These results demonstrate effective multi-domain capability consolidation through two-level on-policy supervision, with a single student deployed across domains.
Lichao Ni, Yuhan Liu, Pengfei Li +1
Huawei Technologies Co., Ltd. · Tsinghua University
Although pre-trained Vision-Language-Action (VLA) models exhibit impressive generalization in robotic manipulation, post-training remains crucial to ensure reliable performance during deployment. However, standard offline Supervised Fine-Tuning (SFT) suffers from distribution shifts and catastrophic forgetting of pre-trained capabilities, while online Reinforcement Learning (RL) struggles with sparse rewards and poor sample efficiency. In this paper, we propose On-Policy VLA Distillation (VLA-OPD), a framework bridging the efficiency of SFT with the robustness of RL. Instead of relying on sparse environmental rewards, VLA-OPD leverages an expert teacher to provide dense, token-level supervision on the student's self-generated trajectories. This enables active error correction on policy-induced states while preserving pre-trained general capabilities through gentle alignment. Crucially, we formulate VLA-OPD via a Reverse-KL objective. Unlike standard Forward-KL that induces mode-covering entropy explosion, or Hard-CE that causes premature entropy collapse, our bounded mode-seeking objective ensures stable policy learning by filtering out the teacher's epistemic uncertainty while maintaining action diversity. Experiments on LIBERO and RoboTwin2.0 benchmarks demonstrate that VLA-OPD significantly improves sample efficiency over RL and robustness over SFT, while effectively mitigating catastrophic forgetting during post-training.