Stochastic Self-Distillation for Flow-Based Vision-Language-Action Models
Organizations: Imperial College London
Abstract
Flow-based vision-language-action (VLA) models achieve strong performance in robotics through imitation learning, yet fine-tuning them with on-policy reinforcement learning (RL) is limited. In group-based RL methods like Group Relative Policy Optimization (GRPO), every rollout step requires multi-step ODE integration, and the sampler's limited stochasticity leads to rollout groups with low variance. We propose VLA-SSD, which addresses both high inference latency and limited stochasticity. In particular, our method self-distills a pretrained flow-matching action head into a stochastic one-step policy by conditioning a single-jump generator on a Brownian path. We derive a conditional action density via our stochastic one-step policy, enabling efficient post-training via likelihood-ratio RL algorithms. We show that distilling VLA-SSD from the VLA reduces the action head's forward-pass time by a factor of 7.7, at the cost of a 5.5% relative decrease in the success rate on the LIBERO benchmark. However, by leveraging the increased sample diversity of VLA-SSD during group generation, post-training a VLA using GRPO yields a 20.6% relative improvement.
Figures & tables
| Policy | Sampler | Passes | Forward (ms) | LIBERO | MetaWorld |
| SFT | ODE, | 10 | 58.4 | 77.1 | 51.8 |
| , GRPO | SDE, | 4 | 24.5 | 91.5 | — |
| , PPO | SDE, | 4 | 24.5 | 98.3 | 70.7 |
| Itô student (ours) | Itô, | 1 | 7.6 | 68.6 | 49.0 |
| Itô + GRPO (ours) | Itô, | 1 | 7.6 | 93.0 | 72.0 |
| Map | Objective | Forward (ms) | spatial | object | goal | long | mean |
| teacher, | — | 58.4 | 96.8 | 98.8 | 95.8 | 85.2 | 94.15 |
| Deterministic flow map | LSD | 7.6 | 96.2 | 96.0 | 94.0 | 79.0 | 91.30 |
| PSD | 7.6 | 96.4 | 95.4 | 93.6 | 77.8 | 90.80 | |
| ESD | 7.6 | 90.2 | 76.8 | 77.0 | 26.0 | 67.50 | |
| Itô flow map, | LSD | 7.6 | 94.2 | 93.6 | 91.0 | 77.0 | 88.95 |
| PSD | 7.6 | 96.6 | 91.2 | 90.8 | 73.2 | 87.95 |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| RL checkpoint | Mean absolute log-ratio error (FP32) |
| Early | 0.000181 |
| Middle | 0.000070 |
| Late | 0.000061 |