cs.RONov 19, 2026

RefreshGrad: An Online Adaptive Data-sampling Method for Diffusion-based Visuomotor Learning

Authors: Dijia Zhan, Jinyi Li, Xuemiao Xu, Yan Zhou, Yu Liu, Shaoshan Liu, Jie Tang

Organizations: South China University of Technology · Tsinghua University · The Chinese University of Hong Kong, Shenzhen

Abstract

Training vision-language-action (VLA) policies from demonstrations typically samples trajectory segments uniformly or relies on static offline data curation before training. However, we find that demonstration data vary sharply in supervision value across temporal windows, and that each window's value to the policy evolves substantially during training rather than being fixed or captured by raw loss alone. Motivated by this finding, we propose RefreshGrad, an online adaptive data-sampling framework that periodically scores temporal windows from the evolving policy using five policy-native signals and dynamically adjusts their sampling weights during training. RefreshGrad focuses optimization on informative, learnable windows without altering the policy architecture, training objective, or inference cost. Across ten LIBERO-Plus manipulation tasks, RefreshGrad boosts average success from 60.7% (Uniform) and 68.2% (FAKTUAL) to 76.1% under visual perturbations, and from 57.4% and 73.3% to 81.3% in clean scenes.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

May 15, 2026

Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillation

Billion-parameter Vision-Language-Action (VLA) policies have recently shown impressive performance in robotic manipulation, yet their size and inference cost remain major obstacles for real-time closed-loop control. We introduce VLA-AD, a distillation framework that uses a Vision-Language Model as an offline semantic supervisor to transfer large VLA teachers into lightweight student policies. Instead of relying only on low-level action imitation, VLA-AD augments teacher-provided 7-DoF action targets with high-level semantic guidance, including task phase anchors and multi-frame operating-direction descriptions. These auxiliary signals are used only during training: at test time, the student policy runs independently, with neither the VLA teacher nor the VLM required. We evaluate VLA-AD on three LIBERO benchmark suites. Using OpenVLA-7B as the teacher, our method produces a 158M-parameter student, yielding a 44×44\times reduction in model size while matching the teacher with only a 0.27%0.27\% average relative gap. The resulting policy runs at 12.5 Hz on an RTX 4090, achieving a 3.28×3.28\times inference speedup over OpenVLA-7B. We further show that the same semantic distillation pipeline generalizes to a different π0.5π_{0.5}-4B teacher, where the student outperforms the teacher on two suites and remains within 0.53%0.53\% on libero\_goal. Additional analysis indicates that phase-level supervision and multi-frame directional cues make the student less sensitive to noisy teacher actions, such as erroneous high-frequency gripper changes. Overall, VLA-AD demonstrates that offline semantic guidance from VLMs can substantially improve the efficiency, robustness, and deployability of VLA policy distillation.
Jun 24, 2026

ROAD-VLA: Robust Online Adaptation via Self-Distillation for Vision-Language-Action Models

Effective online adaptation of vision-language-action (VLA) models remains challenging, as sparse rewards provide weak supervision for high-dimensional autoregressive action policies. Although self-distillation can in principle provide denser training signals, we find that text-based privileged teachers conditioned on demonstrations, retrieved experiences, or high-level plans are ineffective for VLA adaptation, exposing a modality gap between symbolic guidance and low-level robot actions. We propose ROAD-VLA, an advantage-guided self-distillation framework that constructs a proximal teacher directly in action space by perturbing action-token logits with calibrated advantage estimates. This converts sparse rewards into dense token-level supervision while keeping the teacher close to the current policy. We further derive a policy-improvement lower bound under calibrated advantages and accurate teacher matching. Across seven robotic manipulation environments with in-distribution and out-of-distribution shifts, ROADVLA outperforms PPO in nearly all settings, demonstrating robust online VLA adaptation.
Oct 24, 2026

TAD: Distilling Vision-Language-Action Policies from Teacher Trajectories Alone

Online distillation of vision-language-action (VLA) policies commonly relies on teacher action scores that an inference-only interface may not expose. Trajectory-only Adversarial Distillation (\gad{}) instead turns a fixed bank of teacher joint-position features into adaptive feedback: a discriminator compares feature marginals from teacher and student rollouts and supplies rewards for on-policy learning. The student interacts with the environment without online teacher queries or environment task rewards. On ManiSkill PutOnPlateInScene25Main, \gad{} improves OpenVLA-OFT (7B) from 26.2%26.2\% to 76.6%76.6\% \successonce{} (+50.4+50.4 percentage points) and from 20.3%20.3\% to 58.6%58.6\% \successend{}. It trails white-box on-policy distillation (OPD) at 81.6%81.6\% / 59.4%59.4\% by 5.05.0 / 0.80.8 percentage points (n=256n{=}256 evaluation trajectories per checkpoint). The tested four-epoch continuation achieves the highest \gad{} result without additional teacher data. This best run uses 5×5\times OPD's student interaction budget and 12.5×12.5\times its complete-rollout batch-passes. The comparison demonstrates online VLA improvement without teacher scoring while quantifying unequal interaction and optimization budgets.