How (and How Not) to Use Data Augmentation in VLA Post-Training

Bram Grooten; Joaquin Vanschoren

cs.ROOct 5, 2026

How (and How Not) to Use Data Augmentation in VLA Post-Training

Authors: Bram Grooten, Joaquin Vanschoren

Organizations: TU Eindhoven

Abstract

Vision-language-action (VLA) models currently demonstrate strong performance in a wide range of real-world robotics tasks. However, they often still lack the generalization ability to handle large visual out-of-distribution shifts. Post-training of VLAs with reinforcement learning (RL) has been shown to benefit robustness, but significant room for improvement remains. In this work, we systematically study the effect of image augmentation on VLA post-training. We find that it is crucial to augment only the critic module during RL updates, while leaving the actor's input clean during both rollouts and updates. For $π_{0.5}$ and GR00T N1.5 this raises out-of-distribution success on LIBERO-Plus by $7.8$ and $10.0$ points respectively, while augmenting the actor collapses training entirely. We investigate a range of augmentation types and strengths, and provide practical recommendations for improving generalization in VLA post-training.

Figures & tables

Panel of Figure 1 — Figure 1: Augmentation can improve RL post-training of VLAs. Left: one LIBERO observation blended with a random ImageNet image at two strengths. The VLA sees cleanly during rollouts and evaluation, but receives augmented observations for training updates. Right: out-of-distribution success on sensor noise LIBERO-Plus environments for $\pi_{0.5}$ post-trained with PPO. Naively augmenting both actor and critic destroys the policy within 100 epochs, while giving the critic the augmented view and the actor the clean one improves OOD generalization.

Panel of Figure 2 — Figure 2: Five visual perturbation axes of LIBERO-Plus. Each column shows the original base task on top and an out-of-distribution LIBERO-Plus perturbation below. Frames are the full main-camera observation after 20 timesteps with a random policy.

Panel of Figure 3 — Figure 3: The observation before and after augmentation. The base and wrist views (left pair) and the augmented versions the critic receives (right pair), blended with a random ImageNet overlay at $\alpha=0.5$ . Both views draw the same overlay image but crop and flip it independently, as in training.

$\pi_{0.5}$ ( Black et al., 2025a )
			OOD success (LIBERO-Plus)
Augmented modules	Passes	IND	Tex.	Light	Layout	Cam.	Noise	Avg
# tasks		10	258	292	385	376	351	1662
SFT checkpoint (no RL)	–	78.8	89.2	83.6	80.8	17.0	12.8	56.7
no augm. (baseline)	1	93.2	95.0	95.9	92.2	45.7	40.7	73.9
actor + critic	1	0 0.8	0 0.0	0 0.0	0 0.0	0 0.0	0 0.3	0 0.1

Table 1: Where augmentation is applied decides whether RL post-training survives. We post-train VLA models on LIBERO-Spatial with PPO for 300 epochs, using random overlay augmentation at

\alpha=0.5

. All values are success rate percentages. Passes counts forward passes through the VLM backbone per RL training step, and the # tasks row gives the suite sizes. † Collapsed runs were stopped early and are reported at epoch 100.

$\pi_{0.5}$ ( Black et al., 2025a )
		OOD success (LIBERO-Plus)
	IND	Tex.	Light	Layout	Cam.	Noise	Avg
no augmentation	93.2	95.0	95.9	92.2	45.7	40.7	73.9
overlay ( $\alpha=0.5$ )	98.0	95.8	96.9	93.0	55.1	67.8	81.7
shift ( $10$ px)	95.8	95.0	95.2	92.2	57.7	54.4	78.9
overlay + shift	93.6	96.5	93.8	91.4	48.4	49.3	75.9

Table 2: The best augmentation type depends on the model. Training on LIBERO-Spatial with PPO for 300 epochs, using critic-only augmentation. All values are success rate percentages.

		OOD success (LIBERO-Plus)
	IND	Tex.	Light	Layout	Cam.	Noise	Avg
no augmentation	93.2	95.0	95.9	92.2	45.7	40.7	73.9
$\alpha=0.10$	96.0	94.6	94.2	92.7	50.5	53.0	77.0
$\alpha=0.25$	93.8	95.8	90.1	91.4	52.4	55.6	77.0
$\alpha=0.50$	98.0	95.8	96.9	93.0	55.1	67.8	81.7
$\alpha=0.75$	94.8	95.4	95.9	91.4	59.8	60.4	80.6

Table 3: Overlay strength has a single peak, and every setting beats no augmentation. Training

\pi_{0.5}

( Black et al., 2025a ) on LIBERO-Spatial with PPO for 300 epochs, using critic-only augmentation with random overlay. All values are success rate percentages.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Data collection
	$\pi_{0.5}$	GR00T N1.5
Epochs	$300$
Episodes per epoch	$512$ ( $8$ rounds $\times$ $64$ envs)
Parallel environments	$64$
Max. episode length	$240$ steps
Observation resolution	$224\times 224$ , two camera views

Appendix

Table 4: VLA post-training hyperparameters. Settings for the runs of Section 4 , one column per model. Values shared by both are given once.

Placement and schedule
Augmented modules	critic only (default)
Phases augmented	training updates only; rollouts and evaluations stay clean
Augmented views	both cameras (base and wrist)
Draw schedule	fresh sample per observation, shared across the two views
Random overlay
Blend strength $\alpha$	$0.5$ (default)

Appendix

Table 5: Augmentation hyperparameters. Shared by both models. The placement, type and strength rows are the variables studied in Sections 4.1 , 4.2 and 4.3 ; their defaults are given here.

Panel of Figure 4 — Figure 4: Random overlay across the full strength range. Five observations from one LIBERO-Spatial episode, as the policy receives them (top) and after blending with an ImageNet image (bottom), at the $\alpha$ below each column. Every column draws its own overlay, randomly resized, cropped and flipped before blending. Our main experiments use $\alpha=0.5$ , the middle column; Section 4.3 varies it. Rollouts and evaluations always use the clean view, while updates (may) receive augmented versions.

Panel of Figure 5 — Figure 5: Out-of-distribution success during post-training, per perturbation axis. The unaugmented $\pi_{0.5}$ PPO baseline against critic-only overlay augmentation at $\alpha=0.5$ , on each visual axis separately.

Panel of Figure 6 — Figure 6: Training dynamics under each augmentation placement. The four $\pi_{0.5}$ placements of Table 1 . Per-epoch traces are smoothed with a centered nine-epoch moving average; evaluations are plotted as measured.

Panel of Figure 7 — Figure 7: In-distribution against out-of-distribution success. One marker per configuration at its final checkpoint; the dashed crosshair marks the unaugmented baseline. Note that every critic-only augmentation used maintains or improves both IND and OOD performance.

Explore similar work

RL $^2$ -VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models
Jul 29, 2026Derek Ming Siang Tan, Shailesh Shailesh, Srikrishna Iyer +4Inference-Time Steering Video Latents
1National University of Singapore · 3Singapore Technologies Engineering · University of Toronto
Seeing Realism from Simulation: Efficient Video Transfer for Vision-Language-Action Data Augmentation
May 4, 2026Chenyu Hui, Xiaodi Huang, Siyu Xu +5Diffusion-Based Vision-Language-Actions Data Augmentation
Shanghai Jiao Tong University · Xi’an Jiaotong University · Institute of Automation, Chinese Academy of Sciences +3
Scaling by Diversified Experience for Vision-Language-Action Models
Jun 8, 2026Leiyu Wang, Zhaofengnian Wang, Xueqi Li +3Decoupling
Shanghai Jiao Tong University · Shanghai Innovation Institute · Tongji University +3

Jul 29, 2026, cs.ROJ/K move · Enter open · S save

RL $^2$ -VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models

Derek Ming Siang Tan, Shailesh Shailesh, Srikrishna Iyer, William Wei Jie Teo+3

1National University of Singapore · 3Singapore Technologies Engineering · University of Toronto

Inference-Time Steering Video Latents

Open paper AskPDF arXiv

Despite the impressive visuomotor capabilities enabled by Vision-Language-Action (VLA) models, their performance often degrades on challenging and out-of-domain tasks. Recent test-time steering and scaling methods improve performance without extensive data collection and retraining, but action samples often remain concentrated around similar behaviors and therefore inherit correlated failure modes. Moreover, existing methods apply the same intervention strategy at every timestep, regardless of whether the base policy is already likely to succeed. To address these limitations, we introduce

RL^2

, an adaptive inference-time steering framework that leverages Reinforcement Learning on VLA Latents. First, we train a lightweight offline RL policy conditioned on expressive latents extracted from the VLA action expert and compose its flow velocity with that of the frozen VLA during inference. This compositional steering strategy combines the behavioral priors of large-scale imitation learning with the action diversity induced by offline RL beyond dominant demonstration modes. We further discover that inference-time steering follows fundamentally different scaling laws under success and failure states, revealing that action diversity is most beneficial when the base VLA is likely to fail, but can unnecessarily perturb already-accurate actions when success is likely. Building on this insight,

RL^2

activates compositional steering only when failure is predicted. Across the SIMPLER and PolaRiS benchmarks,

RL^2

improves success rates by up to +17.3% in out-of-domain settings, while ablations and scaling studies demonstrate the importance of latent representations and RL training. Finally, real-world experiments demonstrate that these gains transfer beyond simulation, establishing

RL^2

as a practical and modular steering framework for VLA deployment.

Abstract

Explore similar work

RL2^22-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models

Seeing Realism from Simulation: Efficient Video Transfer for Vision-Language-Action Data Augmentation

Scaling by Diversified Experience for Vision-Language-Action Models

RL $^2$ -VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models