Modern learning systems increasingly combine pretrained models with downstream heads. Despite the growing importance of such modular designs, how the downstream model co-adapts to the geometry of pretrained representations remains relatively underexplored. We study this problem through vision-language-action models (VLAs), where a pretrained vision-language model (VLM) conditions an action model for control. We find that the action model progressively concentrates its sensitivity on high-variance dimensions of the VLM outputs. Through controlled experiments across different backbone-head models, we further show that counteracting this sensitivity concentration can be beneficial under representation--task mismatch settings. Motivated by these observations, we introduce Adaptive Interface Equalizer (AdaIE), a parameter-free mechanism that reweights pretrained representations according to the evolving downstream sensitivity during training. AdaIE improves VLA success rates on simulation benchmarks, with gains of up to +5.28 percentage points (pp) on RoboCasa-GR1 and +1.68 pp on RoboTwin 2.0, and achieves an average gain of +19 pp across five real-world manipulation tasks. It further generalizes to diffusion policies and world models, improving downstream performance in both settings. Our results highlight interface co-adaptation as an important optimization problem when coupling pretrained representations with downstream models.
Figures & tables
Figure 1: Action-head sensitivity progressively aligns with the anisotropic VLM representation during training. Top : per-channel variance of the VLM output dimensions at different training steps. Bottom : corresponding per-channel action-head sensitivity. While sensitivity is relatively uniform early in training, it progressively concentrates on high-variance VLM dimensions as training proceeds, revealing increasing alignment between VLM anisotropy and action-head sensitivity.
Task
Dataset
Interface
Metric
Equalize
Baseline
Exploit
Aligned
Depth
NYUv2
DINOv2-S → DPT
δ1 (%) ↑
93.4 ± 0.3
93.5 ± 0.1
93.2 ± 0.3
Classification
CIFAR-100
DeiT-S → Cls.
Acc. (%) ↑
90.0 ± 0.2
89.6 ± 0.4
89.8 ± 0.4
Mismatched
Diffusion Policy
Push-T
ResNet → DP
Score ↑
0.936 ± 0.003
0.922 ± 0.003
0.909 ± 0.009
VLA
RoboCasa
VLM → DiT
Succ. (%) ↑
22.9 ± 0.5
19.4 ± 0.6
16.2 ± 1.1
Table 1: Effect of sensitivity reweighting under aligned and mismatched interfaces.
Figure 2: Overview of Adaptive Interface Equalizer (AdaIE) on VLAs. AdaIE operates at the interface between the VLM and the action DiT. It estimates the action head’s per-channel sensitivity online using stochastic probes, maintains a running sensitivity estimate, and inversely rescales the VLM conditioning channels before they are consumed by the DiT.
Method
Backbone
Steps
Avg. (%)
π0.5
–
–
37.00
GR00T-N1.6
–
–
47.60
StarVLA-GR00T
Qwen3-VL-4B
100K
47.80
StarVLA- π
Qwen3-VL-4B
100K
43.90
StarVLA-OFT
Qwen3-VL-4B
100K
48.80
GR00T
Qwen3-VL-2B
100K
42.36
Table 2: Average success rate on RoboCasa-GR1 (24 tasks). Results above the separator are reported by StarVLA ( Community, 2026 ) , while results below are our runs, evaluated with 60 rollouts per task. Detailed per-task results of our runs are provided in Appendix A.5 .
Method
Backbone
Steps
Success Rate (%)
GR00T
Cosmos-Reason2-2B
150K
66.92
GR00T + AdaIE
Cosmos-Reason2-2B
150K
68.60 (+1.68)
Table 3: Success rates on RoboTwin 2.0 (50 tasks). Both methods are trained for 150K optimization steps using identical training configurations. See per task comparison in the Appendix A.5 .
Figure 3: Real-world manipulation evaluation. For each task, we conduct 10 trials with the left hand and 10 trials with the right hand. See Table 4 for the detailed results.
Baseline
AdaIE
Task
Hand
Task Succ.
Interaction Succ.
Task Succ.
Interaction Succ.
In-Distribution (ID)
Banana
Left
5/10
10/10
6/10
9/10
Right
4/10
10/10
7/10
10/10
Press Bell
Left
6/10
9/10
5/10
10/10
Right
2/10
9/10
4/10
8/10
Table 4: Real-robot evaluation results on in-distribution (ID) and out-of-distribution (OOD) tasks.
Figure 8
Push-T
Reacher
Method
Success (%)
Std.
Method
Success (%)
Std.
DINO-WM
71.4
7.0
DINO-WM
78.6
3.9
DINO-WM + AdaIE
73.2 (+1.8)
6.5
DINO-WM + AdaIE
82.8 (+4.2)
6.3
Table 6: Performance of DINO-WM and DINO-WM + AdaIE.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
GR1
RoboTwin 2.0
VLM backbone
Qwen3-VL-2B-Instruct
Cosmos-Reason2-2B
Action head
28-layer DiT, 1024-d
28-layer DiT, 1024-d
Training steps
100K
150K
Effective batch size
256
192
Optimizer
AdamW
AdamW
VLM learning rate
1×10−5
1×10−5
Appendix
Table 7: Training configurations for the GR1 and RoboTwin 2.0 experiments. Within each benchmark, the baseline and AdaIE models use identical architectures, data, and optimization settings, differing only by the insertion of AdaIE at the VLM–DiT interface (bottom block).
Figure 5: Representative examples from the real-robot training dataset.
Configuration
Baseline
AdaIE
VLM backbone
Qwen3-VL-2B-Instruct
Qwen3-VL-2B-Instruct
Action head
28-layer DiT, 1024-d
28-layer DiT, 1024-d
Training steps
32K
32K
Effective batch size
512
512
Optimizer
AdamW
AdamW
VLM learning rate
1×10−5
1×10−5
Appendix
Table 8: Training configurations for the real-robot baseline and AdaIE models. Both variants use identical model architectures, data, and optimization settings, differing only by the insertion of AdaIE at the VLM–DiT interface.
Configuration
LeWM
DINO-WM
Predictor
6-block AdaLN-Zero DiT, 192-d
6-layer ViT, 16 heads, MLP 2048 ( ∼ 19M)
Training loss
next-emb. MSE + 0.09 SIGReg
latent next-state MSE (DINOv2 space)
Proprioception
unused
excluded (pixels + actions only)
Epochs
10
10
Batch size
128
32
Optimizer
AdamW
AdamW
Appendix
Table 9: Training configurations for the LeWM and DINO-WM experiments. We use pre-trained backbone for two world model, with Tiny-ViT for LeWM and DINOv2 for DINO-WM.
Configuration
DINOv2-S + DPT / NYUv2
Task / dataset
NYU Depth V2, BTS/Eigen split (24,231 train pairs, 654 test images)
Input
480×640 RGB, ImageNet mean/std
Resize / crop (train / eval)
short side to 518 (multiple of 14) + random 518×518 crop / resize only
Table 10: Training configuration for the DINOv2-S + DPT experiments on NYUv2, following the official Depth Anything V2 metric-depth recipe (SiLog loss, AdamW, polynomial decay, 518×518 crops) with the train.py default of 40 epochs and a global batch of 16. The baseline and AdaIE models are identical except for the insertion of AdaIE at the encoder–decoder interface (bottom block). The standard deviation reflects variation across 3 random seed training in Table 1 .
Configuration
DeiT-S / CIFAR-100
Task / dataset
CIFAR-100 (49k train, 1k minival held out from train, 10k test)
Input
32×32 RGB, bicubic upsampling to 224×224 , ImageNet mean/std
Linear 384→100 , random init (trunc. normal, std 0.02; zero bias)
Head input
CLS token after final LayerNorm (pre-logits)
Appendix
Table 11: Training configuration for the DeiT-S experiments on CIFAR-100. The schedule follows the ViT fine-tuning budget (10k steps, batch 512) with DeiT AdamW defaults. The baseline and AdaIE models are identical except for the insertion of AdaIE at the backbone–classifier interface. The standard deviation reflects variation across the 5 random seed training in Table 1 .
Configuration
DiffusionPolicy-C
Task / dataset
PushT image ( pusht_cchi_v7_replay , 90 training episodes, 2% val)
Table 12: Training configuration for the DiffusionPolicy-C experiments on Push-T, following the official Diffusion Policy recipe. The baseline and AdaIE use identical configurations except for the insertion of AdaIE at the observation–denoiser interface. Following the original evaluation protocol, we report the best checkpoint performance for each variant as the mean and standard deviation of test scores across three random seeds.
Task
GR00T-2B
GR00T-2B + AdaIE
GR00T-4B
GR00T-4B + AdaIE
PnPBottleToCabinetClose
50.00
56.67
65.00
48.33
PnPCanToDrawerClose
65.00
78.33
80.00
75.00
PnPCupToDrawerClose
23.33
23.33
23.33
15.00
PnPMilkToMicrowaveClose
53.33
51.67
60.00
60.00
PnPPotatoToMicrowaveClose
26.67
33.33
41.67
43.33
PnPWineToCabinetClose
48.33
48.33
51.67
51.67
Appendix
Table 13: Per-task success rates on RoboCasa-GR1 for our GR00T baselines and AdaIE. 60 rollouts per task.
Task
Clean
Randomized
Avg
GR00T-2B
+ AdaIE
GR00T-2B
+ AdaIE
GR00T-2B
+ AdaIE
adjust_bottle
94.00
99.00
97.00
98.00
95.50
98.50
beat_block_hammer
62.00
76.00
74.00
74.00
68.00
75.00
blocks_ranking_rgb
65.00
71.00
68.00
64.00
66.50
67.50
blocks_ranking_size
60.00
47.00
54.00
46.00
57.00
46.50
click_alarmclock
100.00
99.00
100.00
100.00
100.00
99.50
Appendix
Table 14: Per-task success rates on RoboTwin 2.0 for the GR00T-2B baseline and AdaIE, evaluated with 100 rollouts per task per setup. Clean uses the standard environment, while Randomized introduces variations in backgrounds, lighting, distractors, tabletop height, and language instructions. Avg denotes the mean over both setups. AdaIE improves average performance primarily under Clean. The limited gain under Randomized may reflect additional perceptual and generalization demands that are not directly addressed by interface sensitivity reweighting.
Task
α
Ep. 1
Ep. 2
Ep. 3
Ep. 4
Ep. 5
Ep. 6
Ep. 7
Ep. 8
Ep. 9
Ep. 10
Two Room
None
94
90
88
90
88
88
90
88
90
92
0.05
94
94
94
90
94
94
92
94
92
90
0.10
92
90
90
90
88
90
92
92
92
92
0.15
94
94
92
88
92
90
90
92
94
92
0.25
92
92
90
88
92
94
94
94
94
94
0.35
92
90
88
88
92
94
94
92
96
94
Appendix
Table 15: Ablation of the AdaIE equalization strength α on LeWM (seed 42). We report planning success rates (%) at each training epoch. None denotes the baseline without AdaIE. Each checkpoint is evaluated over 50 episodes using the same evaluation seed.
Figure 6: The co-adaption of DiffusionPolicy-C. The Policy is trained on Push-T.
Figure 7: The co-adaption of DiffusionPolicy-T.The Policy is trained on Push-T.
Figure 8: The co-adaption of DINO-WM.The model is trained on Two Room.
Vision-Language-Action (VLA) models widely adopt pretrained Vision-Language Models (VLMs) as policy backbones, yet it remains unclear what kind of pretrained VLM representation is useful as a VLA initialization. In this paper, we study VLA initialization as a controlled representation-design problem along three axes: capability-level embodied VQA supervision, parameter-update strategy, and robot-data pretraining. Our experiments show that the original pretrained VLM representation is a key source of action performance. However, embodied VQA adaptation does not yield uniform gains: its benefit depends on downstream bottlenecks, and gains from different capability domains are not simply additive. For update strategy, LoRA provides a more reliable initialization than Full Finetune, indicating that overly reshaping the pretrained representation can weaken VLA initialization. Robot-data pretraining further improves VLA initialization, with the strongest variant obtained by staged LoRA-based training. Together, these findings suggest that effective VLM-to-VLA adaptation should inject action-relevant embodied and robot-trajectory signals while preserving the pretrained VLM representation that remains useful for action learning.
Large-scale pretraining has made Vision-Language-Action (VLA) models promising foundations for generalist robot manipulation, yet adapting them to downstream tasks remains necessary. However, the common practice of full fine-tuning treats pretraining as initialization and can shift broad priors toward narrow training-distribution patterns. We propose PriorVLA, a novel framework that preserves pretrained priors and learns to leverage them for effective adaptation. PriorVLA keeps a frozen Prior Expert as a read-only prior source and trains an Adaptation Expert for downstream specialization. Expert Queries capture scene priors from the pretrained VLM and motor priors from the Prior Expert, integrating both into the Adaptation Expert to guide adaptation. Together, PriorVLA updates only 25% of the parameters updated by full fine-tuning. Across RoboTwin 2.0, LIBERO, and real-world tasks, PriorVLA achieves stronger overall performance than full fine-tuning and state-of-the-art VLA baselines, with the largest gains under out-of-distribution (OOD) and few-shot settings. PriorVLA improves over pi0.5 by 11 points on RoboTwin 2.0-Hard and achieves 99.1% average success on LIBERO. Across eight real-world tasks and two embodiments, PriorVLA reaches 81% in-distribution (ID) and 57% OOD success with standard data. With only 10 demonstrations per task, PriorVLA reaches 48% ID and 32% OOD success, surpassing pi0.5 by 24 and 22 points, respectively.
Xinyu Guo, Bin Xie, Wei Chai +4
Institute of Automation, Chinese Academy of Sciences · Dexmal · Nanjing University of Aeronautics and Astronautics +1
Reinforcement learning (RL) is increasingly used to adapt pretrained vision-language-action models (VLAs). We investigate the problem of behavioral steering: adapting a VLA's execution style to preferences such as carrying objects higher. To investigate how little trainable capacity such steering requires and how much pretrained capability can be retained, we freeze an OpenVLA-OFT 7B policy and optimize TinyLoRA adapters with 8 to 384 trainable parameters using either policy-gradient RL (GRPO) or evolutionary search (CMA-ES). With just 256 trainable parameters, we raise carry height by 120 mm on trained tasks at 92% success and by 74 mm on a held-out task at 80% success. By rescaling the learned update, we can tune the trade-off between style gains and capability retention without retraining. Increasing adapter capacity generally improves this trade-off, retaining more pretrained capability at comparable style gains. The choice of optimizer also depends on capacity, with CMA-ES being competitive at low parameter counts, while GRPO offers better trade-offs as capacity grows. These findings show that extremely low-dimensional updates can steer pretrained VLA behavior, with adapter capacity, optimizer choice, and update scaling offering practical control over adaptation and retention.