Pretrained vision language action (VLA) models show strong zero-shot performance but they are rarely deployed directly and are often first post-trained on demonstrations from the target setting. Yet VLAs are typically compared by zero-shot or full-data fine-tuned success which reveals little about the data and compute required for downstream adaptation. We study adaptation efficiency for six VLAs (π0, π0.5, and GR00T N1-N1.7) on four LIBERO suites, varying both demonstrations and training steps. This reveals differences hidden by full-data fine-tuning: models which are within two points on the full LIBERO-Spatial dataset differ by up to 3× in success with 50 demonstrations. π0.5 needs 2-5× fewer demonstrations than π0 and is the most sample-efficient model at every budget, with the largest gap on the long-horizon suite. Within GR00T, N1.5 adapts more efficiently than N1.6 and N1.7 on three of four suites, despite their stronger zero-shot performance. We also study one pretraining factor behind these differences by freezing the VLM during finetuning: this severely degrades the π models whose VLM is jointly trained with the action expert during pretraining, but barely affects GR00T models.
Figures & tables
Figure 1: Best checkpoint success versus number of fine-tuning demonstrations (whole suite, log scale). A model lying above another at a smaller budget is more sample-efficient: e.g., on Spatial, π0.5 with 25 demonstrations (75.0%) outperforms π0 with 50 (63.0%). Within GR00T, N1.5 lies above N1.6 and N1.7 on Spatial, Object, and Goal.
Figure 2: Full-data fine-tuning of all six models (400 evaluation episodes per checkpoint). Most models converge to similar success on the short-horizon suites, but π0.5 gets there 2–5 × faster than any other model. On Long it plateaus about 8 points above GR00T and 13 above π0 . GR00T full-data runs on Object are not yet available.
Suite
B
π0
N1.5
N1.6
N1.7
Spatial
25
35.8 → 10 (2.5 × )
44.8 → 10 (2.5 × )
4.5 → 5 (5.0 × )
27.5 → 5 (5.0 × )
Spatial
50
63.0 → 25 (2.0 × )
73.0 → 25 (2.0 × )
30.8 → 10 (5.0 × )
60.5 → 25 (2.0 × )
Object
25
12.0 → 5 (5.0 × )
46.5 → 15 (1.7 × )
8.2 → 5 (5.0 × )
8.0 → 5 (5.0 × )
Object
50
44.5 → 15 (3.3 × )
69.5 → 25 (2.0 × )
40.2 → 10 (5.0 × )
59.2 → 15 (3.3 × )
Goal
25
21.5 → 10 (2.5 × )
35.8 → 10 (2.5 × )
7.5 → 5 (5.0 × )
19.8 → 10 (2.5 × )
Goal
50
42.8 → 15 (3.3 × )
51.2 → 50 (1.0 × )
46.0 → 15 (3.3 × )
46.0 → 15 (3.3 × )
Table 1: How many demonstrations does π0.5 need to match each model? Each cell reads “best success of the column model with B demonstrations → smallest tested budget B′ at which π0.5 reaches it ( B/B′ ).” Pairwise multipliers, including within GR00T, are listed in Table 3 .
Figure 3: Success over training with a fixed demonstration budget (50. 100 for Long). At a given budget, an epoch is the same number of optimization steps for every model. π0.5 leads from the first epoch on every suite. Among GR00T models, N1.5 learns fastest on Spatial, Object, and Goal. Curves for every budget are in Appendix A .
Figure 4: Best success with full fine-tuning (solid) and with the VLM frozen (hatched) on the full Spatial and Long datasets. Numbers give the change in points. Freezing the VLM costs the π models 23–56 points but changes GR00T N1.5–N1.7 by at most 6. Frozen runs train for 6 ( π ) and 8 (GR00T) epochs.
Figure 5: Full-data learning curves with full fine-tuning (solid) and the VLM frozen (dotted). Freezing slows and caps the π models, whereas the GR00T curves nearly coincide.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: π0 (top) and π0.5 (bottom) budget sweeps with full fine-tuning, 400 evaluation episodes per checkpoint. Darker lines denote larger budgets.
Figure 7: GR00T budget sweeps with full fine-tuning, 400 evaluation episodes per checkpoint. Colors denote models and line styles budgets.
Figure 8: Best success versus fine-tuning demonstrations for π0 and π0.5 with full fine-tuning (opaque) and the VLM frozen (faded).
Figure 9: π0 (top) and π0.5 (bottom) budget sweeps with the VLM frozen. Only the best success (25.0%) is available for the π0 Spatial run with 50 demonstrations, so its curve is not shown.
Spatial
B=5
B=10
B=15
B=25
B=50
π0
1.5
9.2
17.5
35.8
63.0
π0.5
29.8
49.0
52.5
75.0
94.2
N1.5
–
12.5
–
44.8
73.0
N1.6
–
1.0
–
4.5
30.8
N1.7
–
0.5
–
27.5
60.5
Object
B=5
B=10
B=15
B=25
B=50
Appendix
Table 2: Best checkpoint success (%) at every fine-tuning budget B (demonstrations per suite). N1.5–N1.7 denote GR00T models. “–” marks budgets not run.
Reference model
Compared model
Suite
Model
B
Best (%)
Model
B′
Best (%)
B/B′
Spatial
π0
25
35.8
π0.5
10
49.0
2.5 ×
Spatial
π0
50
63.0
π0.5
25
75.0
2.0 ×
Spatial
N1.5
25
44.8
π0.5
10
49.0
2.5 ×
Spatial
N1.5
50
73.0
π0.5
25
75.0
2.0 ×
Spatial
N1.6
25
4.5
π0.5
5
29.8
5.0 ×
Appendix
Table 3: All pairwise data-efficiency multipliers. For a reference model at budget B , B′ is the smallest tested budget at which the compared model reaches at least the reference’s best success. Pairs where the compared model needs the full budget B are omitted.
Suite
Model
70%
80%
90%
Best (%)
Spatial
π0
2.0k
2.0k
3.3k
97.7
π0.5
0.2k
0.4k
0.6k
97.2
N1
3.0k
6.2k
–
84.2
N1.5
1.4k
2.0k
2.6k
98.8
N1.6
2.6k
3.8k
4.6k
97.2
N1.7
3.0k
3.0k
4.2k
97.2
Appendix
Table 4: Full-data fine-tuning: training steps (effective batch size 48) until success is at least 70/80/90% at two consecutive evaluations, and best success. N1–N1.7 denote GR00T models. “–” means the threshold was not reached within the run. To save compute, some runs are evaluated less frequently (every 200–1,100 steps), which bounds the precision of these numbers.
Suite
Model
70%
80%
90%
Best (%)
Spatial
π0
–
–
–
74.8
π0.5
3.6k
–
–
74.0
N1
4.2k
–
–
76.5
N1.5
1.2k
1.2k
3.6k
97.0
N1.6
3.0k
3.0k
7.2k
92.8
N1.7
2.4k
3.0k
5.4k
96.8
Appendix
Table 5: Frozen-VLM fine-tuning on the full datasets: training steps until success is at least 70/80/90% at two consecutive evaluations, and best success. Same conventions as Table 4 .
Fine-tuning a Vision-Language-Action (VLA) model for a new deployment environment is expensive, yet most methods apply uniform-capacity adapters to every network region as if every region requires equal adjustment. This paper tests that assumption on five architecturally diverse VLAs (OpenVLA-OFT, π0, SmolVLA, DTP, Octo; 93M-7B parameters). Measuring per-region adaptation cost as normalized parameter displacement under region-isolated fine-tuning reveals an adaptation spectrum in which appearance shifts concentrate cost in the vision encoder, instruction shifts in the language backbone, and novel-object shifts in the vision encoder together with the action head, across all five architectures. To exploit this structure, we introduce a pipeline that observes, diagnoses, allocates, and adapts. From ten unlabeled target observations and without fine-tuning, the diagnostic estimates per-region cost by combining reference-free gradient and Monte Carlo Dropout signals with a Centered Kernel Alignment score against a cached source reference; the allocator converts the estimates into variable-rank LoRA adapters under a parameter budget and freezes well-calibrated regions; and standard LoRA fine-tuning trains the resulting adapters. The diagnostic ranks regions within each deployment at a median Spearman of 0.91, and the allocation matches or exceeds uniform LoRA at every budget we tested on LIBERO and CALVIN. On a physical xArm-7, the pipeline matches full fine-tuning under an instruction-wording shift with 0.04% of its trainable parameters, and on five held-out scenes evaluated without retraining it leads every baseline, with 11-23 successes of 30 rollouts against 8-18 for the strongest parameter-efficient baseline at equal or larger budgets and 2-11 for full fine-tuning. These results suggest that adaptation cost in VLAs is structured enough to measure before fine-tuning begins.
Shahram Najam Syed, Arthur Jakobsson, Prayuj Sachdev +1
Robotics Institute, Carnegie Mellon University, Pittsburgh, USA · Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, USA
Adapting vision-language-action (VLA) models to deployment-time distribution shifts is important for reliable robotic operation, but conventional first-order adaptation can exceed the memory budget of inference-oriented deployment platforms. Zeroth-order (ZO) optimization offers a forward-only alternative with inference-level memory, but accurate gradient estimation requires many perturbation queries, making naive ZO prohibitively slow for large VLA models. We present VLA-ZO, a framework for fast ZO adaptation that exploits the structure of VLA computation. By confining adaptation to the action side, VLA-ZO keeps the expensive vision-language prefix frozen and reuses its conditioning states across perturbation queries and optimizer steps, while schedule-aware prefetching hides state-transfer overhead. On LIBERO camera-viewpoint shifts, VLA-ZO reduces end-to-end adaptation time by 25.59× at q=16 and 32.54× at q=64 relative to baseline ZO, while improving average task success from 48.27% without adaptation to 58.17% and 63.58%, respectively. These results show that making ZO faster can make larger query budgets practical, providing a promising path toward resource-efficient VLA adaptation on deployment platforms.
Vision-language-action (VLA) models now exceed 95% success on the LIBERO simulation benchmark, yet it remains unclear how much of this performance requires generalization beyond the training demonstrations. In this work, we show that retrieving nearby demonstrations and reusing their action chunks achieves a success rate of 91.7 % on LIBERO without policy training. Under LIBERO-Plus perturbations, retrieval shows the smallest drop of all evaluated policies. Across nine publicly available VLAs, fine-tuning mostly makes the vision-language backbone a better retrieval key, and the actions of the VLAs remain close to the demonstrations. In addition, we introduce Local Action Consistency (LAC), an offline metric that measures whether nearby observations in representation space are also associated with similar future actions. We show that LAC tracks the success rate of retrieval and can guide the training of a retrieval key. Our evaluation and source code are available at https://github.com/sereact/lerobot_knn.