Vision-language-action (VLA) models now exceed 95% success on the LIBERO simulation benchmark, yet it remains unclear how much of this performance requires generalization beyond the training demonstrations. In this work, we show that retrieving nearby demonstrations and reusing their action chunks achieves a success rate of 91.7 % on LIBERO without policy training. Under LIBERO-Plus perturbations, retrieval shows the smallest drop of all evaluated policies. Across nine publicly available VLAs, fine-tuning mostly makes the vision-language backbone a better retrieval key, and the actions of the VLAs remain close to the demonstrations. In addition, we introduce Local Action Consistency (LAC), an offline metric that measures whether nearby observations in representation space are also associated with similar future actions. We show that LAC tracks the success rate of retrieval and can guide the training of a retrieval key. Our evaluation and source code are available at https://github.com/sereact/lerobot_knn.
Figures & tables
Figure 1: Overview of our evaluation pipeline. Both are evaluated with the same rollout protocol. (a) a kNN retrieves neighboring demonstration embeddings and combines their associated action chunks; (b) the VLA predicts an action chunk directly through its backbone and action head.
Figure 2: LAC ( LAC ) is the ratio between the average action distance of the retrieved k nearest neighbors and the average action distance over the same-task set.
LIBERO
LIBERO-Plus
avg
Spat
Obj
Goal
Long
avg
Spat
Obj
Goal
Long
Δ
π0.5 [ 3 ]
97.7
97.0
99.8
97.8
96.2
85.5
90.5
83.8
82.9
84.8
-12.2
MolmoAct2 [ 8 ]
97.5
97.4
99.4
97.6
95.6
82.6
89.5
81.9
76.2
82.9
-14.9
GR00T N1.7 [ 21 ]
94.9
95.8
98.2
94.6
90.8
80.2
89.5
74.3
79.0
78.1
-14.6
VLA-JEPA [ 30 ]
97.2
95.4
100.0
97.4
95.8
74.0
81.0
70.5
77.1
67.6
-23.1
Evo-1 [ 17 ]
95.4
92.0
98.6
97.4
93.6
71.4
74.3
73.3
69.5
68.6
-24.0
Table 1: Success rate ( % ) on LIBERO (official protocol, 2000 episodes) and on the same 420 LIBERO-Plus variants. VLA are ordered by average on LIBERO-Plus. Δ is the difference between the average success rates on LIBERO-Plus and LIBERO, computed before rounding. The kNN rows are training-free, except for the LAC key.
LIBERO
LIBERO-Plus
kNN before
kNN after
VLA
kNN before
kNN after
VLA
π0.5
85.4
81.8
97.7
60.5
65.7
85.5
MolmoAct2
78.6
91.4
97.5
52.6
81.0
82.6
GR00T N1.7*
87.6
87.6
94.9
66.4
66.4
80.2
VLA-JEPA
84.0
83.0
97.2
49.3
65.7
74.0
Evo-1
62.1
87.1
95.4
35.5
72.6
71.4
Table 2: Lookup success using each VLA ’s VLM backbone before and after LIBERO fine-tuning, compared with the full VLA . Bold indicates cases where retrieval on the fine-tuned backbone outperforms the corresponding VLA. ∗ GR00T freezes its VLM , so before = after.
Figure 3: t-SNE maps of retrieval keys for the same 4,150 demonstration frames from LIBERO . Top: before LIBERO fine-tuning (ImageNet ResNet-18 for the first column). Bottom: after fine-tuning ( LAC -trained key for the first column). Colors indicate normalized episode progress. Below each map: median progress gap to the nearest frame from another demonstration of the same task.
Figure 4: Comparison of the action chunks of each VLA , in its LIBERO rollouts, with the training demonstrations. ( 4(a) ) R2 between the VLA chunks and the prediction of the ResNet kNN variant; higher indicates closer agreement. ( 4(b) ) Median squared error to the nearest training chunk of the same task, relative to held-out ground-truth chunks; lower indicates greater proximity to the training data. Dashed lines: held-out ground-truth chunks. † Evaluated on LIBERO-Spatial only. X-VLA is excluded because it predicts absolute poses.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Hyperparameter search for the kNN policy with Optuna, over 24 completed trials of 400 episodes. ( 5(a) ) Hyperparameter importance estimated with fANOVA [ 14 ] . ( 5(b) ) Success rate as a function of the number of neighbors k and of the uniqueness constraint of Equation 3 .
Ablation
avg
Spatial
Object
Goal
Long
Δ
None (multimodal variant)
91.7
88.0
99.8
97.6
81.4
Prompt ignored
64.6
77.6
99.6
8.8
72.4
− 27.1
Raw 16 × 16 pixels
57.2
79.0
0.0
91.0
58.6
− 34.5
10% of demos
77.4
83.8
87.0
90.2
48.6
− 14.3
25% of demos
82.6
83.6
97.8
92.6
56.4
− 9.1
50% of demos
88.5
86.4
99.6
95.4
72.6
− 3.2
Appendix
Table 3: Ablations of the multimodal kNN variant under the standard LIBERO evaluation protocol (50 rollouts per task, 2000 episodes in total). Δ denotes the change in average success relative to the multimodal variant. Raw-pixel retrieval fails on LIBERO-Object, where the ten tasks share a scene containing several small objects.
Model
Repository
Revision
π0.5
lerobot/pi05_libero_finetuned_v044
8e174154
MolmoAct2
allenai/MolmoAct2-LIBERO-LeRobot
f0c5a956
GR00T N1.7
nvidia/gr00t17-lerobot-libero_spatial-640
32a6ec78
nvidia/gr00t17-lerobot-libero_object-640
1499db35
nvidia/gr00t17-lerobot-libero_goal-640
436c57c0
nvidia/gr00t17-lerobot-libero_10-640
5ee08ab0
Appendix
Table 4: Evaluated VLA checkpoints, hosted on the Hugging Face Hub. Revisions are abbreviated commit hashes. GR00T N1.7 uses one checkpoint per LIBERO suite.
Figure 6: Success of every policy on each of the 40 tasks, tasks sorted from easiest to hardest (mean over policies).
Figure 7: Success rate \left(\mathrm{%}\right) of each policy on LIBERO-Plus tasks, per type of perturbation.
Figure 8: t-SNE visualization of each retrieval space, colored by progress through the episode. Below each map: Task, the fraction of nearest neighbors from another demonstration that belong to the same task; Goal, the same fraction within LIBERO-Goal; Gap, the median progress difference to the nearest frame from another demonstration of the same task. GR00T freezes its VLM , so its representation is unchanged by policy fine-tuning.
Figure 9: First two principal components of each retrieval key, colored by LIBERO suite. Below each map: Suite, the fraction of the variance of the two components explained by the suite; PC1+2, the fraction of the total variance captured by the two components.
Figure 10: Offline LAC versus closed-loop retrieval success. Each point is a visual representation used as the key of the ResNet kNN variant. The x-axis reports LAC@8 on held-out demonstrations, and the y-axis the LIBERO success rate. The dashed line is fitted on all keys except the fine-tuned VLM backbones, which were trained on the held-out demonstrations.
Vision-language-action (VLA) models have shown strong potential as generalist robot policies, but adapting them to unseen tasks often requires costly parameter updates. Recent work such as RICL introduces in-context adaptability by retrieving expert demonstrations based on the current VLA observation and providing them as additional context at test time. The effectiveness of this adaptation therefore depends critically on the retrieval mechanism. In this work, we systematically study how different retrieval methods affect both retrieval quality and task performance within the RICL framework. Specifically, we compare four different methods: image-based retrieval, retrieval augmented with VLA's state, retrieval using features from the VLA backbone, and random retrieval. Our experiments yield three main findings. First, no retrieval method consistently dominates the others in task success, while surprisingly, random retrieval achieves a non-trivial success rate. Second, standard retrieval-quality diagnostics do not reliably reflect downstream VLA performance. Third, demonstrations from different but related tasks can provide useful transferable information. Together, these results provide an initial step toward understanding how retrieval mechanisms shape the in-context learning capability of VLA models and their downstream task performance, while highlighting the need for more careful design and evaluation of retrieval mechanisms for reliable test-time adaptation.
Zixuan Liu, Joris Köster, Zizhan Zheng +1
Department of Computer Science, Tulane University · Department of Computer Science, Aalto University · Department of Industrial Engineering and Management, Aalto University
Vision-Language-Action (VLA) models translate natural-language commands into robot action sequences, but leading systems on the LIBERO-Plus robustness benchmark use three- to seven-billion-parameter backbones whose memory demands can exceed embedded robotic budgets. We present CoTinyVLA, a 0.9B-parameter action model on a Qwen3.5-0.8B backbone that obtains that robustness by structuring supervision instead of enlarging the model. Three components target different axes of the problem: dual-view temporal input of 16 history frames per step with textual camera and time markers; hierarchical chain-of-thought (CoT) distillation from a 35B teacher into an episode-level Plan and a chunk-level Think span over task phase, gripper state and next subaction; and paraphrase augmentation expanding 40 base commands into 800 variants. On LIBERO-Plus, spanning 10,030 perturbed tasks across seven perturbation dimensions, CoTinyVLA reaches 90.8% on Spatial, 87.3% on Object, 86.6% on Goal and 80.7% on Long, leading the strongest 7B baseline on all four suites by 4.7, 2.8, 15.9 and 3.0 points, with every margin interval excluding zero. The gains concentrate on the hardest axes of the benchmark: across the eleven published baselines none exceeds 53.2% on Robot Initial States in any suite, whereas CoTinyVLA reaches 73.6% on Goal against 39.9% for the strongest baseline. Ablations show the three components to be separable by perturbation axis, and at a matched image budget how frames are divided between the two cameras and across time accounts for 8.6 points on its own. Closed-loop inference peaks at 2.25 GiB of allocated GPU memory, and paired interventions show the episode Plan to be load-bearing: replacing it with an empty or contradictory span costs 40 to 45 points of success. Structured supervision thus lets a 0.9B backbone exceed all of them. Code: https://github.com/BrainJellyPie/CoTinyVLA
Current Vision-Language-Action (VLA) models typically treat the deepest representation of a vision-language backbone as universally optimal for action prediction. However, robotic manipulation is composed of many frequent closed-loop spatial adjustments, for which excessive abstraction may waste computation and weaken low-level geometric cues essential for precise control. Existing early-exit strategies attempt to reduce computation by stopping at predefined layers or applying heuristic rules such as action consistency, but they do not directly answer when a representation is actually sufficient for action. In this paper, we present LoopVLA, a recurrent VLA architecture that jointly learns representation refinement, action prediction, and sufficiency estimation. LoopVLA iteratively applies a shared Transformer block to refine multimodal tokens, and at each iteration produces both a candidate action and a sufficiency score that estimates whether further refinement is necessary. By sharing parameters across iterations, LoopVLA decouples refinement from absolute layer indices and grounds sufficiency estimation in the evolving representation itself. Since sufficiency has no direct supervision, we introduce a self-supervised distribution alignment objective, where intermediate confidence scores are trained to match the relative action quality across refinement steps, thereby linking sufficiency learning to policy optimization signals. Experiments on LIBERO, LIBERO-Plus, and VLA-Arena show that LoopVLA pushes the efficiency-performance frontier of VLA policies, reducing parameters by 45% and improving inference throughput by up to 1.7 times while matching or outperforming strong baselines in task success.
Boyang Shen, Kaixiang Yang, Hao Wang +4
1Huazhong University of Science and Technology · 2Wuhan United Imaging Surgical Co.,Ltd. (UIS)