How Much of Your VLA Is kNN
Organizations: Sereact GmbH, Stuttgart, Germany · Karlsruher Institut für Technologie, Karlsruhe, Germany
Abstract
Vision-language-action (VLA) models now exceed 95% success on the LIBERO simulation benchmark, yet it remains unclear how much of this performance requires generalization beyond the training demonstrations. In this work, we show that retrieving nearby demonstrations and reusing their action chunks achieves a success rate of 91.7 % on LIBERO without policy training. Under LIBERO-Plus perturbations, retrieval shows the smallest drop of all evaluated policies. Across nine publicly available VLAs, fine-tuning mostly makes the vision-language backbone a better retrieval key, and the actions of the VLAs remain close to the demonstrations. In addition, we introduce Local Action Consistency (LAC), an offline metric that measures whether nearby observations in representation space are also associated with similar future actions. We show that LAC tracks the success rate of retrieval and can guide the training of a retrieval key. Our evaluation and source code are available at https://github.com/sereact/lerobot_knn.
Figures & tables
| LIBERO | LIBERO-Plus | ||||||||||
| avg | Spat | Obj | Goal | Long | avg | Spat | Obj | Goal | Long | ||
| [ 3 ] | 97.7 | 97.0 | 99.8 | 97.8 | 96.2 | 85.5 | 90.5 | 83.8 | 82.9 | 84.8 | -12.2 |
| MolmoAct2 [ 8 ] | 97.5 | 97.4 | 99.4 | 97.6 | 95.6 | 82.6 | 89.5 | 81.9 | 76.2 | 82.9 | -14.9 |
| GR00T N1.7 [ 21 ] | 94.9 | 95.8 | 98.2 | 94.6 | 90.8 | 80.2 | 89.5 | 74.3 | 79.0 | 78.1 | -14.6 |
| VLA-JEPA [ 30 ] | 97.2 | 95.4 | 100.0 | 97.4 | 95.8 | 74.0 | 81.0 | 70.5 | 77.1 | 67.6 | -23.1 |
| Evo-1 [ 17 ] | 95.4 | 92.0 | 98.6 | 97.4 | 93.6 | 71.4 | 74.3 | 73.3 | 69.5 | 68.6 | -24.0 |
| LIBERO | LIBERO-Plus | |||||
|---|---|---|---|---|---|---|
| kNN before | kNN after | VLA | kNN before | kNN after | VLA | |
| 85.4 | 81.8 | 97.7 | 60.5 | 65.7 | 85.5 | |
| MolmoAct2 | 78.6 | 91.4 | 97.5 | 52.6 | 81.0 | 82.6 |
| GR00T N1.7* | 87.6 | 87.6 | 94.9 | 66.4 | 66.4 | 80.2 |
| VLA-JEPA | 84.0 | 83.0 | 97.2 | 49.3 | 65.7 | 74.0 |
| Evo-1 | 62.1 | 87.1 | 95.4 | 35.5 | 72.6 | 71.4 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Ablation | avg | Spatial | Object | Goal | Long | |
|---|---|---|---|---|---|---|
| None (multimodal variant) | 91.7 | 88.0 | 99.8 | 97.6 | 81.4 | |
| Prompt ignored | 64.6 | 77.6 | 99.6 | 8.8 | 72.4 | 27.1 |
| Raw 16 16 pixels | 57.2 | 79.0 | 0.0 | 91.0 | 58.6 | 34.5 |
| 10% of demos | 77.4 | 83.8 | 87.0 | 90.2 | 48.6 | 14.3 |
| 25% of demos | 82.6 | 83.6 | 97.8 | 92.6 | 56.4 | 9.1 |
| 50% of demos | 88.5 | 86.4 | 99.6 | 95.4 | 72.6 | 3.2 |
| Model | Repository | Revision |
|---|---|---|
| lerobot/pi05_libero_finetuned_v044 | 8e174154 | |
| MolmoAct2 | allenai/MolmoAct2-LIBERO-LeRobot | f0c5a956 |
| GR00T N1.7 | nvidia/gr00t17-lerobot-libero_spatial-640 | 32a6ec78 |
| nvidia/gr00t17-lerobot-libero_object-640 | 1499db35 | |
| nvidia/gr00t17-lerobot-libero_goal-640 | 436c57c0 | |
| nvidia/gr00t17-lerobot-libero_10-640 | 5ee08ab0 |
Vision-Language-Action Models
Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.
See all 1,239 papers