cs.RONov 9, 2026

VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess

Authors: Matěj Kripner, Milan Straka

Organizations: Charles University, Faculty of Mathematics and Physics Prague, Czech Republic

Abstract

Vision-language-action (VLA) models are trained as general-purpose foundation models for robot learning, and require fine-tuning for each downstream task and robot. This raises practical questions such as how much data and training updates are needed, how checkpoints should be selected, and whether simulation can be leveraged to alleviate the cost of real data collection. We study these questions on a task that requires both precise manipulation and spatial understanding of the scene. In this task, an SO-101 arm moves a chess king between two squares that are given only by their names, such as e4 and a5. We fine-tune π0.5π_{0.5} with various dataset sizes and update budgets, using teleoperated demonstrations as well as simulated demonstrations from a randomized scripted expert that imitates the variability of human demonstrations. In simulation, the success rate scales with both data and updates. Importantly, we find that a short learning-rate cool-down adds up to 31 points and is necessary for comparing checkpoints, and that validation loss and action error are unreliable for model selection. On the real arm, we fine-tune on up to 256 teleoperated demonstrations and evaluate ten policies in 25 trials each. Partial results indicate that pretraining on the simulated demonstrations helps substantially, most clearly when real data is scarce. We also present a negative result, in which training on the policy's own rollouts with filtered behavior cloning, RECAP-style advantage conditioning, corrections from an automatic coach, or residual reinforcement learning did not yield a measurable improvement over plain behavior cloning in our setting. We release the simulator, the expert and the real demonstrations.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SimVLA: A Simple VLA Baseline for Robotic Manipulation

    Feb 20, 2026Yuankai Luo, Woping Chen, Tong Liang +2Vision-Language-Action ModelsVLMs for Robotics

  2. Rethinking the Practicality of Vision-language-action Model: A Comprehensive Benchmark and An Improved Baseline

    Feb 26, 2026Wenxuan Song, Jiayi Chen, Xiaoquan Sun +12Vision-Language-Action ModelsEfficient VLAs

  3. Rethinking VLM Representation for VLA Initialization

    May 25, 2026Weifeng Lin, Siyuan Huang, Hao Li +5VLM AdaptationLow-Rank Adaptation