cs.RONov 18, 2026

Finetuning VLAs for Dexterous Manipulation: The Right Checkpoint Is All You Need

Authors: Artur Kuramshin, Charlotte Cloutier, Michael Przystupa, Kirsty Ellis, Mohammad S. Akhter, Glen Berseth

Organizations: Mila — Quebec AI Institute, Montréal, Canada · Université de Montréal, Canada · Renesas Electronics Canada Ltd.

Abstract

Vision-Language-Action models have demonstrated strong manipulation capabilities. A limitation of VLA models is that most manipulation datasets focus on parallel-jaw gripper systems, making transfer to multi-finger hands challenging, particularly as teleoperation datasets are harder to collect with multi-fingered robots. In this work, we study how to best adapt VLAs to dexterous hands by pairing supervised fine-tuning (SFT) with diffusion-steering reinforcement learning (DSRL). Using π0.5π_{0.5} and a LEAP hand, we evaluate five fine-tuning recipes across two simulation tasks using only a handful of demonstrations. Our empirical analysis yields three key insights: (1) SFT success is poorly correlated with post-DSRL success, meaning the best SFT checkpoint is rarely the most effective to steer; (2) freezing the vision-language model (VLM) and tuning only the action expert produces the most steerable policy for both training and out-of-distribution placements; and (3) low-rank adaptation (LoRA) is the least steerable approach. Consequently, we recommend an adaptation recipe that freezes the VLM, fully trains the action expert, and eschews SFT rollout success as a checkpoint selection metric.

Explore similar work

CardsList
  1. Towards High-DoF Dexterous Manipulation through VLA Post-Training

    Sep 17, 2026Junlei Zhu, Shenzhe Yao, Chaogui Huang +6Latent Action LearningVLA Model Post-Training

  2. Bridging the Morphology Gap: Adapting VLA Models to Dexterous Manipulation via Intent-Conditioned Fine-Tuning

    Jun 10, 2026Chuanke Pang, Junyi Huang, Jianye Fang +4Contact-Rich Robotic ManipulationVLM Adaptation