cs.RONov 9, 2026

GAP: Generalized Adaptation Prior for Dexterous Piano Playing

Authors: Jinyi Chen, Hongyi Li, Lei Jiang, Xianwei Liu, Yanhong Liang, Yulin Chen, Yucheng Tao, Yongbin Jin, 

Organizations: Center for X-Mechanics, Zhejiang University, Hangzhou, China. · Mirrorme Technology Co., Ltd., Shanghai, China. · ZJU-Hangzhou Global Scientific and Technology Innovation Center, Hangzhou, China.

Abstract

Dexterous hands enable versatile manipulation, but policies trained in simulation often degrade on hardware owing to non-idealities, and the high-dimensional action space makes real-world adaptation costly. Piano playing, which requires precise and coordinated multifinger motion, offers a representative testbed. We propose GAP (Generalized Adaptation Prior for Dexterous Piano Playing), a sample-efficient adaptation framework that learns how to adapt, rather than how to act, during simulation pretraining and transfers this adaptation prior to hardware. Between repeated executions, GAP uses feedback to adjust the actions of the next one: a gradient guidance network learns from paired executions how adjusting actions affects playing performance, and its gradients guide these adjustments. In simulation, GAP yields approximately 2.4 times the F1 scores improvement of the existing fast-adaptation baseline for dexterous piano playing under the same interaction budget. The learned prior also generalizes to musical pieces held out from pretraining. On a physical dexterous hand, the same simulation-pretrained prior improves F1 scores by 3.6%-23% across three pieces of varying difficulty after approximately 7 min of adaptation each. These results suggest a practical route to reduce the real-world interaction needed to deploy dexterous hands.

Figures & tables

Explore similar work

CardsList
  1. HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot Policies

    Mar 12, 2026Amber Xie, Haozhi Qi, Dorsa SadighRobotic ManipulationBimanual Robotic Manipulation

  2. Enforcing Human-like Kinematics in Dexterous Piano Playing via Adversarial Posture Regularization

    Jun 22, 2026Bin Qiu, Yanming Shao, Guanyu Cai +1Adversarial TrainingDexterous Manipulation

  3. Learning Adaptable Dexterous Skills via Hybrid Dexterous Mimicking

    Oct 20, 2026Yuxuan Kuang, Jiashun Wang, Yixuan Sun +5Robot Skill LearningRobotic Manipulation