OmniPiano: Diverse Dexterous Piano-Playing Challenges for Standard, Robust, Safe, and Multi-Agent RL

Fan Xu; Fan Yang; Yifan Xu; Ruiyuan Zhang; Yimou Wu; Nuo Yi; Muning Wen; Guang Chen

cs.AIOct 30, 2026

OmniPiano: Diverse Dexterous Piano-Playing Challenges for Standard, Robust, Safe, and Multi-Agent RL

Authors: Fan Xu, Fan Yang, Yifan Xu, Ruiyuan Zhang, Yimou Wu, Nuo Yi, Muning Wen, Guang Chen,

Organizations: University of California, Berkeley · Shanghai Jiao Tong University · The Chinese University of Hong Kong, Shenzhen · Japan Advanced Institute of Science and Technology ·

Abstract

Dexterous robotic manipulation remains a major challenge in robotics, exemplified by piano playing, which requires coordinated control of fingers and provides a demanding testbed for reinforcement learning (RL). Although previous work established a benchmark for robotic piano playing, it is restricted to two hands and does not support unified evaluation. To address this gap, we introduce OmniPiano, a benchmark that provides one to up to five Shadow Hands and supports standard, robust, safe, and multi-agent RL within a shared piano-playing task family. OmniPiano is developed for scalable and diverse piano-playing tasks with configurable perturbations, explicit safety constraints, and decentralized cooperation settings. Particularly, to facilitate usability and extendability, OmniPiano adopts a highly modular design and provides comprehensive tasks to support RL study of task performance, robustness, safety, and cooperation in dexterous control. With at least 912 task settings and 36 baseline algorithms, OmniPiano further incorporates LLM-based agents to broaden the evaluation scope and reveal new insights. Extensive evaluations show that state-of-the-art RL algorithms and frontier LLM-based agents still struggle on challenging piano-playing tasks, even under standard RL settings, highlighting substantial room for improvement in dexterous control. The code, dataset, tutorial, and demo are available at https://sail-research-lab.github.io/omnipiano/.

Figures & tables

Figure 1: The overview of OmniPiano . A unified benchmark for learning dexterous piano playing.

Feature	RoboPianist	Robust-Gym.	Safety-Gym.	OmniPiano
Action dimension	45	1–30	2–17	23–111
Dexterous piano playing	✓	✗	✗	✓
Robust RL track	✗	✓	✗	✓
Safe RL track	✗	✓	✓	✓
Multi-agent RL track	✗	✓	✓	✓
Unified task setting	✗	✗	✗	✓

Table 1: The comparison of OmniPiano with related reinforcement-learning benchmarks.

Figure 2: The overview of the Robust RL task design. Robustness perturbations are introduced through four independent disruptors acting on the agent–environment interaction loop.

Figure 3: The overview of the safety task design in OmniPiano . Four safety semantics, Joint Range, Actuator Power, Injured Finger, and Hand Collision, define the constrained behaviors. Event, Fraction, and Excess provide alternative measures of constraint violations.

Figure 4: The overview of the multi-agent RL task design. The configuration shown is a typical example. In practice, users can freely configure the MIDI score, number of hands and agents, hand-to-agent assignments, and agent-specific observation and action ranges and overlaps.

Figure 5: Average evaluation reward ( $\bm{\uparrow}$ : top row, higher is better) and mean token usage ( $\bm{\downarrow}$ : bottom row, lower is better) per optimization attempt for LLMs across ten keyframe-optimization iterations and RL methods over 5M training steps in the two-hand setting on three piano pieces.

Figure 6: F1-score comparison of PPO, SAC, DeepSeek-V4.1-Flash, and GPT-6 Astra across eight hand settings on three piano pieces. The details of F1 metric can be found in Appendix D.1.2 .

Figure 7: Periodic evaluation reward of PPO across six robustness channels under Gaussian, uniform, and shift perturbations, with the clean environment shown as a reference.

Panel of Figure 8 — Figure 8: Effect of hand count on safety RL: training reward and safety cost for 2 to 4 hands. Solid lines denote CUP / PPO-Lag; dashed lines denote unconstrained PPO. The dotted line marks the cost budget. In this task, four hands policies achieve lower rewards and show greater difficulty reducing safety costs than two or three hands policies.

Figure 9: Task-wise training performance of representative MARL algorithms across the five settings. Compared with Base, Scalability and Observability variant generally improve performance, while Heterogeneity and especially Coupling variant make policy learning more difficult. Among the variants, Coupling most clearly differentiates the algorithms.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Song	Steps	NPS	F1 Score
Gymnopédie No. 1	657	1.77	0.88
Twinkle Twinkle (Rousseau)	451	1.51	0.86
Clair de Lune	597	2.65	0.63
French Suite No. 1, Allemande	517	10.89	0.59
French Suite No. 5, Sarabande	962	3.56	0.59
Piano Sonata No. 23, 2nd mov.	313	12.80	0.49

Appendix

Table 2: Task statistics and performance across piano pieces.

Figure 10: Periodic evaluation reward during training for PPO and SAC under Restricted and Unrestricted configurations across the three songs.

Figure 11: F1 of LLMs over ten keyframe-optimization iterations and RL method over 5M steps for the two-hand setting on three piano pieces.

Panel of Figure 12 — Figure 12: Cross-algorithm robustness comparison on Clair de Lune .

Figure 13: Composability of Gaussian perturbations on Für Elise with three hands.

Panel of Figure 14 — Figure 14: Additional hand count results for safe RL. Solid lines denote CPO / OnCRPO; dashed lines denote PPO. Training reward and cost show means with $\pm 1$ standard deviation across three seeds. The dotted line marks the cost budget.

Panel of Figure 15 — Figure 15: Overall performance on 2 hand safety tasks: evaluation reward and cost. Shading indicates $\pm 1$ standard deviation across available seeds; dotted lines mark task-specific cost budgets.

Panel of Figure 16 — Figure 16: Overall performance on 3 hand safety tasks: evaluation reward and cost. Shading indicates $\pm 1$ standard deviation across available seeds; dotted lines mark task-specific cost budgets.

Panel of Figure 17 — Figure 17: Overall performance on 4 hand safety tasks: evaluation reward and cost. Shading indicates $\pm 1$ standard deviation across available seeds; dotted lines mark task-specific cost budgets.

Panel of Figure 18 — Figure 18: Key-press F1 on the 2-hand safety tasks. Curves show means with $\pm 1$ standard deviation across available seeds.

Panel of Figure 19 — Figure 19: Key-press F1 on the 3-hand safety tasks. Curves show means with $\pm 1$ standard deviation across available seeds.

Panel of Figure 20 — Figure 20: Key-press F1 on the 4-hand safety tasks. Curves show means with $\pm 1$ standard deviation across available seeds.

Panel of Figure 21 — Figure 21: Sensitivity to the episodic cost limit. Evaluation reward and cost are shown for five budgets; shading indicates $\pm 1$ standard deviation across three seeds.

Panel of Figure 22 — Figure 22: Key-press F1 under different cost limits, shown as means with $\pm 1$ standard deviation across three seeds.

Setting	Agent	Hands	Action	Observation	Visible hands
Base	A1	0,1	1–44	1–49	2
Base	A2	2,3	45–88	40–88	1
Observability	A1	0,1	1–44	1–70	2
Observability	A2	2,3	45–88	19–88	1
Coupling	A1	0,1	1–70	1–49	2
Coupling	A2	2,3	19–88	40–88	1

Appendix

Table 3: MARL test settings.

Figure 23: Detailed MARL results across the five OmniPiano cooperation settings. The first row compares task-wise team return for each MARL algorithm, while the second and third rows compare team return and musical F1 across algorithms for each task.

Figure 24: IPPO performance in the Base MARL setting with and without Gaussian observation noise ( $\sigma=0.05$ ). Reward remains similar, while musical F1 degrades substantially under observation noise.

Figure 25: Rollout reward of llms over ten keyframe-optimization iterations and RL method over 5M steps for the two-hand setting on three piano pieces.

Figure 26: token consumption over ten keyframe-optimization iterations for the two-hand setting on three piano pieces.

Target	Perturbed quantity	Family	Frequency	Low / Medium / High
Action	Canonical actuator target	G/U/S	Step	$0.05/0.10/0.15$
Observation	Proprioception and piano state	G/U/S	Step	$0.05/0.10/0.15$
Reward	Scalar reward received by the learner	G/U/S	Step	$0.10/0.30/0.50$
Gravity	Vertical acceleration $g_{z}$	G/U/S	Episode/step	$0.50/1.00/1.50\,\mathrm{m\,s^{-2}}$
Contact-friction	Fingertip–key sliding coefficient	G/U/S	Episode/step	$0.05/0.10/0.15$
Hand-pose	Per-hand initial $(y,z)$ position	G/U/S	Episode	$(10,5)/(25,10)/(50,20)\,\mathrm{mm}$

Appendix

Table 4: OmniPiano robustness axes and canonical low/medium/high levels. Levels are standard deviations for Gaussian noise, half-widths for symmetric uniform noise, and magnitudes for constant shifts. Hand-pose entries report

(y,z)

offsets.

Hands	Song	Cost formulation	Constraint reference	$T_{\mathrm{nom}}$	Budget $d$	Runs
2	Für Elise	Joint-range fraction	Central 50%	399	19.95	15
2	ClairDeLune	Joint-range excess	Central 50%	588	11.76	15
3	PicturesGreatKiev	Total-power excess	$P_{\max}=12$	720	14.40	15
3	PolonaiseOp40No1	Total-power excess	$P_{\max}=12$	563	11.26	15
4	PicturesGreatKiev	Joint-range excess	Central 50%	720	14.40	15
4	PicturesGreatKiev	Total-power excess	$P_{\max}=16$	720	14.40	15

Appendix

Table 5: Main safety experiments settings. Each task uses PPO, PPO-Lag, OnCRPO, CPO, and CUP, with three seeds and 5M training steps per run.

Track	Baselines
Standard RL	A2C , ARS, CrossQ , DDPG , DroQ , PPO , SAC , TD3 , TQC , TRPO
Robust RL	A2P-SAC , EPPO , OMPO , SCPO
Safe RL	PPO-Lag , TRPO-Lag , RCPO , PDO , FOCOPS , CPO , PCPO , OnCRPO , IPO , P3O , CUP , DDPG-Lag , TD3-Lag , SAC-Lag
Multi-Agent RL	IPPO , MAPPO , HAPPO , MAT , A2PO , HATD3 , FACMAC , MACSAC

Appendix

Table 6: Baseline algorithms supported by OmniPiano across the four evaluation tracks. Italic denotes on-policy methods and underlined denotes off-policy methods.

Robot Manipulation Benchmarks

Robot manipulation benchmarks measure how reliably robots perceive, plan, and execute physical tasks across objects, settings, and embodiments.

See all 346 papers

HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot Policies
Mar 12, 2026Amber Xie, Haozhi Qi, Dorsa SadighRobotic Manipulation Bimanual Robotic Manipulation
Stanford University · Amazon FAR (Frontier AI & Robotics)
CANTABILE: Learning Expressive Dynamics for Robotic Piano Performance
Sep 16, 2026Woosik Kim, Wonhyeok Choi, Sunghoon ImBimanual Robotic Manipulation Dynamic Robotic Manipulation
KAIST, Daejeon, Republic of Korea · DGIST, Daegu, Republic of Korea
Enforcing Human-like Kinematics in Dexterous Piano Playing via Adversarial Posture Regularization
Jun 22, 2026Bin Qiu, Yanming Shao, Guanyu Cai +1Adversarial Training Dexterous Manipulation
Shanghai Jiao Tong University · The University of Hong Kong · Shanghai AI Laboratory

Mar 12, 2026J/K move · Enter open · S save

HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot Policies

Amber Xie, Haozhi Qi, Dorsa Sadigh

Stanford University · Amazon FAR (Frontier AI & Robotics)

Robotic Manipulation Bimanual Robotic Manipulation Sim-to-Real Transfer Data-Efficient Robot Learning

Open paper AskPDF arXiv

Mastering dexterous manipulation with multi-fingered hands has been a grand challenge in robotics for decades. Despite its potential, the difficulty of collecting high-quality data remains a primary bottleneck for high-precision tasks. While reinforcement learning and simulation-to-real-world transfer offer a promising alternative, the transferred policies often fail for tasks demanding millimeter-scale precision, such as bimanual piano playing. In this work, we introduce HandelBot, a framework that combines a simulation policy and rapid adaptation through a two-stage pipeline. Starting from a simulation-trained policy, we first apply a structured refinement stage to correct spatial alignments by adjusting lateral finger joints based on physical rollouts. Next, we use residual reinforcement learning to autonomously learn fine-grained corrective actions. Through extensive hardware experiments across five recognized songs, we demonstrate that HandelBot can successfully perform precise bimanual piano playing. Our system outperforms direct simulation deployment by a factor of 1.8x and requires only 30 minutes of physical interaction data.

Abstract

Robot Manipulation Benchmarks

Explore similar work

HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot Policies

CANTABILE: Learning Expressive Dynamics for Robotic Piano Performance

Enforcing Human-like Kinematics in Dexterous Piano Playing via Adversarial Posture Regularization