Score-Informed Transformer for Refining MIDI Velocity in Automatic Piano Transcription
Organizations: David · Signal Processing and Recognition Lab, University of Western Australia, Perth, WA 6000 Australia
Abstract
MIDI velocity is crucial for capturing expressive dynamics in human performances. In practical scenarios, a music score with inaccurate velocities may be available alongside the performance audio (e.g., music education and free online archives), enabling the task of score-informed MIDI velocity estimation. In this work, we propose a modular, lightweight score-informed Transformer correction module that refines the velocity estimates of Automatic Music Transcription (AMT) systems. Leveraging high-quality expressive datasets for rigorous evaluation, we focus on piano transcription, integrating our module into multiple existing systems. Trained exclusively on the MAESTRO dataset, our method consistently reduces velocity errors and improves cross-dataset generalization to SMD and MAPS datasets. Under this training protocol, integrating our score-informed module with HPT (named Score-HPT) establishes a new state-of-the-art, outperforming existing score-informed methods and piano transcription models while adding only 1 M parameters. Furthermore, our framework remains robust to temporal misalignments caused by imperfect audio-score synchronization, a common real-world challenge.
Figures & tables
| Model & Inputs | MAESTRO test set | ||||
|---|---|---|---|---|---|
| MAE | STD | # Params | |||
| HPT | |||||
| audio | 4.6 | 6.1 | 4.1 M | ||
| Score-HPT | |||||
| audio | + onset | 4.0 | 4.8 | 5.0 M | |
| audio | + onset | + frame | 4.0 | 4.7 | 5.0 M |
| Model | SMD dataset | ||
|---|---|---|---|
| MAE | STD | # Params | |
| Score-Inf. Methods | |||
| DiffVel [ 27 ] | 19.7 | 13.1 | – |
| FiLM Conv [ 15 ] | 15.1 | 12.3 | – |
| FiLM U-Net [ 14 ] | 9.9 | 7.8 | 8.8 M |
| Proposed | |||
Automatic Music Transcription
Automatic music transcription converts recorded music into symbolic notes, rhythms, instruments, and sometimes readable scores or tablature.
See all 39 papers