Full-Page Optical Music Recognition of Handwritten Monophonic Scores
Authors: Adrian Rosello, Antonio Ríos-Vila, David Rizo, Jorge Calvo-Zaragoza
Organizations: Pattern Recognition and Artificial Intelligence group, University of Alicante, Alicante, Spain · Instituto Superior de Enseñanzas Artísticas de la Comunidad Valenciana, Alicante, Spain
Full-page end-to-end Optical Music Recognition seeks to transcribe entire music pages directly into symbolic notation, avoiding the limitations of traditional pipelines that rely on accurate staff segmentation. Recent Transformer-based architectures have achieved strong performance on typeset scores, relying on large-scale synthetic data for pretraining. However, their applicability to handwritten music remains largely unexplored. In this work, we study full-page transcription on handwritten monophonic collections and analyze the impact of synthetic pretraining in this setting. To investigate which factors are most relevant during pretraining, we introduce a generator capable of producing visually coherent full-page scores in both typeset and handwritten styles. Experiments on three real handwritten datasets provide a comparative evaluation of several full-page pipelines and different synthetic pretraining strategies. The results suggest that the benefits of synthetic pretraining are primarily associated with learning structural layout conventions rather than with visual similarity to the target handwriting.
Figures & tables
Figure 1: 10-fold error rates after directly adjusting the Unfolding [ 10 ] and SMT [ 15 ] models to the target handwritten corpora.
Figure 2: Diagram of the two-step pipeline used.
Figure 3: Diagrams of the holistic pipelines used.
Figure 4: Representative examples of the generated music in both typeset (a) and handwritten (b) style.
Name
FMT-C
FMT-M
Jazzmus
Vocabulary size
60
75
96
Number of pages
86
205
163
Table 1: Statistics of the real full-page datasets.
Figure 5: Comparison of the SMT, Unfolding, and CRNNf scenarios in terms of SER across 10 folds for the FMT-C , FMT-M , and Jazzmus datasets.
FMT-C
FMT-M
Jazzmus
CRNNf
45.7±27.8
81.6±25.4
63.1±2.8
Unfolding
40.4±14.9
68.3±18.0
61.0±3.5
SMTpr
31.6±9.9
32.8±8.7
26.9±3.4
SMThw
38.4±10.4
32.9±5.2
30.6±3.1
Table 2: 10-fold cross-validated SER (%) on FMT-C , FMT-M , and Jazzmus . Values report mean and standard deviation across folds. Best results per dataset are in bold.
Figure 6: Qualitative result of the model SMTpr performance on a test split of the 10k-fold of Jazzmus . Red denotes false positives, while blue represents false negatives.
Optical Music Recognition (OMR), the task of transcribing sheet music into a structured textual representation, is currently bottlenecked by a lack of large-scale, annotated datasets of real scans. This forces models to rely on either few-shot transfer or synthetic training pipelines that remain overly simplistic. A secondary challenge is encoding non-uniqueness: in the popular Humdrum **kern format for transcribing music, multiple different text encodings can render into the same visual sheet music. This one-to-many mapping creates a harder learning task and introduces high uncertainty during decoding. We propose Transcoda, an OMR system built on (i) an advanced synthetic data generation pipeline, (ii) a normalization of the **kern encoding to enforce a unique normal form and (iii) grammar-based decoding to ensure the syntactic correctness of the output. This approach allows us to train a compact 59M-parameter model in just 6 hours on a single GPU that outperforms billion-parameter baselines. Transcoda achieves the best score among state of the art baselines on a newly curated benchmark of synthetically rendered scores at 18.46% OMR-NED (compared to 43.91% for the next-best system, Legato) and reduces the error rate on historical Polish scans to 63.97% OMR-NED (down from 80.16% for SMT++).
Transcribing music into a human-readable score requires a coherent understanding of rhythm, harmony, melody, and form. Two obstacles limit this goal: annotated recordings are scarce, and accurate local predictions can still produce inconsistent musical sequences. We present SheetSage2, a unified music transcription framework that combines synthetic data, task-specific structured decoding, and autoregressive distillation. Automatically annotated MIDI, rendered into audio, provides scalable supervision across music understanding tasks. Task-specific structured decoders integrate complementary musical cues and their temporal dependencies to produce musically coherent scores. Autoregressive distillation further retains transcription accuracy without task-specific dynamic programming at inference. Across eight benchmark collections, a single SheetSage2-AR model exceeds the listed prior systems on 12 of 15 benchmark--metric pairs in our evaluation, substantially improving over SheetSage1 and surpassing task-specific models on several benchmarks. Model weights and inference code are publicly available.
We propose a novel pipeline, Legato 2, for extracting symbolic notation and semantic knowledge from images of sheet music. Legato 2 features the first large-scale neural model for optical music recognition (OMR) to operate sequentially on a system-by-system basis, following the horizontal lines of notation as they are read on the page, rather than treating the page as an undifferentiated image, enabling better scaling to arbitrarily long inputs. It is also the first OMR model capable of generating symbolic transcriptions that include embedded textual content, such as titles and annotations. The pipeline combines system-level segmentation with an autoregressive vision-LM to capture both local notation details and score structure. Across multiple datasets, Legato 2 consistently outperforms prior state of the art. We also show that symbolic transcriptions complement visual inputs for frontier language models, improving their interpretation of dense musical documents. Legato 2 establishes new state-of-the-art performance in both OMR and downstream sheet music understanding.
Guang Yang, Brian Siyuan Zheng, Victoria Ebert +1
Paul G. Allen School of Computer Science & Engineering, University of Washington · Allen Institute for AI