Data-Driven Glyph Positioning for Handwritten Music Synthesis

Elona Shatri; Jiří Mayer; Jan Hajič

cs.CVOct 20, 2026

Data-Driven Glyph Positioning for Handwritten Music Synthesis

Authors: Elona Shatri, Jiří Mayer, Jan Hajič

Organizations: Charles University, Prague, Czech Republic

Abstract

Synthesised data is a common strategy to address the lack of annotated handwritten music scores in Optical Music Recognition (OMR). Smashcima is a recent synthesis framework that assembles pages of handwritten scores by sampling glyph images from MUSCIMA++ and positioning them with hard-coded rules. In this work we ask whether these rules are grounded empirically and whether a learned model for context-dependent positioning can further improve synthesis quality. We extract inter-glyph distributions from around 285 thousand annotated relationships across MUSCIMA++ and OmniOMR, finding that several key parameters deviate systematically from real handwriting. Replacing these constants with empirical means reduces positioning RMSE by up to 60%. We then train a Graph Neural Network (GNN) on the Music Notation Graph structure to predict context-dependent offsets, achieving an additional 50 to 85% RMSE reduction on stems and beams by modelling how neighbouring glyphs constrain placement. Cross-dataset analysis shows that spacing differences between the two corpora are explained by note density (3.3), not style, and that duration-proportional spacing does not hold in handwriting (R2 = 0.001). In downstream evaluation, a YOLOv8 detector pretrained on GNN-positioned synthetic data achieves +30% mAP50 over the baseline and +11% over empirical means when fine-tuned on 200 real handwritten staves, showing that positioning accuracy matters for downstream OMR.

Figures & tables

Figure 1: Empirical positioning distributions. Row 1: cross-dataset scatterplots (blue: MUSCIMA++, orange: OmniOMR). Row 2: key findings with Smashcima baselines (red dashed) and empirical means (green). Row 3: writer-dependent beam style, inter-note spacing, and the absence of duration-proportional spacing ( $\text{R}^{2}\approx 0.001$ ).

Relationship	Baseline	Empirical	RMSE	Improv.
	value	mean	(global)
Stem length (up)	$5.0\text{\,}\mathrm{m}\mathrm{m}$	$3.8\text{\,}\mathrm{m}\mathrm{m}$	$1.44\text{\,}\mathrm{m}\mathrm{m}$	+60%
Stem length (down)	$5.0\text{\,}\mathrm{m}\mathrm{m}$	$4.2\text{\,}\mathrm{m}\mathrm{m}$	$1.32\text{\,}\mathrm{m}\mathrm{m}$	+57%
Beam angle (global)	$0\text{\,}\mathrm{\SIUnitSymbolDegree}$	$4.0\text{\,}\mathrm{\SIUnitSymbolDegree}$	$4.51\text{\,}\mathrm{\SIUnitSymbolDegree}$	+40%
Beam angle (per-writer)	$0\text{\,}\mathrm{\SIUnitSymbolDegree}$	writer int.	$3.81\text{\,}\mathrm{\SIUnitSymbolDegree}$	+49%
Inter-note gap	$1.2\text{\,}\mathrm{m}\mathrm{m}$	$5.07\text{\,}\mathrm{m}\mathrm{m}$	$3.94\text{\,}\mathrm{m}\mathrm{m}$	+29%

Table 1: Summary of positioning corrections. RMSE improvement is relative to Smashcima’s hard-coded baseline, evaluated under LOWO cross-validation.

Edge type	Baseline	Empirical	GNN	vs Emp.	$n$
notehead $\to$ stem	3.07	1.44	0.60	+80.6%	1984
notehead $\to$ beam	6.89	7.13	1.07	+84.4%	1743
notehead $\to$ flag	3.68	3.68	0.75	+79.5%	323
notehead $\to$ ledger line	1.16	1.16	0.55	+52.7%	600
notehead $\to$ staccato	2.59	2.59	0.91	+65.0%	289
notehead $\to$ duration dot	0.62	0.62	0.61	+1.5%	131

Table 2: Per-edge-type positioning RMSE (mm). The GNN improves most on relationships where placement is constrained by local context.

Figure 2: LOWO cross-validation: per-writer RMSE distribution across 50 held-out writers. The GNN generalises consistently (mean $2.06\pm 0.30$ mm), with only two outliers above 3 mm.

Panel of Figure 3 — Figure 3: Synthesised output from the same MusicXML input and random seed. Left: baseline (hard-coded constants). Centre: empirical (data-driven means). Right: GNN (context-dependent positioning). Note the progressively shorter stems, tilted beams, and wider inter-note spacing from left to right. The GNN output shows more realistic stem length variation in beamed groups.

Shots	Baseline	Empirical	GNN	$\Delta$ vs Base	$\Delta$ vs Emp
5	0.093	0.101	0.114	+22.6%	+12.9%
10	0.104	0.106	0.125	+20.8%	+18.3%
20	0.110	0.109	0.126	+15.1%	+16.1%
50	0.115	0.116	0.136	+18.7%	+17.5%
100	0.123	0.137	0.153	+24.0%	+11.4%
200	0.168	0.196	0.218	+30.0%	+11.3%

Table 3: Few-shot fine-tuning on real handwritten staves (mAP50). The GNN-pretrained model consistently outperforms both baselines across all data regimes.

Figure 4: Few-shot domain adaptation on real OmniOMR staves. The GNN-pretrained model (green) consistently outperforms both the empirical (orange) and baseline (red) models across all data regimes. The advantage is largest in the low-data regime where pretraining quality matters most.

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 625 papers

Optical Music Recognition for Real-World Manuscripts with Synthetic Data
Jun 8, 2026Jiří Mayer, Martina Dvořáková, Vojtěch Dvořák +6Optical Music Recognition Synthetic-to-Real Domain Adaptation
Institute of Formal and Applied Linguistics, Charles University · Moravian Library
Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training
May 11, 2026Daniel Dratschuk, Paul SwobodaOptical Music Recognition Synthetic Data Generation
Heinrich Heine University Düsseldorf, Germany
Full-Page Optical Music Recognition of Handwritten Monophonic Scores
Sep 4, 2026Adrian Rosello, Antonio Ríos-Vila, David Rizo +1Optical Music Recognition Synthetic Data Pretraining
Pattern Recognition and Artificial Intelligence group, University of Alicante, Alicante, Spain · Instituto Superior de Enseñanzas Artísticas de la Comunidad Valenciana, Alicante, Spain

Jun 8, 2026J/K move · Enter open · S save

Optical Music Recognition for Real-World Manuscripts with Synthetic Data

Jiří Mayer, Martina Dvořáková, Vojtěch Dvořák, Markéta Herzánová Vlková+5

Institute of Formal and Applied Linguistics, Charles University · Moravian Library

Optical Music Recognition Synthetic-to-Real Domain Adaptation Synthetic Data Augmentation

Open paper AskPDF arXiv

Optical Music Recognition (OMR) has seen major progress in model design, with end-to-end methods now capable of recognising notation at all levels of complexity. However, the impact of this progress has been limited by the visual domains of available training datasets, which are largely born-digital. Existing large collections of sheet music in libraries and other heritage institutions contain predominantly manuscripts, whose visual domains are highly diverse and different, so existing OMR systems fail when applied in the real world. These institutions are often resource-constrained, so large in-domain datasets cannot be expected. We provide a first baseline on real-world manuscripts with complex piano notation in the resource-constrained scenario. Using fine-grained music notation graph (MuNG) annotations and the Smashcima synthesis tool, we then show that while some direct transcriptions of in-domain data remain essential, domain adaptation using synthetic musical manuscript images brings significant improvement. Furthermore, the symbols used do not need to be in-domain, so the expensive fine-grained annotation can be avoided. We thus bring OMR closer to one of its stated goals: preserving and promoting musical cultural heritage.

Abstract

Synthetic Data Generation

Explore similar work

Optical Music Recognition for Real-World Manuscripts with Synthetic Data

Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training

Full-Page Optical Music Recognition of Handwritten Monophonic Scores