BMdataset: A Musicologically Curated LilyPond Dataset

Matteo Spanio; Ilay Guler; Antonio Rodà

cs.SDApr 12, 2026

BMdataset: A Musicologically Curated LilyPond Dataset

Authors: Matteo Spanio, Ilay Guler, Antonio Rodà

Organizations: Department of Information Engineering, University of Padua, Padua, Italy · Boston University, Boston, USA

Abstract

Symbolic music research has relied almost exclusively on MIDI-based datasets; text-based engraving formats such as LilyPond remain unexplored for music understanding. We present BMdataset, a musicologically curated dataset of 393 LilyPond scores (2,646 movements) transcribed by experts directly from original Baroque manuscripts, with metadata covering composer, musical form, instrumentation, and sectional attributes. Building on this resource, we introduce LilyBERT (weights can be found at https://huggingface.co/csc-unipd/lilybert), a CodeBERT-based encoder adapted to symbolic music through vocabulary extension with 115 LilyPond-specific tokens and masked language model pre-training. Linear probing on the out-of-domain Mutopia corpus shows that, despite its modest size (~90M tokens), fine-tuning on BMdataset alone outperforms continuous pre-training on the full PDMX corpus (~15B tokens) for both composer and style classification, demonstrating that small, expertly curated datasets can be more effective than large, noisy corpora for music understanding. Combining broad pre-training with domain-specific fine-tuning yields the best results overall (84.3% composer accuracy), confirming that the two data regimes are complementary. We release the dataset, tokenizer, and model to establish a baseline for representation learning on LilyPond.

Figures & tables

Dataset	Format	Size	License
LMD [ 4 ]	MIDI	174K	CC-BY †
SymphonyNet [ 7 ]	MIDI	46K	–
MAESTRO [ 8 ]	MIDI	1.3K	CC-BY-NC-SA
Wikifonia [ 9 ]	MusicXML	6.4K	–
POP909 [ 10 ]	MIDI	909	–
Nottingham [ 11 ]	ABC	1.2K	PD

Table 1: Comparison of symbolic music datasets. BMdataset is the first musicologically curated LilyPond-native dataset.

Panel of Figure 1 — Figure 1: BMdataset dataset statistics. The dataset is dominated by Late Baroque works, string-centric instrumentation, and concerto forms. Vivaldi accounts for 30.5% of compositions, while the remaining 70 composers contribute 1–18 pieces each.

	Composer (30 classes)			Style (10 classes)
Model	Acc.	Prec.	Recall	Acc.	Prec.	Recall
CB + PDMX full	$.808{\scriptstyle\pm.021}$	$.751{\scriptstyle\pm.025}$	$.751{\scriptstyle\pm.032}$	$.826{\scriptstyle\pm.015}$	$\mathbf{.838}{\scriptstyle\pm.044}$	$.781{\scriptstyle\pm.031}$
CB + BMdataset	$\underline{.829}{\scriptstyle\pm.013}$	$.750{\scriptstyle\pm.039}$	$\underline{.752}{\scriptstyle\pm.036}$	$\mathbf{.837}{\scriptstyle\pm.004}$	$.813{\scriptstyle\pm.047}$	$\underline{.794}{\scriptstyle\pm.016}$
CB + PDMX 90M	$.817{\scriptstyle\pm.017}$	$\mathbf{.767}{\scriptstyle\pm.035}$	$.747{\scriptstyle\pm.036}$	$.823{\scriptstyle\pm.009}$	$.805{\scriptstyle\pm.053}$	$.775{\scriptstyle\pm.039}$
CB + PDMX $\rightarrow$ BM	$\mathbf{.843}{\scriptstyle\pm.010}$	$\underline{.765}{\scriptstyle\pm.031}$	$\mathbf{.775}{\scriptstyle\pm.028}$	$\underline{.829}{\scriptstyle\pm.011}$	$\underline{.825}{\scriptstyle\pm.013}$	$\mathbf{.797}{\scriptstyle\pm.035}$

Table 2: Linear probing results (layer 6, mean

\pm

std over 5 folds). Best in bold , second best underlined .

Figure 2: Layer-wise probing accuracy for composer and style classification. Composer information is best captured at middle layers (6–9), while style features are strongest in earlier layers and degrade towards the output.

Figure 3: t-SNE projection of layer-6 embeddings (CB + PDMX $\rightarrow$ BM) on Mutopia, colored by style (left) and composer (right). Stylistic clusters (Baroque, Classical, Romantic) are clearly separated; within-style composer clusters indicate that the model captures both global period conventions and individual compositional idioms.

Figure 4: Row-normalised confusion matrix for composer classification (CB + PDMX $\rightarrow$ BM, layer 6, aggregated over 5 folds). Most misclassifications occur between stylistically related composers (e.g., Grieg $\leftrightarrow$ Chopin, Haydn $\leftrightarrow$ Mozart), suggesting that errors reflect genuine stylistic proximity rather than random noise.

Computational Musicology

See all 11 papers

Can LLMs understand LilyPond? A benchmark for symbolic music generation and understanding
Jun 7, 2026Matteo Spanio, Mohammad Torabi, Andrea Poltronieri +1Music Generation Evaluation LLM Evaluation
Centro di Sonologia Computazionale, University of Padova, Padova, Italy · Music Technology Group, Universitat Pompeu Fabra, Barcelona, Spain
PianoCoRe: Combined and Refined Piano MIDI Dataset
May 7, 2026Ilya BorovikMusic Information Retrieval
*Skolkovo Institute of Science and Technology, Moscow, Russia
HumMusQA: A Human-written Music Understanding QA Benchmark Dataset
Mar 29, 2026Benno Weck, Pablo Puentes, Andrea Poltronieri +2Music Understanding Audio-Language Model Evaluation
Universitat Pompeu Fabra · Universitat Autònoma de Barcelona

Jun 7, 2026J/K move · Enter open · S save

Can LLMs understand LilyPond? A benchmark for symbolic music generation and understanding

Matteo Spanio, Mohammad Torabi, Andrea Poltronieri, Antonio Rodà

Centro di Sonologia Computazionale, University of Padova, Padova, Italy · Music Technology Group, Universitat Pompeu Fabra, Barcelona, Spain

Music Generation Evaluation LLM Evaluation Language Model Generation Evaluation Symbolic Music Generation Music Understanding

Open paper AskPDF arXiv

Symbolic music evaluation for large language models remains fragmented across representations, datasets, and metrics. We introduce LilyBench, a LilyPond-based benchmark that jointly evaluates symbolic music generation and music understanding on the same family of open-weight LLMs. The benchmark includes a 200-prompt generation suite and ten understanding tasks adapted from ABC-Eval, covering syntax, metadata prediction, structural sequencing, and music recognition. Generation quality is evaluated using compile rate, MusPy descriptor distributions via Jensen-Shannon similarity, and LilyBERT-based Fréchet Music Distance (FMD). Experiments on four open-weight models show that executable LilyPond generation is achievable in zero-shot settings, while structural understanding tasks remain challenging despite strong performance on composer and genre recognition. Our experiments also reveal systematic disagreements between descriptor-based and embedding-based metrics, suggesting that symbolic music evaluation benefits from metric triangulation rather than single-score ranking. We release the benchmark, prompt bank, and evaluation code to support future research in symbolic music generation and understanding at https://github.com/CSCPadova/lilybench

Abstract

Computational Musicology

Explore similar work

Can LLMs understand LilyPond? A benchmark for symbolic music generation and understanding

PianoCoRe: Combined and Refined Piano MIDI Dataset

HumMusQA: A Human-written Music Understanding QA Benchmark Dataset