KunSet: A Corpus and Benchmark for Lyrics Alignment and Rhyme Retrieval in Kunqu Opera
Authors: Lejun Min, Siqi Chen, Delong Wang, Tong Zheng, Yunqi Lu, Zhishan Zhou, Ke Chen, Yuting Zhong, Junrong Li, Shengran Jin, Beixuan Pei, Jun Chen, Yang Li
Organizations: Stanford University · Peking University · University of California San Diego · Chinese Opera Channel CCTV · Indiana University · Sun Yat-sen University
Kunqu is one of the oldest extant forms of Chinese opera, yet its digitization remains scarce and difficult to annotate, with two challenges standing out: first, close analysis of Kunqu performance demands syllable-level lyrics alignment; second, analyzing Kunqu pronunciation requires its phonological attributes (e.g., rhyme) to be linked with each syllable. To address these challenges, we present KunSet, the first Kunqu performance corpus comprising 482,164 fine-grained annotations of syllable-level lyrics and phonological attributes in total media duration of 155.2 h with 106.3 h annotated. To ensure annotation quality, KunSet was preannotated by an iterative ML-assisted pipeline and corrected by over 200 trained annotators via a purpose-built online annotation system under a two-round review process. To assess KunSet as a music information retrieval (MIR) benchmark, we evaluate two tasks: lyrics alignment, by finetuning a forced-alignment model with an acoustic anchor decoder; and same-rhyme retrieval, using frozen Chinese HuBERT features. Models built with KunSet outperform their baselines. We release KunSet and model checkpoints to support MIR research on traditional-music corpora.
Figures & tables
Figure 1: Annotation pipeline for the syllable-aligned corpus.
Measure
Value
Recordings (the number of)
281
Media / lyrics-covered time (h)
155.19 / 106.34
Sung / spoken / mixed (h)
50.30 / 55.15 / 0.90
Sentence / syllable records
78,505 / 482,164
Distinct transcript characters
4,357
Sentence time coverage (known media)
68.5%
Table 1: Descriptive statistics of the 281-recording KunSet corpus. Time coverage uses interval unions.
Figure 2: Excerpt–sentence–syllable annotation hierarchy (selected fields). Sentence and syllable annotations are linked by lineId , with all times referring to the recording timeline. Each syllable links to dictionary candidates through phonology .
Figure 3: Overview of 281 recordings. (a) Time by type; mixed denotes conflicting labels, outside denotes unannotated time. (b) Per-recording time composition, sorted by singing share. (c, d) Sentence and syllable duration CDFs (log-scaled). (e) All six attributes resolved, linked but not all resolved, or unlinked. (f) Attributes resolved by a single candidate or candidate consensus.
Overall ↓
Sung ↓
Spoken ↓
Pair
Onset
Offset
Onset
Offset
Onset
Offset
M–A
158.8
206.4
272.1
354.1
115.8
156.7
M–B
172.5
225.5
309.1
408.5
126.2
165.0
A–B
158.1
207.3
282.8
384.0
108.2
147.8
Table 2: Production (M) versus blind annotations (A/B), grouped by A’s delivery labels. Onset and offset MAEs (ms) are recording-macro averages; lower is better.
Figure 4: Acoustics around syllable starts and random times in the same sentence. Zero marks the sampled time; shading shows pointwise 95% confidence intervals from 2,000 recording-level bootstrap resamples.
Overall
Sung
Spoken
Model
Decoder
MAE ↓
Acc@200 ↑
MAE ↓
Acc@200 ↑
MAE ↓
Acc@200 ↑
Pretrained
Original
391.30
61.93
797.83
48.78
288.63
64.72
Pretrained
Anchor
337.70
65.12
702.34
50.76
241.07
68.45
Ours (FT7)
Original
372.91
64.49
753.40
53.66
267.69
67.25
Ours (FT7)
Anchor
312.90
67.60
620.62
56.64
233.98
70.35
Ours (FT251)
Original
224.04
79.52
529.21
65.35
144.95
83.46
Table 3: Test alignment overall and by delivery mode. MAE (ms) and Acc@200 (%) are recording-macro averages. Ours (FT251): 3-seed means, overall MAE seed SDs ≤ 4.5 ms; Ours (FT7): one model.
Supervision
P@10 ↑
None (HuBERT)
11.36
Shuffled rhyme groups
8.81 ± 0.07
Modern Mandarin finals
16.35 ± 0.14
Traditional rhyme groups (ours)
17.16 ± 0.11
Table 4: Same-rhyme retrieval on the 10-recording Test pool: macro P@10 (%), mean ± sample SD over three seeds.
Text-to-music alignment is commonly evaluated by the global similarity between text and audio embeddings, which collapses all described attributes into a single score and obscures which ones are missing. We instead introduce MQAScore, which verifies each atomic musical concept along five clip-level dimensions (genre, instrument, mood/theme, vocal, and usage) by the probability that a large audio language model (LALM) answers affirmatively to a binary question. To assess evaluator reliability, we construct verification labels from presence/absence annotations and LLM-guided concept substitutions, and propose the Sensitivity-Invariance Ratio, which measures how selectively an evaluator responds to meaning-changing substitutions. Experiments show that concept-level scoring improves both question-answering and similarity-based evaluators, and MQAScore can be a more reliable and interpretable fine-grained complement to global similarity for evaluating text-to-music alignment.
Wei-Jaw Lee, Chun-Ping Wang, Xuanjun Chen +1
GICE, National Taiwan University · AI-CoRE, National Taiwan University · Taiwan AILabs
We present Gugak-VocalSet, the first publicly available audio dataset for Korean traditional singing with dense annotations of sigimsae, a structured pitch contour system of vocal expressions. Gugak-VocalSet contains 24.5 hours of unaccompanied solo vocal recordings across 305 tracks of representative repertoire curated by the National Gugak Center, performed by 20 professional singers in a controlled studio environment with fixed key and tempo. In addition to genre, lyrics, and mood/timbre tags, the dataset provides a unified sigimsae ontology and approximately 68,000 segments (71,700 instances including multi-label) of audio-grounded sigimsae annotations; the ontology adopts a functional-position-based hierarchy designed for consistent application across genres, and 14 expert annotators specializing in Korean traditional music performed segment-level labeling accordingly. Baseline experiments on sigimsae classification and event detection show that domain-informed F0 representations remain competitive with, and on classification surpass, frozen probing of a general-purpose music foundation model, while pitch and spectral cues play complementary roles in event-level detection. Companion page: https://www.danbinaerin.com/Gugak-VocalSet/
Danbinaerin Han, Wonil Kim, Seah Hong +5
KAIST, Daejeon · Neutune, Seoul · National Gugak Center, Seoul
Aligned audio, performance MIDI, and musical scores provide valuable supervision for music analysis and performance modeling, yet datasets that jointly align all three modalities remain scarce and limited in scale and genre diversity. We introduce Gangqinpu, a multi-genre piano dataset containing 18,786 score-performance pairs and 1,007.1 hours of source audio, with automatic note-level alignments and derived beat/downbeat annotations. It spans diverse genres, including pop, classical, jazz, folk/traditional, rock, and blues. Our automated pipeline transcribes piano audio, aligns the resulting MIDI with musical scores, assigns genre labels, and filters out non-expressive performances with a score-informed binary classifier. We further construct joint splits with ASAP using conservative score-similarity grouping to reduce data leakage. We evaluate Gangqinpu on performance MIDI-to-score transcription, expressive performance rendering, and beat tracking. Downstream experiments demonstrate the value of Gangqinpu as complementary training data, with cross-dataset gains in beat tracking and primarily in-domain benefits for transcription and expressive performance rendering. The code and dataset are publicly available at https://github.com/wei-zeng98/gangqinpu.
Wei Zeng, Anvay Mathur, Ye Wang
Integrative Sciences and Engineering Programme, NUS Graduate School · School of Computing, National University of Singapore · Singapore American School