KunSet: A Corpus and Benchmark for Lyrics Alignment and Rhyme Retrieval in Kunqu Opera
Authors: Lejun Min, Siqi Chen, Delong Wang, Tong Zheng, Yunqi Lu, Zhishan Zhou, Ke Chen, Yuting Zhong, Junrong Li, Shengran Jin, Beixuan Pei, Jun Chen, Yang Li
Organizations: Stanford University · Peking University · University of California San Diego · Chinese Opera Channel CCTV · Indiana University · Sun Yat-sen University
Kunqu is one of the oldest extant forms of Chinese opera, yet its digitization remains scarce and difficult to annotate, with two challenges standing out: first, close analysis of Kunqu performance demands syllable-level lyrics alignment; second, analyzing Kunqu pronunciation requires its phonological attributes (e.g., rhyme) to be linked with each syllable. To address these challenges, we present KunSet, the first Kunqu performance corpus comprising 482,164 fine-grained annotations of syllable-level lyrics and phonological attributes in total media duration of 155.2 h with 106.3 h annotated. To ensure annotation quality, KunSet was preannotated by an iterative ML-assisted pipeline and corrected by over 200 trained annotators via a purpose-built online annotation system under a two-round review process. To assess KunSet as a music information retrieval (MIR) benchmark, we evaluate two tasks: lyrics alignment, by finetuning a forced-alignment model with an acoustic anchor decoder; and same-rhyme retrieval, using frozen Chinese HuBERT features. Models built with KunSet outperform their baselines. We release KunSet and model checkpoints to support MIR research on traditional-music corpora.
Figures & tables
Figure 1: Annotation pipeline for the syllable-aligned corpus.
Measure
Value
Recordings (the number of)
281
Media / lyrics-covered time (h)
155.19 / 106.34
Sung / spoken / mixed (h)
50.30 / 55.15 / 0.90
Sentence / syllable records
78,505 / 482,164
Distinct transcript characters
4,357
Sentence time coverage (known media)
68.5%
Table 1: Descriptive statistics of the 281-recording KunSet corpus. Time coverage uses interval unions.
Figure 2: Excerpt–sentence–syllable annotation hierarchy (selected fields). Sentence and syllable annotations are linked by lineId , with all times referring to the recording timeline. Each syllable links to dictionary candidates through phonology .
Figure 3: Overview of 281 recordings. (a) Time by type; mixed denotes conflicting labels, outside denotes unannotated time. (b) Per-recording time composition, sorted by singing share. (c, d) Sentence and syllable duration CDFs (log-scaled). (e) All six attributes resolved, linked but not all resolved, or unlinked. (f) Attributes resolved by a single candidate or candidate consensus.
Overall ↓
Sung ↓
Spoken ↓
Pair
Onset
Offset
Onset
Offset
Onset
Offset
M–A
158.8
206.4
272.1
354.1
115.8
156.7
M–B
172.5
225.5
309.1
408.5
126.2
165.0
A–B
158.1
207.3
282.8
384.0
108.2
147.8
Table 2: Production (M) versus blind annotations (A/B), grouped by A’s delivery labels. Onset and offset MAEs (ms) are recording-macro averages; lower is better.
Figure 4: Acoustics around syllable starts and random times in the same sentence. Zero marks the sampled time; shading shows pointwise 95% confidence intervals from 2,000 recording-level bootstrap resamples.
Overall
Sung
Spoken
Model
Decoder
MAE ↓
Acc@200 ↑
MAE ↓
Acc@200 ↑
MAE ↓
Acc@200 ↑
Pretrained
Original
391.30
61.93
797.83
48.78
288.63
64.72
Pretrained
Anchor
337.70
65.12
702.34
50.76
241.07
68.45
Ours (FT7)
Original
372.91
64.49
753.40
53.66
267.69
67.25
Ours (FT7)
Anchor
312.90
67.60
620.62
56.64
233.98
70.35
Ours (FT251)
Original
224.04
79.52
529.21
65.35
144.95
83.46
Table 3: Test alignment overall and by delivery mode. MAE (ms) and Acc@200 (%) are recording-macro averages. Ours (FT251): 3-seed means, overall MAE seed SDs ≤ 4.5 ms; Ours (FT7): one model.
Supervision
P@10 ↑
None (HuBERT)
11.36
Shuffled rhyme groups
8.81 ± 0.07
Modern Mandarin finals
16.35 ± 0.14
Traditional rhyme groups (ours)
17.16 ± 0.11
Table 4: Same-rhyme retrieval on the 10-recording Test pool: macro P@10 (%), mean ± sample SD over three seeds.
Integrative Sciences and Engineering Programme, NUS Graduate School · School of Computing, National University of Singapore · Singapore American School