Gugak-VocalSet: A Multi-Genre Studio-Quality Dataset of Korean Traditional Singing with Sigimsae Annotations

Danbinaerin Han; Wonil Kim; Seah Hong; Dongjoo Moon; Jaeok Lee; Jongpil Lee; Chaewon Kim; Juhan Nam

cs.SDOct 11, 2026

Gugak-VocalSet: A Multi-Genre Studio-Quality Dataset of Korean Traditional Singing with Sigimsae Annotations

Authors: Danbinaerin Han, Wonil Kim, Seah Hong, Dongjoo Moon, Jaeok Lee, Jongpil Lee, Chaewon Kim, Juhan Nam

Organizations: KAIST, Daejeon · Neutune, Seoul · National Gugak Center, Seoul

Abstract

We present Gugak-VocalSet, the first publicly available audio dataset for Korean traditional singing with dense annotations of sigimsae, a structured pitch contour system of vocal expressions. Gugak-VocalSet contains 24.5 hours of unaccompanied solo vocal recordings across 305 tracks of representative repertoire curated by the National Gugak Center, performed by 20 professional singers in a controlled studio environment with fixed key and tempo. In addition to genre, lyrics, and mood/timbre tags, the dataset provides a unified sigimsae ontology and approximately 68,000 segments (71,700 instances including multi-label) of audio-grounded sigimsae annotations; the ontology adopts a functional-position-based hierarchy designed for consistent application across genres, and 14 expert annotators specializing in Korean traditional music performed segment-level labeling accordingly. Baseline experiments on sigimsae classification and event detection show that domain-informed F0 representations remain competitive with, and on classification surpass, frozen probing of a general-purpose music foundation model, while pitch and spectral cues play complementary roles in event-level detection. Companion page: https://www.danbinaerin.com/Gugak-VocalSet/

Figures & tables

Figure 1: Genre hierarchy of Gugak-VocalSet. Numbers show track counts per category.

Level	Type	Content
Song	Metadata	Title, Genre (top/mid/sub), Singer, Jangdan, Tempo, Time Sig., EN/KR Captions
	Expert Tags	Key, Scale, Mood (80 tags), Timbre (38 tags)
Segment	Lyrics	Phrase-level (7–20 s)
	Alignment	Lyrics–Audio Alignment
Event	Sigimsae	17 Ornament Types w/ Timestamps

Table 1: Summary of metadata and annotations in Gugak-VocalSet.

Name	Sym.	Contour	Description
Onset Sigimsae (앞시김새)
onset asc. slide (밀어내기)			Sliding up from a lower pitch to the main note
onset desc. flick (꺾어내기)			Briefly touching a higher pitch before flicking down to the main note
onset upper turn (감아내기 위)			Winding upward, passing through a higher pitch, then returning to the main note
onset lower turn (감아내기 아래)			Winding downward, passing through a lower pitch, then returning to the main note
Vibrato Sigimsae / (요성 시김새) / yoseong / 搖聲

Figure 2: The 17 sigimsae types in the unified ontology grouped by functional position, with notation symbol, pitch contour, and description.

Figure 3: Annotation examples: jeongak (top) and minsogak (bottom). Each panel shows lyric–audio alignment, F0 contour overlaid on a spectrogram, and sigimsae annotations color-coded by functional category.

Panel of Figure 4 — Figure 4: Statistics of 71,742 sigimsae instances across 305 tracks. (a) Annotation count per type (log scale). (b) Duration distribution per type; labels show means $\mu$ .

Task-specific CNNs (trained from scratch)
Model	Input	Macro F1	Weighted F1
F0CNN	F0 contour	.441 $\pm$ .015	.530 $\pm$ .022
MelCNN	Mel spec	.400 $\pm$ .018	.510 $\pm$ .025
Frozen foundation models (best layer, MARBLE 2-layer MLP probe)
MERT-v1 95M (L7)	MERT emb.	.358 $\pm$ .011	.454 $\pm$ .017
CultureMERT-TA 95M (L7)	MERT emb.	.334 $\pm$ .022	.427 $\pm$ .031

Table 2: 17-category sigimsae classification (song-level 5-fold CV, mean±std). MERT-family models: best layer among L

\in\{5,7,9,11,12\}

Figure 5: Row-normalized (recall) confusion matrix for 17-type classification (F0CNN, aggregated over 5 folds).

	Within		Cross
Direction	M.F1	Acc	M.F1	Acc
J $\rightarrow$ M	.423 $\pm$ .007	.558 $\pm$ .005	.279 $\pm$ .002	.429 $\pm$ .010
M $\rightarrow$ J	.354 $\pm$ .004	.500 $\pm$ .006	.286 $\pm$ .013	.432 $\pm$ .035

Table 3: Cross-genre Macro F1 (F0CNN, 17-cat, 5-fold CV, 102 songs per genre, mean

\pm

std over 3 seeds).

Model	Frame F1 M	Ev.F1 200ms	IoU F1 0.5
EDTCN (F0)	.324 $\pm$ .015	.675 $\pm$ .014	.661 $\pm$ .014
EDTCN (mel)	.269 $\pm$ .013	.763 $\pm$ .012	.712 $\pm$ .015
MERT-v1 95M (L5)	.311 $\pm$ .015	.756 $\pm$ .026	.706 $\pm$ .028
CultureMERT 95M (L5)	.294 $\pm$ .016	.747 $\pm$ .019	.700 $\pm$ .022

Table 4: Sigimsae event detection results on the 17-category ontology. MERT-family: best layer in L

\in\{5,7,9,12\}

by frame Macro F1.

Cultural Heritage

Cultural heritage uses AI to document, analyze, preserve, and interpret artifacts, sites, archives, and living traditions.

See all 57 papers

Pitch Contour Tokenization using VQ-VAE and Its Application on Korean Traditional Music Analysis
Aug 11, 2026Seonguk Ju, Seola Cho, Sooin Chung +2Audio Representation Learning Vector-Quantized Variational Autoencoder
Music & Arts Learning (MALer) Lab, Sogang University, South Korea · Graduate School of Culture Technology, KAIST, South Korea
Deep Supervised Contrastive Learning of Pitch Contours for Robust Pitch Accent Classification in Seoul Korean
Apr 21, 2026Hyunjung Joo, GyeongTaek LeeSpeech Prosody Supervised Contrastive Learning
Department of Linguistics, Rutgers University · Hanyang Institute for Phonetics and Cognitive Sciences of Language (HIPCS) · Department of Smart Factory, Gachon University
KunSet: A Corpus and Benchmark for Lyrics Alignment and Rhyme Retrieval in Kunqu Opera
Oct 21, 2026Lejun Min, Siqi Chen, Delong Wang +10Music Information Retrieval Audio-Text Alignment
Stanford University · Peking University · University of California San Diego +3

Aug 11, 2026J/K move · Enter open · S save

Pitch Contour Tokenization using VQ-VAE and Its Application on Korean Traditional Music Analysis

Seonguk Ju, Seola Cho, Sooin Chung, Danbinaerin Han+1

Music & Arts Learning (MALer) Lab, Sogang University, South Korea · Graduate School of Culture Technology, KAIST, South Korea

Audio Representation Learning Vector-Quantized Variational Autoencoder Audio Tokenization Music Information Retrieval

Open paper AskPDF arXiv

Computational analysis of music often relies on discrete representations, yet many musical traditions are organized around continuous pitch movement that resists segmentation into note-like units. For such traditions, the discrete units that analysis would build on are not given in advance. We address this gap by learning a vocabulary of local pitch-contour patterns directly from unlabeled audio, using a VQ-VAE that quantizes fixed-length contour segments into a finite codebook. To make the learned tokens stable across segmentation positions and small variations in timing and pitch range, we train the model with a reconstruction objective evaluated under the best alignment among a set of candidate temporal and pitch-domain transformations. Applied to Korean traditional music, the learned tokens recover information about expert-defined sigimsae categories without supervision, and in pansori individual tokens align with the two principal modes, Gyemyeonjo and Ujo, supporting their use as units for corpus-level analysis of contour-centric traditions.

Abstract

Cultural Heritage

Explore similar work

Pitch Contour Tokenization using VQ-VAE and Its Application on Korean Traditional Music Analysis

Deep Supervised Contrastive Learning of Pitch Contours for Robust Pitch Accent Classification in Seoul Korean

KunSet: A Corpus and Benchmark for Lyrics Alignment and Rhyme Retrieval in Kunqu Opera