cs.SDFeb 24, 2026

Voices of the Mountains: Deep Learning-Based Vocal Error Detection System for Kurdish Maqams

Authors: Darvan Shvan Khairaldeen, Hossein Hassani

Organizations: University of Kurdistan Hewlêr Kurdistan Region - Iraq

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Maqam, a singing type, is a significant component of Kurdish music. A maqam singer receives training in a traditional face-to-face or through self-training. Automatic Singing Assessment (ASA) uses machine learning (ML) to provide the accuracy of singing styles and can help learners to improve their performance through error detection. Currently, the available ASA tools follow Western music rules. The musical composition requires all notes to stay within their expected pitch range from start to finish. The system fails to detect micro-intervals and pitch bends, so it identifies Kurdish maqam singing as incorrect even though the singer performs according to traditional rules. Kurdish maqam requires recognizing performance errors within microtonal spaces, which is beyond Western equal temperament. This research is the first attempt to address the mentioned gap. While many error types happen during singing, our focus is on pitch, rhythm, and modal stability errors in the context of Bayati-Kurd. We collected 50 songs from 13 vocalists ( 2-3 hours) and annotated 221 error spans (150 fine pitch, 46 rhythm, 25 modal drift). The data was segmented into 15,199 overlapping windows and converted to log-mel spectrograms. We developed a two-headed CNN-BiLSTM with attention mode to decide whether a window contains an error and to classify it based on the chosen errors. Trained for 20 epochs with early stopping at epoch 10, the model reached a validation macro-F1 of 0.468. On the full 50-song evaluation at a 0.750 threshold, recall was 39.4% and precision 25.8% . Within detected windows, type macro-F1 was 0.387, with F1 of 0.492 (fine pitch), 0.536 (rhythm), and 0.133 (modal drift); modal drift recall was 8.0%. The better performance on common error types shows that the method works, while the poor modal-drift recall shows that more data and balancing are needed.

Figures & tables

Figure 1: Overview of the research design we decided to use in this study.
Figure 2: The three error types: (a) fine pitch; (b) rhythm; (c) modal drift.
Figure 3: Model architecture with CNN feature extractor, bidirectional LSTM encoder, attention pooling, and two-task output heads.
ComponentDetailsOutput size
InputLog-mel spectrogram1×128×T1\times 128\times T
CNN Blocks 1–43 ×\times 3 Conv, BN, ReLU, MaxPool 2 ×\times 2128×8×T/16128\times 8\times T/16
BiLSTM2 layers, 256 per direction, dropout 0.3Sequence of 512 dims
Attention512 →\rightarrow 256 (Tanh), dropout 0.2, 256 →\rightarrow 1, softmax512 dims
Detection headDropout 0.3, Linear 512 →\rightarrow 1, sigmoid1 (probability)
Type headLinear 512 →\rightarrow 256, ReLU, dropout 0.3, Linear 256 →\rightarrow 3, softmax3 classes
Table 1: Summary of the model architecture.
Figure 4: The Vocal Annotator interface used for creating training annotations.
StatisticValue
Total songs50
Total singers13
Total annotations221
Average errors per song4.4
Songs with modal drift8
Table 2: Dataset collection statistics.
Error typeCountPercentage
Fine pitch error15067.9%
Rhythm error4620.8%
Modal drift error2511.3%
Total221100%
Table 3: Error type distribution in the collected dataset.
Figure 6: Error type distribution in the collected dataset (221 annotations).
SplitWindowsPercentage
Training13,36388.0%
Validation7184.7%
Test1,1187.4%
Total15,199100%
Table 4: Dataset split statistics.
Error typeCountPercentage
Fine pitch error53767.5%
Rhythm error21226.6%
Modal drift error475.9%
Total errors796–
Clean windows12,56794.0%
Table 5: Error distribution in training set.
MetricValue
Detection threshold0.750
Detection recall39.4%
Detection precision25.8%
Detection F10.311
Type macro-F10.387
Table 6: Comprehensive evaluation results on all 50 songs.
Error typeDetectedPrecisionRecallF1 score
Fine pitch error5789.5%34.0%0.492
Rhythm error2576.0%41.3%0.536
Modal drift error540.0%8.0%0.133
Table 7: Per-class classification performance on all 50 songs.
Figure 7: Validation macro-F1 and detection F1 over 20 epochs. Best validation macro-F1 (0.468) at epoch 10.
Figure 8: Confusion matrix for error-type classification (87 matched detections). Diagonal cells (shaded) are correct predictions.
More in this topic

Automated Evaluation

Automated evaluation develops metrics, benchmarks, and AI judges to assess model outputs and system behavior at scale.

See all 396 papers

Explore similar work

CardsList
Jun 24, 2026

Listening Like a Judge: A Music-Aware Framework for Automatic Singing Performance Evaluation

Automatic singing quality assessment (SQA) requires evaluating lyrical correctness and musical fidelity while handling expressive variations. However, existing systems largely rely on either acoustic cues or lyric transcriptions exclusively, limiting holistic performance evaluation. Furthermore, their integration is non-trivial due to challenges in robust singing transcription amid melisma, vibrato, and tempo elasticity. To this end, we propose MusicJudge, a modality-guided framework for automated SQA that performs block-aligned multimodal analysis by coupling lyric correctness with pitch-rhythm fidelity. It detects semantically meaningful lyric blocks using multi-signal matching that integrates semantic embeddings, lexical similarity, and phonetic alignment. To improve singing audio transcription, we introduce Modality-Guided LoRA for ASR fine-tuning. Experiments across datasets demonstrate strong agreement with human expert judgments and validate the generalizability of MusicJudge.

Neelam Saini, Sourav Ghosh

Samsung R&D Institute Bangalore, India

Music Information Retrieval
AskPDF
Jun 18, 2026

PASQA: Pitch-Accent-Focused Speech Quality Assessment Model Trained on Synthetic Speech with Accent Errors

Existing mean opinion score (MOS) prediction models typically predict utterance-level naturalness MOS and can be insensitive to localized pitch-accent errors. We propose Pitch-Accent-focused Speech Quality Assessment (PASQA), which explicitly targets pitch-accent correctness. To train our model, we construct a controlled Japanese accent-error dataset by changing accent patterns using an accent-controllable text-to-speech system, and compute a pseudo accent-quality score from the accent-error rate. PASQA builds on self-supervised representations and employs mora-conditioned fusion, ranking loss, an auxiliary accent-error localization task, and speaker-invariant training. Experiments show that conventional models fail to preserve the ordering by accent-error severity, whereas PASQA achieves high ordering accuracy on both seen and unseen speakers. Further, PASQA shows stronger agreement with human accent-correctness judgments. The code is available at https://github.com/lycorp-jp/PASQA.

Masaya Kawamura, Yuma Shirahata, Kentaro Mitsui +1

LY Corporation, Japan

Speech Quality AssessmentSpeech Prosody
AskPDF
Oct 14, 2026

PEDBench: Can Speech Models Detect Phonetic Errors?

Phonetic error detection (PED) aims to identify phoneme-level substitutions, deletions, and insertions and is essential for educational and clinical speech assessment and intervention. Despite recent advances in speech models for phoneme recognition, their ability to reliably detect fine-grained phonetic errors remains unclear because of a lack of high-quality evaluation data. To address this gap, we introduce PEDBench, the first PED benchmark spanning a continuum from natural to controlled speech. It comprises real-world recordings of spoken errors made by children (N=334), scripted error productions by human actors (N=1,188), and synthetic error productions generated using phone-conditioned text-to-speech (N=1,083). All phonetic annotations were transcribed and verified by two phonetically trained annotators. Our evaluation of 13 state-of-the-art speech models shows that they fail to reliably detect phonetic errors, even under controlled conditions with acted and synthetic speech, where acoustic variability is reduced. The models also exhibit low precision, frequently misclassifying correctly pronounced phonemes as errors. Our speech-language models, fine-tuned on synthetic data, outperform commercial and open-source baselines on word-level PED, although a sim-to-real domain gap remains. We plan to publicly release our code and recipes, along with our acted and synthetic datasets, to support the development of more accurate PED systems.

Zhonghao Shi, Zackary Rackauckas, Jasmine Yip +9

Harvard University · University of Southern California · University of Michigan +1

Speech Language Models
AskPDF