This paper introduces a computational workflow for automatically transcribing microtonal, free-rhythm vocal music, with Iranian classical music as a case study. Our approach is based on performances by the renowned vocalist Karimi and ground truth transcriptions by the prominent ethnomusicologist Masoudieh [14], which were subsequently incorporated into the IRMA Audio-MIDI dataset [20]. To accurately extract melodies, we employ pitch histograms in conjunction with Dynamic Time Warping (DTW). Additionally, we introduce specialized musical notations to capture the intricate ornamentations characteristic of the genre, with particular emphasis on the vocal technique tahrir. The transcription process is implemented in Python using the music21 library for symbolic music representation [5]. This study not only advances the field of computational ethnomusicology but also highlights the potential of computational methods in preserving and analyzing complex musical traditions. The transcription system also generates a combined visualization of the audio pitch contour and the DTW-aligned MIDI representation, enabling users to inspect the correspondence between the performance and the generated transcription. A companion visual editor supports expert-in-the-loop correction of the resulting notation.
Figures & tables
Figure 1: Masoudieh’s Transcript of Example 1
Figure 2: Time-Frequency Data Processing
Figure 3: Pitch Histogram for Example 1
Figure 4: Time-Frequency with Marked Peaks of the Histogram in Example 1
Figure 5: Histogram of Second Intervals in the Scales of the First Five Dastgāhs in Karimi’s Radif.
Figure 6: The Time-Frequency Graph of the First Frame of Example 1 with Marked Notes and Ornamentations
Figure 7: Error and Derivative in Finding the Tempo for Example 1
Figure 8: Generated Transcription of the First Two Sentences of Example 1
Figure 9: Visualizing the Pitch and Generated Transcription, Example 1, Sentence 1
Figure 10: Visualizing the Pitch and Generated Transcription, Example 1, Sentence 2
Figure 11: Expert-in-the-loop correction framework. Program A generates the initial transcription and audio–MIDI alignment. Program B supports revision of notes, ornament markers, and alignment boundaries, including segmented DTW recomputation, before corrected symbolic files are exported. Program A is revisited only when a change to the shāhed requires the transcription to be regenerated.
Existing methods for automatic music transcription are often limited to single-instrument recordings or fail on complex, real music mixes. Although previous work utilizes synthetic training data, the resulting models generalize poorly, leading to largely unusable transcription output in realistic, multi-instrument settings. In this work, we analyze the effectiveness of synthetic data for pre-training while combining it with fine-tuning on real music audio and post-training using reinforcement learning. We further introduce conditioning on instrument presence to customize transcriptions. Finally, we release MuScriptor, an open-weight multi-instrument music transcription model that works on real-world music recordings from across a diverse range of musical genres.
Simon Rouard, Michael Krause, Axel Roebel +2
Kyutai · Mirelo AI · UMR STMS, IRCAM-CNRS Sorbonne Univ.
This paper presents the results of the 2025 Automatic Music Transcription (AMT) Challenge, an online competition to benchmark progress in multi-instrument transcription. Eight teams submitted valid solutions; two outperformed the baseline MT3 model. The results highlight both advances in transcription accuracy and the remaining difficulties in handling polyphony and timbre variation. We conclude with directions for future challenges: broader genre coverage and stronger emphasis on instrument detection.
Transcribing music into a human-readable score requires a coherent understanding of rhythm, harmony, melody, and form. Two obstacles limit this goal: annotated recordings are scarce, and accurate local predictions can still produce inconsistent musical sequences. We present SheetSage2, a unified music transcription framework that combines synthetic data, task-specific structured decoding, and autoregressive distillation. Automatically annotated MIDI, rendered into audio, provides scalable supervision across music understanding tasks. Task-specific structured decoders integrate complementary musical cues and their temporal dependencies to produce musically coherent scores. Autoregressive distillation further retains transcription accuracy without task-specific dynamic programming at inference. Across eight benchmark collections, a single SheetSage2-AR model exceeds the listed prior systems on 12 of 15 benchmark--metric pairs in our evaluation, substantially improving over SheetSage1 and surpassing task-specific models on several benchmarks. Model weights and inference code are publicly available.