A Learned Musical-Time Coordinate for Piano Audio-to-Score Transcription

Min-Hsing Wang; Tsung-Ren Huang

cs.SDOct 21, 2026

A Learned Musical-Time Coordinate for Piano Audio-to-Score Transcription

Authors: Min-Hsing Wang, Tsung-Ren Huang

Organizations: National Taiwan University

Abstract

End-to-end piano audio-to-score (A2S) transcription is the task of reconstructing a notated score from an audio recording in a single system. Existing A2S approaches usually rely on pre-segmented excerpts, and few handle whole pieces without external segmentation. This raises the question of whether a musical-time coordinate can guide score transcription while also helping the system segment whole pieces. Previous work on jazz saxophone A2S uses an external beat tracker to obtain such a coordinate for score prediction. We introduce downbeat phase, a musical-time coordinate learned jointly with score transcription that represents each audio frame's relative position within a notated bar. With this coordinate, our system can not only condition its score decoder on downbeat phase but also self-segment at bar boundaries and stitch the segments into a whole piece. Evaluation suggests that the coordinate improves metrical structure and supports whole-piece transcription with comparable notation quality.

Figures & tables

Figure 1: The encoder extracts content features and estimates downbeat phase, while the decoder combines these features with audio- and score-side Fourier positional encodings of downbeat phase to generate a score autoregressively.

Syn (9,958 excerpts, 28 works)
	MV2H components					MV2H
System	$F_{\mathrm{mp}}$	$F_{\mathrm{voi}}$	$F_{\mathrm{meter}}$	$F_{\mathrm{val}}$	$F_{\mathrm{harm}}$	4	5
Piano-A2S	.681	.876	.352	.901	.638	.774	.690
Ours	.792 ∗	.885	.675 †	.930 ∗	.749 ∗	.839 ∗	.806 ∗
w/o coordinate	.793 ∗	.890 ∗	.557 ∗	.936 ∗	.768 ∗	.847 ∗	.789 ∗
ASAP (633 excerpts, 8 works)

Table 1: Pre-segmented MV2H evaluation on Syn and ASAP.

			Syn (180 renderings, 45 works)				ASAP (74 recordings, 39 works)
System	External Segmentation	Coordinate	$F_{\mathrm{meter}}$	$\mathrm{MV2H}_{5}$	OMR-NED	Transcribed (%)	$F_{\mathrm{meter}}$	$\mathrm{MV2H}_{5}$	OMR-NED	Transcribed (%)
Piano-A2S	Beat This!	n/a	.236	.614	75.4	98	.143	.584	84.0	97
	oracle	n/a	.337	.650	67.9	77	.168	.566	72.6	78
Ours	—	predicted	.667 ∗	.829 ∗	61.8 ∗	100	.318 ∗	.718 ∗	85.4	100
	oracle	predicted	.696 ∗	.837 ∗	60.0 ∗	100	.287 ∗	.701 ∗	85.1	100
	oracle	oracle	.807 †	.870 ∗	53.8 ∗	100	.518 †	.772 †	75.1 ∗	100

Table 2: Whole-piece transcription quality on Syn and ASAP.

Figure 2: Coordinate ablation using one training seed on common scored excerpts from the test splits in Sec. 3.1 .

\Delta F_{\mathrm{meter}}

denotes the difference from Ours w/o coordinate.

Automatic Music Transcription

Automatic music transcription converts recorded music into symbolic notes, rhythms, instruments, and sometimes readable scores or tablature.

See all 34 papers

DuET: Note-Frame Dual Encoding for Piano Performance-to-Score Transcription
Oct 15, 2026Wei Zeng, Ye WangAutomatic Music Transcription Transformer
Integrative Sciences and Engineering Programme, NUS Graduate School · School of Computing, National University of Singapore
TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data
Sep 1, 2026Jianhuai Hu, Yashan Wang, Shangda Wu +7Domain Generalization Automatic Music Transcription
Central Conservatory of Music · Independent Researcher · Wuhan Conservatory of Music +2
SheetSage2: Coherent Lead-Sheet Transcription with Synthetic Supervision
Oct 4, 2026Junyan Jiang, Ruibin Yuan, Jiahao Pan +4Automatic Music Transcription Music Understanding
New York University · MBZUAI · ACE Studio +1

Oct 15, 2026J/K move · Enter open · S save

DuET: Note-Frame Dual Encoding for Piano Performance-to-Score Transcription

Wei Zeng, Ye Wang

Integrative Sciences and Engineering Programme, NUS Graduate School · School of Computing, National University of Singapore

Automatic Music Transcription Transformer Rotary Positional Embeddings

Open paper AskPDF arXiv

Transcribing expressive piano performances into scores requires interpreting individual note timings within an underlying metrical structure. However, score attributes are associated with notes, whereas beats may occur without coincident note onsets. We introduce DuET, a Dual-Encoder Transformer that jointly learns note-level performance MIDI-to-score (PM2S) transcription and frame-level beat and downbeat tracking. Bidirectional cross-attention connects note and frame encoders, while an autoregressive decoder alternates between their representations to predict structured score attributes. We further introduce TimeRoPE and Time-Pitch Rotary Positional Encoding (TPRoPE) to establish a shared performance-time reference and explicitly encode relative temporal and pitch relationships. Evaluations on ASAP, a classical piano dataset, and our curated Gangqinpu dataset, which spans multiple genres with a focus on popular piano music, demonstrate improvements in score timing and notated duration. Compared with the strongest baseline on each dataset, DuET reduces MUSTER mean error from 6.56% to 3.64% on Gangqinpu and from 11.30% to 11.22% on ASAP. Ablations support the benefits of frame-level context, time-based positional encoding, and decoder-history masking. Code, weights, and demos are available at https://github.com/wei-zeng98/DuET.

Abstract

Automatic Music Transcription

Explore similar work

DuET: Note-Frame Dual Encoding for Piano Performance-to-Score Transcription

TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data

SheetSage2: Coherent Lead-Sheet Transcription with Synthetic Supervision