Transcribing expressive piano performances into scores requires interpreting individual note timings within an underlying metrical structure. However, score attributes are associated with notes, whereas beats may occur without coincident note onsets. We introduce DuET, a Dual-Encoder Transformer that jointly learns note-level performance MIDI-to-score (PM2S) transcription and frame-level beat and downbeat tracking. Bidirectional cross-attention connects note and frame encoders, while an autoregressive decoder alternates between their representations to predict structured score attributes. We further introduce TimeRoPE and Time-Pitch Rotary Positional Encoding (TPRoPE) to establish a shared performance-time reference and explicitly encode relative temporal and pitch relationships. Evaluations on ASAP, a classical piano dataset, and our curated Gangqinpu dataset, which spans multiple genres with a focus on popular piano music, demonstrate improvements in score timing and notated duration. Compared with the strongest baseline on each dataset, DuET reduces MUSTER mean error from 6.56% to 3.64% on Gangqinpu and from 11.30% to 11.22% on ASAP. Ablations support the benefits of frame-level context, time-based positional encoding, and decoder-history masking. Code, weights, and demos are available at https://github.com/wei-zeng98/DuET.
Figures & tables
Attribute
Name
Values
Description
pi
Pitch
{0,…,127}
MIDI pitch copied from the input note
bi
Bar index
bi−a∈{−256,…,255}
Encoded relative to window anchor a
qi
Onset integer part
{0,…,63}
Whole quarter-note units from bar start
fi
Onset fractional part
F⊂[0,1)
Fractional quarter-note offset
ℓi
Duration
D⊂[0,51]
Notated duration; zero denotes a grace note
si
Staff
{0,1}
Upper or lower staff, respectively
Table 1: Attributes in each note-aligned score record. Score-time values are measured in quarter-note units.
Figure 1: Overview of DuET within the audio-to-score pipeline. Each dual-encoder layer combines branch-specific self-attention, bidirectional cross-attention, and feed-forward networks. Successive decoder layers attend to note and frame memories in alternation, predicting one score record per input note. Rotary positional encoding is applied to queries and keys. Residual connections and layer normalization are omitted for clarity.
Figure 2: Positional coordinates for the same four notes. IndexRoPE uses sequence indices, TimeRoPE uses scaled performance timestamps, and TPRoPE uses both performance timestamps and pitch coordinates. The dashed vertical lines indicate the 20 ms frame grid; note onsets need not coincide with this grid.
MUSTER [ 31 , 20 ]
ScoreSimilarity [ 32 , 33 ]
Method
Pitch
Miss
Extra
Onset
Offset
Mean
Miss
Extra
Duration
Staff
ASAP
MuseScore [ 16 ]
2.41
7.35
9.64
47.90
49.44
23.35
16.17
16.74
55.23
21.87
HMMs (J-pop) [ 19 ]
2.09
6.38
8.67
25.02
29.21
14.27
10.80
11.39
71.38
–
HMMs (classical) [ 19 ]
2.11
6.47
8.75
22.58
29.84
13.95
10.74
11.28
64.73
–
Neural beat tracking [ 10 ]
2.02
6.81
9.01
68.28
54.11
28.04
17.10
17.67
66.98
6.86
Table 2: PM2S transcription errors (%). Lower is better. Bold and underlined values indicate the best and second-best results within each dataset, ranked before rounding where full-precision results are available. ASAP baseline results are taken from Beyer and Dai [11] .
MUSTER [ 31 , 20 ]
ScoreSimilarity [ 32 , 33 ]
Model
Pitch
Miss
Extra
Onset
Offset
Mean
Miss
Extra
Duration
Staff
ASAP
DuET
2.25
5.95
11.10
7.66
16.42
8.68
9.28
12.69
30.58
3.99
Note encoder only
2.35
5.94
10.94
10.85
22.42
10.50
9.71
13.11
52.23
4.74
Index / Alternating †
2.35
6.92
12.41
8.67
19.00
9.87
10.27
13.67
36.62
4.73
Time / Alternating
2.84
11.41
16.95
13.21
24.00
13.68
15.88
19.28
42.98
4.93
Table 3: PM2S ablations on fixed common-valid subsets of 39 ASAP and 72 Gangqinpu performances. Errors are percentages; lower is better. Bold and underlined values indicate the best and second-best results within each dataset, ranked before rounding. Positional/memory variants are named as “RoPE / decoder memory”. DuET uses Time–Pitch RoPE with alternating memory access. Thin rules separate encoder, positional/memory, and masking controls.
ASAP
Gangqinpu
Model
Beat
Downbeat
Beat
Downbeat
DuET
68.01
65.55
77.34
79.94
Frame-only tracker
61.19
55.51
78.86
81.37
Dual-encoder tracker
62.89
53.97
80.24
80.73
Index / Alternating †
62.34
61.66
70.60
73.09
Time / Alternating
65.73
60.45
81.24
82.50
Table 4: Beat-tracking ablations on 59 ASAP and 100 Gangqinpu performances. Values are macro F1 scores (%); higher is better. Bold and underlined values indicate the best and second-best results in each column. Positional/memory variants are named as “RoPE / decoder memory”. DuET uses Time–Pitch RoPE with alternating memory access.
Figure 3: Qualitative comparison of score excerpts reconstructed from the same performance MIDI: (a) DuET, (b) Beyer and Dai, and (c) Liu’s PM2S. All outputs are rendered with MuseScore 4.
Integrative Sciences and Engineering Programme, NUS Graduate School · School of Computing, National University of Singapore · Singapore American School