A High-Accuracy Optical Music Recognition Method Based on Bottleneck Residual Convolutions
Authors: Junwen Ma, Huhu Xue, Xingyuan Zhao, and Weicheng Fu
Organizations: Department of Artificial Intelligence, Tianshui Normal University, Tianshui 741001, Gansu, China. · Department of Physics, Tianshui Normal University, Tianshui 741001, Gansu, China. · School of Music and Dance, Tianshui Normal University, Tianshui 741001, Gansu, China.
Optical Music Recognition (OMR) aims to convert printed or handwritten music score images into editable symbolic representations. This paper presents an end-to-end OMR framework that combines residual bottleneck convolutions with bidirectional gated recurrent unit (BiGRU)-based sequence modeling. A convolutional neural network with ResNet-v2-style residual bottleneck blocks and multi-scale dilated convolutions is used to extract features that encode both fine-grained symbol details and global staff-line structures. The extracted feature sequences are then fed into a BiGRU network to model temporal dependencies among musical symbols. The model is trained using the Connectionist Temporal Classification loss, enabling end-to-end prediction without explicit alignment annotations. Experimental results on the Camera-PrIMuS and PrIMuS datasets demonstrate the effectiveness of the proposed framework. On Camera-PrIMuS, the proposed method achieves a sequence error rate (SeER) of 7.52% and a symbol error rate (SyER) of 0.45%, with pitch, type, and note accuracies of 99.33%, 99.60%, and 99.28%, respectively. The average training time is 1.74~s per epoch, demonstrating high computational efficiency while maintaining strong recognition performance. On PrIMuS, the method achieves a SeER of 8.11% and a SyER of 0.49%, with pitch, type, and note accuracies of 99.27%, 99.58%, and 99.21%, respectively. A fine-grained error analysis further confirms the effectiveness of the proposed model.
Figures & tables
Fig. 1: Architecture of the proposed end-to-end Optical Music Recognition model based on bottleneck residual convolution and BiGRU. The model consists of a bottleneck residual CNN feature extractor, a BiGRU sequence modeling module, and a fully connected output layer for end-to-end symbol sequence recognition.
Augmentation Type
Probability
Brightness/Contrast/Gamma Adjustment
0.9
Blur Perturbation
0.5
JPEG Compression Artifacts
0.5
Morphological Erosion
0.08
Morphological Dilation
0.08
Small-Angle Rotation and Shearing
0.6
TABLE I: Data augmentation operations and activation probabilities
Fig. 2: Comparison between the original music score image and its augmented versions.
Category
Name
Value
Input
Image height
128
Number of channels
1
Batch size
16
Convolutional encoder
Number of residual blocks
5
Convolution channels
[32, 64, 128, 256, 256]
Kernel size
3×3
TABLE II: Main hyperparameter configuration of the model.
Category
Name
Value
Input
Img. height
128
Num. channels
1
Batch size
16
Conv. encoder
Num. residual blocks
5
Conv. channels
[32, 64, 128, 256, 256]
Kernel size
3×3
TABLE III: Main hyperparameter configuration of the model
Data type
Model
Clean
Distortions
Agnostic
Semantic
Agnostic
Semantic
Clean
CRNN
1.10 / 21.7
0.80 / 12.5
44.3 / 95.1
59.7 / 97.9
R2-CRNN
0.59 / 16.2
0.56 / 13.4
7.63 / 20.9
38.19 / 91.0
Ours
0.49 / 8.66
0.49 / 7.80
23.41 / 87.91
39.05 / 93.6
Distortions
CRNN
1.40 / 24.9
3.30 / 44.6
1.60 / 24.7
3.40 / 38.3
R2-CRNN
0.68 / 18.7
1.06 / 25.4
1.01 / 27.8
1.50 / 35.8
TABLE IV: Recognition performance of different models on the Camera-PrIMuS dataset under Clean and Distortions conditions (without data augmentation, in %)
Model
Agnostic
Semantic
CRNN
1.00 / 17.9
0.80 / 12.5
Ours
0.53 / 9.09
0.50 / 8.33
TABLE V: Recognition performance of different models on the PrIMuS dataset under Clean and Distortions conditions (without data augmentation, in %)
Methods
SeER (%)
SyER (%)
Semantic
7.80
0.49
Semantic+Augmentation
7.52
0.45
Agnostic
8.66
0.49
Agnostic+Augmentation
7.64
0.44
TABLE VI: Performance comparison of our model with and without data augmentation on the Camera-PrIMuS dataset.
Methods
SeER (%)
SyER (%)
Semantic
8.33
0.50
Semantic+Augmentation
8.11
0.49
Agnostic
9.09
0.53
Agnostic+Augmentation
8.55
0.48
TABLE VII: Performance comparison of our model with and without data augmentation on the PrIMuS dataset
Methods
SeER
SyER
Pitch accuracy
Type accuracy
Note accuracy
Training time (s/epoch)
ConvNet-STN
27.0298
9.3975
70.0000
72.3987
68.3274
2.79
DWD
25.1475
7.9030
82.1123
80.9635
78.5064
3.50
MCRR
5.1488
1.0612
90.7544
91.5520
88.2094
1.93
Ours
7.52
0.45
99.33
99.60
99.28
1.74
TABLE VIII: Performance comparison of SOTA models on the Camera-PrIMuS dataset
Methods
SeER
SyER
Pitch accuracy
Type accuracy
Note accuracy
Training time (s/epoch)
ConvNet-STN
16.8056
5.0208
89.7107
94.0273
85.5620
0.98
DWD
18.5609
8.7811
95.2074
96.0049
91.1255
1.21
MCRR
1.4571
0.3234
97.0000
97.1298
94.4447
0.56
Ours
8.11
0.49
99.27
99.58
99.21
1.80
TABLE IX: Performance comparison of SOTA models on the PrIMuS dataset
Category
I
D
N1
N2
OMR-NED (%)
Errors (%)
Barlines
76
190
29,516
29,402
0.45
22.77
Clefs
3
3
7,935
7,935
0.04
0.51
GraceNotes
186
172
2,043
2,057
8.73
30.65
KeySignatures
2
0
6,095
6,097
0.02
0.17
MultiRests
4
5
1,990
1,989
0.23
0.77
Notes
202
209
120,423
120,416
0.17
35.19
TABLE X: Fine-grained OMR-NED analysis across different symbol categories (Semantic encoding) on the Camera-PrIMuS dataset
Category
I
D
N1
N2
OMR-NED (%)
Errors (%)
Accidentals
148
77
15,663
15,734
0.72
18.53
Barlines
66
208
29,619
29,477
0.46
22.57
Clefs
1
1
7,957
7,957
0.01
0.16
Fermatas
1
0
394
395
0.13
0.08
GraceNotes
154
152
2,028
2,030
7.54
25.21
MeterSigns
0
1
4,230
4,229
0.01
0.08
TABLE XI: Fine-grained OMR-NED analysis across different symbol categories (Agnostic encoding) on the Camera-PrIMuS dataset.
Category
I
D
N1
N2
OMR-NED (%)
Errors (%)
Barlines
90
219
29,413
29,284
0.53
22.91
Clefs
4
2
7,950
7,952
0.04
0.44
GraceNotes
207
200
2,102
2,109
9.67
30.17
KeySignatures
4
3
6,054
6,055
0.06
0.52
MultiRests
5
5
1,947
1,947
0.26
0.74
Notes
239
243
120,676
120,672
0.20
35.73
TABLE XII: Fine-grained OMR-NED analysis across different symbol categories (Semantic encoding) on the PrIMuS dataset
Category
I
D
N1
N2
OMR-NED (%)
Errors (%)
Accidentals
140
61
15,415
15,494
0.65
16.24
Barlines
93
178
29,347
29,262
0.46
21.89
Clefs
1
0
7,953
7,954
0.01
0.08
Fermatas
5
0
406
411
0.61
0.40
GraceNotes
210
201
2,232
2,241
9.19
33.20
MeterSigns
0
0
4,233
4,233
0.00
0.00
TABLE XIII: Fine-grained OMR-NED analysis across different symbol categories (Agnostic encoding) on the PrIMuS dataset