Organizations: Fudan University, Shanghai, China · University of Macau, Macau, China · The Chinese PLA General Hospital, Beijing, China · Shanghai Jiao Tong University, Shanghai, China
Fine-grained action segmentation during renorrhaphy in robot-assisted partial nephrectomy requires frame-level recognition of visually similar suturing gestures with variable duration and substantial class imbalance. The SIA-RAPN benchmark defines this problem on 50 clinical videos acquired with the da Vinci Xi system and annotated with 12 frame-level labels. The benchmark compares four temporal models built on I3D features: MS-TCN++, AsFormer, TUT, and DiffAct. Evaluation uses balanced accuracy, edit score, segmental F1 at overlap thresholds of 10, 25, and 50, frame-wise accuracy, and frame-wise mean average precision. In addition to the primary evaluation across five released split configurations on SIA-RAPN, the benchmark reports cross-domain results on a separate single-port RAPN dataset. Across the strongest reported values over those five runs on the primary dataset, DiffAct achieves the highest F1, frame-wise accuracy, edit score, and frame mAP, while MS-TCN++ attains the highest balanced accuracy.
Figures & tables
Model
F1@10
F1@25
F1@50
Acc
Bal-Acc
Edit
Frame mAP
MS-TCN++
76.54
71.96
58.02
68.83
67.54
74.93
48.79
AsFormer
79.60
73.78
57.87
64.55
58.73
78.00
40.08
TUT
76.04
71.03
57.62
61.86
54.57
76.28
37.61
DiffAct
83.14
80.23
66.86
72.12
67.16
82.30
51.28
Table 1: Summary comparison using the strongest reported value for each metric across the five splits.
Split
F1@{10,25,50}
Acc
Bal-Acc
Edit
Frame mAP
Split 1
0.7845 / 0.7378 / 0.5635
0.6348
0.5752
0.7568
0.3936
Split 2
0.7804 / 0.7147 / 0.5751
0.6302
0.5664
0.7800
0.3927
Split 3
0.7131 / 0.6346 / 0.4477
0.5498
0.4953
0.7271
0.3079
Split 4
0.7653 / 0.6937 / 0.5279
0.6304
0.5873
0.7461
0.4008
Split 5
0.7960 / 0.7309 / 0.5787
0.6455
0.5745
0.7872
0.3968
Table 2: AsFormer per-split performance on the primary SIA-RAPN benchmark.
Split
F1@{10,25,50}
Acc
Bal-Acc
Edit
Frame mAP
Split 1
0.7604 / 0.7103 / 0.5762
0.6186
0.5457
0.7628
0.3761
Split 2
0.7498 / 0.6885 / 0.5415
0.6147
0.5447
0.7242
0.3677
Split 3
0.7295 / 0.6749 / 0.5076
0.5975
0.5167
0.7364
0.3398
Split 4
0.7361 / 0.6729 / 0.5279
0.5866
0.5119
0.7542
0.3471
Split 5
0.7421 / 0.7022 / 0.5567
0.5992
0.5203
0.7421
0.3621
Table 3: TUT per-split performance on the primary SIA-RAPN benchmark.
Split
F1@{10,25,50}
Acc
Bal-Acc
Edit
Frame mAP
Split 1
0.7491 / 0.6961 / 0.5318
0.6446
0.6120
0.7357
0.4203
Split 2
0.7542 / 0.7149 / 0.5630
0.6651
0.5994
0.7426
0.4300
Split 3
0.7609 / 0.7144 / 0.5637
0.6836
0.6534
0.7477
0.4768
Split 4
0.7259 / 0.6879 / 0.5534
0.6499
0.5866
0.7200
0.4148
Split 5
0.7654 / 0.7196 / 0.5802
0.6883
0.6754
0.7493
0.4879
Table 4: MS-TCN++ per-split performance on the primary SIA-RAPN benchmark.
Split
F1@{10,25,50}
Acc
Bal-Acc
Edit
Frame mAP
Split 1
0.8274 / 0.8023 / 0.6654
0.7166
0.6541
0.8158
0.5017
Split 2
0.8012 / 0.7600 / 0.6268
0.6679
0.5982
0.8021
0.4291
Split 3
0.8207 / 0.7836 / 0.6686
0.7103
0.6573
0.8230
0.4942
Split 4
0.8314 / 0.7926 / 0.6589
0.7059
0.6563
0.8132
0.4861
Split 5
0.8291 / 0.7918 / 0.6536
0.7212
0.6716
0.8205
0.5128
Table 5: DiffAct per-split performance on the primary SIA-RAPN benchmark.
Split
F1@{10,25,50}
Acc
Bal-Acc
Edit
Frame mAP
Split 1
0.7414 / 0.6729 / 0.4361
0.5985
0.5157
0.7547
0.2947
Split 2
0.7099 / 0.6049 / 0.4444
0.5724
0.5040
0.7368
0.3066
Split 3
0.7203 / 0.6302 / 0.4759
0.6009
0.5369
0.6984
0.3129
Split 4
0.7143 / 0.6211 / 0.4596
0.5731
0.5855
0.6749
0.3179
Split 5
0.7055 / 0.6196 / 0.3926
0.5894
0.5270
0.7523
0.3084
Table 6: DiffAct per-split performance on the single-port cross-domain dataset.
Split
F1@{10,25,50}
Acc
Bal-Acc
Edit
Frame mAP
Split 1
0.7439 / 0.6201 / 0.4012
0.5489
0.5479
0.7514
0.2835
Split 2
0.6748 / 0.5636 / 0.3273
0.4980
0.4031
0.7241
0.2315
Split 3
0.6647 / 0.5612 / 0.2866
0.4400
0.3782
0.6592
0.1962
Split 4
0.6890 / 0.5775 / 0.3222
0.5060
0.5425
0.6994
0.2630
Split 5
0.7089 / 0.5849 / 0.3270
0.4876
0.4548
0.7531
0.2229
Table 7: AsFormer per-split performance on the single-port cross-domain dataset.
Split
F1@{10,25,50}
Acc
Bal-Acc
Edit
Frame mAP
Split 1
0.6609 / 0.5698 / 0.3605
0.5383
0.4811
0.6170
0.2763
Split 2
0.6952 / 0.6132 / 0.4126
0.5249
0.4767
0.6425
0.2816
Split 3
0.6301 / 0.5533 / 0.3919
0.4820
0.4051
0.6597
0.2183
Split 4
0.6762 / 0.5886 / 0.3438
0.5508
0.5221
0.6684
0.2781
Split 5
0.6592 / 0.5690 / 0.3662
0.5407
0.4791
0.6683
0.2738
Table 8: TUT per-split performance on the single-port cross-domain dataset.
Split
F1@{10,25,50}
Acc
Bal-Acc
Edit
Frame mAP
Split 1
0.5277 / 0.4286 / 0.2468
0.4330
0.4221
0.5577
0.2078
Split 2
0.5696 / 0.4401 / 0.2330
0.4710
0.4470
0.5192
0.2264
Split 3
0.5263 / 0.4637 / 0.2561
0.4754
0.4401
0.5449
0.2326
Split 4
0.4986 / 0.4000 / 0.2197
0.4721
0.4219
0.5053
0.2283
Split 5
0.4354 / 0.3618 / 0.2041
0.3864
0.3474
0.4231
0.1726
Table 9: MS-TCN++ per-split performance on the single-port cross-domain dataset.