Geometrical Cross-Attention and Nonvoid Voxelization for Efficient 3D Medical Image Segmentation
Authors: Chenxin Yuan, Shoupeng Chen, Haojiang Ye, Yiming Miao, Limei Peng, Pin-Han Ho
Organizations: Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, 518100, Guangdong, China · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore · School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, Shenzhen, 518172, Guangdong, China · School of Computer Science and Engineering, Kyungpook National University, Daegu, 37224, South Korea · Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, N2L3G1, Ontario, Canada
Accurate segmentation of 3D medical scans is crucial for clinical diagnostics and treatment planning, yet existing methods often fail to achieve both high accuracy and computational efficiency across diverse anatomies and imaging modalities. To address these challenges, we propose GCNV-Net, a novel 3D medical segmentation framework that integrates a Tri-directional Dynamic Nonvoid Voxel Transformer (3DNVT), a Geometrical Cross-Attention module (GCA), and Nonvoid Voxelization. The 3DNVT dynamically partitions relevant voxels along the three orthogonal anatomical planes, namely the transverse, sagittal, and coronal planes, enabling effective modeling of complex 3D spatial dependencies. The GCA mechanism explicitly incorporates geometric positional information during multi-scale feature fusion, significantly enhancing fine-grained anatomical segmentation accuracy. Meanwhile, Nonvoid Voxelization processes only informative regions, greatly reducing redundant computation without compromising segmentation quality, and achieves a 56.13% reduction in FLOPs and a 68.49% reduction in inference latency compared to conventional voxelization. We evaluate GCNV-Net on multiple widely used benchmarks: BraTS2021, ACDC, MSD Prostate, MSD Pancreas, and AMOS2022. Our method achieves state-of-the-art segmentation performance across all datasets, outperforming the best existing methods by 0.65% on Dice, 0.63% on IoU, 1% on NSD, and relatively 14.5% on HD95. All results demonstrate that GCNV-Net effectively balances accuracy and efficiency, and its robustness across diverse organs, disease conditions, and imaging modalities highlights strong potential for clinical deployment.
Figures & tables
Figure 1: Overview of the proposed GCNV-Net. The Nonvoid Voxelization converts dense volumes to sparse nonvoid voxels. The encoder applies 3DNVT blocks with GCA-based downsampling. The decoder fuses multi-scale features using GCA-based upsampling and residual convolution to produce dense segmentation.
Figure 3: Tri-directional Dynamic Nonvoid Voxel Transformer (3DNVT): dynamic window partitioning followed by plane-wise (transverse/sagittal/coronal) voxel grouping
Figure 4: Geometrical Cross-Attention (GCA) modules. GCA-Down uses pooled features as queries with high-resolution features as keys/values, and GCA-Up reverses the direction for decoder fusion.
Dataset
Non-zero Ratio (%) ↓
Cropped Ratio (%) ↓
Nonvoid Voxels (k) ↓
Traditional Voxels (k) ↓
Embedded Voxel Saving (%) ↑
MSD Prostate
99.50
99.82
57.19
131.1
56.38
BraTS2021
16.54
63.18
38.84
262.1
85.18
ACDC
97.72
99.92
6.71
16.4
59.11
SIMON BIDS
79.78
97.62
138.10
262.1
47.31
SPIDER
52.20
47.27
110.53
262.1
57.83
ISLES2022
19.10
70.77
2.58
16.4
84.26
Table 1: Nonvoid Voxelization efficiency across eleven 3D medical image datasets. Non-zero Ratio: percentage of voxels with non-zero intensity. Cropped Ratio: percentage of voxels remaining after cropping to the minimum bounding box of all non-zero voxels. Nonvoid/Traditional Voxels (k): average embedded voxel count using our Nonvoid Voxelization or classical voxelization, respectively. Embedded Voxel Saving: 1 - Nonvoid Voxels / Traditional Voxels. ↑ / ↓ : preferred direction.
Figure 5: Nonvoid Voxelization visualization. First five columns: dataset-specific trained embeddings. Last two columns: zero-shot transfer (ACDC → SPIDER, BraTS2021 → MSD BrainTumor). Nonvoid/total voxel counts and ratios are shown below each volume.
Method
BraTS2021
ACDC
MSD Prostate
MSD Pancreas
AMOS2022
FLOPs (G) ↓
Params (M) ↓
Latency (ms) ↓
QEA ↑
Rank ↓
Stat. Sign.
Dice ↑
HD95 ↓
Dice ↑
HD95 ↓
Dice ↑
HD95 ↓
Dice ↑
HD95 ↓
Dice ↑
HD95 ↓
nnU-Net-ResEncL
91.19
1.33
91.65
0.78
73.91
15.79
67.86
5.89
89.42
4.27
1432.0
102.3
44.6
0.873
8
✓
nnU-Net-ResEncM
91.24
1.37
92.01
0.77
74.07
15.26
68.12
5.70
88.73
4.61
1432.0
102.3
44.6
1.107
5
✓
nnU-Net-NoResEnc
91.23
1.39
91.58
0.78
73.32
17.82
66.54
5.99
88.65
4.71
951.0
31.2
33.7
1.660
3
✓
MedNeXt-L
91.46
1.33
92.60
0.74
72.90
19.25
69.30
5.77
89.77
3.94
584.7
63.0
189.3
1.191
4
✓
MedNeXt-M
90.39
1.60
91.73
0.78
73.64
16.73
69.74
5.71
89.07
4.39
254.6
17.6
103.1
1.743
2
✓
Table 2: Quantitative comparison on BraTS2021, ACDC, MSD Pancreas, MSD Prostate, and AMOS2022 datasets. Dice is given in %. Efficiency related metrics are evaluated on input volumes of size 1×128×128×128 (RTX4090, batch=1). QEA denotes the normalized quality-efficiency polygon enclosed area. ✓indicates the statistical significance of GCNV-Net versus each methods under paired Wilcoxon signed-rank test ( p<0.05 , two-sided, Holm-corrected). ↑ / ↓ denotes the preferred direction.
Figure 6: Qualitative segmentation comparison across all five benchmarks. Per-sample Dice (D) and HD95 (H) are shown below each prediction.
Figure 7: Quality-efficiency trade-off polygons (top-5 by enclosed area).
Figure 8: Per-case Dice distributions for different 3D segmentation methods. Brackets show representative p-values (top-4) from paired two-sided Wilcoxon signed-rank tests (Holm-corrected) comparing GCNV-Net with each competing methods. All comparisons yield p-values below 0.05, indicating statistically significant improvements.
Figure 9: Mean Dice, HD95, FLOPs, and inference latency of the variants for ablation studies all benchmark datasets. FLOPs and latency are evaluated on input 3D volumes of size 1×128×128×128 (RTX4090, batch=1 ).
Figure 1: Per-case IoU, HD95, NSD distributions for different 3D segmentation methods. Brackets show representative p-values (top-4) from paired two-sided Wilcoxon signed-rank tests (Holm-corrected) comparing GCNV-Net with each competing methods. All comparisons yield p-values below 0.05, indicating statistically significant improvements.
Figure 2: Visualized segmentation results on three representative samples from (a) BraTS2021, (b) ACDC, (c) MSD Prostate, (d) MSD Pancreas and (e) AMOS2022 datasets. Each subfigure includes predictions from twelve baseline models, our GCNV-Net, original input images (Origin), and ground truth. GCNV-Net demonstrates superior segmentation accuracy and boundary preservation across varied anatomical structures and imaging modalities.
Figure 3: The quality efficiency trade off polygon.
Figure 4: Embedded voxel saving of nonvoid voxelization versus under different ϵ (threshold hyperparameter).
Method
Whole Tumor
Tumor Core
Enhancing Tumor
Average
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
nnU-Net-ResEncL
93.23
88.27
1.32
0.97
92.11
87.28
1.24
0.96
88.22
81.71
1.41
0.96
91.19
85.75
1.33
0.96
nnU-Net-ResEncM
93.64
88.75
1.36
0.97
92.31
87.58
1.29
0.96
87.77
81.27
1.46
0.96
91.24
85.87
1.37
0.96
nnU-Net-NoResEnc
93.87
89.00
1.32
0.97
92.31
87.57
1.30
0.96
87.51
80.91
1.56
0.96
91.23
85.83
1.39
0.96
MedNeXt-L
93.19
88.38
1.36
0.98
91.67
86.81
1.19
0.97
89.52
83.26
1.45
0.96
91.46
86.15
1.33
0.97
MedNeXt-M
92.71
87.85
1.49
0.97
90.03
84.88
1.41
0.96
88.44
82.08
1.89
0.94
90.39
84.94
1.60
0.96
Table 1: Segmentation performance (Dice (%), IoU (%), HD95, and NSD) of different 3D segmentation methods on BraTS2021 datasets across three tumor substructures.
Method
Right Ventricular Cavity
Myocardium
Left Ventricular Cavity
Average
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
nnU-Net-ResEncL
90.21
82.85
0.94
0.98
89.75
81.63
1.02
0.99
94.98
90.64
0.38
0.99
91.65
85.04
0.78
0.99
nnU-Net-ResEncM
90.79
83.72
0.93
0.98
90.06
82.10
1.00
0.99
95.17
90.92
0.37
1.00
92.01
85.58
0.77
0.99
nnU-Net-NoResEnc
90.37
83.10
0.94
0.98
89.72
81.59
1.02
0.99
94.66
90.14
0.39
0.99
91.58
84.94
0.78
0.99
MedNeXt-L
91.49
84.76
0.91
0.98
90.49
82.72
0.98
0.99
95.83
91.93
0.34
1.00
92.60
86.47
0.74
0.99
MedNeXt-M
90.89
83.90
0.92
0.98
89.85
81.79
1.01
0.99
94.45
89.80
0.40
0.99
91.73
85.17
0.78
0.99
Table 2: Segmentation performance (Dice (%), IoU (%), HD95, and NSD) of different 3D segmentation methods on ACDC dataset across three cardiac structures.
Method
Peripheral Zone
Transition Zone
Average
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
nnU-Net-ResEncL
66.11
51.63
16.97
0.81
81.72
72.02
14.62
0.90
73.91
61.83
15.79
0.85
nnU-Net-ResEncM
66.35
51.81
15.91
0.81
81.79
72.12
14.61
0.90
74.07
61.97
15.26
0.85
nnU-Net-NoResEnc
65.57
51.24
19.56
0.80
81.07
71.34
16.09
0.89
73.32
61.29
17.82
0.84
MedNeXt-L
65.44
51.18
20.42
0.79
80.36
70.65
18.07
0.88
72.90
60.92
19.25
0.84
MedNeXt-M
65.98
51.56
17.68
0.80
81.30
71.61
15.77
0.89
73.64
61.58
16.73
0.85
Table 3: Segmentation performance (Dice (%), IoU (%), HD95, and NSD) of different 3D segmentation methods on the MSD Prostate dataset across Peripheral Zone, Transition Zone, and their average.
Method
Pancreas
Pancreatic Tumor
Average
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
Dice ↑
IoU ↑
HD95 ↓
NSD ↑
nnU-Net-ResEncL
82.85
71.23
3.34
0.91
52.86
42.00
8.43
0.73
67.86
56.62
5.89
0.82
nnU-Net-ResEncM
83.01
71.33
3.92
0.91
53.23
42.38
7.48
0.73
68.12
56.85
5.70
0.82
nnU-Net-NoResEnc
81.89
69.99
3.20
0.91
51.20
40.74
8.79
0.69
66.54
55.37
5.99
0.80
MedNeXt-L
83.76
72.31
3.82
0.93
54.84
43.94
7.72
0.76
69.30
58.13
5.77
0.84
MedNeXt-M
84.18
72.48
3.84
0.93
55.29
44.28
7.58
0.77
69.74
58.38
5.71
0.85
Table 4: Segmentation performance (Dice (%), IoU (%), HD95, and NSD) of different 3D segmentation methods on the MSD Pancreas dataset across Pancreas, Pancreatic Tumor, and their average.
Table 5: Segmentation performance (Dice (%), IoU (%), HD95, and NSD) of different 3D segmentation methods on the AMOS2022 dataset across Spleen (Spl), Right Kidney (RK), Left Kidney (LK), Gall Bladder (GB), Esophagus (Eso), Liver (Liv), Stomach (Sto), Aorta (Aor), Postcava (IVC), Pancreas (Pan), Right Adrenal Gland (RAG), Left Adrenal Gland (LAG), Duodenum (Duo), Bladder (Bla), Prostate/Uterus (Pro/Ute), and their average (AVG).
Institute of Artificial Intelligence Innovation, Industry Academia Innovation School, National Yang Ming Chiao Tung University · Department of Computer Science and Information Engineering, National Chung Cheng University
National Institute of Technology Kurukshetra Department of Electronics and Communication Engineering · National Institute of Technology Kurukshetra Department of Computer Engineering · Indian Institute of Technology Ropar Department of Computer Science and Engineering