cs.CVSep 27, 2026
SaveParameter-Efficient 3D Segmentation of Liver and Liver tumors: Depthwise factorization Scales Better Than Dense Convolution with Spatial Dimensionality
Organizations: Department of Molecular Biosciences and Bioengineering, University of Hawaii at Manoa, Honolulu, HI, USA · Faculty of Computer Studies, Arab Open University, Riyadh 11681, Kingdom of Saudi Arabia
Abstract
Three-dimensional dense convolutional networks are the strongest performers on volumetric medical image segmentation, but their parameter counts scale poorly: moving a dense k x k convolution to k x k x k multiplies its weights by k. We observe that depthwise separable factorization does not share this penalty. Because the cubic kernel term applies only to the depthwise stage while the pointwise projection, which dominates the parameter count, is unchanged, the same architecture grows by 5 % from 2D to 3D where a dense convolutional U-Net grows by 200 %. We exploit this asymmetry to build a 3D U-Net with 536,990 parameters, 24x fewer than an identical dense 3D U-Net. On MSD Task03 Liver (the Medical Segmentation Decathlon liver task, derived from LiTS), evaluated per case under five-fold cross-validation over all 131 public volumes, the model reaches a tumor Dice of 0.577 (95% CI [0.518, 0.633]) and a liver Dice of 0.947 (95% CI [0.941, 0.952]). Its liver Dice exceeds previously reported performance. Its tumor Dice exceeds their low-resolution configuration (0.4701) by 0.107 and their 2D configuration (0.5394), at approximately one twenty-fourth of the parameters and roughly half the in-plane resolution. Trained under identical conditions on a common held-out split, it exceeds a dense 3D U-Net on liver by +0.031 Dice (paired p = 0.006) and on tumor by +0.041 (95 % CI [+0.005, +0.081], paired p = 0.056), suggesting the factorization also acts as a regularizer in the small-data regime characteristic of medical imaging. We further show, on both LiTS and a 2D endoscopy benchmark, that a large fraction of the network's learnable spatial filters can be replaced by fixed shifts at no cost in accuracy, but that replacing all of them is measurably worse, the placement of spatial capacity matters more than its total amount.
Figures & tables
| Convolution | 2D | 3D | Growth |
|---|---|---|---|
| Dense | |||
| Depthwise separable |
Table 1: Parameter count of an identical U-Net topology under 2D and 3D convolution. The depthwise separable variant pays the cubic kernel term only on the depthwise stage; the pointwise projection, which dominates, is unchanged.
Figure 1: The scaling asymmetry, computed from Equation 1 over the four encoder stages of the architecture in Figure 2 . (a) Moving a dense U-Net from to kernels multiplies its convolution weights by . The depthwise separable form does not pay this, because the cubic term applies only to the depthwise stage while the pointwise projection, which dominates the count, is unchanged. (b) The same data as growth rates: against .
Figure 2: (A) The 3D U-Net backbone. Numbers give the channel width at each level; the base width of 32 is halved relative to the 64 conventional in 2D because activation memory, not parameter count, binds in 3D. Dashed arrows are skip connections. (B) The three convolutional block variants compared in this work. All share the structure spatial mixing followed by channel mixing and differ only in the spatial stage, so any difference in accuracy is attributable to that stage alone.
| Model | Parameters | Liver Dice | tumor Dice |
|---|---|---|---|
| Ours (hybrid) | 0.947 [0.941, 0.952] | 0.577 [0.518, 0.633] |
Table 2: MSD Liver per-case Dice under five-fold cross-validation over all 131 public volumes. Every volume is held out exactly once. No checkpoint selection is performed: each fold trains for a fixed 200 epochs and the final weights are scored, because the held-out volumes serve as both the validation and the reported set. Post-processing applied as described in Section 3.6. Intervals are 95% percentile bootstrap over cases ( resamples), which is the uncertainty relevant to generalization; see Section 4.2 . Parameter counts are exact.
| Method | Params | Liver | tumor |
|---|---|---|---|
| nnU-Net, 2D [ 13 ] | — | 0.9437 | 0.5394 |
| nnU-Net, 3D low-res only [ 13 ] | — | 0.9469 | 0.4701 |
| nnU-Net, 3D cascade [ 13 ] | — | 0.9538 | 0.5849 |
| nnU-Net, 3D [ 13 ] | — | 0.9411 | 0.6174 |
| nnU-Net, 3D + 3D cascade ensemble [ 13 ] | — | 0.9543 | 0.6182 |
| Dense 3D U-Net (this work, single split) | 0.9140 | 0.4578 |
Table 3: Comparison on MSD Task03_Liver. All rows are five-fold cross-validation over the same 131 public volumes, with no separate held-out set, so the figures are directly comparable. We omit nnU-Net’s challenge-submission entry, which was scored on withheld cases we have no access to. nnU-Net does not report parameter counts for these configurations.
| Fold | Volumes | tumor-free | tumor (all) | tumor (bearing) |
|---|---|---|---|---|
| 0 | 27 | 3 | 0.5801 | 0.5276 |
| 1 | 26 | 1 | 0.6543 | 0.6405 |
| 2 | 26 | 3 | 0.5753 | 0.5634 |
| 3 | 26 | 6 | 0.4872 | 0.4834 |
| 4 | 26 | 0 | 0.5892 | 0.5892 |
| Pooled | 131 | 13 | 0.577 | 0.565 |
Table 4: Per-fold tumor Dice (all-volume LiTS convention). Fold means span to . A permutation test that reassigns the same 131 per-case scores to folds at random reproduces this spread exactly: the observed standard deviation of sits on the null median of ( ), and a fold as low as occurs in of random assignments. The variation between folds is therefore attributable to which cases fell together, not to differences between the trained models.
Figure 3: (a) Per-fold tumor Dice, with the pooled mean (dashed) and its bootstrap interval over cases (shaded). (b) Permutation null: the same 131 per-case scores reassigned to folds of the same sizes at random, times. The observed standard deviation across folds falls at the median of that null, so the fold-to-fold spread carries no information about the trained models, it reflects only which cases fell together.
| Variant | Parameters | Liver | tumor (all 19) | tumor (17 bearing) |
|---|---|---|---|---|
| Dense 3D U-Net | 0.914 | 0.458 | 0.512 | |
| (a) Fixed shift | 0.941 | 0.574 | 0.524 | |
| (b) Learnable depthwise | 0.945 | 0.596 | 0.548 | |
| (c) Hybrid | 0.945 | 0.600 | 0.553 |
Table 5: Block variant ablation on MSD Liver: one seed per variant, identical training, identical post-processing ( voxels), evaluated on the same held-out split of 19 volumes. Comparisons against the dense baseline are paired over volumes. A single seed is used for every row so that the paired structure is preserved; the three-seed figures for the hybrid variant are reported separately in Section 4.9 .
| Seed | Encoder | Deep decoder | Shallow decoder |
|---|---|---|---|
| 0 | 5.47 | 1.94 | 3.80 |
| 1 | 5.66 | 2.04 | 3.87 |
| 2 | 5.47 | 2.17 | 3.90 |
Table 6: Mean number of open taps per depthwise filter under gating ( ), by network depth, on the 2D endoscopy benchmark. Three seeds; per-layer standard deviation across seeds 0.16.
Figure 4: gate allocation on the 2D endoscopy benchmark. (a) Mean open taps per depthwise filter against network depth, at six penalty strengths, with error bars over three seeds. The profile is V-shaped at every : the gates retain spatial capacity in the shallow encoder, shed it fastest at up1.conv1 , the first decoder stage, fed directly by the bottleneck, and restore it through the shallow decoder as resolution returns. The ordering is preserved across a 500-fold range of and across seeds (mean per-layer standard deviation 0.16 at , maximum 0.38). Table 6 reports the grouped means at . (b) Total tap budget against accuracy, by seed. Reducing spatial parameters sevenfold produces no detectable change, while the gap between seeds ( Dice between seed means) is nearly twice the largest variation any produces within a seed ( ).
| Variant | Parameters | Dice |
|---|---|---|
| Dense U-Net | 0.805 0.010 | |
| Fixed shift, with padding projection | 0.798 0.023 | |
| Fixed shift, no projection | 0.794 0.003 | |
| Learnable depthwise | 0.816 0.022 | |
| Hybrid | 0.811 0.004 |
Table 7: 2D endoscopy benchmark (Kvasir-SEG [ 15 ] ), Dice on a fixed held-out split, mean standard deviation over three random initializations. The split is held fixed across all runs ( split_seed , independent of the run seed) so that variation reflects initialization only, and comparisons against the dense baseline are paired by seed.
Figure 5: Kvasir-SEG segmentations from the hybrid configuration ( up1,up2,up3 frozen, parameters), on the fixed test split of 100 images ( split_seed ). Rows are the 5th, 50th and 95th percentile of test Dice, ranked, so they are representative rather than selected: , and . Across the split the mean is but the median is , with a minimum of and a maximum of , as on MSD Liver, most cases are segmented well and a tail of failures accounts for most of the shortfall in the mean.
| Frozen stages | Parameters | Filters frozen | Dice |
|---|---|---|---|
| up1 | 0.819 0.011 | ||
| up1,up2 | 0.810 0.013 | ||
| up1,up2,up3 | 0.815 0.008 | ||
| down4,up1,up2,up3 | 0.808 0.015 | ||
| down3,down4,up1,up2,up3 | 0.814 0.012 |
Table 8: Freezing configuration sweep on the 2D endoscopy benchmark, five seeds each, fixed split. Configurations are cumulative, beginning at the deepest decoder stage. No configuration differs significantly from any other (one-way ANOVA , ), and all exceed the fully frozen model ( ).
Figure 6: Per-case Dice under five-fold cross-validation over all 131 volumes. (a) tumor Dice for every tumor-bearing volume, ranked. The error is not distributed around the mean: a body of adequate segmentations is followed by volumes ( ) in which the model detects no tumor at all, shown in grey. (b) Liver Dice for the same volumes, on the same evaluation, for contrast, case-level standard deviation against for tumor.
Figure 7: Per-case tumor Dice against ground-truth lesion volume, log scale. Open circles are the volumes missed entirely. Performance degrades systematically with decreasing burden (Spearman , , ): the failures are concentrated in small-lesion cases rather than distributed at random.
Figure 8: Representative segmentations spanning the range in Figure 6 . Each row shows one axial slice at the largest tumor cross-section: CT left, ground truth center, prediction right. Top, volume 100, a large confluent lesion, tumor Dice . Middle, volume 122, multiple smaller lesions of which one is recovered, . Bottom, volume 55, a -voxel lesion for which the model predicts no tumor at all, ; the liver is nonetheless segmented accurately at . All panels use a single fixed intensity window spanning the full preprocessed range, HU, so brightness is comparable across the three volumes.
| Volume | CNR | Liver coverage | Recall | Precision | Interpretation |
|---|---|---|---|---|---|
| 39 | 0.02 | 0.00 | 0.05 | lesion isodense with parenchyma | |
| 104 | 3.65 | 0.00 | 0.19 | hypodense lesion excluded from liver | |
| 55 | 5.17 | 0.00 | 0.00 | hypodense lesion excluded from liver | |
| 49 | 1.07 | 0.69 | 0.11 | detected, over-segmented | |
| 21 | 1.07 | 0.42 | 0.42 | genuinely difficult |
Table 9: The weakest test volumes, with contrast-to-noise ratio (CNR) between tumor and surrounding parenchyma, and the fraction of ground-truth tumor covered by the predicted liver region.
| Dense 3D U-Net | Ours | Ratio | |
|---|---|---|---|
| Parameters | |||
| Model file size (float32) | |||
| Training activation memory | 4.58 GB | 6.06 GB | |
| CPU inference, s/patch | 20.6 | 16.8 | |
| CPU inference, whole volume |
Table 10: Measured consequences of the parameter reduction. Inference timed on a single CPU core at , forward pass only; activation memory measured by summing module output tensors during a forward pass at , batch size 1, base width 32. A ratio below one indicates the factorized model is worse on that axis.