cs.CVNov 5, 2026

Ultra-Lightweight 3D Medical Image Segmentation via Knowledge Distillation in Label-Scarce Domains

Authors: Rahim Ullah, Hasan Abbas, Syed Waqar Nabi

Organizations: School of Computing, University of Glasgow, Glasgow, United Kingdom · James Watt School of Engineering, University of Glasgow, Glasgow, United Kingdom

Abstract

Mamba is a neural network architecture that reduces computational complexity, and has been shown to be effective for 3D medical image segmentation. However, existing models still remain computationally expensive. We propose compressing Mamba by distillation while achieving competitive performance. Specifically, we demonstrate compression of a 139M-parameter SegMamba-V2 teacher into a lightweight TinyUNet3D student with about 0.96M parameters, achieving about 145×\times compression with a performance drop of just over one Dice point. We further investigate the utility of knowledge distillation in a range of data-labeling scenarios under identical experimental settings, and show that its advantage is most prominent when labeled data is scarce. To demonstrate the generalizability of our approach, experiments are conducted on three datasets: BraTS 2023, MSD Spleen and MSD Hippocampus. The code for TinyUNet3D is publicly available at: https://github.com/2802990u/tinyunet3d-dist-kd.

Figures & tables

Explore similar work

CardsList
  1. GMD-Mamba: Lightweight 3D Medical Image Segmentation Using a Grouped Multi-Directional Mamba

    Nov 5, 2026Rahim Ullah, Hasan Abbas, Hazrat Ali +1Medical Image Segmentation3D Medical Image Segmentation

  2. Detail Consistent Stage-Wise Distillation for Efficient 3D MRI Segmentation

    May 25, 2026Mengchen Fan, Baocheng Geng, Xi Xiao +5Image SegmentationMedical Image Segmentation

  3. DAMamba-UNet3D: A Parameter-Efficient Mamba State Space U-Net with Dynamic Adaptive Scan for 3D Medical Image Segmentation

    Jul 21, 2026Mohammad Arafat Hussain, Ellen Grant, Yangming OuMambaU-Net