Leveraging Segmentation Representations for Efficient Classification: A Self-Configuring Framework Based on nnU-Net
Authors: Renjun Cai, Antonio Javier Gallego, Félix Fuentes-Hurtado
Organizations: Au Reikura School of Computing and Mathematical Sciences, University of Waikato, New Zealand · University Institute for Computing Research, University of Alicante, Spain
We propose a self-configuring framework that extends the nnU-Net workflow to both segmentation and classification in 3D medical imaging. Building on nnU-Net's self-configuring method, the framework reuses representations learned by a segmentation model for image-level classification. Its encoder features and predicted segmentation probabilities are used by a lightweight classifier. Two information-sharing strategies are investigated. In the frozen-backbone branch, features are augmented with region-of-interest information, predicted foreground volume, and segmentation confidence. In the adaptation branch, the final encoder stage is partially fine-tuned for classification, with an auxiliary segmentation loss to preserve segmentation-related information. Additional five-fold experiments showed that more complex configurations did not consistently improve classification performance. On the public validation leaderboard, our selected framework achieved an Overall score of 0.8056. In the official FLARE 2026 Task 2 final evaluation, our framework ranked first overall, achieving the highest DSC, NSD, and AUROC across all four testing tasks, including two hidden testing tasks. Local inference measurements also remained below the challenge limits of 60s per case and 4GB GPU memory. These results demonstrate the practical value of reusing segmentation representations for classification while maintaining a simple and efficient framework.
Figures & tables
Figure 1: Overview of the proposed framework.
Config.
Global
ROI
Volume
Confidence
Trainable components
Aux. seg. loss
A0
✓
–
–
–
Classification head
–
A1
✓
✓
–
–
Classification head
–
A2
✓
✓
✓
–
Classification head
–
A3
✓
✓
✓
✓
Classification head
–
A4
✓
–
–
–
Final encoder stage + classification head
–
A5
✓
–
–
–
Final encoder stage + classification head
✓
Table 1: Summary of the A0-A5 classification configurations.
Table 2: Medical imaging datasets used in this study. Dataset001 to Dataset005, Dataset061, and Dataset062 are the training datasets. Dataset007 and Dataset072 contain the same images but different classification tasks. Dataset008 and Dataset009 are final testing datasets. Hidden testing datasets are not included.
Item
Setting
Operating system
Ubuntu 24.04.1 LTS
CPU
AMD EPYC 9B45, 48 logical CPU cores
RAM
176.9 GiB
GPU
1 × NVIDIA RTX PRO 6000 Blackwell Server Edition, 96 GB
CUDA version
12.8
Programming language
Python 3.12.14
Table 3: Development environment and software requirements.
Name
Details
Batch size
Segmentation: automatically calculated according to the nnU-Net plan and available GPU memory, with a minimum of 2. Classification: 64 for cached features; 1 for image-based processing.
Patch size
Automatically determined from dataset characteristics and available GPU memory using nnU-Net planning.
Total epochs
Segmentation: 2500 for original development; 200 for additional five-fold evaluation. Classification: A0–A3 use 150–250 depending on dataset size and class imbalance; A4–A5 use 100 by default, with optional continuation.
Optimiser
Segmentation: SGD with Nesterov momentum ( 0.99 ) and weight decay ( 3×10−5 ). Classification: AdamW with weight decay determined by the training configuration.
Initial learning rate
Segmentation: 10−2 with polynomial decay. A0–A3: 5×10−4 to 10−3 , depending on dataset size and class imbalance. A4–A5: 10−4 for the classification head and 10−5 for the unfrozen encoder stage by default.
Loss function
Segmentation: Dice and cross-entropy loss. Classification: cross-entropy with class weighting and label smoothing. A5 additionally uses 0.2Lsegaux .
Table 4: Training protocols for the proposed framework.
Figure 2: Qualitative segmentation results for Dataset007&072. Ground-truth and predicted tumour regions are shown in green and red, respectively.
Dataset
Config.
DSC
NSD
BA
AUROC
Overall
EGCT
A0
0.8385 (0.821–0.856)
0.7749 (0.756–0.794)
0.6748 (0.623–0.722)
0.8202 (0.786–0.852)
0.7771 (0.756–0.798)
A1
0.8385 (0.821–0.856)
0.7749 (0.756–0.794)
0.6917 (0.645–0.741)
0.8253 (0.790–0.859)
0.7826 (0.761–0.805)
A2
0.8385 (0.821–0.856)
0.7749 (0.756–0.794)
0.6820 (0.635–0.732)
0.8267 (0.792–0.860)
0.7805 (0.759–0.803)
A3
0.8385 (0.821–0.856)
0.7749 (0.756–0.794)
0.6724 (0.624–0.721)
0.8273 (0.793–0.861)
0.7783 (0.756–0.800)
A4
0.8390 (0.822–0.857)
0.7762 (0.757–0.795)
0.6600 (0.609–0.707)
0.8274 (0.794–0.860)
0.7757 (0.754–0.797)
A5
0.8394 (0.822–0.857)
0.7763 (0.758–0.795)
0.6700 (0.620–0.719)
0.8284 (0.795–0.862)
0.7785 (0.757–0.801)
Table 5: Five-fold evaluation on Dataset008 EGCT and Dataset009 PHLF. Values are reported with 95% confidence intervals. DSC and NSD are case-level means, while BA and AUROC are calculated from pooled out-of-fold predictions. Overall is the mean of the four metrics. Bold indicates the highest observed value for each metric within a dataset.
Dataset
DSC
NSD
BA
AUROC
EGCT
0.8628
0.8158
0.7027
0.8581
PHLF
0.8741
0.8074
0.9189
0.8919
Liver Metastasis PET/CT
0.8362
0.6885
0.2285
0.6861
ICC & HCC CT
0.8426
0.6800
0.6066
0.6875
Table 6: Official FLARE 2026 Task 2 testing results of our submitted framework.
Requirements
Answer
A meaningful title
Yes
The number of authors ( ≤ 6)
5
Author affiliations and ORCID
Yes
Corresponding author email is presented
Yes
Validation scores are presented in the abstract
Yes
Introduction includes at least three parts: background, related work, and motivation
Yes
Table 10: Checklist Table.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Selected config.
DSC
NSD
BA
AUROC
Overall
Baseline Overall
Dataset001 BrainMets
A2
0.8156±0.1285
0.9090±0.0894
0.6104
0.5920
0.7318
0.7475
Dataset002 MU_Glioma_Post
A1
0.7900±0.2944
0.8368±0.2893
0.6553
0.7818
0.7660
0.7443
Dataset003 UCSD_PTGB
A0
0.9222±0.0370
0.9645±0.0249
0.6364
0.6061
0.7823
0.5840
Dataset004 PI-CAI
A3
0.6253±0.4274
0.6080±0.4139
0.3149
0.7248
0.5683
0.5980
Dataset005 LUNA25
A1
0.7722±0.1557
0.8621±0.1795
0.7812
0.8312
0.8117
0.8193
Dataset061 PETWB_Lung
A2
0.9879±0.0033
0.9413±0.0135
0.7472
0.7417
0.8545
0.7758
Appendix
Table 7: Exploratory development results across seven datasets. DSC and NSD are reported as mean ± standard deviation. Overall is the mean of DSC, NSD, BA, and AUROC. Baseline Overall values are from the original baseline results.
Submission ID
Configuration
DSC
NSD
IDH BA
IDH AUROC
Grade BA
Grade AUROC
Overall
889951
Unified classification head
0.8849
0.7777
0.8140
0.8473
0.5832
0.8425
0.7916
890034
Additional radiological features
0.8809
0.7740
0.7598
0.8768
0.5921
0.8383
0.7870
891215
nnSSL pre-training
0.8787
0.7703
0.7305
0.7862
0.4299
0.6907
0.7144
894490
Selected framework (ResEnc-M)
0.8811
0.7750
0.8141
0.8929
0.5811
0.8894
0.8056
895685
Larger backbone (ResEnc-XL)
0.8856
0.7795
0.8282
0.8713
0.5811
0.8665
0.8020
Appendix
Table 8: Public validation leaderboard results on Dataset007 and Dataset072 GliomaIDHType. Overall is the mean of DSC, NSD, IDH BA, IDH AUROC, Grade BA, and Grade AUROC. Bold indicates the highest observed score in each metric.
Dataset
Average runtime (s)
Maximum runtime (s)
Peak GPU memory (MB)
Dataset001 BrainMets
4.18
5.72
2578.32
Dataset002 MU_Glioma_Post
3.62
3.96
2102.84
Dataset003 UCSD_PTGB
15.51
49.47
3158.06
Dataset004 PI-CAI
23.33
57.26
3185.04
Dataset005 LUNA25
8.00
10.45
3645.15
Dataset061 PETWB_Lung
3.89
5.26
3106.72
Appendix
Table 9: Inference runtime and peak GPU memory usage measured on the development machine.
Accurate and efficient 3D medical image segmentation is essential for clinical AI, where models must remain reliable under stringent memory, latency, and data availability constraints. Transformer-based methods achieve strong accuracy but suffer from excessive parameters, high FLOPs, and limited generalization. We propose LightMedSeg, a modular UNet-style segmentation architecture that integrates anatomical priors with adaptive context modeling. Anchor-conditioned FiLM modulation enables anatomy-aware feature calibration, while a local structural prior module and texture-aware routing dynamically allocate representational capacity to boundary-rich regions. Computational redundancy is minimized through ghost and depthwise convolutions, and multi-scale features are adaptively fused via a learned skip router with anchor-relative spatial position bias. Despite requiring only 0.48M parameters and 14.64~GFLOPs, LightMedSeg achieves segmentation accuracy within a few Dice points of heavy transformer baselines. Therefore, LightMedSeg is a deployable and data-efficient solution for 3D medical image segmentation. Code will be released publicly upon acceptance.
Kavyansh Tyagi, Vishwas Rathi, Puneet Goyal
National Institute of Technology Kurukshetra, India · Indian Institute of Technology Ropar, India
We present APRIL-MedSeg, a YAML-driven modular framework for 2D medical image segmentation. It provides a unified and extensible ecosystem that decomposes segmentation networks into reusable components. Also, the framework integrates a broad spectrum of advanced paradigms, including semi-supervised learning, domain adaptation, knowledge distillation, weakly supervised learning, and text-guided segmentation as well as foundation model support. A registry-based configuration system with inheritance enables flexible and reproducible experiment management, supporting seamless switching across models, datasets, and training strategies. In addition, the framework provides a unified interface for medical datasets, augmentation pipelines, deployment utilities and model ensembling. Overall, APRIL-MedSeg is designed as a general-purpose research and development platform that bridges algorithmic innovation and practical deployment, while also serving as a structured ecosystem for systematically organizing and reproducing advances in medical image segmentation. The code is available at https://github.com/juntaoJianggavin/APRIL-MedSeg under an Apache 2.0 license.
Juntao Jiang, Jinsheng Bai, Linxuan Fan +3
Zhejiang University, APRIL Lab · Vanderbilt University · University of Science and Technology of China
We propose parameter-efficient SSM-based U-Net architectures for 3D medical image segmentation. Convolutional U-Nets afford O(n) local mixing per layer but lack explicit global context; transformers provide global reasoning at O(n^2) cost in sequence length n. State-space models (SSMs), such as Mamba, offer O(n) global propagation per block. Yet, existing medical SSM segmenters rely on fixed scan patterns and large parameter budgets. Dynamic Adaptive Scan (DAS), which learns data-dependent reordering before selective scan, has not been applied to medical imaging or extended to 3D volumes. We propose DAMamba-UNet3D, a hybrid encoder-decoder that integrates tri-plane 3D-DAS blocks at encoder stages E2-E4 while retaining convolutions elsewhere (~5.3M parameters). On BraTS 2020 five-fold cross-validation, DAMamba-UNet3D achieves mean Dice 0.815+/-0.013 (full-volume per-case evaluation) at ~13x lower parameter cost than SegMamba (0.824+-0.014, ~70M). At comparable scale, DAMamba-L (~70M), a wide DAS-native variant with encoder-only DAMamba and a convolutional bottleneck, reaches 0.829+-0.012, surpassing retrained SegMamba by 0.5pt. Component ablations show that encoder-only DAS placement is critical as bottleneck and decoder SSM blocks lower Dice. Together, the results suggest that learned tri-plane DAS in a hybrid U-Net is competitive with, and under our large-scale design may improve upon, SegMamba's fixed Tri-orientated Mamba (ToM) scanning on BraTS 2020. Code: https://github.com/marafathussain/DAMamba-UNet3D.
Mohammad Arafat Hussain, Ellen Grant, Yangming Ou
Boston Children’s Hospital, Harvard Medical School