L2GSSL: Label-Efficient Local-to-Global Self-Supervised Learning for Volumetric Healthcare Data

Po-Cheng Cheng; Hung-Hsuan Chen

cs.CVOct 15, 2026

L2GSSL: Label-Efficient Local-to-Global Self-Supervised Learning for Volumetric Healthcare Data

Authors: Po-Cheng Cheng, Hung-Hsuan Chen

Organizations: Computer Science and Information Engineering, National Central University

Abstract

Healthcare institutions routinely accumulate high-dimensional computed tomography (CT) volumes, but only a fraction can be annotated by experts. We present L2GSSL, a local-to-global self-supervised learning framework that leverages unlabeled volumes by decomposing pretraining into intra-slice and inter-slice representation spaces. A slice encoder learns view-invariant local features, and a global encoder aggregates the resulting tokens to model volumetric context. The stop-gradient isolates the local space from the global objective, while slice-wise execution and a three-phase schedule support training under constrained GPU memory. On LIDC-IDRI lung nodule malignancy classification, L2GSSL reaches $0.9229$ AUROC with only $10\%$ of the labels and $0.9377$ with $25\%$ , exceeding the volume-level VICReg by $0.0229$ and $0.0294$ , respectively. With all labels, the strongest supervised baseline exceeds L2GSSL by only $0.0044$ AUROC, consistent with self-supervised pretraining providing its greatest value when annotations are limited. Without adaptation to the external LNDb cohort, a model fine-tuned on $25\%$ of LIDC-IDRI obtains $0.8138$ AUROC and $0.7589$ AUPRC, improving over the strongest evaluated self-supervised baseline by $0.1130$ and $0.0954$ . Experiments across data sources, thoracic and abdominal anatomy, downstream tasks, and two backbones examine healthcare-data variety, while lesion-slice masking shows that the predictions depend on lesion-bearing content.

Figures & tables

Method	Pretraining input	Alignment target	Explicit negatives	Auxiliary network state	Local/global split
MoCo v2 [ 17 ]	Slice-aggregated volume	Final volume embedding	Yes	Momentum encoder and queue	No
SimSiam [ 16 ]	Slice-aggregated volume	Final volume embedding	No	Predictor	No
Volume VICReg [ 4 ]	Slice-aggregated volume	Final volume embedding	No	None	No
VoCo [ 3 ]	Overlapping 3D crops	Volumetric spatial relation	Spatially defined	EMA teacher	No
2D-VoCo [ 15 ]	Individual CT slices	Slice-level spatial-semantic relation	Spatially defined	None	No
L2GSSL	Slices and ordered tokens	Slice and volume embeddings	No	None	Yes

TABLE I: Design comparison of the evaluated SSL baselines and closely related volumetric pretraining approaches. “Local/global split” denotes separate objectives and controlled gradient paths, not merely a hierarchical backbone.

Fig. 1: Computational graph of L2GSSL. Local embeddings are inputs to the global encoder, but the global loss is detached at the local encoder. Solid arrows show forward computation; colored arrows show parameter updates.

Dataset	Anatomy/modality	Available scale	Use in this study	Label use	Evaluation role
LIDC-IDRI [ 5 ]	Thoracic CT	$1{,}018$ cases; $1{,}625$ processed nodules	SSL and downstream training	$104$ , $260$ , or $1{,}040$ training nodules	Internal discrimination and reader analysis
LNDb [ 6 ]	Thoracic CT	$294$ scans	External testing only	No target-domain training	Direct cross-dataset transfer
RSNA 2023 [ 7 ]	Abdominal CT	$3{,}147$ patients	Downstream trauma pipeline	Competition task labels	Multi-organ task transfer
AbdomenCT-1K [ 8 ]	Abdominal CT	Auxiliary volumetric collection	Additional SSL source	Labels not used by SSL	Anatomically aligned pretraining

TABLE II: Datasets and their roles in the evaluation.

Protocol	Encoder update	Supervision	Controlled factor	Primary question
Fixed-batch linear probe	Frozen	LIDC-IDRI labels	Local batch $8$ ; two accumulation steps	Are pretrained features linearly separable?
VRAM-capped linear probe	Frozen	LIDC-IDRI labels	Peak VRAM constrained to $47$ – $48$ GB	How effectively is a fixed memory budget used?
Label-fraction fine-tuning	Full network	$10\%$ , $25\%$ , or $100\%$ of LIDC-IDRI	Shared split and downstream pipeline	Does pretraining reduce dependence on labels?
Direct LNDb evaluation	Frozen at inference	No LNDb training labels	LIDC-IDRI model evaluated without adaptation	Does performance transfer across data sources?
RSNA pipeline integration	Full downstream pipeline	RSNA task labels	Original multi-expert inference design retained	Does initialization transfer to abdominal trauma CT?

TABLE III: Evaluation protocols and the claims they are designed to test.

	$10\%$ labels ( $104$ )		$25\%$ labels ( $260$ )		$100\%$ labels ( $1{,}040$ )
Method	AUROC	AUPRC	AUROC	AUPRC	AUROC	AUPRC
Supervised MST-DINOv2	$0.6564\pm 0.0789$	$0.4557\pm 0.0706$	$0.8695\pm 0.0499$	$0.7875\pm 0.0781$	$\mathbf{0.9448}\pm 0.0070$	$0.9044\pm 0.0070$
MoCo v2	$0.5986\pm 0.0175$	$0.4240\pm 0.0190$	$0.8221\pm 0.0727$	$0.7290\pm 0.1276$	$0.9360\pm 0.0073$	$0.8893\pm 0.0128$
SimSiam	$0.6847\pm 0.1026$	$0.5033\pm 0.1212$	$0.8007\pm 0.1462$	$0.6936\pm 0.1966$	$0.9389\pm 0.0024$	$\mathbf{0.9123}\pm 0.0055$
Volume VICReg	$0.9000\pm 0.0048$	$0.8573\pm 0.0034$	$0.9083\pm 0.0022$	$0.8686\pm 0.0133$	$0.9296\pm 0.0077$	$0.8867\pm 0.0092$
L2GSSL	$\mathbf{0.9229}\pm 0.0100$	$\mathbf{0.8769}\pm 0.0042$	$\mathbf{0.9377}\pm 0.0039$	$\mathbf{0.9003}\pm 0.0064$	$0.9404\pm 0.0024$	$0.9043\pm 0.0038$

TABLE IV: LIDC-IDRI full fine-tuning under identical batch-size constraints. Best values are bold.

Method	AUROC	AUPRC
MoCo v2	$.6749\pm.0039$	$.5086\pm.0012$
SimSiam	$.6226\pm.0092$	$.4545\pm.0167$
Volume VICReg	$.8485\pm.0092$	$.7740\pm.0109$
L2GSSL	$\mathbf{.8875}\pm.0061$	$\mathbf{.8111}\pm.0118$

TABLE V: LIDC-IDRI linear probing with identical batch size.

Method	AUROC	AUPRC
Supervised	$.6869\pm.0553$	$.6289\pm.0632$
MoCo v2	$.6717\pm.0245$	$.6215\pm.0075$
SimSiam	$.6313\pm.1005$	$.5551\pm.1603$
Volume VICReg	$.7008\pm.0154$	$.6635\pm.0515$
L2GSSL	$\mathbf{.8138}\pm.0401$	$\mathbf{.7589}\pm.0461$

TABLE VI: Direct LNDb evaluation after fine-tuning on

25\%

of LIDC-IDRI; no LNDb adaptation.

Method	Peak VRAM (GB)	AUROC	AUPRC
MoCo v2	$47$ – $48$	$.6661\pm.0104$	$.4936\pm.0037$
SimSiam	$47$ – $48$	$.6226\pm.0092$	$.4545\pm.0167$
Volume VICReg	$47$ – $48$	$.8485\pm.0092$	$.7740\pm.0109$
L2GSSL	$47$ – $48$	$\mathbf{.8599}\pm.0038$	$\mathbf{.7926}\pm.0039$

TABLE VII: LIDC-IDRI linear probing with the MST-based methods constrained to a common peak VRAM budget. Best results are bold.

		LP		FT, $10\%$ labels		FT, $25\%$ labels		FT, $100\%$ labels
Method	Peak VRAM (GB)	AUROC	AUPRC	AUROC	AUPRC	AUROC	AUPRC	AUROC	AUPRC
VoCo (scratch)	$27.57$	$.6770\pm.0085$	$.4775\pm.0193$	$.6324\pm.0669$	$.4523\pm.0734$	$.7001\pm.0290$	$.5286\pm.0436$	$.7805\pm.0166$	$.6482\pm.0010$
VoCo (MONAI)	$27.57$	$.6905\pm.0065$	$.5361\pm.0069$	$.6508\pm.0561$	$.4641\pm.0512$	$.7058\pm.0231$	$.5491\pm.0267$	$.7654\pm.0118$	$.6182\pm.0188$
L2GSSL	$24.37$ – $24.48$	$\mathbf{.8875}\pm.0061$	$\mathbf{.8111}\pm.0118$	$\mathbf{.9229}\pm.0100$	$\mathbf{.8769}\pm.0042$	$\mathbf{.9377}\pm.0039$	$\mathbf{.9003}\pm.0064$	$\mathbf{.9404}\pm.0024$	$\mathbf{.9043}\pm.0038$

TABLE VIII: Comparison with the fully 3D VoCo reference on LIDC-IDRI. Linear probing (LP), fine-tuning (FT), and measured peak VRAM are reported for the evaluated configurations. The comparison is not controlled for backbone, input formulation, or initialization. Best discrimination results are bold.

Method	AUROC	AUPRC
MoCo v2	$.6999\pm.0513$	$.6842\pm.0357$
SimSiam	$.7510\pm.0054$	$.7253\pm.0057$
Volume VICReg	$.7567\pm.0562$	$.7400\pm.0542$
L2GSSL	$\mathbf{.8919}\pm.0051$	$\mathbf{.8850}\pm.0034$

TABLE IX: LIDC-IDRI linear probing with the MaxViT backbone. Best results are bold.

Condition	Malignant TPR	AUROC	AUPRC
No mask	$87.84\%$	$0.9404$	$0.9043$
Tumor-slice mask	$14.57\%$	$0.6169$	$0.4307$
Change	$-73.27\%$	$-0.3235$	$-0.4736$

TABLE X: Effect of masking malignant lesion-bearing slices.

Panel of Figure 12 — Fig. 2: Per-nodule held-out-radiologist ROC analysis under two treatments of an uncertain score from the selected radiologist. Each reference label is constructed from the remaining radiologists. The diamond shows the sensitivity and false-positive rate computed after pooling the held-out judgments across nodules; curves show L2GSSL runs.

Initialization	Configuration	AUROC	AUPRC
Random	Full L2GSSL	$0.5968\pm 0.0011$	$0.3938\pm 0.0026$
Random	Without stop-gradient	$0.5767\pm 0.0202$	$0.3757\pm 0.0054$
Random	Without phased schedule	$0.5877\pm 0.0191$	$0.3801\pm 0.0183$
DINOv2	Full L2GSSL	$0.8599\pm 0.0038$	$0.7926\pm 0.0039$
DINOv2	Without stop-gradient	$0.5749\pm 0.0299$	$0.3806\pm 0.0207$
DINOv2	Without phased schedule	$0.8967\pm 0.0028$	$0.8260\pm 0.0060$

TABLE XI: Linear-probing component ablation on LIDC-IDRI. Results are mean

\pm

standard deviation; the best result within each initialization is bold.

Lung Nodule Detection

See all 10 papers

NEMESIS: Noise-suppressed Efficient MAE with Enhanced Superpatch Integration Strategy
Apr 2, 2026Kyeonghun Kim, Hyeonseok Jung, Youngung Han +9Medical Image Representation Learning Masked Autoencoders
OUTTA · Chung-Ang University · Seoul National University +3
DALE-CT: Depth-Aware Foundation Models for Computed Tomography
Jun 5, 2026Evan W. Damron, Mahmut S. Gokmen, Mitchell A. Klusty +3Medical Imaging Foundation Models Self-Supervised Pre-Training
College of Medicine Office of Research, University of Kentucky, Lexington, KY 40506 USA
Where to Look Matters: Rethinking Sub-Volume Sampling in 3D Medical Self-supervised Learning
Oct 9, 2026Junkai Liu, Le ZhangMedical Image Representation Learning Self-Supervised Pre-Training
School of Engineering, University of Birmingham

Apr 2, 2026J/K move · Enter open · S save

NEMESIS: Noise-suppressed Efficient MAE with Enhanced Superpatch Integration Strategy

Kyeonghun Kim, Hyeonseok Jung, Youngung Han, Hyunsu Go+8

OUTTA · Chung-Ang University · Seoul National University+3

Medical Image Representation Learning Masked Autoencoders Self-Supervised Pre-Training 3D ViTs Medical Image Classification

Open paper AskPDF arXiv

Volumetric CT imaging is essential for clinical diagnosis, yet annotating 3D volumes is expensive and time-consuming, motivating self-supervised learning (SSL) from unlabeled data. However, applying SSL to 3D CT remains challenging due to the high memory cost of full-volume transformers and the anisotropic spatial structure of CT data, which is not well captured by conventional masking strategies. We propose NEMESIS, a masked autoencoder (MAE) framework that operates on local 128x128x128 superpatches, enabling memory-efficient training while preserving anatomical detail. NEMESIS introduces three key components: (i) noise-enhanced reconstruction as a pretext task, (ii) Masked Anatomical Transformer Blocks (MATB) that perform dual-masking through parallel plane-wise and axis-wise token removal, and (iii) NEMESIS Tokens (NT) for cross-scale context aggregation. On the BTCV multi-organ classification benchmark, NEMESIS with a frozen backbone and a linear classifier achieves a mean AUROC of 0.9633, surpassing fully fine-tuned SuPreM (0.9493) and VoCo (0.9387). Under a low-label regime with only 10% of available annotations, it retains an AUROC of 0.9075, demonstrating strong label efficiency. Furthermore, the superpatch-based design reduces computational cost to 31.0 GFLOPs per forward pass, compared to 985.8 GFLOPs for the full-volume baseline, providing a scalable and robust foundation for 3D medical imaging.

Abstract

Lung Nodule Detection

Explore similar work

NEMESIS: Noise-suppressed Efficient MAE with Enhanced Superpatch Integration Strategy

DALE-CT: Depth-Aware Foundation Models for Computed Tomography

Where to Look Matters: Rethinking Sub-Volume Sampling in 3D Medical Self-supervised Learning