L2GSSL: Label-Efficient Local-to-Global Self-Supervised Learning for Volumetric Healthcare Data

Po-Cheng Cheng; Hung-Hsuan Chen

cs.CVOct 15, 2026

L2GSSL: Label-Efficient Local-to-Global Self-Supervised Learning for Volumetric Healthcare Data

Authors: Po-Cheng Cheng, Hung-Hsuan Chen

Organizations: Computer Science and Information Engineering, National Central University

Abstract

Healthcare institutions routinely accumulate high-dimensional computed tomography (CT) volumes, but only a fraction can be annotated by experts. We present L2GSSL, a local-to-global self-supervised learning framework that leverages unlabeled volumes by decomposing pretraining into intra-slice and inter-slice representation spaces. A slice encoder learns view-invariant local features, and a global encoder aggregates the resulting tokens to model volumetric context. The stop-gradient isolates the local space from the global objective, while slice-wise execution and a three-phase schedule support training under constrained GPU memory. On LIDC-IDRI lung nodule malignancy classification, L2GSSL reaches $0.9229$ AUROC with only $10\%$ of the labels and $0.9377$ with $25\%$ , exceeding the volume-level VICReg by $0.0229$ and $0.0294$ , respectively. With all labels, the strongest supervised baseline exceeds L2GSSL by only $0.0044$ AUROC, consistent with self-supervised pretraining providing its greatest value when annotations are limited. Without adaptation to the external LNDb cohort, a model fine-tuned on $25\%$ of LIDC-IDRI obtains $0.8138$ AUROC and $0.7589$ AUPRC, improving over the strongest evaluated self-supervised baseline by $0.1130$ and $0.0954$ . Experiments across data sources, thoracic and abdominal anatomy, downstream tasks, and two backbones examine healthcare-data variety, while lesion-slice masking shows that the predictions depend on lesion-bearing content.

Figures & tables

Method	Pretraining input	Alignment target	Explicit negatives	Auxiliary network state	Local/global split
MoCo v2 [ 17 ]	Slice-aggregated volume	Final volume embedding	Yes	Momentum encoder and queue	No
SimSiam [ 16 ]	Slice-aggregated volume	Final volume embedding	No	Predictor	No
Volume VICReg [ 4 ]	Slice-aggregated volume	Final volume embedding	No	None	No
VoCo [ 3 ]	Overlapping 3D crops	Volumetric spatial relation	Spatially defined	EMA teacher	No
2D-VoCo [ 15 ]	Individual CT slices	Slice-level spatial-semantic relation	Spatially defined	None	No
L2GSSL	Slices and ordered tokens	Slice and volume embeddings	No	None	Yes

TABLE I: Design comparison of the evaluated SSL baselines and closely related volumetric pretraining approaches. “Local/global split” denotes separate objectives and controlled gradient paths, not merely a hierarchical backbone.

Fig. 1: Computational graph of L2GSSL. Local embeddings are inputs to the global encoder, but the global loss is detached at the local encoder. Solid arrows show forward computation; colored arrows show parameter updates.

Dataset	Anatomy/modality	Available scale	Use in this study	Label use	Evaluation role
LIDC-IDRI [ 5 ]	Thoracic CT	$1{,}018$ cases; $1{,}625$ processed nodules	SSL and downstream training	$104$ , $260$ , or $1{,}040$ training nodules	Internal discrimination and reader analysis
LNDb [ 6 ]	Thoracic CT	$294$ scans	External testing only	No target-domain training	Direct cross-dataset transfer
RSNA 2023 [ 7 ]	Abdominal CT	$3{,}147$ patients	Downstream trauma pipeline	Competition task labels	Multi-organ task transfer
AbdomenCT-1K [ 8 ]	Abdominal CT	Auxiliary volumetric collection	Additional SSL source	Labels not used by SSL	Anatomically aligned pretraining

TABLE II: Datasets and their roles in the evaluation.

Protocol	Encoder update	Supervision	Controlled factor	Primary question
Fixed-batch linear probe	Frozen	LIDC-IDRI labels	Local batch $8$ ; two accumulation steps	Are pretrained features linearly separable?
VRAM-capped linear probe	Frozen	LIDC-IDRI labels	Peak VRAM constrained to $47$ – $48$ GB	How effectively is a fixed memory budget used?
Label-fraction fine-tuning	Full network	$10\%$ , $25\%$ , or $100\%$ of LIDC-IDRI	Shared split and downstream pipeline	Does pretraining reduce dependence on labels?
Direct LNDb evaluation	Frozen at inference	No LNDb training labels	LIDC-IDRI model evaluated without adaptation	Does performance transfer across data sources?
RSNA pipeline integration	Full downstream pipeline	RSNA task labels	Original multi-expert inference design retained	Does initialization transfer to abdominal trauma CT?

TABLE III: Evaluation protocols and the claims they are designed to test.

	$10\%$ labels ( $104$ )		$25\%$ labels ( $260$ )		$100\%$ labels ( $1{,}040$ )
Method	AUROC	AUPRC	AUROC	AUPRC	AUROC	AUPRC
Supervised MST-DINOv2	$0.6564\pm 0.0789$	$0.4557\pm 0.0706$	$0.8695\pm 0.0499$	$0.7875\pm 0.0781$	$\mathbf{0.9448}\pm 0.0070$	$0.9044\pm 0.0070$
MoCo v2	$0.5986\pm 0.0175$	$0.4240\pm 0.0190$	$0.8221\pm 0.0727$	$0.7290\pm 0.1276$	$0.9360\pm 0.0073$	$0.8893\pm 0.0128$
SimSiam	$0.6847\pm 0.1026$	$0.5033\pm 0.1212$	$0.8007\pm 0.1462$	$0.6936\pm 0.1966$	$0.9389\pm 0.0024$	$\mathbf{0.9123}\pm 0.0055$
Volume VICReg	$0.9000\pm 0.0048$	$0.8573\pm 0.0034$	$0.9083\pm 0.0022$	$0.8686\pm 0.0133$	$0.9296\pm 0.0077$	$0.8867\pm 0.0092$
L2GSSL	$\mathbf{0.9229}\pm 0.0100$	$\mathbf{0.8769}\pm 0.0042$	$\mathbf{0.9377}\pm 0.0039$	$\mathbf{0.9003}\pm 0.0064$	$0.9404\pm 0.0024$	$0.9043\pm 0.0038$

TABLE IV: LIDC-IDRI full fine-tuning under identical batch-size constraints. Best values are bold.

Method	AUROC	AUPRC
MoCo v2	$.6749\pm.0039$	$.5086\pm.0012$
SimSiam	$.6226\pm.0092$	$.4545\pm.0167$
Volume VICReg	$.8485\pm.0092$	$.7740\pm.0109$
L2GSSL	$\mathbf{.8875}\pm.0061$	$\mathbf{.8111}\pm.0118$

TABLE V: LIDC-IDRI linear probing with identical batch size.

Method	AUROC	AUPRC
Supervised	$.6869\pm.0553$	$.6289\pm.0632$
MoCo v2	$.6717\pm.0245$	$.6215\pm.0075$
SimSiam	$.6313\pm.1005$	$.5551\pm.1603$
Volume VICReg	$.7008\pm.0154$	$.6635\pm.0515$
L2GSSL	$\mathbf{.8138}\pm.0401$	$\mathbf{.7589}\pm.0461$

TABLE VI: Direct LNDb evaluation after fine-tuning on

25\%

of LIDC-IDRI; no LNDb adaptation.

Method	Peak VRAM (GB)	AUROC	AUPRC
MoCo v2	$47$ – $48$	$.6661\pm.0104$	$.4936\pm.0037$
SimSiam	$47$ – $48$	$.6226\pm.0092$	$.4545\pm.0167$
Volume VICReg	$47$ – $48$	$.8485\pm.0092$	$.7740\pm.0109$
L2GSSL	$47$ – $48$	$\mathbf{.8599}\pm.0038$	$\mathbf{.7926}\pm.0039$

TABLE VII: LIDC-IDRI linear probing with the MST-based methods constrained to a common peak VRAM budget. Best results are bold.

		LP		FT, $10\%$ labels		FT, $25\%$ labels		FT, $100\%$ labels
Method	Peak VRAM (GB)	AUROC	AUPRC	AUROC	AUPRC	AUROC	AUPRC	AUROC	AUPRC
VoCo (scratch)	$27.57$	$.6770\pm.0085$	$.4775\pm.0193$	$.6324\pm.0669$	$.4523\pm.0734$	$.7001\pm.0290$	$.5286\pm.0436$	$.7805\pm.0166$	$.6482\pm.0010$
VoCo (MONAI)	$27.57$	$.6905\pm.0065$	$.5361\pm.0069$	$.6508\pm.0561$	$.4641\pm.0512$	$.7058\pm.0231$	$.5491\pm.0267$	$.7654\pm.0118$	$.6182\pm.0188$
L2GSSL	$24.37$ – $24.48$	$\mathbf{.8875}\pm.0061$	$\mathbf{.8111}\pm.0118$	$\mathbf{.9229}\pm.0100$	$\mathbf{.8769}\pm.0042$	$\mathbf{.9377}\pm.0039$	$\mathbf{.9003}\pm.0064$	$\mathbf{.9404}\pm.0024$	$\mathbf{.9043}\pm.0038$

TABLE VIII: Comparison with the fully 3D VoCo reference on LIDC-IDRI. Linear probing (LP), fine-tuning (FT), and measured peak VRAM are reported for the evaluated configurations. The comparison is not controlled for backbone, input formulation, or initialization. Best discrimination results are bold.

Method	AUROC	AUPRC
MoCo v2	$.6999\pm.0513$	$.6842\pm.0357$
SimSiam	$.7510\pm.0054$	$.7253\pm.0057$
Volume VICReg	$.7567\pm.0562$	$.7400\pm.0542$
L2GSSL	$\mathbf{.8919}\pm.0051$	$\mathbf{.8850}\pm.0034$

TABLE IX: LIDC-IDRI linear probing with the MaxViT backbone. Best results are bold.

Condition	Malignant TPR	AUROC	AUPRC
No mask	$87.84\%$	$0.9404$	$0.9043$
Tumor-slice mask	$14.57\%$	$0.6169$	$0.4307$
Change	$-73.27\%$	$-0.3235$	$-0.4736$

TABLE X: Effect of masking malignant lesion-bearing slices.

Panel of Figure 12 — Fig. 2: Per-nodule held-out-radiologist ROC analysis under two treatments of an uncertain score from the selected radiologist. Each reference label is constructed from the remaining radiologists. The diamond shows the sensitivity and false-positive rate computed after pooling the held-out judgments across nodules; curves show L2GSSL runs.

Initialization	Configuration	AUROC	AUPRC
Random	Full L2GSSL	$0.5968\pm 0.0011$	$0.3938\pm 0.0026$
Random	Without stop-gradient	$0.5767\pm 0.0202$	$0.3757\pm 0.0054$
Random	Without phased schedule	$0.5877\pm 0.0191$	$0.3801\pm 0.0183$
DINOv2	Full L2GSSL	$0.8599\pm 0.0038$	$0.7926\pm 0.0039$
DINOv2	Without stop-gradient	$0.5749\pm 0.0299$	$0.3806\pm 0.0207$
DINOv2	Without phased schedule	$0.8967\pm 0.0028$	$0.8260\pm 0.0060$

TABLE XI: Linear-probing component ablation on LIDC-IDRI. Results are mean

\pm

standard deviation; the best result within each initialization is bold.

Lung Nodule Detection

See all 10 papers

Apr 2, 2026

NEMESIS: Noise-suppressed Efficient MAE with Enhanced Superpatch Integration Strategy

Volumetric CT imaging is essential for clinical diagnosis, yet annotating 3D volumes is expensive and time-consuming, motivating self-supervised learning (SSL) from unlabeled data. However, applying SSL to 3D CT remains challenging due to the high memory cost of full-volume transformers and the anisotropic spatial structure of CT data, which is not well captured by conventional masking strategies. We propose NEMESIS, a masked autoencoder (MAE) framework that operates on local 128x128x128 superpatches, enabling memory-efficient training while preserving anatomical detail. NEMESIS introduces three key components: (i) noise-enhanced reconstruction as a pretext task, (ii) Masked Anatomical Transformer Blocks (MATB) that perform dual-masking through parallel plane-wise and axis-wise token removal, and (iii) NEMESIS Tokens (NT) for cross-scale context aggregation. On the BTCV multi-organ classification benchmark, NEMESIS with a frozen backbone and a linear classifier achieves a mean AUROC of 0.9633, surpassing fully fine-tuned SuPreM (0.9493) and VoCo (0.9387). Under a low-label regime with only 10% of available annotations, it retains an AUROC of 0.9075, demonstrating strong label efficiency. Furthermore, the superpatch-based design reduces computational cost to 31.0 GFLOPs per forward pass, compared to 985.8 GFLOPs for the full-volume baseline, providing a scalable and robust foundation for 3D medical imaging.

Kyeonghun Kim, Hyeonseok Jung, Youngung Han +9

OUTTA · Chung-Ang University · Seoul National University +3

Medical Image Representation Learning Masked Autoencoders Self-Supervised Pre-Training 3D ViTs Medical Image Classification

Ask PDF

Jun 5, 2026

DALE-CT: Depth-Aware Foundation Models for Computed Tomography

Recent breakthroughs in self-supervised learning (SSL), such as the Latent-Euclidean Joint-Embedding Predictive Architecture (LeJEPA), alongside successes in integrating visual encoders with language models, have driven the demand for adaptable, high-capacity vision encoders in Computed Tomography (CT). In this work, we explore 2D slice-based architectures as a flexible alternative to native 3D models for processing volumetric CT data. Using the CT-RATE dataset, we trained DALE-CT (Depth-Aware Latent-Euclidean Computed Tomography), a 2D model family built entirely from scratch using LeJEPA, and compared its performance against a continually pre-trained DINOv2 baseline. To enhance representation quality, we developed a novel 3D depth-aware pre-training strategy anchored by dense auxiliary supervision from both automated anatomical masks and human-annotated abnormalities. Under linear probe evaluation with Multiple Instance Learning (MIL) for multi-abnormality detection, the frozen backbone of this dual-supervised model (DALE-CT-2S) achieves a Macro AUROC of 0.833. This performance demonstrates near-parity with state-of-the-art 3D vision-language models, achieved entirely from scratch with significantly less data and no textual supervision. To ensure reproducibility, all training code, evaluation scripts, and model weights have been made publicly available.

Evan W. Damron, Mahmut S. Gokmen, Mitchell A. Klusty +3

College of Medicine Office of Research, University of Kentucky, Lexington, KY 40506 USA

Medical Imaging Foundation Models Self-Supervised Pre-Training Medical Image Anomaly Detection Computed Tomography

Ask PDF

Oct 9, 2026

Where to Look Matters: Rethinking Sub-Volume Sampling in 3D Medical Self-supervised Learning

Self-supervised learning on 3D medical images commonly relies on random sub-volume sampling during pretraining. However, random cropping implicitly treats all spatial regions as equally worth observing, despite anatomical priors and spatial redundancy that make regions inherently unequal in their value for representation learning. To this end, we first show through controlled crop-quality manipulations that better sub-volumes lead to better downstream transfer, revealing the observation policy as an overlooked bottleneck in 3D medical SSL. Motivated by this finding, we revisit sub-volume sampling as a where-to-look problem. We propose VolumeProbe, a plug-and-play framework that replaces random cropping with an offline coarse-to-fine probe bank guided by anatomy-awareness, informativeness, and diversity. We instantiate VolumeProbe as a lightweight Lite variant using geometric and image-statistical cues, and a diagnostic Feat variant using frozen visual features, to examine whether lightweight volume-intrinsic priors suffice or richer semantic cues reliably provide additional value. Extensive experiments across multiple 3D medical SSL backbones and downstream tasks demonstrate that VolumeProbe-Lite consistently improves transfer performance over random sampling. Mechanistic analyses further link these gains to more anatomically plausible and informative observations, reduced spatial redundancy, more efficient learning dynamics, and more structured representations.

Junkai Liu, Le Zhang

School of Engineering, University of Birmingham

Medical Image Representation Learning Self-Supervised Pre-Training 3D Medical Imaging

Ask PDF

Abstract

Lung Nodule Detection

Explore similar work

NEMESIS: Noise-suppressed Efficient MAE with Enhanced Superpatch Integration Strategy

DALE-CT: Depth-Aware Foundation Models for Computed Tomography

Where to Look Matters: Rethinking Sub-Volume Sampling in 3D Medical Self-supervised Learning