DINOv2 for PET: A Pre-Training Domain Benchmark Across Classification, Segmentation, and Prognosis

Roxanne De Maeyer; Yannick Brackenier; Vicky De Ridder; Simon DeKeyser; Maarten Larmuseau; Jef Vandemeulebroucke

cs.CVOct 29, 2026

DINOv2 for PET: A Pre-Training Domain Benchmark Across Classification, Segmentation, and Prognosis

Authors: Roxanne De Maeyer, Yannick Brackenier, Vicky De Ridder, Simon DeKeyser, Maarten Larmuseau, Jef Vandemeulebroucke

Organizations: Nuclivision bv, Edward Pynaertkaai 87, 9000 Gent, Belgium · Vrije Universiteit Brussel (VUB), Pleinlaan 2, 1050 Brussel, Belgium · imec, Kapeldreef 75, 3001 Leuven, Belgium

Abstract

DINO (self-DIstillation with NO labels) has become a stan- dard for self-supervised pre-training, producing features that generalise across datasets and downstream tasks. In medical imaging it has been adopted mainly for radiology (X-ray, CT and MRI), where even natural- image DINO features transfer well. In the literature, these features have been reported to transfer worse to functional imaging such as PET. Yet PET-specific DINO models have barely been introduced, and even less validated across multiple tasks and datasets. We present PET-DINOv2-2D, the first DINOv2 model pre-trained on 2D PET, and benchmark it against open-source encoders: the origi- nal DINOv2 (pre-trained on the LVD-142M natural-image dataset) and OmniRad (pre-trained on CT, MRI and ultrasound). Using the same lightweight heads for every backbone, and despite far less pre-training data, PET-DINOv2-2D matches or outperforms both on three down- stream PET tasks: normal/abnormal classification, lesion segmentation and recurrence-free survival prognosis. Its advantage is largest when few labels are available, suggesting that PET pre-training gives more transferable, label-efficient features. Code is made available at https://github.com/nuclivision/pet-dinov2-2d/.

Figures & tables

Figure 1: Study overview. We ask whether the pre-training domain of a DINO encoder governs how well its frozen features transfer to PET. Three ViT-S encoders share the same architecture but differ in pre-training data: natural images (DINOv2), CT/MRI (OmniRad) and PET (our PET-DINOv2-2D). Each is used as a frozen feature extractor for three downstream PET tasks: patient-level Normal/Abnormal classification, lesion segmentation, and recurrence-free survival prognosis. Every encoder is paired with the same lightweight task head.

Dataset name	PET tracer	# Subjects	# Scans
ACRIN-NSCLC [ 14 ]	FDG	242	435
AutoPET [ 7 ]	FDG	1014	1014
HECKTOR [ 9 ]	FDG	680	680
NSCLC-Radiogenomics [ 3 ]	FDG	211	240
UDPET [ 15 ]	FDG	1447	1447
Private datasets	FDG, PSMA, DOTA	378	378

Table 1: Pre-training data

	Centre 1	Centre 2	Centre 3
Dataset size (patients)	521	692	698
Number of abnormal patients (AN)	238 (45.7%)	347 (50.1%)	349 (50.0%)

Table 2: Downstream classification dataset

Figure 2: (a) Classification AUC vs. downstream data fraction on the triage PET-MIP task, (b) Held-out test Dice vs. labelled-data fraction on the DeepPSMA lesion task. Colour = backbone. Bands are $\pm 1$ SD over 8 training seeds in (a) and $\pm$ std across the 5 cross-validation models in (b).

	Image only		$\Delta$ vs. PET-DINOv2 (image only)	Img+clin.
Encoder	Har.	Uno [95% CI]	$\Delta$ Uno ( $p$ )	Har. / Uno
PET-DINOv2-2D	0.610	0.617 [.566,.667]	—	0.622 / 0.634
OmniRad	0.552	0.564 [.511,.618]	$-.054$ ( .043 ∗ )	0.596 / 0.606
DINOv2-natural	0.547	0.558 [.505,.611]	$-.059$ ( .022 ∗ )	0.584 / 0.596
Clinical only (reference): Harrell 0.618, Uno 0.623

Table 3: Recurrence-free survival discrimination on HECKTOR (centre-grouped 5-fold CV; PCA

{=}16

). Image-only C-indices are pooled out-of-fold; 95% CIs and

p

(vs. PET-DINOv2-2D) are from a paired patient bootstrap. Best image-only value in bold ;

{}^{\ast}p<0.05

. Harrell-based

\Delta

(

p

) vs. PET-DINOv2-2D: DINOv2-natural

+.063

(.016), OmniRad

+.058

(.028).

Medical Image Analysis

Medical image analysis uses computational methods to interpret clinical images, measure anatomy and disease, and support diagnosis and treatment.

See all 295 papers

SegDINO: Introducing Multi-Scale Structure into DINO for Efficient Medical Image Segmentation
Jun 16, 2026Sicheng Yang, Hongqiu Wang, Zhaohu Xing +5Medical Image Segmentation
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Department of Radiology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou 510060, China · Department of Pancreatobiliary Surgery, State Key Laboratory of Oncology in South China, Sun Yat-sen University Cancer Center, Guangzhou, Guangdong, China +1
BRACHIO: Broad Representations Across Centres for Head-and-neck Imaging Oncology (HECKTOR 2026)
Oct 29, 2026Roxanne De Maeyer, Ine Dirks, Yannick Brackenier +3Oncology PET/CT Tumor Segmentation
Nuclivision bv, Edward Pynaertkaai 87, 9000 Gent, Belgium · Vrije Universiteit Brussel (VUB), Pleinlaan 2, 1050 Brussel, Belgium · imec, Kapeldreef 75, 3001 Leuven, Belgium
BrainDINO: A Brain MRI Foundation Model for Generalizable Clinical Representation Learning
Apr 30, 2026Yizhou Wu, Shansong Wang, Yuheng Li +5Medical Imaging Foundation Models Self-Supervised Pre-Training
Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA, USA. · Department of Radiation and Cellular Oncology, The University of Chicago, Chicago, IL, USA. · Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, GA, USA. +2

Jun 16, 2026J/K move · Enter open · S save

SegDINO: Introducing Multi-Scale Structure into DINO for Efficient Medical Image Segmentation

Sicheng Yang, Hongqiu Wang, Zhaohu Xing, Sixiang Chen+4

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Department of Radiology, State Key Laboratory of Oncology in South China, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Guangzhou 510060, China · Department of Pancreatobiliary Surgery, State Key Laboratory of Oncology in South China, Sun Yat-sen University Cancer Center, Guangzhou, Guangdong, China+1

Medical Image Segmentation

Open paper AskPDF arXiv

Self-supervised DINO models provide strong transferable visual representations, yet applying them directly to image segmentation remains challenging. Existing approaches commonly rely on heavy decoders with complex upsampling, introducing substantial parameter and computational overhead. We observe that introducing scale into DINO features is far more critical than increasing decoder capacity. In this work, we present SegDINO, an efficient segmentation framework that integrates a DINOv3 backbone with lightweight scale modeling. SegDINO introduces Token Pyramid Adaptation (TPA) to reorganize intermediate DINO features into a pseudo multi-scale hierarchy, and Scale-Aware Decoding (SAD) for efficient intra-scale refinement and top-down multi-scale propagation. We further curate PanCT, a new CT dataset containing 284 patients with expert-annotated pancreatic tumors, to assess SegDINO's ability to handle difficult small-lesion cases. Extensive experiments on PanCT and three public benchmarks demonstrate that SegDINO achieves state-of-the-art results with high efficiency. The code is available at https://github.com/script-Yang/segdino_v2.

Abstract

Medical Image Analysis

Explore similar work

SegDINO: Introducing Multi-Scale Structure into DINO for Efficient Medical Image Segmentation

BRACHIO: Broad Representations Across Centres for Head-and-neck Imaging Oncology (HECKTOR 2026)

BrainDINO: A Brain MRI Foundation Model for Generalizable Clinical Representation Learning