DINOv2 for PET: A Pre-Training Domain Benchmark Across Classification, Segmentation, and Prognosis
Organizations: Nuclivision bv, Edward Pynaertkaai 87, 9000 Gent, Belgium · Vrije Universiteit Brussel (VUB), Pleinlaan 2, 1050 Brussel, Belgium · imec, Kapeldreef 75, 3001 Leuven, Belgium
Abstract
DINO (self-DIstillation with NO labels) has become a stan- dard for self-supervised pre-training, producing features that generalise across datasets and downstream tasks. In medical imaging it has been adopted mainly for radiology (X-ray, CT and MRI), where even natural- image DINO features transfer well. In the literature, these features have been reported to transfer worse to functional imaging such as PET. Yet PET-specific DINO models have barely been introduced, and even less validated across multiple tasks and datasets. We present PET-DINOv2-2D, the first DINOv2 model pre-trained on 2D PET, and benchmark it against open-source encoders: the origi- nal DINOv2 (pre-trained on the LVD-142M natural-image dataset) and OmniRad (pre-trained on CT, MRI and ultrasound). Using the same lightweight heads for every backbone, and despite far less pre-training data, PET-DINOv2-2D matches or outperforms both on three down- stream PET tasks: normal/abnormal classification, lesion segmentation and recurrence-free survival prognosis. Its advantage is largest when few labels are available, suggesting that PET pre-training gives more transferable, label-efficient features. Code is made available at https://github.com/nuclivision/pet-dinov2-2d/.
Figures & tables
| Dataset name | PET tracer | # Subjects | # Scans |
|---|---|---|---|
| ACRIN-NSCLC [ 14 ] | FDG | 242 | 435 |
| AutoPET [ 7 ] | FDG | 1014 | 1014 |
| HECKTOR [ 9 ] | FDG | 680 | 680 |
| NSCLC-Radiogenomics [ 3 ] | FDG | 211 | 240 |
| UDPET [ 15 ] | FDG | 1447 | 1447 |
| Private datasets | FDG, PSMA, DOTA | 378 | 378 |
| Centre 1 | Centre 2 | Centre 3 | |
|---|---|---|---|
| Dataset size (patients) | 521 | 692 | 698 |
| Number of abnormal patients (AN) | 238 (45.7%) | 347 (50.1%) | 349 (50.0%) |
| Image only | vs. PET-DINOv2 (image only) | Img+clin. | ||
| Encoder | Har. | Uno [95% CI] | Uno ( ) | Har. / Uno |
| PET-DINOv2-2D | 0.610 | 0.617 [.566,.667] | — | 0.622 / 0.634 |
| OmniRad | 0.552 | 0.564 [.511,.618] | ( .043 ∗ ) | 0.596 / 0.606 |
| DINOv2-natural | 0.547 | 0.558 [.505,.611] | ( .022 ∗ ) | 0.584 / 0.596 |
| Clinical only (reference): Harrell 0.618, Uno 0.623 | ||||
Medical Image Analysis
Medical image analysis uses computational methods to interpret clinical images, measure anatomy and disease, and support diagnosis and treatment.
See all 295 papers