cs.CVNov 17, 2026

Impact of Pretraining Length, Model Size, and Annotation Budget on Downstream Clinical Tasks of Foundation Models in Radiotherapy

Authors: Alejandro Cortina Uribe, Sidsel Højklint Poulsen, Ivan Richter Vogelius, Jens Petersen

Organizations: Department of Computer Science, University of Copenhagen, Denmark · Department of Oncology, Copenhagen University Hospital, Denmark

Abstract

Cone-beam CT (CBCT) scans acquired during radiotherapy capture changes such as tumor shrinkage and radiation-induced tissue changes, offering great potential for foundation models that help monitor individual treatment responses. However, to our knowledge, no large-scale pretrained models for radiotherapy CBCT exist, and evaluating modeling choices is difficult since downstream tasks are not known in advance. We examine how pretraining length, model size, and finetuning strategy relate to downstream clinical performance, and how metrics tracked during pretraining correlate with it. We pretrain vision transformer encoders with masked image modeling on about 150,000 raw 3D radiotherapy CBCT scans, tracking representation quality via validation loss, effective rank, and linear probing. Downstream tasks are classification of feeding tube presence (n=51,135n=51,135 CBCTs, 1,5621,562 patients) and clinical characteristics (T-, N-stage, sex, age, tumor site; n=1,278n=1,278 patients) in head and neck cancer. We compare low-rank adaptation (LoRA) and full finetuning against no pretraining, varying annotated dataset size. Pretext validation loss was the most consistent predictor of downstream performance; effective rank and linear probing were also associated, but not consistently across model sizes and finetuning methods. LoRA performed comparably to full finetuning while training only 0.5-2.5% of the parameters. Fully finetuned models consistently outperformed models without pretraining across all sizes, with AUROC 0.91-0.97 vs. 0.58-0.85 for feeding tube classification and 0.99-1.00 vs. 0.71-0.97 for sex classification. Model size had only a small, inconsistent effect, suggesting smaller models suffice at lower computational cost. Code and pretrained models (non-commercial license): https://github.com/alexCortinaU/RT-CBCT_SSL

Figures & tables

Explore similar work

CardsList
  1. nnFoundation: 3D Foundation Models for Radiology

    Sep 22, 2026Constantin Ulrich Harsy, Tassilo Wald, Karol Gotkowski +803D Foundation ModelsMedical Imaging Foundation Models

  2. Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis

    Feb 28, 2026Youngjin Yoo, Han Liu, Bogdan Georgescu +14Medical Imaging Foundation ModelsLow-Rank Adaptation

  3. CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT

    Oct 9, 2026Séléna Jean-Mactoux, Sophie Loizillon, Ghina Jardali +3Medical Image RetrievalRadiology VLMs