cs.CVOct 9, 2026

CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT

Authors: Séléna Jean-Mactoux, Sophie Loizillon, Ghina Jardali, Amandine Crombé, Nathalie Lassau, Alexandre Bône

Organizations: Biomaps, UMR1281, INSERM, CNRS, CEA, Université Paris-Saclay, Villejuif, France · Guerbet Research, Villepinte, France · Department of Diagnostic Oncologic Imaging, Gustave Roussy Institute, Villejuif, France · Department of Radiology, Pellegrin University Hospital, Bordeaux, France · Bordeaux Institute of Oncology, BRIC U1312, INSERM, Team SARCOTARGET, Bordeaux, France

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Purpose : To develop and evaluate CAPTOR, a deep-learning vision-language foundation model for oncologic chest-abdomen-pelvis (CAP) CT interpretation. Materials & Methods : CAPTOR (Chest-Abdomen-Pelvis foundaTion model for Oncologic Radiology) is a contrastive vision-language model, trained using a pretraining strategy for oncologic CAP CT (in single- or multiple-series acquisitions), on a multicenter retrospective dataset from North America, Europe and Asia, predominantly from tumor lesion examinations. CAPTOR was compared with state-of-the-art CT foundation models on three downstream tasks: disease detection, survival prediction using TumorImagingBench and image retrieval. Classification and retrieval were both evaluated on held-out internal and public test sets, stratified by anatomical coverage and indication. Bootstrap resampling was used to estimate confidence intervals, and pairwise comparisons were performed using permutation tests. Results : CAPTOR was trained and tested on 25,512 and 10,794 CT-report pairs respectively. TumorImagingBench datasets comprised between 134 and 421 studies each. CAPTOR outperformed all baseline models, on the classification task, on the CAP subset (AUC 0.767, 95%CI: 0.734-0.793, vs. 0.695, 95%CI: 0.645-0.736 for the next-best, p<0.001) and oncologic subset (AUC 0.784, 95%CI: 0.759-0.805, vs. 0.734, 95%CI: 0.703-0.767 p<0.001), ranked first on three of the four oncologic prognostic benchmarks, and achieved superior retrieval performance on CAP and oncologic subsets. Conclusion : A foundation model trained on oncology-enriched CAP CT data outperformed relevant state-of-the-art models in its target clinical context, underscoring the importance of clinically aligned pretraining. We release our preprocessing, training and inference code : https://github.com/Guerbet-AI/CAPTOR and model weights : https://huggingface.co/Guerbet-AI/CAPTOR

Figures & tables

Figure 1: Overview of the CAPTOR pipeline. CT volumes are cropped into 12 anatomical sub-regions using TotalSegmentator28 masks (A); regional crops are paired with the Findings section of the corresponding report and CAPTOR is trained with a rotating-region contrastive objective (B). At inference, TotalSegmentator masks are used to generate regional crops, and each CT yields one embedding per region (C). The resulting frozen image and text encoders are evaluated via XGBoost classification, k-nearest neighbor (kNN) survival prediction, and cross-modal retrieval (D, E, F). MAP@K = mean average precision at K, with K the number of retrieved images, std = Study
Figure 2: Flow chart showing the datasets used for training CAPTOR and for training/testing downstream tasks. CAP= Chest-Abdomen-Pelvis, AP= Abdomen-Pelvis
Table 1: Composition of the internal and public datasets used for training and testing. Percentages are reported at the study level. Train/test sets for TumorImagingBench1 are not included, as their original training/test splits were not used: survival prediction was instead assessed via k-nearest-neighbor classifiers across 10 random splits (Materials and Methods). The median age is shown, with inter-quartile range in brackets. CTPA = CT pulmonary angiogram; F=Female, M=Male, NSCLC = non-small cell lung cancer; GTV = gross tumor volume; CRLM = colorectal liver metastases.
Figure 3: Bar plot showing the classification scores of CAPTOR and five state-of-the art foundation models on our test set, stratified by visible anatomical region and clinical indication. The number of studies in each subset is in parentheses. One XGBoost was trained for each medical condition and model using the image embedding extracted with the model’s image encoder. For each subset and model, the macro-AUC is the unweighted mean of the AUC scores for each medical condition relevant to the subset among the 26 conditions studied. The confidence intervals and the pairwise comparison were obtained with a patient-level bootstrap and a patient-level permutation test respectively, both based on 2,000 iterations. AP = Abdomen-Pelvis, CAP=Chest-Abdomen-Pelvis. *** = p < 0.001, ** = p < 0.01, * = p < 0.05
Figure 4: Per-condition classification performance of CAPTOR and five baselines on the oncological subset, for each of the 26 medical conditions studied. One XGBoost was trained for each medical condition and model, using the appropriate image embedding (i.e. the corresponding regional crop for CAPTOR and the corresponding series for the baselines) extracted with the model’s image encoder.
Figure 5: Two-year survival-prediction performance (AUC) of CAPTOR and five baseline models on four oncologic TumorImagingBench1 cohorts (NSCLC-Radiomics21, NSCLC-Radiogenomics22, C4KC-KiTS23, Colorectal-Liver-Metastases24), using kNN classifiers on frozen image embeddings, averaged over 10 random splits. Statistical significance was assessed using a permutation test (2,000 permutations). The number of studies for each dataset is indicated between parentheses. ** = p < 0.01, * = p < 0.05
Datasets TaskSubsetCAPTORCAPTOR-100K
 Whole0.849 (0.828–0.866)0.846 (0.824–0.862)
 Chest0.838 (0.789–0.865)0.828 (0.778–0.860)
Global test set Medical condition classificationAP0.807 (0.797–0.816)0.808 (0.798–0.818)
 CAP0.767 (0.734–0.793)0.744 (0.703–0.779)
 Oncological0.784 (0.759–0.805)0.771 (0.743–0.796)
 Non-oncological0.859 (0.833–0.879)0.856 (0.831–0.874)
Table 3: Effect of training-data volume on CAPTOR performance. Comparison of CAPTOR and CAPTOR-100K performances across all downstream tasks with 95%confidence intervals in parentheses. Retrieval results are shown for K=10, with K the number of retrieved images considered.
Figure 6: Classification performance (macro-AUC) as a function of labeled training-set fraction (10-100%) for CAPTOR, CAPTOR-100K, and five baseline models, on (left) the whole test set, (middle) the CAP subset, and (right) the oncologic subset.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Radar plot comparing the per-condition classification F1-score of Mistral Large 3, GPT-5.4, and GPT-4o mini on the CT-RATE dataset, evaluated against the dataset’s original labels (extracted with a fine-tuned RadBERT-RoBERTa-4m on 1,000 reports).
Figure 8: Bar plot showing the classification scores of CAPTOR, CAPTOR-100K and five state-of-the art foundation models, on our test set stratified by anatomical region visible and indication. One XGBoost was trained for each medical condition and model using the image embedding extracted with the model’s image encoder. For each subset and model, the macro-AUC is the unweighted mean of the AUC scores for each medical condition relevant to the subset among the 26 conditions we studied. The confidence intervals and the pairwise comparison were obtained with a patient-level bootstrap and a patient-level permutation test respectively, both of size 2,000. ***= p < 0.001, ** = p < 0.01, * = p < 0.05, the stars are the color of the comparison model (orange for CAPTOR-100K and blue for CAPTOR)
Figure 9: Per-condition classification performance of CAPTOR, CAPTOR-100K, and baselines for the 26 medical conditions, shown separately for the CAP subset (left) and the oncological subset (right). One XGBoost was trained for each medical condition and model using the appropriate image embedding (i.e. the right regional crop for CAPTOR and the right series for the baselines) extracted with the model’s image encoder
Figure 10: Two-year survival-prediction performance (AUC) of CAPTOR, CAPTOR-100K and five baseline models on four oncologic TumorImagingBench cohorts (NSCLC-Radiomics, NSCLC-Radiogenomics, C4KC-KiTS, Colorectal-Liver-Metastases), using kNN classifiers on frozen image embeddings averaged over 10 random splits. Statistical significance was assessed using a permutation test (2,000 permutations).
Figure 11: Classification macro-AUC of CAPTOR, CAPTOR-100K, and five baseline models for each abnormality label on the external, non-oncologic CT-RATE [ 11 ] and RAD-Chest CT [ 33 ] datasets, using each dataset’s original labels. One XGBoost was trained for each medical condition and model using the image embedding extracted with the model’s image encoder. For each subset and model, the macro-AUC is the unweighted mean of the AUC scores for each medical condition relevant to the subset among the 26 conditions we studied. The confidence intervals and the pairwise comparison were obtained with a patient-level bootstrap and a patient-level permutation test respectively, both of size 2,000. *** = p < 0.001, ** = p < 0.01, * = p < 0.05

Explore similar work

CardsList

  1. CT-CLIP Representations for Multimodal Lung Cancer Survival Prediction

    Jul 9, 2026Sofie Allgöwer, Mikael Johansson, Andreas Hallqvist +4Medical Imaging Foundation ModelsRadiology VLMs

    Chalmers University of Technology, Gothenburg, Sweden · Umeå University, Umeå, Sweden · Sahlgrenska University Hospital, Gothenburg, Sweden +1

  2. Learning Robust Visual Features in Computed Tomography Enables Efficient Transfer Learning for Clinical Tasks

    Apr 5, 2026Rubén Moreno-Aguado, Alba Magallón, Victor Moreno +2Medical Imaging Foundation ModelsMedical Image Representation Learning

    Bioengineering Department and Imperial-X, Imperial College London, London, UK · Department of Computer Science, University of Manchester, Manchester, UK · Oncology Data Analytics Program, Catalan Institute of Oncology, Barcelona, Spain +6

  3. Anatomy Contextualized Adaption of CT Foundation Models

    Jul 29, 2026Roshan Kenia, Stephanie L McNamara, William LotterMedical Imaging Foundation ModelsVLM Adaptation

    Department of Biomedical Informatics, Harvard Medical School, Boston, MA · Department of Data Science, Dana-Farber Cancer Institute, Boston, MA · Department of Radiology, Massachusetts General Hospital, Boston, MA +2