Universal CT Representations from Anatomy to Disease Phenotype through Agglomerative Pretraining

Yuheng Li; Yuan Gao; Haoyu Dong; Yuxiang Lai; Shansong Wang; Mojtaba Safari; James E. Baciak; Xiaofeng Yang

cs.CVMay 21, 2026

Universal CT Representations from Anatomy to Disease Phenotype through Agglomerative Pretraining

Authors: Yuheng Li, Yuan Gao, Haoyu Dong, Yuxiang Lai, Shansong Wang, Mojtaba Safari, James E. Baciak, Xiaofeng Yang

Organizations: Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA 30332, USA · Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA 30322, USA · Department of Electrical and Computer Engineering, Duke University, Durham, NC 27705, USA · Department of Computer Science and Informatics, Emory University, Atlanta, GA 30322, USA · Department of Materials Science & Engineering, Nuclear Engineering Program, University of Florida, Gainesville, FL 32611, USA

Abstract

Computed tomography (CT) is a central to three-dimensional medical imaging, yet CT-based artificial intelligence remains fragmented across task-specific models for segmentation, classification, registration, and report analysis. Here we present FlexiCT, a family of CT foundation models trained by agglomerative continual pretraining on 266,227 CT volumes from 56 publicly available datasets, forming a large-scale public resource for CT representation learning. FlexiCT uses agglomerative pretraining across three stages: two-dimensional axial pretraining, three-dimensional anatomical pretraining and report-guided semantic alignment. This training strategy supports slice-level, volume-level and vision-language analysis. Across five downstream task families (segmentation, classification, registration, vision-language understanding and clinical retrieval), FlexiCT matches or exceeds prior task-specific approaches on multiple benchmarks. Its embeddings further organize CT scans along gradients associated with various tumor stages, suggesting that CT foundation models can capture imaging features relevant to disease phenotype characterization. Project page and code are available at: https://ricklisz.github.io/flexict.github.io and https://github.com/ricklisz/FlexiCT.

Explore similar work

Anatomy Contextualized Adaption of CT Foundation Models
Jul 29, 2026Roshan Kenia, Stephanie L McNamara, William LotterMedical Vision-Language Models Vision-Language Alignment
OrganLens: Organ-Specific Representation Learning for CT Foundation Models
Jul 28, 2026Zhixuan Ge, Anqi Li, Sadeer Al-Kindi +2Computed Tomography Dataset Representation Learning
DALE-CT: Depth-Aware Foundation Models for Computed Tomography
Jun 5, 2026Evan W. Damron, Mahmut S. Gokmen, Mitchell A. Klusty +3Computed Tomography Dataset Dinov3

Jul 29, 2026 · cs.CVJ/K move · Enter open · S save

Anatomy Contextualized Adaption of CT Foundation Models

Roshan Kenia, Stephanie L McNamara, William Lotter

Department of Biomedical Informatics, Harvard Medical School, Boston, MA · Department of Data Science, Dana-Farber Cancer Institute, Boston, MA · Department of Radiology, Massachusetts General Hospital, Boston, MA · Department of Pathology, Brigham and Women’s Hospital, Boston, MA · Department of Pathology, Harvard Medical School, Boston, MA

CT vision-language foundation models have demonstrated promising performance across downstream tasks, but are typically trained with whole-volume representations that dilute fine-grained anatomical signals. Fine-grained vision-language pre-training addresses this by aligning anatomy-level visual features with anatomy-specific text, but in doing so discards the global context that whole-volume models provide. Furthermore, existing fine-grained approaches train from scratch, making them computationally expensive. We introduce Anatomy Contextualized Adaptation (ACA), a lightweight framework that adapts frozen CT foundation model representations for anatomy-level vision-language alignment while enhancing global contextualization. ACA uses TotalSegmentator to decompose CT volumes into anatomy-level embeddings, which are refined via a transformer that captures cross-anatomy relationships, and aligned to both per-anatomy and scan-level text extracted from radiology reports. Evaluated on Merlin and CT-RATE, ACA consistently outperforms both the frozen foundation model baselines and existing fine-grained methods in zero-shot finding classification, while requiring less than one hour of training once embeddings are cached. The attention weights learned by ACA's inter-anatomy transformer additionally indicate plausible cross-anatomy context routing. Altogether, these results support ACA as a lightweight approach for adapting CT foundation models to anatomically grounded vision-language alignment while preserving and enhancing global anatomical context.

Medical Vision-Language Models Vision-Language Alignment Anatomy Foundation Model

Open paper Ask about it PDF

Abstract

Explore similar work

Anatomy Contextualized Adaption of CT Foundation Models

OrganLens: Organ-Specific Representation Learning for CT Foundation Models

DALE-CT: Depth-Aware Foundation Models for Computed Tomography