cs.CVNov 18, 2026

How Well Do Generalist CT Foundation Models Differentiate Liver Tumors?

Authors: Yosuke Yamagishi, Tomohiro Kikuchi, Shouhei Hanaoka, Takeharu Yoshikawa, Osamu Abe

Organizations: Division of Radiology and Biomedical Engineering, Graduate School of Medicine, The University of Tokyo, Bunkyo-ku, Tokyo, Japan · Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, Tokyo, Japan · Department of Radiology, School of Medicine, Jichi Medical University, Shimotsuke, Tochigi, Japan · Department of Radiology, The University of Tokyo Hospital, Tokyo, Japan

Abstract

Generalist CT foundation models detect many findings with high performance, but their ability to differentiate detected lesions remains largely unevaluated. This retrospective study evaluated two generalist models, RADAR and Merlin, for differentiating liver tumors using a public single-institution multiphase CT dataset (517 cases: hepatocellular carcinoma, intrahepatic cholangiocarcinoma, breast and colorectal cancer liver metastases, and hemangioma). The models were evaluated in a zero-shot setting using their existing outputs and with linear probing, in which only a linear classifier was trained on frozen image features (stratified fivefold cross-validation repeated five times). In the zero-shot setting, RADAR achieved an accuracy of 54.4% (95% CI: 49.9, 58.8) and a κκ of 0.37 for four-class differentiation at the portal venous phase and predicted 31% (129 of 421) of malignant tumors as hemangioma. With linear probing of four-phase features, its accuracy improved to 79.9% (95% CI: 76.6, 83.4) with a κκ of 0.72, and 1% (five of 421) of malignant tumors were predicted as hemangioma; Merlin achieved an accuracy of 60.9% (95% CI: 56.9, 65.0). Generalist CT foundation models showed limited zero-shot ability to differentiate liver tumors, which improved substantially with linear probing alone.

Explore similar work

Oct 21, 2026

An interpretable deep learning method for early hepatocellular carcinoma diagnosis on multiphase CT

Early diagnosis of hepatocellular carcinoma is critical to preserve access to curative treatment options. The Liver Imaging Reporting and Data System (LI-RADS) provides a standardized framework for non-invasive diagnosis on multiphase computed tomography, based on major visual imaging features. However, these features may be difficult to assess consistently, particularly in small liver lesions. We developed dLI-RADS, an interpretable artificial intelligence method designed to estimate the probability of hepatocellular carcinoma from multiphase computed tomography images of manually segmented liver lesions, while remaining aligned with LI-RADS-related imaging concepts. dLI-RADS combines neural network-derived predictions with handcrafted quantitative imaging measurements reflecting LI-RADS major features and integrates these inputs using logistic regression. In this multicenter retrospective study, patients with multiphase computed tomography and at least one liver lesion were included, while lesions larger than 40 mm were excluded. Histology served as the reference standard for hepatocellular carcinoma, and a composite reference standard was used for non-hepatocellular carcinoma lesions. The lesion-level accuracy of dLI-RADS and LI-RADS were compared on the internal evaluation set using 10,000-repetition bootstrap non-inferiority tests, with 5% margins and significance levels. On the external test set, sensitivities were similarly compared. dLI-RADS was non-inferior to LI-RADS for lesion-level accuracy in the internal evaluation set, achieving 80% accuracy compared with 74% for LI-RADS (p = 0.04)...
May 8, 2026

Benchmarking Foundation Models for Renal Lesion Stratification in CT

The rapid proliferation of open-source medical foundation models (FMs) raises a practical question: how well do their pre-trained representations transfer to clinically relevant but data-scarce classification tasks? Particularly in CT-based renal lesion classification, a push toward greater generalizability would be meaningful, as the field is constrained by inherently limited training data. We addressed this through a benchmark of three medical FMs on this specific task. This six-class problem spans common entities like cysts and clear cell renal cell carcinoma, alongside rare subtypes. Using a frozen feature-probing protocol, we compared FM embeddings against a handcrafted radiomics classifier and a 3D ResNet-50 trained from scratch. Models were trained on a composite dataset of 2,854 lesions and evaluated on an external test set of 234 lesions from The Cancer Imaging Archive. Our results reveal two key findings. First, FM performance (AUC 0.70-0.77) matched the from-scratch ResNet (AUC 0.72) while drastically reducing hardware demand, requiring only seconds on a CPU after feature extraction. However, the conventional radiomics baseline significantly outperformed all deep learning approaches, achieving an AUC of 0.88 (all p ≤\leq 0.002). This suggests that current generalist FM embeddings do not yet capture the fine-grained texture and shape heterogeneity driving histological subtype discrimination. Despite their potential in data-scarce settings, medical FMs did not surpass established models for renal lesion stratification, leaving radiomics as the current state-of-the-art.
Oct 9, 2026

CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT

Objectives: Radiologists interpreting oncologic chest-abdomen-pelvis (CAP) CT must also detect non-oncologic conditions, yet existing CT foundation models are trained on single anatomical regions or on non-oncologic data. We developed and evaluated CAPTOR, a vision-language foundation model for oncologic CAP CT that uses an anatomically structured preprocessing strategy to learn from whole examinations. Materials & Methods: CAPTOR is a contrastive vision-language model pretrained on 25,512 predominantly oncologic CAP CT-report pairs (single or multiple series) from a retrospective multicenter dataset (North America, Europe, Asia). It was compared with five CT foundation models, a larger-scale variant (CAPTOR-100K) and an ablation without anatomical preprocessing (CAPTOR-NoPre) on classification and retrieval using held-out internal and public test sets, and on survival prediction using TumorImagingBench. Confidence intervals and pairwise comparisons used patient-level bootstrap and permutation tests. Results: On 10,794 test studies, CAPTOR achieved higher AUC than all five models on CAP (0.767 vs. 0.695 for the next-best model; p<0.001) and oncologic subsets (0.784 vs. 0.734; p<0.001), with superior retrieval on both, and ranked first on three of four oncologic survival benchmarks. Despite substantially fewer training examinations, it performed comparably to CAPTOR-100K. Removing the anatomically structured preprocessing reduced classification performance, including on the institutionally held-out test subset (macro-AUC 0.737 vs. 0.691 for CAPTOR-NoPre, p<0.01). Conclusion: Anatomically structured preprocessing enabled whole-examination CAP CT interpretation, including multiple-series acquisitions, and improved downstream performance. CAPTOR outperformed state-of-the-art CT foundation models on CAP and oncologic subsets while matching a substantially larger-scale variant.