eess.IVAug 22, 2025

A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer

Authors: Yuhui Tao, Zhongwei Zhao, Zilong Wang, Xufang Luo, Feng Chen, Kang Wang, Chuanfu Wu, Xue Zhang, 

Organizations: Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai, 200032, China. · Shanghai Key Laboratory of Medical Imaging Computing and Computer Assisted Intervention, Shanghai, 200032, China. · Department of Urology, Qilu Hospital of Shandong University, Jinan, Shandong, 250012, China. · Microsoft Research Asia, Shanghai, 200232, China. · 

Abstract

The non-invasive assessment of renal masses remains a critical challenge in urologic oncology, where diagnostic uncertainty frequently causes overtreatment. Here, we develop RenalCLIP, a vision-language foundation model for precision oncology in kidney cancer. Utilizing 27,866 computed tomography scans from 8,809 patients across diverse multi-center cohorts, we employ a two-stage pre-training strategy to align domain-specific visual and textual representations. RenalCLIP achieves enhanced performance and generalizability across ten core clinical tasks, spanning anatomical assessment, diagnostic classification, and survival prediction, significantly outperforming state-of-the-art general-purpose foundation models. Furthermore, RenalCLIP demonstrates strong data efficiency in diagnostic classification, achieving peak baseline performance using only 20% of the training data. The model also exhibits robust zero-shot diagnostic capabilities, effective image-text retrieval, and high-quality medical report generation. Our findings establish RenalCLIP as a powerful, generalizable tool to enhance diagnostic precision, refine prognostic stratification, and personalize the management of renal masses.

Explore similar work

CardsList
  1. Radiomics-Conditioned Modulation of RenalCLIP Features for Clear Cell Renal Cell Carcinoma Classification

    Sep 22, 2026Yuan Liang, Sourav Bhattacharjee, Abraham CampbellMedical Imaging Foundation ModelsRadiomics

  2. A Comprehensive Benchmark of Histopathology Foundation Models for Kidney Digital Pathology Images

    Mar 16, 2026Harishwar Reddy Kasireddy, Patricio S. La Rosa, Akshita Gupta +12Pathology Foundation ModelsComputational Pathology

  3. Benchmarking Foundation Models for Renal Lesion Stratification in CT

    May 8, 2026Hartmut Häntze, Sarah de Boer, Myrthe Buser +7Medical Imaging Foundation ModelsMedical Image Classification