How Well Do Generalist CT Foundation Models Differentiate Liver Tumors?
Organizations: Division of Radiology and Biomedical Engineering, Graduate School of Medicine, The University of Tokyo, Bunkyo-ku, Tokyo, Japan · Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, Tokyo, Japan · Department of Radiology, School of Medicine, Jichi Medical University, Shimotsuke, Tochigi, Japan · Department of Radiology, The University of Tokyo Hospital, Tokyo, Japan
Abstract
Generalist CT foundation models detect many findings with high performance, but their ability to differentiate detected lesions remains largely unevaluated. This retrospective study evaluated two generalist models, RADAR and Merlin, for differentiating liver tumors using a public single-institution multiphase CT dataset (517 cases: hepatocellular carcinoma, intrahepatic cholangiocarcinoma, breast and colorectal cancer liver metastases, and hemangioma). The models were evaluated in a zero-shot setting using their existing outputs and with linear probing, in which only a linear classifier was trained on frozen image features (stratified fivefold cross-validation repeated five times). In the zero-shot setting, RADAR achieved an accuracy of 54.4% (95% CI: 49.9, 58.8) and a of 0.37 for four-class differentiation at the portal venous phase and predicted 31% (129 of 421) of malignant tumors as hemangioma. With linear probing of four-phase features, its accuracy improved to 79.9% (95% CI: 76.6, 83.4) with a of 0.72, and 1% (five of 421) of malignant tumors were predicted as hemangioma; Merlin achieved an accuracy of 60.9% (95% CI: 56.9, 65.0). Generalist CT foundation models showed limited zero-shot ability to differentiate liver tumors, which improved substantially with linear probing alone.
Medical Imaging Foundation Models
Medical imaging foundation models learn reusable representations for tasks across clinical images, modalities, and institutions.
See all 258 papers