cs.LGOct 9, 2026

Clinical value claims in multimodal AI outpace supporting evidence

Authors: Jiayu Feng

Organizations: Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, MA, USA

Abstract

Multimodal prediction models add a data source, such as imaging or an assay, to clinical information, and health systems must decide whether that added test is worth acquiring. We audited 177 studies comparing multimodal with unimodal models: 81 from a systematic review and 96 from a probability sample of 2025-2026 PubMed records. Gains in the area under the receiver operating characteristic curve (AUROC) were smaller when a study fitted a separate model for every data source (adjusted difference, -0.030; 95% CI, -0.056 to -0.005). Within 78 such studies, leaving out the strongest source's model would have raised the average gain from 0.046 to 0.108. This complete benchmarking fell from 69.1% of review studies to an estimated 46.0% of recent ones. An estimated 22% assessed decision utility and none evaluated a model prospectively, yet most claimed potential clinical usefulness or improved care (84.0% and an estimated 87.7%).

Explore similar work

CardsList
  1. An Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification

    Mar 3, 2026L. Julián Lechuga López, Farah E. Shamout, Tim G. J. RudnerSelective PredictionUncertainty Calibration

  2. When Are Multimodal Predictions Biologically Supported? A Diagnostic Evaluation Framework

    May 29, 2026Dylan Steiner, Gustavo Arango-Argoty, Gerald Sun +1Pathology Foundation ModelsMultimodal Embedding

  3. Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

    Jul 10, 2026Nikkie Hooman, Zhongjie Wu, Eric C. Larson +1Multimodal RobustnessClinical Prediction