DeepCORO-CLIP: A Multi-View Foundation Model for Comprehensive Coronary Angiography Video-Text Analysis and External Validation
Organizations: Division of Cardiology, Department of Medicine, Montreal Heart Institute, 5000 Belanger Street, QC, H1T 1C8 Canada · HeartWise.ai, 5000 Belanger Street, QC, H1T 1C8 Canada · McGill, Faculty of Engineering, Montreal, Quebec, Canada. · Division of Cardiology, Department of Medicine, University of California, San Francisco, Cardiology (San Francisco, CA, United States), 505 Parnassus Avenue, San Francisco, CA, 94143, United States of America · Cardiovascular Research Institute, University of California, San Francisco, CA, 94143, United States of America · Bakar Computational Health Sciences Institute, University of California, San Francisco, 94158, United States of America · Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA · Department of Bioengineering, University of California Los Angeles, Los Angeles, CA · Department of Medicine, Kaiser Permanente, San Francisco, USA · Department of Medicine, Université de Montréal, 2900 Edouard Montpetit Blvd, Montreal, Quebec H3T 1J4, Canada
Abstract
Coronary angiography is the reference standard for evaluating coronary artery disease, yet visual interpretation remains variable between readers. Existing artificial intelligence methods typically analyze single frames or projections and focus mainly on stenosis, limiting comprehensive coronary assessment. We present DeepCORO-CLIP, a multi-view foundation model trained with video-text contrastive learning on 203,808 angiography videos from 28,117 patients across 32,473 studies at the Montreal Heart Institute and externally validated on 4,249 studies from the University of California, San Francisco. DeepCORO-CLIP integrates multiple projections with attention-based pooling for study-level assessment across diagnostic, prognostic, and disease progression tasks. For significant stenosis detection, the model achieved an AUROC of 0.888 internally and 0.89 on external validation. Mean absolute error against core laboratory quantitative coronary angiography was 13.6%, lower than clinical reports at 19.0%. The model also performed strongly for chronic total occlusion, intracoronary thrombus, and coronary calcification detection. Transfer learning enabled prediction of one-year major adverse cardiovascular events with AUROC 0.79 and estimation of left ventricular ejection fraction with mean absolute error 7.3%. Embeddings also captured disease progression across serial examinations. With a mean inference time of 4.2 seconds in hospital deployment, DeepCORO-CLIP provides a foundation for automated coronary angiography interpretation at the point of care. Code, sample data, model weights, and deployment infrastructure are publicly released.