Triple-Phase Multimodal Knowledge Aggregation Framework for Microbial Keratitis Subtype Diagnosis on Slit-Lamp Photography
Organizations: Department of Biomedical Engineering, Duke University, Durham, NC, USA. · Kellogg Eye Center, Department of Ophthalmology and Visual Sciences, University of Michigan, Ann Arbor, MI, USA. · Department of Cornea and Refractive Surgery Services, Aravind Eye Care System, Madurai, Tamil Nadu, India. · Bascom Palmer Eye Institute, Department of Ophthalmology, University of Miami Miller School of Medicine, Miami, FL, USA. · Flaum Eye Institute, Department of Ophthalmology, University of Rochester Medical Center, Rochester, NY, USA. · Department of Ophthalmology, Henry Ford Hospital, Detroit, MI, USA. · Duke Eye Center, Duke University School of Medicine, Durham, NC, USA.
Abstract
Microbial keratitis requires rapid pathogen identification to guide treatment, but culture- and PCR-based diagnostics are slow and resource-intensive. We developed a triple-phase multimodal framework for bacterial-versus-fungal keratitis classification using slit-lamp photographs acquired under blue-light, sclerotic-scatter, and white-light illumination, together with clinical metadata. The model combines cross-modality contrastive learning, modality-specific fine-tuning, and feature-level multimodal ensemble learning for patient-level prediction. We evaluated the framework on a multicenter dataset of 1,645 patients and 17,158 images from India and the United States. The model achieved 85.84% accuracy, 84.46% average F1-score, and 0.885 AUC. Site-specific evaluation showed that pooled results were overly optimistic, whereas resampling- and balance-based re-evaluation provided a more realistic assessment of cross-site generalization. Under all settings, our framework remained the top-performing approach. The code is available at https://github.com/yqwang01/TPMKA and dataset access will be provided subject to University of Michigan data-sharing clearance.