ReMoE: Report-Guided Mixture-of-Experts for Multimodal OCT/OCTA Anomaly Detection
Authors: Zihan Nie, Qincheng Qiao, Muhao Xu, Wei Feng, Xinguo Hou, Weiye Song, Zongyuan Ge
Organizations: School of Mechanical Engineering, Shandong University, Jinan, China · 2Key Laboratory of High Efficiency and Clean Mechanical Manufacture, Shandong University, Ministry of Education, Jinan, China · 6Airdoc-Monash Research, Monash University, Clayton, VIC 3800, Australia · Department of Endocrinology and Metabolism, Qilu Hospital, Shandong University, Jinan 250012, China · 5The First Clinical Medical College, Cheeloo College of Medicine, Shandong University, Jinan 250012, China · 3Monash University, Clayton, VIC 3800, Australia
Multimodal medical anomaly detection identifies samples deviating from normal patterns, where scarce abnormal cases make normality modeling from normal data practical. In retinal Optical Coherence Tomography (OCT) and OCT Angiography (OCTA) anomaly detection, existing unsupervised methods rely on visual feature distributions, reconstruction residuals, or encoder-decoder discrepancies, making anomaly scores depend on appearance-level deviations, while multimodal normality also contains semantic organization described in normal medical reports. To this end, we propose Report-Guided Mixture-of-Experts (ReMoE), which distills normal report semantics into an image-to-text prior student, builds modality-aware priors, and uses Report-Guided Modality Modulation (RMM) to modulate features through mixture-of-experts routing. Experiments on a private OCT/OCTA dataset with paired normal reports and a public OCTA500-3MM setting using a fixed normal report demonstrate state-of-the-art performance.