Interpretable Context-Aware Multi-Label Retinal Classification with Dual-Hub Heterogeneous Graph Neural Network

Proshanto Chanda; Munib Mesinovic; Tingting Zhu; Abhirup Banerjee

cs.CVNov 6, 2026

Interpretable Context-Aware Multi-Label Retinal Classification with Dual-Hub Heterogeneous Graph Neural Network

Authors: Proshanto Chanda, Munib Mesinovic, Tingting Zhu, Abhirup Banerjee

Organizations: ⋆Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, UK

Abstract

Multi-label retinal screening requires capturing co-occurring pathologies and clinical context, yet standard multimodal fusion remains opaque. We propose a dual-hub heterogeneous graph neural network (GNN) that couples spatial fundus features and structured clinical covariates through cross-modal message passing while providing inherent interpretability. Rather than relying on post-hoc saliency approximations, our architecture exposes multi-level decision pathways during forward aggregation: spatially biased self-attention captures lesion-level patch interactions, while modality-specific graph attention coefficients explicitly quantify the contribution of visual regions and clinical risk factors to each disease node. Evaluated on the BRSET dataset (16,266 photographs from 8,524 patients), the proposed model achieves 76.1% Macro-AUPRC across seven conditions, matching or exceeding image-only (75.9%) and ResNet-50+MLP fusion (75.9%) baselines, with largest contextual gains in macular edema (+3.5%) and diabetic retinopathy (+2.1%). Codebase is available at: https://github.com/proshanto-c/ca-mlc-refactor.

Figures & tables

Figure 1: Architecture overview. Within each examination, patch tokens and clinical covariates are decoupled into modality hubs ( $v_{\text{img}}$ , $v_{\text{pat}}$ ) that exchange cross-modal evidence before routing to disease label nodes. FC edges denote full connectivity.

	Context-Only		Single-Label (Img)		Multi-Label (Img)		ResNet-50 (Img+Ctx)		Dual-Hub GNN
Target	AUPRC	$F_{1}$	AUPRC	$F_{1}$	AUPRC	$F_{1}$	AUPRC	$F_{1}$	AUPRC	$F_{1}$
ICD	$31.0\pm 1.3$	$39.8\pm 0.9$	$\mathbf{78.7}\pm 1.7$	$\mathbf{71.2}\pm 1.4$	$73.4\pm 3.2$	$67.9\pm 3.3$	$73.1\pm 2.3$	$68.2\pm 1.7$	$74.0\pm 2.3$	$68.6\pm 1.5$
Drusen	$33.5\pm 0.5$	$34.3\pm 1.0$	$\mathbf{76.4}\pm 0.7$	$\mathbf{70.2}\pm 1.0$	$75.2\pm 1.8$	$68.1\pm 1.0$	$72.2\pm 2.3$	$68.0\pm 0.6$	$75.3\pm 1.4$	$68.3\pm 1.1$
DR	$44.8\pm 1.8$	$51.0\pm 2.2$	$\mathbf{87.3}\pm 2.0$	$\mathbf{81.3}\pm 3.3$	$84.5\pm 1.3$	$78.8\pm 2.3$	$87.2\pm 1.3$	$81.2\pm 1.0$	$86.6\pm 1.4$	$81.2\pm 1.5$
ME	$27.2\pm 4.8$	$30.0\pm 6.3$	$77.1\pm 5.7$	$68.6\pm 7.0$	$81.7\pm 2.4$	$75.7\pm 2.1$	$\mathbf{86.0}\pm 2.1$	$77.1\pm 2.4$	$85.2\pm 0.8$	$\mathbf{77.4}\pm 0.9$
Scar	$2.9\pm 0.5$	$4.0\pm 0.5$	$83.7\pm 1.9$	$80.6\pm 4.7$	$\mathbf{84.2}\pm 3.4$	$\mathbf{81.1}\pm 2.6$	$83.0\pm 4.3$	$78.4\pm 6.0$	$83.8\pm 1.7$	$78.9\pm 0.7$

Table 1: Test performance (mean

\pm

std across five seeds, in %). Context-Only, Single-Label (Img), Multi-Label (Img), and Proposed are GNN configurations; ResNet-50 (Img+Ctx) is the ResNet-50+MLP feature fusion baseline. Bold denotes the best score among image-based and multimodal configurations.

Figure 2: Visualisation of interpretability approaches on ICD, DR, and AMD examples. Original image and Grad-CAM on the ResNet50+MLP are Column 1 and 2 (left-to-right) respectively. Right two columns show Dual-Hub GNN attention visualisations. Column 3 shows attention for patch-to-hub edges. Column 4 shows patch-to-patch attention. Stars mark the two highest-attended GAT patches.

Retinal Disease Classification

See all 36 papers

Interpretable Retinal Disease Prediction Using Biology-Informed Heterogeneous Graph Representations
Feb 23, 2025Laurin Lux, Alexander H. Berger, Maria Romeo Tricas +8Medical Imaging Diabetic Retinopathy Grading
Munich Center for Machine Learning (MCML), Munich, Germany · School of Medicine and Health, TUM University Hospital, Munich, Germany · School of Computation, Information and Technology, TUM, Munich, Germany
Disentangling Co-Occurring Retinal Pathologies with Saliency-Guided Sparse Expert Routing
Aug 10, 2026Nagur Shareef Shaik, Jeongwoo Park, Yeong-Jin Kim +3Medical Imaging Sparse Mixture-of-Experts
Department of Computer Science, Georgia State University, Atlanta, GA 30324, USA · Piehealthcare Inc., Suwon-si, Gyeonggi-do, 16311, Republic of Korea
Benchmarking Convolutional, Transformer, Hybrid, and Vision Language Models for Multi Disease Retinal Screening
May 25, 2026Durjoy Dey, Aymane Ajbar, Yuhong YanMedical Image Classification Vision Transformer
Department of Computer Science and Software Engineering Concordia University Montreal, QC, Canada · Ebovir Biotechnologie Inc. Montreal, QC, Canada

Feb 23, 2025J/K move · Enter open · S save

Interpretable Retinal Disease Prediction Using Biology-Informed Heterogeneous Graph Representations

Laurin Lux, Alexander H. Berger, Maria Romeo Tricas, Richard Rosen+7

Munich Center for Machine Learning (MCML), Munich, Germany · School of Medicine and Health, TUM University Hospital, Munich, Germany · School of Computation, Information and Technology, TUM, Munich, Germany

Medical Imaging Diabetic Retinopathy Grading Explainable Medical Image Analysis Retinal Disease Classification

Open paper AskPDF arXiv

Interpretability is crucial for utilizing machine learning models as clinical decision support tools for medical diagnostics. However, most state-of-the-art image classifiers based on neural networks are not interpretable. As a result, clinicians often resort to known biomarkers to guide diagnosis, although biomarker-based classification often suffers from drastic information loss compared to raw medical images. This work proposes a method that preserves the rich imaging information while simultaneously enhancing the interpretability of predictions for diabetic retinopathy staging from optical coherence tomography angiography (OCTA) images. The core contribution of our method is a novel biology-informed heterogeneous graph representation that models retinal vessel segments, intercapillary areas, and the foveal avascular zone (FAZ) in a human-interpretable way. This graph representation allows us to frame diabetic retinopathy staging as a graph-level classification task, which we solve using an established, efficient graph neural network architecture. We compare our method against established methods, including classical biomarker-based classifiers, convolutional neural networks (CNNs), and vision transformers in predicting the clinically assigned DR stage based on color fundus photography images. We find stage agreement rates of our method and alternative vision model based classifiers saturating at AUC-ROC values of 84%. Crucially, we use our biology-informed graph to provide explanations of great detail. Our approach surpasses existing methods in precisely localizing and identifying abnormal vessels and non-perfusion areas. Our approach sets the stage for the interpretable identification of patients who require special attention due to their traceable microvascular changes, only observable using the details of OCTA images.

Abstract

Retinal Disease Classification

Explore similar work

Interpretable Retinal Disease Prediction Using Biology-Informed Heterogeneous Graph Representations

Disentangling Co-Occurring Retinal Pathologies with Saliency-Guided Sparse Expert Routing

Benchmarking Convolutional, Transformer, Hybrid, and Vision Language Models for Multi Disease Retinal Screening