Learning to Fuse and Reconstruct Multi-View Graphs for Diabetic Retinopathy Grading
Authors: Haoran Li, Yuxin Lin, Huan Wang, Xiaoling Luo, Qi Zhu, Jiahua Shi, Huaming Chen, Bo Du, Johan Barthelemy, Zongyan Xue, Jun Shen, Yong Xu
Organizations: Department of Data Science and Artificial Intelligence, Monash University, Melbourne, Australia · ARC Centre of Excellence for the Weather of the 21st Century · Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology (Shenzhen), Shenzhen, China · School of Computing and Information Technology, University of Wollongong, Wollongong, Australia · College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China · College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · Centre for Nutrition and Food Sciences, The University of Queensland, Brisbane, Australia · School of Electrical and Computer Engineering, University of Sydney, Sydney, Australia · Department of Management, Griffith University, Brisbane, Australia · NVIDIA, Santa Clara, USA · The University of New South Wales, Sydney, Australia
Diabetic retinopathy (DR) is one of the leading causes of vision loss worldwide, making early and accurate DR grading critical for timely intervention. Recent clinical practices leverage multi-view fundus images for DR detection with a wide coverage of the field of view (FOV), motivating deep learning methods to explore the potential of multi-view learning for DR grading. However, existing methods often overlook the inter-view correlations when fusing multi-view fundus images, failing to fully exploit the inherent consistency across views originating from the same patient. In this work, we present MVGFDR, an end-to-end Multi-View Graph Fusion framework for DR grading. Different from existing methods that directly fuse visual features from multiple views, MVGFDR is equipped with a novel Multi-View Graph Fusion (MVGF) module to explicitly disentangle the shared and view-specific visual features. Specifically, MVGF comprises three key components: (1) Multi-view Graph Initialization, which constructs visual graphs via residual-guided connections and employs Discrete Cosine Transform (DCT) coefficients as frequency-domain anchors; (2) Multi-view Graph Fusion, which integrates selective nodes across multi-view graphs based on frequency-domain relevance to capture complementary view-specific information; and (3) Masked Cross-view Reconstruction, which leverages masked reconstruction of shared information across views to facilitate view-invariant representation learning. Extensive experimental results on MFIDDR, by far the largest multi-view fundus image dataset, demonstrate the superiority of our proposed approach over existing state-of-the-art approaches in diabetic retinopathy grading.
Figures & tables
Fig. 1: Illustration of our motivation. (a) Semantic information captured by different DCT frequency components in fundus images. (b) Comparison between (i) existing multi-view methods and (ii) our approach. Unlike prior methods, our approach disentangles the feature embeddings into shared and unique components, applying feature fusion and masked reconstruction learning separately to each. (c) Comparison of state-of-the-art multi-view DR methods and our proposed MVGFDR on seven evaluation metrics.
Fig. 2: Overall framework of MVGFDR. (a) Multi-view Graph Initialization (MVGI) for each view based on DCT frequency. (b) Graph nodes selection from each view according to their corresponding high-, mid- and low-frequency DCT components. (c) Multi-view Graph Fusion (MGF) of high-frequency information using a GCN. (d) Masked Cross-View Reconstruction (MCVR) on low- and mid-frequency information with the random node masking rate η .
Fig. 3: Visualization of view-specific representations extracted by MVGI from different views. A representative Grade-4 sample is shown for illustration. The visualization focuses on the high-frequency (DCT) representations, which are associated with lesion-related patterns.
Fig. 4: Proposed Masked Cross-View Reconstructor (MCVR) (a) We set a simple cross-graph reconstructor as the comparison baseline. (b) The proposed MCVR adopts a decoder-only transformer to perform masked view reconstruction guided by view-positional embeddings (VP, ) and frequency-positional embeddings (FP, ).
Grade 0
Grade 1
Grade 2
Grade 3
Grade 4
Method
Pre.
Sens.
F 1
Pre.
Sens.
F 1
Pre.
Sens.
F 1
Pre.
Sens.
F 1
Pre.
Sens.
F 1
Inception_ResNet_k_2_BSV
74.83
97.15
84.54
46.28
12.53
19.72
48.98
39.34
43.64
63.38
60.81
62.07
66.67
15.38
25.00
Mobile_Net_k2_BSV
76.60
96.70
85.49
48.55
14.99
22.91
56.85
45.36
50.46
64.16
75.00
69.16
60.00
15.38
24.49
ResNet50_BSV
79.38
94.68
86.36
48.34
22.82
31.00
51.48
47.54
49.43
62.94
72.30
67.30
70.00
17.95
28.57
ResNet50_32x4d_BSV
79.51
95.13
86.62
46.64
27.96
34.97
55.04
38.80
45.51
71.23
70.27
70.75
75.00
23.08
35.29
ConvNeXt-B_BSV
82.62
95.13
88.43
57.14
34.00
42.64
55.83
49.73
52.60
64.57
76.35
69.97
81.82
23.08
36.00
TABLE I: Comparison of precision (Prec.), sensitivity (Sens.), and F 1 scores between our method and single-view approaches in different DR grades. Best results are highlighted in bold.
Method
Acc.
Spe.
Kappa
F 1
AUC
Inception_ResNet_k_2_BSV [AAAI17] [ 59 ]
70.66
67.13
38.69
65.43
85.55
Mobile_Net_k2_BSV [CVPR18] [ 60 ]
72.38
68.73
43.61
67.28
86.88
ResNet50_BSV [CVPR16] [ 61 ]
73.22
73.20
45.29
69.35
86.76
ResNext50_32x4d_BSV [CVPR17] [ 62 ]
73.36
73.04
47.15
70.37
88.22
ConvNeXt-B_BSV [CVPR22] [ 63 ]
75.96
77.81
53.72
73.65
89.77
Swin-B_BSV [CVPR21] [ 64 ]
75.08
75.53
51.32
72.42
88.83
TABLE II: Comparison results against single-view approaches with our proposed multi-view approach. Best results are in bold font.
Grade 0
Grade 1
Grade 2
Grade 3
Grade 4
Method
Pre.
Sens.
F 1
Pre.
Sens.
F 1
Pre.
Sens.
F 1
Pre.
Sens.
F 1
Pre.
Sens.
F 1
RETFound
80.11
96.33
87.47
50.20
27.96
35.92
54.41
40.44
46.39
65.79
67.57
66.67
90.00
23.08
36.73
MVCINN
86.71
96.33
91.26
68.25
48.10
56.43
57.44
61.20
59.26
70.00
66.22
68.06
68.42
33.33
44.83
ETMC
86.79
97.53
91.82
73.26
56.38
63.72
66.41
47.54
55.41
64.41
77.03
70.15
0.12
90.15
0.87
LFMVDR (with lesion)
89.69
95.20
92.36
69.53
63.31
66.28
62.05
56.28
59.03
69.48
72.30
70.86
54.55
61.54
57.83
MVTSM (with lesion)
89.23
95.65
92.33
73.86
54.36
62.63
61.08
67.76
64.25
72.67
73.65
73.15
64.10
64.10
64.10
TABLE III: Comparison of precision (Prec.), sensitivity (Sens.), and F 1 scores between our method and multi-view approaches in different DR grades. The best results are highlighted in bold.
Method (End-to-end)
Acc.
Spe.
Kappa
F 1
AUC
RETFound [Nature23] [ 66 ]
74.06
73.83
48.44
70.91
89.04
MVCINN [AAAI23] [ 12 ]
80.10
83.32
62.45
78.86
91.07
ETMC [TPAMI22] [ 14 ]
81.54
83.44
64.76
79.74
93.53
LFMVDR (w/o lesion) [JBHI24] [ 32 ]
80.38
85.90
64.04
79.39
-
MVTSM (w/o lesion) [PR2025] [ 13 ]
71.73
82.76
-
70.93
-
SMVDR (w/o prompts) [AAAI25] [ 33 ]
81.64
89.92
66.94
81.38
-
TABLE IV: Comparison of experimental results between multi-view approaches and our proposed method. For fair comparison, we divide existing methods into two types: End-to-end and Expert-guided. The best results are in bold font.
DRTiD
Acc.
Spe.
Kappa
F 1
MVCINN [AAAI23] [ 12 ]
85.87
85.08
74.14
84.28
PLReMix [WACV25] [ 71 ]
87.94
88.93
76.70
87.64
Qmix [TMI25] [ 72 ]
89.29
91.15
80.80
88.31
MVGFDR (Ours)
92.68
92.74
83.99
90.71
CheXpert
Acc.
Spe.
Kappa
F 1
MVCINN [AAAI23] [ 12 ]
73.04
79.61
56.90
71.73
TABLE V: Comparison of experimental results on DRTiD and CheXpert datasets.
Fig. 5: Examples of V1-V4 views of different datasets. (a) MFIDDR. (b) The generated MVG-DDR.
MVG-DDR (Single-view)
Acc.
Pre.
Spe.
Kappa
F 1
ResNet-50_BSV [CVPR16] [ 61 ]
76.59
74.63
78.80
65.00
74.12
DenseNet-121_BSV [CVPR17] [ 78 ]
73.57
71.85
72.58
60.53
69.54
SE-BN-Inception_BSV [CVPR18] [ 79 ]
76.39
74.83
80.94
64.72
74.62
Swin-B_BSV [CVPR21] [ 64 ]
78.75
77.09
79.17
68.88
76.47
PVT-M_BSV [ICCV21] [ 49 ]
79.08
77.72
78.52
69.08
76.62
PVT-L_BSV [ICCV21] [ 49 ]
80.85
79.74
81.96
73.43
79.25
TABLE VI: Comparison of experimental results on the generated MVG-DDR dataset. We re-conduct experiments on single-veiw methods and multi-view methods. The best results are in bold font.
Fig. 6: (a) t-SNE [ 80 ] visualization of feature representations learned by the model. (b-c) Evaluation of the hyper-parameter settings. (b) Joint accuracy under different masking rates η and reconstruction loss control rate α . (c) Joint F 1 score under varying numbers of clustering center M and frequency range thresholds kθ .
Candidate
MFIDDR
PVT+GF
Node Select.
View Recon.
Acc.
Pre.
Spe.
Kappa
F 1
No.1
✔
79.96
78.63
80.17
61.28
77.91
No.2
✔
✔
81.10
79.87
81.93
63.52
78.76
No.3
✔
✔
82.52
80.74
85.11
67.39
80.86
Ours
✔
✔
✔
85.87
85.13
93.00
73.14
84.49
TABLE VII: Ablation study of the key components on MFIDDR.
Diabetic retinopathy (DR) remains a leading cause of vision loss among working-age adults worldwide, making reliable severity grading clinically important. Despite strong performance, most deep models formulate DR grading as image-level classification and do not explicitly model clinically grounded evidence, such as lesion types and spatial relations. In this paper, we propose PARAGraph, a Pathology-Anatomy-Aware Hierarchical Graph framework for DR grading. PARAGraph represents each image as a three-level hierarchical graph with lesion-level nodes, intermediate category and region nodes, and global anatomical and semantic nodes. To incorporate medical priors into nodes, we construct an optic disc-fovea-anchored coordinate frame that provides a scale- and rotation-normalized retinal reference system. Within this frame, lesion nodes are encoded with category, normalized area, and anatomical coordinates. To mitigate noisy lesion segmentation, PARAGraph uses a dual-fusion strategy that introduces global visual context into a graph semantic node and a decision-level prediction branch, improving robustness when lesion evidence is unreliable. Extensive experiments on Messidor-2, APTOS, and DDR show that PARAGraph achieves consistent DR grading performance over state-of-the-art methods. Interpretability and robustness analyses further demonstrate that its predictions are clinically grounded, closely associated with lesion evidence and robust to lesion segmentation noise.
Ziyang Zhang, Yuankai Huo, Yalin Zheng +1
Vanderbilt University, USA · University of Liverpool, UK
Diabetic retinopathy (DR) is a leading cause of vision impairment worldwide, and automated grading systems play a crucial role in large-scale screening programs. However, deep learning models often exhibit degraded performance when deployed across datasets acquired under different imaging conditions. This study presents a robust dual-resolution deep learning framework for DR grading that integrates attention-based feature fusion with ordinal regression to improve cross-dataset generalization. The proposed method employs two parallel EfficientNet backbones operating at different spatial resolutions to capture complementary retinal features. A learnable attention mechanism adaptively fuses multi-resolution representations, while an ordinal regression formulation based on the cumulative link model (CORAL) explicitly accounts for the ordered nature of DR severity levels. To mitigate domain discrepancies between datasets, a preprocessing strategy combining circular cropping, contrast enhancement, and histogram matching is applied. The model was trained on the APTOS 2019 dataset and evaluated on both an internal validation split and an external Messidor-2 test set. Experimental results demonstrate strong grading performance, achieving a quadratic weighted kappa (QWK) of 0.88 on the APTOS validation set and 0.68 on the unseen Messidor-2 dataset, indicating improved robustness for cross-dataset DR grading applications.
Afshan Hashmi
Tuwaiq Academy, Tuwaiq Research and Development Centre, Riyadh, Saudi Arabia
Diabetic retinopathy (DR) is one of the main causes of impaired vision. A good and reliable automated grading system can make the screening process safer and more accurate. Because DR stages progress gradually, the task of grading disease severity naturally follows an ordinal structure in which neighboring classes share similar visual characteristics. In this study, ORViT-DR, a hybrid deep learning framework, is designed to improve DR grading from low-resolution retinal images. The proposed approach combines convolutional feature extraction with transformer-based global context modeling through a pre-trained ViT-Hybrid backbone, which integrates BiT-ResNetv2 with a Vision Transformer architecture. The approach is tested on the RetinaMNIST subset of the MedMNISTv2 dataset, which contains 28x28 retinal fundus images annotated with five levels of disease severity. To promote stable training and better feature learning, the training strategy applies progressive layer unfreezing, layer-wise learning rate decay, exponential moving average (EMA) parameter updates, and ensemble-based prediction during inference. Experimental results on the official RetinaMNIST test set show that the proposed method achieves 57.00% classification accuracy, along with a quadratic weighted kappa score of 0.5963 and a macro-F1 score of 0.4293. These results suggest that hybrid CNN-Transformer architectures can provide effective representations for ordinal retinal image analysis.
Soumit Kumar Kundu, Nabil Ashab, Bidhan Biswas +4
Department of Computer Science and Engineering, Dhaka International University, Dhaka, Bangladesh · KUET, Khulna, Bangladesh · Bangladesh Bank, Dhaka, Bangladesh