Diabetic retinopathy (DR) is a leading cause of preventable blindness, and automated fundus image grading can play an important role in large-scale screening. In this work, we investigate three CLIP-based approaches for five-class DR severity grading: (1) a zero-shot baseline using prompt engineering, (2) a hybrid FCN-CLIP model augmented with CBAM attention, and (3) a ranking-aware prompting model that encodes the ordinal structure of DR progression. We train and evaluate on a combined dataset of APTOS 2019 and Messidor-2 (n=5,406), addressing class imbalance through resampling and class-specific optimal thresholding. Our experiments show that the ranking-aware model achieves the highest overall accuracy (93.42%, AUROC 0.9845) and strong recall on clinically critical severe cases, while the hybrid FCN-CLIP model (92.49%, AUROC 0.99) excels at detecting proliferative DR. Both substantially outperform the zero-shot baseline (55.17%, AUROC 0.75). We analyze the complementary strengths of each approach and discuss their practical implications for screening contexts.
Figures & tables
Dataset
No DR
Mild
Moderate
Severe
Proliferative
Total
APTOS 2019
1805
370
999
193
295
3662
Messidor-2
1017
270
347
75
35
1744
Combined
2822
640
1346
268
330
5406
Percentage
52.2%
11.8%
24.9%
5.0%
6.1%
100%
Table 1: Class distribution in the combined dataset (APTOS 2019 + Messidor-2).
Hyperparameter
Value
Batch size
32
Learning rate
2e-4
Optimizer
AdamW
Weight decay
1e-3
Epochs
20
Early stopping
patience 7
Table 2: Training hyperparameters.
Metric
Zero-shot
FCN-CLIP
Ranking
Metric
Zero-shot
FCN-CLIP
Ranking
No DR
Severe DR
Precision
0.82
0.97
0.99
Precision
0.85
0.65
0.52
Recall
0.61
0.98
0.99
Recall
0.50
0.82
0.83
F1
0.70
0.97
0.99
F1
0.63
0.73
0.64
Mild DR
Proliferative DR
Precision
0.20
0.93
0.91
Precision
0.86
0.66
0.81
Table 3: Per-class and overall metrics for the three DR classification approaches.
Model
Exact match
Off-by-one
Off-by-two+
Zero-shot CLIP
60%
30%
10%
Hybrid FCN-CLIP
75%
22%
3%
Ranking-aware
80%
18%
2%
Table 4: Error distribution across models (percentage of test predictions).
Figure 1: Attention map comparison for a mild DR case. Left: zero-shot CLIP (diffuse, anatomy-focused). Right: Hybrid FCN-CLIP with CBAM (concentrated on lesion regions).
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Training and validation curves: Ranking-aware prompting model.
Figure 3: Training and validation curves: Hybrid FCN-CLIP model.
Distribution
No DR
Mild
Moderate
Severe
Proliferative
Original
1957
449
961
188
229
Percentage
51.7%
11.9%
25.4%
5.0%
6.0%
Resampled
445
200
200
180
180
Percentage
37.0%
16.6%
16.6%
15.0%
15.0%
Appendix
Table 5: Original vs. resampled class distribution in the training set.
Department of Computer Science and Engineering, Dhaka International University, Dhaka, Bangladesh · KUET, Khulna, Bangladesh · Bangladesh Bank, Dhaka, Bangladesh