Weakly Supervised Patch Annotation for Improved Screening of Diabetic Retinopathy

Shramana Dey; Abhirup Banerjee; B. Uma Shankar; Ramachandran Rajalakshmi; Sushmita Mitra

cs.CVMar 4, 2026

Weakly Supervised Patch Annotation for Improved Screening of Diabetic Retinopathy

Authors: Shramana Dey, Abhirup Banerjee, B. Uma Shankar, Ramachandran Rajalakshmi, Sushmita Mitra

Organizations: Indian Statistical Institute Kolkata, West Bengal, India · Department of Engineering Science, University of Oxford Oxford, United Kingdom · Radcliffe Department of Medicine, University of Oxford Oxford, United Kingdom · Techno India University Kolkata, West Bengal, India ·

Abstract

Diabetic Retinopathy (DR) requires timely screening to prevent irreversible vision loss. However, its early detection remains a significant challenge since often the subtle pathological manifestations (lesions) get overlooked due to insufficient annotation. Existing literature primarily focuses on image-level supervision, weakly-supervised localization, and clustering-based representation learning, which fail to systematically annotate unlabeled lesion region(s) for refining the dataset. Expert-driven lesion annotation is labor-intensive and often incomplete, limiting the performance of deep learning models. We introduce Similarity-based Annotation via Feature-space Ensemble (SAFE), a two-stage framework that unifies weak supervision, contrastive learning, and patch-wise embedding inference, to systematically expand sparse annotations in the pathology. SAFE preserves fine-grained details of the lesion(s) under partial clinical supervision. In the first stage, a dual-arm Patch Embedding Network learns semantically structured, class-discriminative embeddings from expert annotated patches. Next, an ensemble of independent embedding spaces extrapolates labels to the unannotated regions based on spatial and semantic proximity. An abstention mechanism ensures trade-off between highly reliable annotation and noisy coverage. Experimental results demonstrate reliable separation of healthy and diseased patches, achieving upto 0.9886 accuracy. The annotation generated from SAFE substantially improves downstream tasks such as DR classification, demonstrating a substantial increase in F1-score of the diseased class and a performance gain as high as 0.545 in Area Under the Precision-Recall Curve (AUPRC). Qualitative analysis, with explainability, confirms that SAFE focuses on clinically relevant lesion patterns; and is further validated by ophthalmologists.

Figures & tables

Figure 1: Representative patches from fundus images of Messidor dataset, illustrating key DR-related pathologies and healthy retinal regions. White bounding boxes highlight the lesions associated with DR.

Figure 2: Overview of the SAFE framework for patch-wise label inference using weak supervision. Stage 1 takes labeled patches from weakly annotated fundus images to train multiple embedding models. Stage 2 uses the embedding spaces from Stage 1, together with unlabeled patches, to generate refined annotations for downstream tasks.

Figure 3: Overview of the patch embedding network (PEN) architecture. The labeled input patch and its corresponding augmented version are passed through the encoder $f_{\theta}(\cdot)$ , and a combined loss is optimized. The resulting embedding space $(\mathcal{E})$ is utilized in the second stage of SAFE.

n_{H}=\sum_{r\in\mathcal{N}_{K}(r^{p}_{q})}\mathbf{1}_{[\ell(r)=\texttt{Healthy}]},

Algorithm 1 Generating Annotations using Feature-space Ensemble

Dataset	Images		Patches
	Images		Labeled		Unlabeled
	No-DR	DR	Healthy	Unhealthy	Unlabeled
Messidor ∗	546	654	41386	9387	39598
IDRiD (-)	168	81	87987	9292	33490
e-ophtha (-)	99	58	10522	511	8933
DDR (-)	2506	757	223542	18468	117448

Table 1: Distribution of No-DR and DR -affected images in each dataset, along with the corresponding labeled and unlabeled patches.

Figure 4: The extended confusion matrix used for computing metrics, such as Acc, BAcc, PR, RE, F1-score, AUPRC, D ${}_{\text{rate}}$ and MR

Dataset	Method	Acc ( $\uparrow$ )	BAcc ( $\uparrow$ )	Healthy				Unhealthy				$\mathbf{D}_{\text{rate}}$ ( $\uparrow$ )
Dataset	Method	Acc ( $\uparrow$ )	BAcc ( $\uparrow$ )	PR ( $\uparrow$ )	RE ( $\uparrow$ )	F1 ( $\uparrow$ )	MR ( $\downarrow$ )	PR ( $\uparrow$ )	RE ( $\uparrow$ )	F1 ( $\uparrow$ )	MR ( $\downarrow$ )	$\mathbf{D}_{\text{rate}}$ ( $\uparrow$ )
Messidor ∗	SAFE	0.9599	0.9062	0.9601	0.9919	0.9758	0.0077	0.9589	0.8205	0.8844	0.1529	0.9375
	LCL	0.9347	0.8932	0.9546	0.9638	0.9592	0.0362	0.8546	0.8227	0.8384	0.1773	1.0000
	ResNet18	0.9238	0.8853	0.9533	0.9507	0.9520	0.0493	0.8114	0.8200	0.8157	0.1800	1.0000
	PEN + KNN	0.9460	0.8881	0.9495	0.9849	0.9668	0.0150	0.9292	0.7914	0.8548	0.2007	1.0000
	PLT	0.7765	0.7037	0.8800	0.8300	0.8500	0.1700	0.4700	0.5800	0.5200	0.4200	1.0000

Table 2: Comparative analysis of SAFE with baseline approaches.

Dataset	SAFE	BAcc ( $\uparrow$ )	F1-Healthy ( $\uparrow$ )	F1-Unhealthy ( $\uparrow$ )	D ${}_{\text{rate}}$ ( $\uparrow$ )
Messidor	without Ensemble	0.8745	0.9582	0.8421	0.9754
Messidor	Ensemble	0.9062	0.9758	0.8844	0.9375
IDRiD	without Ensemble	0.8211	0.9674	0.7812	0.9882
IDRiD	Ensemble	0.8637	0.9859	0.8205	0.9568
e-ophtha	without Ensemble	0.7358	0.9721	0.6485	0.9821
e-ophtha	Ensemble	0.7729	0.9912	0.6931	0.9557

Table 3: Comparative performance of SAFE variants, without and with ensemble.

Dataset	Loss Optimized	DB	WD
Dataset	Loss Optimized	DB	Healthy	Unhealthy
Messidor ∗	$\mathcal{L}_{BCE}$	0.8695 ± 0.1068	0.0130 ± 0.0099	0.0563 ± 0.0384
	$\mathcal{L}_{SCL}$	0.9160 ± 0.0359	0.0014 ± 0.0006	0.0075 ± 0.0045
	$\mathcal{L}$	0.9096 ± 0.0130	0.0013 ± 0.0004	0.0036 ± 0.0028
IDRiD (-)	$\mathcal{L}_{BCE}$	0.8485 ± 0.0087	0.0042 ± 0.0009	0.1467 ± 0.0241
	$\mathcal{L}_{SCL}$	1.6056 ± 0.6535	0.0004 ± 0.0002	0.0078 ± 0.0071

Table 4: Comparative performance of PEN variants, optimized with loss functions

\mathcal{L}_{\text{BCE}}

\mathcal{L}_{\text{SCL}}

, and their combination

\mathcal{L}

Figure 5: t-SNE embeddings of labeled training (seen) and test (unseen) patches from a Messidor split, obtained using PEN optimized with (a) $\mathcal{L}_{\text{BCE}}$ , (b) $\mathcal{L}_{\text{SCL}}$ , (c) $\mathcal{L}$ .

Figure 6: Visualization of patch-level explanation. Rows (a)–(c) are initially labeled patches Healthy , and (d)–(f) are initially labeled patches Unhealthy .

Panel of Table 5 — Table 5: Comparative analysis of annotation generated using SAFE in downstream task.

Panel of Figure 7 — Figure 7: Sensitivity analysis of the weighting factor $\lambda$ on the Messidor ∗ dataset.

Figure 8: Sensitivity analysis across $K$ – $\tau$ configurations on the Messidor ∗ dataset.

Figure 9: Representative fundus images, annotated with SAFE-generated patch labels, highlighting (a) the strength, and (b) some limitations of the algorithm.

Weakly Supervised Learning

Weakly supervised learning trains models from incomplete, noisy, coarse, or indirect labels instead of dense ground truth.

See all 156 papers

Jul 22, 2026

PRISM-DR: Per-lesion Retinal Inference with Specialist Models for Diabetic Retinopathy

Diabetic retinopathy is a leading cause of preventable blindness; its early lesions are small, low contrast, and easily missed in manual screening. Most automated detectors handle the four non-proliferative DR lesions: microaneurysms, hemorrhages, hard exudates, and soft exudates, with a single multi-class model, even though these lesions differ sharply in size, color, morphology, and prevalence, so a shared model favors common, easy classes over rare, difficult ones. We present PRISM-DR, a lesion-specific pipeline that trains one single-class detector per lesion, each with its own configuration. From a raw fundus image, the pipeline applies region of interest cropping, fundus-specific preprocessing, four parallel YOLO detectors, tiling, per-lesion ensembling of five cross-validation folds, and an inter-lesion suppression step that resolves overlaps by physical lesion size and clinical priority rather than confidence. Per lesion, the best of five YOLO generations is selected, and augmentation is tuned by Bayesian optimization. Trained on IDRiD with stratified five-fold cross-validation, the system reaches a test mAP50 of 0.527 and F1 of 0.529, highest AP50 on hard exudates with 0.561. Without fine-tuning, the models transfer well where the imaging scale is close to IDRiD and degrade as field of view and resolution depart. These modest absolute results reflect a small single-source training set and a difficult task; however, treating each lesion as a separate detection problem is a practical alternative to a single multi-class model.

Zübeyr Özeren, Tansel Uyar

Department of Biomedical Engineering, Baskent University, Ankara, Türkiye · Department of Biomedical Engineering, Ankara University, Ankara, Türkiye

Medical Image Object Detection Diabetic Retinopathy Grading Medical Image Anomaly Detection Object Detection

Ask PDF

Apr 2, 2026

Managing Diabetic Retinopathy with Deep Learning: A Data Centric Overview

Diabetic Retinopathy (DR) is a serious microvascular complication of diabetes, and one of the leading causes of vision loss worldwide. Although automated detection and grading, with Deep Learning (DL), can reduce the burden on ophthalmologists, it is constrained by the limited availability of high-quality datasets. Existing repositories often remain geographically narrow, contain limited samples, and exhibit inconsistent annotations or variable image quality; thereby, restricting their clinical reliability. This paper presents a comprehensive review and comparative analysis of fundus image datasets used in the management of DR. The study evaluates their usability across key tasks, including binary classification, severity grading, lesion localization, and multi-disease screening. It also categorizes the datasets by size, accessibility, and annotation type (such as image-level, lesion-level, and multi-disease). Finally, a recently published dataset is presented as a case study to illustrate broader challenges in dataset curation and usage. The review consolidates current knowledge while highlighting persistent gaps such as the lack of standardized lesion-level annotations and longitudinal data. It also outlines recommendations for future dataset development to support clinically reliable and explainable solutions in DR screening.

Shramana Dey, Zahir Khan, T. A. PramodKumar +5

Indian Statistical Institute, Kolkata, West Bengal, India · Dr. Mohan’s Diabetes Specialties Centre, Madras Diabetes Research Foundation, Chennai, Tamil Nadu, India · Department of Electrical Engineering, National Institute of Technology, Durgapur, West Bengal, India +1

Medical Image Benchmarks Diabetic Retinopathy Grading Fundus Imaging

Ask PDF

Aug 8, 2026

PARAGraph: Pathology-Anatomy-Aware Hierarchical Graph for Diabetic Retinopathy Grading

Diabetic retinopathy (DR) remains a leading cause of vision loss among working-age adults worldwide, making reliable severity grading clinically important. Despite strong performance, most deep models formulate DR grading as image-level classification and do not explicitly model clinically grounded evidence, such as lesion types and spatial relations. In this paper, we propose PARAGraph, a Pathology-Anatomy-Aware Hierarchical Graph framework for DR grading. PARAGraph represents each image as a three-level hierarchical graph with lesion-level nodes, intermediate category and region nodes, and global anatomical and semantic nodes. To incorporate medical priors into nodes, we construct an optic disc-fovea-anchored coordinate frame that provides a scale- and rotation-normalized retinal reference system. Within this frame, lesion nodes are encoded with category, normalized area, and anatomical coordinates. To mitigate noisy lesion segmentation, PARAGraph uses a dual-fusion strategy that introduces global visual context into a graph semantic node and a decision-level prediction branch, improving robustness when lesion evidence is unreliable. Extensive experiments on Messidor-2, APTOS, and DDR show that PARAGraph achieves consistent DR grading performance over state-of-the-art methods. Interpretability and robustness analyses further demonstrate that its predictions are clinically grounded, closely associated with lesion evidence and robust to lesion segmentation noise.

Ziyang Zhang, Yuankai Huo, Yalin Zheng +1

Vanderbilt University, USA · University of Liverpool, UK

Graph Neural Networks Diabetic Retinopathy Grading Medical Image Classification Explainable Medical Image Analysis Retinal Disease Classification

Ask PDF

Abstract

Weakly Supervised Learning

Explore similar work

PRISM-DR: Per-lesion Retinal Inference with Specialist Models for Diabetic Retinopathy

Managing Diabetic Retinopathy with Deep Learning: A Data Centric Overview

PARAGraph: Pathology-Anatomy-Aware Hierarchical Graph for Diabetic Retinopathy Grading