Weakly Supervised Patch Annotation for Improved Screening of Diabetic Retinopathy

Shramana Dey; Abhirup Banerjee; B. Uma Shankar; Ramachandran Rajalakshmi; Sushmita Mitra

cs.CVMar 4, 2026

Weakly Supervised Patch Annotation for Improved Screening of Diabetic Retinopathy

Authors: Shramana Dey, Abhirup Banerjee, B. Uma Shankar, Ramachandran Rajalakshmi, Sushmita Mitra

Organizations: Indian Statistical Institute Kolkata, West Bengal, India · Department of Engineering Science, University of Oxford Oxford, United Kingdom · Radcliffe Department of Medicine, University of Oxford Oxford, United Kingdom · Techno India University Kolkata, West Bengal, India ·

Abstract

Diabetic Retinopathy (DR) requires timely screening to prevent irreversible vision loss. However, its early detection remains a significant challenge since often the subtle pathological manifestations (lesions) get overlooked due to insufficient annotation. Existing literature primarily focuses on image-level supervision, weakly-supervised localization, and clustering-based representation learning, which fail to systematically annotate unlabeled lesion region(s) for refining the dataset. Expert-driven lesion annotation is labor-intensive and often incomplete, limiting the performance of deep learning models. We introduce Similarity-based Annotation via Feature-space Ensemble (SAFE), a two-stage framework that unifies weak supervision, contrastive learning, and patch-wise embedding inference, to systematically expand sparse annotations in the pathology. SAFE preserves fine-grained details of the lesion(s) under partial clinical supervision. In the first stage, a dual-arm Patch Embedding Network learns semantically structured, class-discriminative embeddings from expert annotated patches. Next, an ensemble of independent embedding spaces extrapolates labels to the unannotated regions based on spatial and semantic proximity. An abstention mechanism ensures trade-off between highly reliable annotation and noisy coverage. Experimental results demonstrate reliable separation of healthy and diseased patches, achieving upto 0.9886 accuracy. The annotation generated from SAFE substantially improves downstream tasks such as DR classification, demonstrating a substantial increase in F1-score of the diseased class and a performance gain as high as 0.545 in Area Under the Precision-Recall Curve (AUPRC). Qualitative analysis, with explainability, confirms that SAFE focuses on clinically relevant lesion patterns; and is further validated by ophthalmologists.

Figures & tables

Figure 1: Representative patches from fundus images of Messidor dataset, illustrating key DR-related pathologies and healthy retinal regions. White bounding boxes highlight the lesions associated with DR.

Figure 2: Overview of the SAFE framework for patch-wise label inference using weak supervision. Stage 1 takes labeled patches from weakly annotated fundus images to train multiple embedding models. Stage 2 uses the embedding spaces from Stage 1, together with unlabeled patches, to generate refined annotations for downstream tasks.

Figure 3: Overview of the patch embedding network (PEN) architecture. The labeled input patch and its corresponding augmented version are passed through the encoder $f_{\theta}(\cdot)$ , and a combined loss is optimized. The resulting embedding space $(\mathcal{E})$ is utilized in the second stage of SAFE.

n_{H}=\sum_{r\in\mathcal{N}_{K}(r^{p}_{q})}\mathbf{1}_{[\ell(r)=\texttt{Healthy}]},

Algorithm 1 Generating Annotations using Feature-space Ensemble

Dataset	Images		Patches
	Images		Labeled		Unlabeled
	No-DR	DR	Healthy	Unhealthy	Unlabeled
Messidor ∗	546	654	41386	9387	39598
IDRiD (-)	168	81	87987	9292	33490
e-ophtha (-)	99	58	10522	511	8933
DDR (-)	2506	757	223542	18468	117448

Table 1: Distribution of No-DR and DR -affected images in each dataset, along with the corresponding labeled and unlabeled patches.

Figure 4: The extended confusion matrix used for computing metrics, such as Acc, BAcc, PR, RE, F1-score, AUPRC, D ${}_{\text{rate}}$ and MR

Dataset	Method	Acc ( $\uparrow$ )	BAcc ( $\uparrow$ )	Healthy				Unhealthy				$\mathbf{D}_{\text{rate}}$ ( $\uparrow$ )
Dataset	Method	Acc ( $\uparrow$ )	BAcc ( $\uparrow$ )	PR ( $\uparrow$ )	RE ( $\uparrow$ )	F1 ( $\uparrow$ )	MR ( $\downarrow$ )	PR ( $\uparrow$ )	RE ( $\uparrow$ )	F1 ( $\uparrow$ )	MR ( $\downarrow$ )	$\mathbf{D}_{\text{rate}}$ ( $\uparrow$ )
Messidor ∗	SAFE	0.9599	0.9062	0.9601	0.9919	0.9758	0.0077	0.9589	0.8205	0.8844	0.1529	0.9375
	LCL	0.9347	0.8932	0.9546	0.9638	0.9592	0.0362	0.8546	0.8227	0.8384	0.1773	1.0000
	ResNet18	0.9238	0.8853	0.9533	0.9507	0.9520	0.0493	0.8114	0.8200	0.8157	0.1800	1.0000
	PEN + KNN	0.9460	0.8881	0.9495	0.9849	0.9668	0.0150	0.9292	0.7914	0.8548	0.2007	1.0000
	PLT	0.7765	0.7037	0.8800	0.8300	0.8500	0.1700	0.4700	0.5800	0.5200	0.4200	1.0000

Table 2: Comparative analysis of SAFE with baseline approaches.

Dataset	SAFE	BAcc ( $\uparrow$ )	F1-Healthy ( $\uparrow$ )	F1-Unhealthy ( $\uparrow$ )	D ${}_{\text{rate}}$ ( $\uparrow$ )
Messidor	without Ensemble	0.8745	0.9582	0.8421	0.9754
Messidor	Ensemble	0.9062	0.9758	0.8844	0.9375
IDRiD	without Ensemble	0.8211	0.9674	0.7812	0.9882
IDRiD	Ensemble	0.8637	0.9859	0.8205	0.9568
e-ophtha	without Ensemble	0.7358	0.9721	0.6485	0.9821
e-ophtha	Ensemble	0.7729	0.9912	0.6931	0.9557

Table 3: Comparative performance of SAFE variants, without and with ensemble.

Dataset	Loss Optimized	DB	WD
Dataset	Loss Optimized	DB	Healthy	Unhealthy
Messidor ∗	$\mathcal{L}_{BCE}$	0.8695 ± 0.1068	0.0130 ± 0.0099	0.0563 ± 0.0384
	$\mathcal{L}_{SCL}$	0.9160 ± 0.0359	0.0014 ± 0.0006	0.0075 ± 0.0045
	$\mathcal{L}$	0.9096 ± 0.0130	0.0013 ± 0.0004	0.0036 ± 0.0028
IDRiD (-)	$\mathcal{L}_{BCE}$	0.8485 ± 0.0087	0.0042 ± 0.0009	0.1467 ± 0.0241
	$\mathcal{L}_{SCL}$	1.6056 ± 0.6535	0.0004 ± 0.0002	0.0078 ± 0.0071

Table 4: Comparative performance of PEN variants, optimized with loss functions

\mathcal{L}_{\text{BCE}}

\mathcal{L}_{\text{SCL}}

, and their combination

\mathcal{L}

Figure 5: t-SNE embeddings of labeled training (seen) and test (unseen) patches from a Messidor split, obtained using PEN optimized with (a) $\mathcal{L}_{\text{BCE}}$ , (b) $\mathcal{L}_{\text{SCL}}$ , (c) $\mathcal{L}$ .

Figure 6: Visualization of patch-level explanation. Rows (a)–(c) are initially labeled patches Healthy , and (d)–(f) are initially labeled patches Unhealthy .

Panel of Table 5 — Table 5: Comparative analysis of annotation generated using SAFE in downstream task.

Panel of Figure 7 — Figure 7: Sensitivity analysis of the weighting factor $\lambda$ on the Messidor ∗ dataset.

Figure 8: Sensitivity analysis across $K$ – $\tau$ configurations on the Messidor ∗ dataset.

Figure 9: Representative fundus images, annotated with SAFE-generated patch labels, highlighting (a) the strength, and (b) some limitations of the algorithm.

Weakly Supervised Learning

Weakly supervised learning trains models from incomplete, noisy, coarse, or indirect labels instead of dense ground truth.

See all 156 papers

PRISM-DR: Per-lesion Retinal Inference with Specialist Models for Diabetic Retinopathy
Jul 22, 2026Zübeyr Özeren, Tansel UyarMedical Image Object Detection Diabetic Retinopathy Grading
Department of Biomedical Engineering, Baskent University, Ankara, Türkiye · Department of Biomedical Engineering, Ankara University, Ankara, Türkiye
Managing Diabetic Retinopathy with Deep Learning: A Data Centric Overview
Apr 2, 2026Shramana Dey, Zahir Khan, T. A. PramodKumar +5Medical Image Benchmarks Diabetic Retinopathy Grading
Indian Statistical Institute, Kolkata, West Bengal, India · Dr. Mohan’s Diabetes Specialties Centre, Madras Diabetes Research Foundation, Chennai, Tamil Nadu, India · Department of Electrical Engineering, National Institute of Technology, Durgapur, West Bengal, India +1
PARAGraph: Pathology-Anatomy-Aware Hierarchical Graph for Diabetic Retinopathy Grading
Aug 8, 2026Ziyang Zhang, Yuankai Huo, Yalin Zheng +1Graph Neural Networks Diabetic Retinopathy Grading
Vanderbilt University, USA · University of Liverpool, UK

Jul 22, 2026J/K move · Enter open · S save

PRISM-DR: Per-lesion Retinal Inference with Specialist Models for Diabetic Retinopathy

Zübeyr Özeren, Tansel Uyar

Department of Biomedical Engineering, Baskent University, Ankara, Türkiye · Department of Biomedical Engineering, Ankara University, Ankara, Türkiye

Medical Image Object Detection Diabetic Retinopathy Grading Medical Image Anomaly Detection Object Detection

Open paper AskPDF arXiv

Diabetic retinopathy is a leading cause of preventable blindness; its early lesions are small, low contrast, and easily missed in manual screening. Most automated detectors handle the four non-proliferative DR lesions: microaneurysms, hemorrhages, hard exudates, and soft exudates, with a single multi-class model, even though these lesions differ sharply in size, color, morphology, and prevalence, so a shared model favors common, easy classes over rare, difficult ones. We present PRISM-DR, a lesion-specific pipeline that trains one single-class detector per lesion, each with its own configuration. From a raw fundus image, the pipeline applies region of interest cropping, fundus-specific preprocessing, four parallel YOLO detectors, tiling, per-lesion ensembling of five cross-validation folds, and an inter-lesion suppression step that resolves overlaps by physical lesion size and clinical priority rather than confidence. Per lesion, the best of five YOLO generations is selected, and augmentation is tuned by Bayesian optimization. Trained on IDRiD with stratified five-fold cross-validation, the system reaches a test mAP50 of 0.527 and F1 of 0.529, highest AP50 on hard exudates with 0.561. Without fine-tuning, the models transfer well where the imaging scale is close to IDRiD and degrade as field of view and resolution depart. These modest absolute results reflect a small single-source training set and a difficult task; however, treating each lesion as a separate detection problem is a practical alternative to a single multi-class model.

Abstract

Weakly Supervised Learning

Explore similar work

PRISM-DR: Per-lesion Retinal Inference with Specialist Models for Diabetic Retinopathy

Managing Diabetic Retinopathy with Deep Learning: A Data Centric Overview

PARAGraph: Pathology-Anatomy-Aware Hierarchical Graph for Diabetic Retinopathy Grading