cs.CVNov 8, 2026

Dropping Tokens Without Dropping Them: Causal Attribution for Diabetic Retinopathy

Authors: Pramath Girish Haritz, Chandra Sekhar Seelamantula

Organizations: Indian Institute of Science

Abstract

In this work, we investigate whether attention-based attribution maps in a vision transformer provide causal explanations for diabetic retinopathy (DR) classification. We introduce a principled decomposition of attention into two attribution maps: the contribution (ctc_t), which satisfies the completeness axiom in explainable artificial intelligence (xAI), and the pull (ptp_t), which quantifies causal explanations. We define an identity relating ctc_t and ptp_t that characterizes when ctc_t fails to capture causality. We test whether each map is causally faithful on the MAPLES-DR dataset by dropping/inserting tokens, and report Necessity (NN), Sufficiency (SuSu), and Faithfulness Correlation (FCFC). The pull ptp_t achieves the highest causal alignment (FC=0.53FC = 0.53, N=0.71%N = 0.71\%, Su=3.0%Su = 3.0\%), outperforming Attention Rollout, Grad-CAM, and AttnLRP. Occlusion retains better sufficiency (Su=1.6%Su = 1.6\%) but is about 800 times slower. Furthermore, on MAPLES-DR, ptp_t achieves 70%70\% of the average precision (AP) of HRDecoder, a state-of-the-art supervised lesion segmentation model, despite the absence of pixel-level supervision. Together, these findings show that the pull ptp_t and the contribution ctc_t form computationally efficient attribution maps for reliable causal interpretation in safety-critical medical diagnostics. The code and demo are available at https://github.com/pramath-haritz/dr-xai.

Figures & tables

Explore similar work

Aug 28, 2026

Explainable Diabetic Retinopathy Classification Using Vision Foundation Models

Diabetic retinopathy (DR) is a leading cause of preventable blindness, and automated screening must be both accurate and trustworthy. We study an explainable DR classification framework built on vision foundation models and three transfer learning strategies. Three backbones, DINOv2, CLIP, and Vision Transformer (ViT), were adapted by full fine-tuning, linear probing, and Low-Rank Adaptation (LoRA). Models were trained and internally evaluated on the ODIR dataset and externally evaluated on APTOS to assess generalization. DINOv2-LoRA achieved the highest internal AUROC (0.758), whereas DINOv2 and ViT full fine-tuning achieved the highest external AUROC (0.920). Probability calibration was assessed with reliability analysis after isotonic regression. For explainability, Grad-CAM and HiResCAM maps were compared with expert-annotated lesion masks from the IDRiD dataset using Dice, Intersection over Union (IoU), and Pointing Game metrics. The results show that foundation models, particularly DINOv2, provide strong predictive performance, and that LoRA is a parameter-efficient alternative to full fine-tuning. Spatial overlap between explanation maps and annotated lesions was limited (mean Dice of about 0.13 for the combined lesion mask), which highlights the value of quantitatively testing whether model attention corresponds to clinically relevant retinal lesions.
Apr 25, 2026

From Pixels to Explanations: Interpretable Diabetic Retinopathy Grading with CNN-Transformer Ensembles, Visual Explainability and Vision-Language Models

The quality of diabetic retinopathy (DR) screening relies on the ability to correctly grade severity; however, many deep-learning (DL) classifiers cannot be easily interpreted in the clinical context. This study presents a methodology that combines strong discriminative models with multimodal explanations, converting retinal pixels into clinically interpretable outputs. Using the APTOS 2019 benchmark, we evaluated six representative CNN- and transformer-based backbones under a controlled protocol with stratified five-fold cross-validation. We then compared ensembling strategies (hard voting, weighted soft voting, stacking) and investigated a hybrid class-level fusion variant to exploit grade-specific advantages. For interpretability, we produced Grad-CAM++ visual attribution maps and short textual rationales using vision-language models (VLMs) conditioned on the fundus image and classifier outputs under conservative prompting constraints. Modern CNN backbones (ResNet-50 and ConvNeXt-Tiny) provided the strongest single-model baselines, with cross-validated QWK up to 0.919 and 0.914, respectively. Ensembling improved ordinal agreement, and weighted soft voting was the most consistent across folds (QWK 0.934 +/- 0.017). Hybrid class-level fusion was competitive but did not yield a statistically reliable improvement over standard fusion in paired fold comparisons (Holm-adjusted p >= 1.000). For explanation quality, Grad-CAM++ offered plausible but coarse localization, and VLM rationales were generally grade-consistent. Quantitatively, VLM variants showed a trade-off between clinical completeness and template-level semantic similarity (coverage 0.700 vs. BERTScore 0.072), while image-text alignment was comparable (CLIPScore approximately 0.34).
Jul 6, 2026

Causal-RetiGraph: Cross-Cohort Retinal Support and Same-Subject Pathway Analysis for Diabetic Retinopathy

Diabetic retinopathy (DR) is a local retinal lesion process and a visible manifestation of systemic microvascular injury. Modern retinal AI can grade images accurately, but often leaves unanswered how local lesion evidence, retinal vascular structure, and systemic disease pathways are connected. This paper introduces Causal-RetiGraph, a compact biomedical informatics framework that links retinal graph phenotypes with NHANES-anchored pathway modelling. The retinal-image fold constructs an interpretable X1234X1234 phenotype from vessel maps, lesion evidence, image embeddings, and AutoMorph biomarkers through spatial X12X_{12} and Jacobian X34X_{34} branches. The NHANES fold models systemic exposures, covariates, a same-subject retinal mediator family R∗R^*, and downstream outcome families. X1234X1234 is used for retinal support and pathway prioritisation, while R∗R^* is used for participant-level pathway summaries. On the retinal fold, X1234X1234 achieves 0.9055 binary DR accuracy and 0.9711 AUROC, with graded DR QWK of 0.8312. The results show that lesion and biomarker streams improve contextual retinal representation under scarce and imbalanced data. In NHANES, HbA1c, urine albumin, pulse pressure, fasting glucose, and systolic blood pressure are the strongest binary DR anchors. Participant-level pathway analysis identifies glycaemic--renal and glycaemic--haemodynamic pathways as the clearest mediator-style signals. These results suggest that retinal graph phenotypes can help prioritise systemic pathways in DR while preserving the distinction between image-derived support and same-subject mediation.