Braided Vision Transformer for Stroke Detection in Multi-view Retinal Fundus Imaging

Aysen Degerli; Mika Hilvo

cs.CVAug 12, 2026

Braided Vision Transformer for Stroke Detection in Multi-view Retinal Fundus Imaging

Authors: Aysen Degerli, Mika Hilvo

Organizations: VTT Technical Research Centre of Finland, Tampere and Espoo, Finland

Abstract

Stroke remains a leading cause of mortality and morbidity worldwide, emphasizing the importance of its accurate and immediate assessment. Retinal fundus imaging has emerged as a promising modality for stroke assessment, as the retina reflects cerebrovascular and neurological risk factors. Contrary to conventional neuroimaging techniques, retinal fundus imaging offers a non-invasive, cost-effective, and portable alternative for rapid screening. This paper explores the feasibility of retinal fundus imaging for stroke and transient ischemic attack (TIA) detection using macula-centric and optic nerve head-centric views captured from both eyes. Our study introduces, to the best of our knowledge, the first vision transformer model for retinal fundus imaging in stroke assessment, offering a novel approach for capturing retinal patterns. Thereby, we propose the Braided Vision Transformer (BViT) model, which extracts representative features from the given multi-view images while simultaneously capturing inter-view relationships across both eyes, enabling a more informative understanding of retinal biomarkers associated with cerebrovascular events. Experiments conducted on our collected Stroke-Data dataset demonstrate that BViT achieves an AUC score of 0.75 for stroke detection, outperforming regular vision transformers.

Figures & tables

Figure 1: The illustration of the proposed Braided Vision Transformer model structure that is designed to detect stroke from macula-centric and optic nerve head-centric retinal fundus images collected from both the left and right eye.

Figure 2: The structure of the proposed braided transformer module.

Model	Transformer Module	Sensitivity	Specificity	Precision	F1-Score	AUC	Accuracy
B-ViT	Braided (ours)	$0.647$	$0.727$	0.660	0.653	0.752	0.691
ViT	Vanilla [ 8 ]	$0.576$	0.744	$0.648$	$0.610$	$0.729$	$0.668$
B-ViT	Mutual [ 22 ]	$0.576$	$0.702$	$0.613$	$0.594$	$0.717$	$0.645$
ViT	Mutual [ 22 ]	$0.636$	$0.727$	$0.656$	$0.646$	$0.729$	$0.686$
MVS-Net-v1 [ 7 ]	-	0.667	$0.661$	$0.617$	$0.641$	$0.707$	$0.664$
MVS-Net-v2 [ 7 ]	-	$0.606$	$0.678$	$0.606$	$0.606$	$0.710$	$0.646$

Table 1: Average stroke and TIA detection performance results of vision transformers with different transformer modules computed from 5-folds, where the highest scores are highlighted in bold .

Figure 3: Overall performance comparison of the proposed and baseline models for stroke detection.

Class	Model	Transformer Module	Sensitivity	Specificity	Precision	F1-Score	AUC	Accuracy
Control	B-ViT	Braided (ours)	$0.736$	$0.586$	$0.685$	0.709	0.716	0.600
	ViT	Vanilla [ 8 ]	0.752	$0.475$	$0.636$	$0.689$	$0.708$	$0.568$
	B-ViT	Mutual [ 22 ]	$0.736$	$0.586$	$0.685$	0.709	$0.715$	0.600
	ViT	Mutual [ 22 ]	$0.727$	$0.596$	0.688	$0.707$	$0.714$	$0.595$
	MVS-Net-v1 [ 7 ]	-	$0.636$	0.626	$0.675$	$0.655$	$0.675$	$0.555$
	MVS-Net-v2 [ 7 ]	-	$0.694$	$0.505$	$0.632$	$0.661$	$0.668$	$0.546$

Table 2: Average performance results of vision transformers with different transformer modules reported for each class computed from 5-folds, where the highest scores are highlighted in bold .

Vision Transformer

Vision Transformers (ViTs) use self-attention over image patches to learn visual representations for recognition, segmentation, and other tasks.

See all 365 papers

Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images
May 12, 2024Fatema Tuj Johora Faria, Mukaffi Bin Moin, Pronay Debnath +2Explainable Medical Image Analysis Retinal Vessel Segmentation
aDepartment of Computer Science and Engineering, Ahsanullah University of Science and Technology, Dhaka, Bangladesh
Enabling Vision and Cross-Modal Learning for Multimodal Stroke Recurrence Prediction: An Interpretable Two-Step Framework
Sep 10, 2026Christian Gapp, Elias Tappeiner, Martin Welk +8Multimodal Learning Multimodal Fusion
Institute of Biomedical Image Analysis UMIT TIROL – Private University for Health Sciences and Health Technology, Eduard-Wallnöfer-Zentrum 1, 6060 Hall in Tirol, Austria · VASCage – Centre on Clinical Stroke Research, 6020 Innsbruck, Austria · Department of Radiology, Medical University of Innsbruck, 6020 Innsbruck, Austria +2
Interpretable Fundus Image Classification via Ring-Based Retinal Vasculature Features
Aug 25, 2026Xiaoyan Li, Shixin Xu, Arvind Gupta +1Ophthalmic Image Analysis Medical Image Classification
Computer Science, University of Toronto, 27 King’s College Cir., Toronto, M5S 1A1, Ontario, Canada. · Zu Chongzhi Center, Duke Kunshan University, No.8 Duke Ave., Kunshan, 215300, Jiangsu, China. · Mathematics and Statistics, York University, 4700 Keele St., Toronto, M3J 1P3, Ontario, Canada. +1

May 12, 2024J/K move · Enter open · S save

Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images

Fatema Tuj Johora Faria, Mukaffi Bin Moin, Pronay Debnath, Asif Iftekher Fahim+1

aDepartment of Computer Science and Engineering, Ahsanullah University of Science and Technology, Dhaka, Bangladesh

Explainable Medical Image Analysis Retinal Vessel Segmentation Convolutional Neural Networks Image Segmentation Hybrid CNN-Transformer Architectures U-Net+5

Open paper AskPDF arXiv

Early detection of vision-threatening conditions such as diabetic retinopathy, glaucoma, and age-related macular degeneration depends on retinal fundus image analysis, but manual assessment is slow and expert-dependent. Automated convolutional neural networks classify fundus images accurately yet act as black boxes, and existing retinal vessel segmentation methods lose discriminative power under pathology and seldom exploit attention or transformer backbones. Using the FIVES and DRIVE fundus datasets, we develop a two-pipeline framework that pairs four-class disease classification with attention- and transformer-based vessel segmentation, organised in three stages: (1) FIVES images are augmented by rotation and horizontal and vertical flips and used to fine-tune eight ImageNet-pretrained CNNs: ResNet101, DenseNet169, Xception, InceptionV3, DenseNet121, InceptionResNetV2, ResNet50, and EfficientNetB0. (2) Five gradient-based explanation methods, Grad-CAM, Grad-CAM++, Score-CAM, Faster Score-CAM, and Layer-CAM, are computed on the final convolutional block of each classifier and compared qualitatively across architectures. (3) Ten U-Net variants are benchmarked for vessel segmentation: TransUNet (hybrid CNN--Transformer encoder) and Attention U-Net (gated skip connections), evaluated with ResNet50V2, ResNet101V2, and ResNet152V2 backbones, along with additional Attention U-Net configurations using DenseNet backbones, and the fully transformer-based Swin-UNet. ResNet101 gives the highest classification accuracy: 94.17% (F1 0.942)

>

88.33% for EfficientNetB0. For segmentation, the architecture ranking is consistent on both datasets: Attention U-Net

>

TransUNet

>

Swin-UNet. The strongest configuration is Attention U-Net with a ResNet101V2 backbone: FIVES IoU 0.722, Dice 0.838; DRIVE IoU 0.648, Dice 0.787, lifting DRIVE IoU 60.80

\rightarrow

64.83 over a prior custom U-Net.

Abstract

Vision Transformer

Explore similar work

Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images

Enabling Vision and Cross-Modal Learning for Multimodal Stroke Recurrence Prediction: An Interpretable Two-Step Framework

Interpretable Fundus Image Classification via Ring-Based Retinal Vasculature Features