ISLA: Ensemble learning and domain adaptation for generalizable acute ischemic stroke lesion segmentation in diffusion MRI with clinical validation
Authors: Vincent Roca, Hakim Daouairi, Sevak Yepremian, Hilde Henon, Nicolas Martinage, Laurent Puy, Grégory Kuchcinski, Martin Bretzner, Renaud Lopes
Organizations: Univ. Lille, CNRS, Inserm, CHU Lille, Institut Pasteur de Lille, US 41 - UAR 2014 - PLBS, F-59000 Lille, France · CHU Lille, Département de Neurologie vasculaire, F-59037 Lille, France · Univ. Lille, Inserm, CHU Lille, U1172 - Lille Neurosciences & Cognition, F-59037 Lille, France · CHU Lille, Département de Neuroradiologie, F-59037 Lille, France · CHU Lille, Département de Médecine Nucléaire, F-59037 Lille, France
Accurate segmentation of acute ischemic stroke (AIS) lesions on MRI is essential for clinical decision-making. Deep learning methods offer a promising approach to automating this task. However, they often generalize poorly to unseen clinical sites, and few AIS lesion segmentation methods have been externally validated. In this study, we propose ISLA (Ischemic Stroke Lesion Analyzer), a framework for AIS lesion segmentation built on the design principles of nnU-Net. ISLA combines attention modules, deep supervision, and ensemble learning across different input resolutions, convolutional block types, and loss functions. We also introduce ISLA-MT, a variant that leverages unlabeled target-domain images through semi-supervised learning. We trained both approaches on the SOOP and ISLES 2022 datasets and evaluated on a held-out internal validation set of 253 cases, as well as on 590 cases from an independent multicenter clinical cohort. We compared them with three state-of-the-art pre-trained models and with nnU-Net trained on the same data. ISLA outperformed the pre-trained models on the internal validation set, while both ISLA and ISLA-MT outperformed them on the external cohort. Although nnU-Net performed slightly better than ISLA internally, both ISLA variants demonstrated better generalization to the external cohort. Furthermore, ISLA-MT significantly improved performance over ISLA on the external cohort. The proposed models performed particularly well on low-contrast lesions, and qualitative error analyses suggested greater robustness to acquisition artifacts. Both models are fast (∼ 5 s per case), and their code and weights are publicly available at https://github.com/RocaVincent/isla_segmentation.
Figures & tables
Figure 1: Overview of the ISLA framework. 8 sub-models are trained independently on the preprocessed images. The sub-models differ in input resolution, convolutional block design, and loss function. Final ISLA predictions are obtained by averaging the predictions of all sub-models.
Figure 2: Illustration of the U-Net architecture used in ISLA . The model takes a trace diffusion-weighted image (DWI) and the corresponding apparent diffusion coefficient map (ADC) as inputs. Lesion predictions are generated at multiple levels of the decoder for deep supervision. The number of feature maps at each level is indicated below the gray boxes. The lower-right table reports the spatial dimensions at each U-Net depth and the convolutional kernel sizes for the two input resolutions used in ISLA .
Figure 3: Illustration of ISLA-MT training. Student training is guided by a weighted sum of segmentation and consistency losses. The segmentation loss is computed on labeled source-domain images, whereas the consistency loss is computed on unlabeled target-domain images. The teacher weights are updated using an exponential moving average of the student weights.
dataset
number of participants
field strength, Tesla
manufacturer
in-plane pixel spacing, mm a
slice thickness, mm a
lesion size, mL b
SOOP
1265
1.5 (80.1%) 3.0 (19.9%)
Philips (98.3%) GE (1.2%) Siemens (0.5%)
0.95 ± 0.16
6.00 ± 0.04
7.98 (31.91)
ISLES
250
3.0 (57.6%) unknown (42.4%)
Philips (50.0 %) unknown (48.4%) Siemens (1.6%)
1.88 ± 0.28
2.60 ± 1.15
6.66 (19.59)
external evaluation cohort
590
1.5 (88.1%) 3.0 (11.9%)
Philips (46.6%) Siemens (29.8%) GE (23.6%)
1.01 ± 0.26
4.98 ± 0.85
15.28 (35.16)
external adaptation cohort
1505
1.5 (91.5%) 3.0 (8.5%)
Philips (52.6%) GE (28.3%) Siemens (19.1%)
0.98 ± 0.22
5.12 ± 0.81
-
a Values are expressed as mean ± standard deviation.
b Values are expressed as median (interquartile range).
Table 1: Characteristics of populations, MRI scanners, diffusion-weighted imaging, and lesions for each dataset.
Figure 4: Proportions of ischemic stroke lesions across brain regions in the annotated datasets. The manual lesion delineations were registered to the 1 mm 3 JHU ICBM template (Section 4.5.3 ), and voxel-wise proportions were computed. The proportion maps were then smoothed using a Gaussian filter (FWHM = 4 mm), and values below 0.02 were masked.
Figure 5: Segmentation metrics in the internal validation dataset. Asterisks indicate significant pairwise comparisons ( ∗∗∗ : p<0.001 , ∗∗ : p<0.01 , ∗ : p<0.05 ) with ISLA . Boxplots display the median and interquartile range only; whiskers and outliers are omitted for clarity. Abbreviations → DSC: Dice similarity coefficient; HD95: 95th percentile Hausdorff distance; AVD: absolute volume difference; ALD: absolute lesion count difference; LF1: lesion detection F1 score.
Figure 6: Segmentation metrics in the external evaluation cohort. Asterisks indicate significant pairwise comparisons ( ∗∗∗ : p<0.001 , ∗∗ : p<0.01 , ∗ : p<0.05 ), with a letter indicating the compared ISLA model (A: ISLA , B: ISLA-MT ). Boxplots display the median and interquartile range only; whiskers and outliers are omitted for clarity. Abbreviations → DSC: Dice similarity coefficient; HD95: 95th percentile Hausdorff distance; AVD: absolute volume difference; ALD: absolute lesion count difference; LF1: lesion detection F1 score.
Figure 7: Difference in Dice similarity coefficient ( Δ DSC) according to lesion volume and contrast-to-noise ratio (CNR) in the external evaluation cohort. For each comparison method and participant, the DSC difference relative to ISLA or ISLA-MT is reported ( DSCISLA(-MT)−DSCcomparison ), along with the Spearman’s rank correlation coefficient ( ρ ) and corresponding significance levels ( ∗∗∗ : p<0.001 , ∗∗ : p<0.01 , ∗ : p<0.05 ). Curves show locally weighted linear regression fits of Δ DSC against lesion volume and CNR for each ISLA and ISLA-MT variant, computed with a span of 0.3 and 3 residual reweightings.
Figure 8: Voxel-wise proportions of false positives (FPs) and false negatives (FNs) in the external evaluation cohort. Distant FPs (resp. FNs) correspond to predicted (resp. ground-truth) lesion voxels located more than 5 mm from any ground-truth (resp. predicted) lesion voxel. Error maps were transformed to MNI space before averaging. The average maps were smoothed using a Gaussian filter (FWHM = 4×4×8 mm) and thresholded at 0.5% for the overall error maps and 0.2% for the distant error maps.
Figure 9: Illustrative examples of the different categories of model errors. Axial slices of the trace diffusion-weighted image (DWI), apparent diffusion coefficient map (ADC), and ground-truth annotations (GT) are shown alongside true-positive, false-negative (FN), and false-positive (FP) voxels. All volumes were resampled to 0.8 × 0.8 × 3.0 mm 3 for visualization. A Large FN produced by DeepISLES . B FP produced by DAGMNet in the temporal poles and anterior frontal regions. C FP produced by DAGMNet anterior to the temporal poles. D FP produced by PAT / DAGMNet / nnU-Net in the posterior superior part of the brain. E FN produced by PAT / DAGMNet in the deep sylvian region. F FN produced by nnU-Net / ISLA in the deep sylvian region and detected by ISLA-MT . G FN produced by PAT / DAGMNet / nnU-Net in the left superior region of the brain. H FP produced by ISLA-MT in the deep sylvian region.
ablated variant
DSC
HD95
AVD
ALD
LF1
ISLA
1.0×1.0×2.0mm 3
−0.246∗∗∗
+0.131∗
−0.043
+0.079
−0.074
0.8×0.8×3.0mm 3
+0.037
−0.026
−0.109
+0.006
−0.101
StdUNet
−0.056
+0.067
−0.025
+0.063
−0.097
ResUNet
−0.015
+0.051
−0.112
+0.059
−0.052
DiceBCE
−0.415∗∗∗
+0.063
+0.395∗∗∗
+0.036
−0.111
Table 2: Comparison of segmentation metrics between the ablated and the full ISLA variants in the external evaluation cohort.
runtime
peak GPU memory (GiB)
peak CPU RAM (GiB)
total
average / case
DeepISLES
18 h 11 min 39 s
1 min 51 s
19.80
9.54
PAT
57 min 24 s
6 s
8.87
3.55
DAGMNet
27 h 9 min 31 s
2 min 46 s
8.62
4.62
nnU-Net
52 min 11 s
5 s
3.77
8.87
ISLA
51 min 58 s
5 s
8.48
22.62
Table 3: Computational requirements for inference on the external evaluation cohort.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 1: Illustration of the implemented attention gate module. xd denotes the decoder feature maps, xe denotes the encoder feature maps, and xm the output modulated feature maps. Conv[k1×k2×k3,s1×s2×s3,F] represents a convolution with F filters, kernel size k1×k2×k3 , and stride s1×s2×s3 . TLU[u1×u2×u3] denotes trilinear upsampling with factors u1×u2×u3 .
DSC
HD95
AVD
ALD
LF1
ISLA
0.852 (0.150)
2.950 (5.950)
1.521 (4.145)
1 (2)
0.833 (0.333)
ResUNet-DiceBCE-0.8
0.850 (0.144)
3.067 (6.611)
1.379 (3.705)
1 (2)
0.842 (0.333)
ResUNet-UFL-0.8
0.848 (0.161)
3.088 (6.031)
1.463 (4.322)
1 (2)
0.800 (0.333)
StdUNet-DiceBCE-1.0
0.847 (0.156)
3.128 (7.216)
1.321 (3.720)
1 (2)
0.833 (0.333)
StdUNet-DiceBCE-0.8
0.844 (0.161)
3.128 (9.163)
1.394 (3.806)
1 (2)
0.800 (0.333)
ResUNet-DiceBCE-1.0
0.839 (0.156)
3.128 (8.682)
1.306 (3.667)
1 (2)
0.800 (0.333)
Appendix
Table 1: Comparison of ISLA and its sub-models on the internal validation set.
Figure 1: Segmentation metrics on the 28 double-annotated cases from the external evaluation cohort. Asterisks indicate significant pairwise differences ( ∗∗∗ : p<0.001 , ∗∗ : p<0.01 , ∗ : p<0.05 ) from the corresponding inter-rater metric. Boxplots display the median and interquartile range only; whiskers and outliers are omitted for clarity. Abbreviations → DSC: Dice similarity coefficient; HD95: 95th percentile Hausdorff distance; AVD: absolute volume difference; ALD: absolute lesion count difference; LF1: lesion detection F1 score.
Figure 1: Illustrative examples of DeepISLES false negatives across the brain. Axial slices of the trace diffusion-weighted image (DWI), apparent diffusion coefficient map (ADC), and ground-truth annotations (GT) are shown alongside true-positive, false-negative, and false-positive voxels. All volumes were resampled to 0.8 × 0.8 × 3.0 mm 3 for visualization.
Figure 2: Illustrative examples of DAGMNet false positives in the temporal poles and anterior frontal regions. Axial slices of the trace diffusion-weighted image (DWI), apparent diffusion coefficient map (ADC), and ground-truth annotations (GT) are shown alongside true-positive, false-negative, and false-positive voxels. All volumes were resampled to 0.8 × 0.8 × 3.0 mm 3 for visualization.
Figure 3: Illustrative examples of DAGMNet false positives anterior to the temporal poles. Axial slices of the trace diffusion-weighted image (DWI), apparent diffusion coefficient map (ADC), and ground-truth annotations (GT) are shown alongside true-positive, false-negative, and false-positive voxels. All volumes were resampled to 0.8 × 0.8 × 3.0 mm 3 for visualization.
Figure 4: Illustrative examples of false positives produced by PAT , DAGMNet and/or nnU-Net in the posterior superior part of the brain. Axial slices of the trace diffusion-weighted image (DWI), apparent diffusion coefficient map (ADC), and ground-truth annotations (GT) are shown alongside true-positive, false-negative, and false-positive voxels. All volumes were resampled to 0.8 × 0.8 × 3.0 mm 3 for visualization.
Figure 5: Illustrative examples of false negatives produced by PAT and/or DAGMNet in the deep sylvian region. Axial slices of the trace diffusion-weighted image (DWI), apparent diffusion coefficient map (ADC), and ground-truth annotations (GT) are shown alongside true-positive, false-negative, and false-positive voxels. All volumes were resampled to 0.8 × 0.8 × 3.0 mm 3 for visualization.
Figure 6: Illustrative examples of false negatives produced by nnU-Net and/or ISLA in the deep sylvian region and detected by ISLA-MT . Axial slices of the trace diffusion-weighted image (DWI), apparent diffusion coefficient map (ADC), and ground-truth annotations (GT) are shown alongside true-positive, false-negative, and false-positive voxels. All volumes were resampled to 0.8 × 0.8 × 3.0 mm 3 for visualization.
Figure 7: Illustrative examples of false negatives produced by PAT , DAGMNet and/or nnU-Net in the left superior region of the brain. Axial slices of the trace diffusion-weighted image (DWI), apparent diffusion coefficient map (ADC), and ground-truth annotations (GT) are shown alongside true-positive, false-negative, and false-positive voxels. All volumes were resampled to 0.8 × 0.8 × 3.0 mm 3 for visualization.
Figure 8: Illustrative examples of false positives produced by ISLA-MT in the deep sylvian region. Axial slices of the trace diffusion-weighted image (DWI), apparent diffusion coefficient map (ADC), and ground-truth annotations (GT) are shown alongside true-positive, false-negative, and false-positive voxels. All volumes were resampled to 0.8 × 0.8 × 3.0 mm 3 for visualization.
DeepISLES
PAT
DAGMNet
nnU-Net
ISLA
ISLA-MT
DSC
0.644 (0.743)
0.762 (0.233)
0.735 (0.233)
0.765 (0.185)
0.772 (0.186)
0.781 (0.176)
precision
0.865 (0.264)
0.768 (0.212)
0.749 (0.260)
0.740 (0.235)
0.764 (0.201)
0.743 (0.213)
recall
0.606 (0.843)
0.820 (0.343)
0.804 (0.291)
0.880 (0.243)
0.860 (0.279)
0.891 (0.181)
Values are expressed as median (interquartile range).
Appendix
Table 1: Dice similarity coefficient (DSC), precision and recall in the external evaluation cohort.
DSC
HD95
AVD
ALD
LF1
DeepISLES
0.645 (0.745)
12.153 (39.426)
5.210 (14.167)
1 (2)
0.667 (0.478)
SEALS
0.552 (0.787)
19.321 (54.089)
6.385 (15.488)
1 (2)
0.667 (0.548)
NVAUTO
0.679 (0.427)
11.146 (32.509)
6.486 (17.027)
3 (4)
0.545 (0.427)
SWAN
0.595 (0.754)
14.290 (47.519)
5.216 (14.868)
2 (2)
0.667 (0.524)
Values are expressed as median (interquartile range).
Appendix
Table 1: Segmentation metrics of DeepISLES and its constituent models in the 588 participants with available T2-weighted fluid-attenuated inversion recovery images in the external evaluation cohort.
ablated variant
precision
recall
ISLA
1.0×1.0×2.0mm 3
+0.149∗∗
−0.469∗∗∗
0.8×0.8×3.0mm 3
−0.225∗∗∗
+0.172∗∗∗
StdUNet
−0.102∗
−0.035
ResUNet
−0.037
−0.090
DiceBCE
−0.867∗∗∗
+0.748∗∗∗
Appendix
Table 1: Comparison of precision and recall between the ablated and the full ISLA variants in the external evaluation cohort.
Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.
Computational Radiology & Artificial Intelligence Unit, Department of Physics and Computational Radiology, Oslo University Hospital, Oslo, Norway · Center for Lifespan Changes in Brain and Cognition, Department of Psychology, University of Oslo, Oslo, Norway · Division of Radiology and Nuclear Medicine, Oslo University Hospital, Oslo, Norway +5
College of Computing and Data Science, Nanyang Technological University, Singapore · Department of Radiology and Diagnostic Imaging, University of Alberta, Edmonton, Canada