ODySSeI: An Open-Source End-to-End Framework for Automated Detection, Segmentation, and Severity Estimation of Lesions in Invasive Coronary Angiography Images
Authors: Anand Choudhary, Xiaowu Sun, Thabo Mahendiran, Ortal Senouf, Denise Auberson, Bernard De Bruyne, Stephane Fournier, Olivier Muller, +3 more
Organizations: LTS4 Laboratory, EPFL, Lausanne, Switzerland · Chair of Mathematical Data Science, EPFL, Lausanne, Switzerland · School of AIAC, Xi’an Jiaotong-Liverpool University, Suzhou, China · Cardiology Department, CHUV, Lausanne, Switzerland · Cardiovascular Center, OLV, Aalst, Belgium · EPFL AI Center, Lausanne, Switzerland
Invasive Coronary Angiography (ICA) is the clinical gold standard for the assessment of coronary artery disease. However, its interpretation remains subjective and prone to intra- and inter-operator variability. In this work, we introduce ODySSeI: an Open-source end-to-end framework for automated Detection, Segmentation, and Severity estimation of lesions in ICA images. ODySSeI integrates deep learning-based lesion detection and lesion segmentation models trained using a novel Pyramidal Augmentation Scheme (PAS) to enhance robustness and real-time performance across diverse patient cohorts (2149 patients from Europe, North America, and Asia). Furthermore, we propose a quantitative coronary angiography-free Lesion Severity Estimation (LSE) technique that directly computes the Minimum Lumen Diameter (MLD) and diameter stenosis from the predicted lesion geometry. Extensive evaluation on both in-distribution and out-of-distribution clinical datasets demonstrates ODySSeI's strong generalizability. Our PAS yields large performance gains in highly complex tasks as compared to relatively simpler ones, notably, a 2.5-fold increase in lesion detection performance versus a 1-3% increase in lesion segmentation performance over their respective baselines. Our LSE technique achieves high accuracy, with predicted MLD values differing by only ± 2-3 pixels from the corresponding ground truths. On average, ODySSeI processes a raw ICA image within only a few seconds on a CPU and in a fraction of a second on a GPU and is available as a plug-and-play web interface at swisscardia.epfl.ch. Overall, this work establishes ODySSeI as a comprehensive and open-source framework which supports automated, reproducible, and scalable ICA analysis for real-time clinical decision-making.
Figures & tables
Figure 1: Overview of ODySSeI . (a) During inference, a raw ICA image undergoes detection, segmentation (after cropping and resizing the lesion instances detected in the image), and severity estimation of any lesions present in it. (b) Raw training data is constantly augmented using all the three tiers of our Pyramidal Augmentation Scheme (PAS) to iteratively fine-tune our lesion detection model for the identification of lesions and prediction of their bounding box coordinates. The fine-tuned lesion detection model is then integrated into ODySSeI for inference. (c) Cropped training data (only lesion instances) is constantly augmented using two tiers of our PAS to iteratively fine-tune our lesion segmentation model for the prediction of lesion geometry. The fine-tuned lesion segmentation model is then integrated into ODySSeI for inference. (d) The three tiers of our PAS: static, dynamic, and composite augmentations. (e) Our lesion severity estimation technique involves computing the arterial radius of every point on the centerline skeleton of the segmented lesion. Twice of the minimum between two radii peaks yields the Minimum Lumen Diameter (MLD). Complement of the ratio of the MLD to the highest peak’s diameter yields the Diameter Stenosis (DS).
Table 1: PAS-based data augmentation substantially boosts ODySSeI’s lesion detection performance . The static augmentation tier consists of seven domain-specific augmentations, the dynamic augmentation tier consists of five probabilistic configurational augmentations, and the composite augmentation tier consists of a scene-based augmentation. Note that the ‘none’ augmentation suite refers to the baseline model which is trained without any data augmentations.
a Mean ± standard deviation are reported over per-image metrics;
Table 2: Addition of an external patient cohort (ARCADE dataset) together with PAS-based data augmentation increases ODySSeI’s lesion detection performance . The AUG S -FAME2 dataset refers to the statically augmented FAME2 dataset. Dynamic and composite augmentations are always applied during training, on all datasets.
Figure 2: PAS-based data augmentation changes the learning dynamics of ODySSeI’s lesion detection model and encourages structured overfitting via static augmentations, imparts robustness and stabilization via dynamic augmentations, and improves precision in lesion localization via composite augmentations. Plots show (a) lesion-level [email protected] on the FAME2 validation set, (b) lesion-level [email protected] on the FAME2 validation set, and (c) loss on the FAME2 validation set.
Figure 3: Lesion Detection using ODySSeI: Visualization of predictions and the corresponding ground truths on a representative sample of the FAME2 validation set show the improved robustness of our best performing lesion detection model over the baseline.
a Mean ± standard deviation are reported over per-image metrics;
b Total values are reported (all instances of lesions across all images are considered together);
Table 3: Lesion Detection using ODySSeI: Test set performance.
Figure 4: Lesion Detection using ODySSeI: Visualization of predictions and the corresponding ground truths on a representative sample of the FAME2 test set show that our lesion detection model identifies and localizes lesions accurately and in some cases, identifies lesions which are missing from the corresponding ground truths.
Figure 5: Lesion Detection using ODySSeI: Visualization of predictions and the corresponding ground truths on a representative sample of the FC dataset show that our model identifies and precisely localizes most of the narrow lesions while occasionally missing out on identifying some wide lesions.
MLD-Based Metrics
Dataset
Treating CTPs as FPs
Treating CTPs as TPs
MLD-Precision
MLD-Recall
MLD-F1 Score
MLD-Precision
MLD-Recall
MLD-F1 Score
FAME2 (Validation)
0.764
0.545
0.636
0.982
0.607
0.750
FAME2 (Test)
0.593
0.427
0.496
1.000
0.557
0.715
FC
0.542
0.198
0.290
1.000
0.313
0.476
Table 4: High values of MLD-based metrics support ODySSeI’s clinical reliability. Treating CTPs as (actual) TPs leads to a further boost in ODySSeI’s lesion detection performance.
Augmentation Suite
Validation Set Metrics
Acc
Prec
Rec
Dice
IoU
clDice
MHD
None
0.947±0.033
0.860±0.094
0.912±0.074
0.883±0.076
0.798±0.106
0.934±0.081
1.542±3.351
Static
0.949±0.031
0.867±0.078
0.916±0.054
0.889±0.058
0.805±0.085
0.939±0.069
1.291±2.670
Static + Dynamic
0.952±0.028
0.876±0.068
0.921±0.049
0.896±0.051
0.816±0.073
0.948±0.060
0.740±1.743
Table 5: PAS-based data augmentation boosts ODySSeI’s lesion segmentation performance . The static augmentation tier consists of seven domain-specific augmentations and the dynamic augmentation tier consists of five probabilistic configurational augmentations. Note that the ‘none’ augmentation suite refers to the baseline model which is trained without any data augmentations.
Figure 6: PAS-based data augmentation changes the learning dynamics of ODySSeI’s lesion segmentation model and encourages structured overfitting via static augmentations while imparting robustness and stabilization via dynamic augmentations. Plots show (a) F1-Score on the FAME2 validation set and (b) loss on the FAME2 validation set.
Test Set Metrics
Dataset
Acc
Prec
Rec
Dice
IoU
clDice
MHD
FAME2 (Test)
0.941±0.040
0.867±0.065
0.935±0.031
0.898±0.038
0.817±0.060
0.948±0.048
0.796±1.309
FC
0.958±0.034
0.913±0.052
0.946±0.061
0.929±0.052
0.870±0.075
0.963±0.057
0.713±2.343
Table 6: Lesion Segmentation: Test set performance.
Figure 7: Lesion Segmentation using ODySSeI: Visualization of predictions and the corresponding ground truths on a representative sample of (a) the cropped FAME2 test set and (b) the cropped FC dataset show that our model segments lesions well across different cohorts.
Dataset
Metrics
MAD a ± SD b
Prec d (95% CI e )
Rec d (95% CI e )
F1 d (95% CI e )
Bal Acc d (95% CI e )
FAME2 (Validation)
2.378 ± 2.023
1.000 (0.657-1.000)
0.656 (0.400-0.664)
0.792 (0.505-0.794)
0.828 (0.363-0.835)
FAME2 (Test)
2.470 ± 2.133
1.000 (0.643-1.000)
0.656 (0.394-0.659)
0.792 (0.495-0.798)
0.828 (0.366-0.836)
FC
2.877 ± 2.444
1.000 (0.567-1.000)
0.684 (0.378-0.700)
0.812 (0.466-0.877)
0.842 (0.449-0.851)
a Mean Absolute Difference; b Standard Deviation; c Prec, Rec, F1, and Bal Acc are used to assess our
algorithm’s performance in determining lesions with an MLD ≤ 4 px. Note that a threshold of 6 px is used
Table 7: MLDs predicted with our QCA-free lesion severity estimation technique are in high agreement with the corresponding ground truths for both ID as well as OOD datasets.
Figure 8: Lesion Severity Estimation: Bland-Altman plots show high agreement between predicted and ground truth MLDs for the (a) FAME2 validation set, the (b) FAME2 test set, and the (c) FC dataset.
DTU Compute, Technical University of Denmark, Kongens Lyngby, Denmark · Cardiovascular Research Unit, Copenhagen University Hospital – Rigshospitalet, Copenhagen, Denmark
Department of Radiology, Northwestern University, Chicago, IL, USA · Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA · Department of Cardiology, Northwestern University, Chicago, IL, USA +1