The MYOSAIQ Challenge: Myocardial Segmentation with Automated Infarct Quantification
Organizations: INSA-Lyon, Universite Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1294, F-42023, Saint Etienne, France · Institut Universitaire de France (IUF), France · Department of Computer Science, University of Sherbrooke, Sherbrooke, QC, Canada · Wuhan National Library of Optoelectronics, Huazhong University of Science and Technology, Wuhan, China · Sunnybrook Research Institute, University of Toronto, Toronto, Canada · National Heart and Lung Institute, Faculty of Medicine, Imperial College London, London, United Kingdom · Centre for Medical Image Computing, Department of Computer Science, University College London, London, United Kingdom · Fudan University, Shanghai, 200433, China · Gottfried Schatz Research Center: Medical Physics and Biophysics, Medical University of Graz, Graz, Austria · Institute of Computer Graphics and Vision, Graz University of Technology, Graz, Austria · Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Graz, Austria · Instituto de Investigaci´on en Ingenier´ıa de Arag´on, Universidad de Zaragoza; Instituto de Investigaci´on Sanitaria de Arag´on (IIS Arag´on); CIBER-BBN, Zaragoza, Arag´on, Spain · Centre Hospitalier Annecy Genevois, France · Hˆopital Universitaire Cardiologique Louis Pradel, Bron, France · CarMeN INSERM U1060, Universite Claude Bernard Lyon 1, Lyon, France · Hˆopital Universitaire de Saint-Etienne, Saint-Etienne, France.
Abstract
Late gadolinium enhancement (LGE) cardiac magnetic resonance (MR) imaging is the modality of choice to assess myocardial infarction (MI) lesions. Nowadays MI volume quantification is not performed routinely in clinical practice. Numerous deep learning (DL) methods have been developed to automate the segmentation of the myocardium and infarct regions. However, most studies rely on relatively small datasets which typically undergo pre-processing steps to standardize images and focus on a specific phase of myocardial infarction following reperfusion therapy. These limitations have impeded the development of models that are generalizable across diverse conditions and thus suitable for routine clinical use. To advance research and establish benchmarks in generalizable learning for myocardial infarct quantification, this paper presents findings from the Myocardial Segmentation with Automated Infarct Quantification (MYOSAIQ) challenge. The dataset set up for the challenge combines 439 CMR volumes from two multicenter clinical trials, with representative data acquired in acute and chronic phases after acute MI. Data were acquired in 16 centers using MRI scanners from three different vendors. Six teams participated until the end of the challenge, employing various baseline models, data augmentation techniques, and confidence strategies. To enhance the significance of this study, we compare the challengers' results with those of fine-tuned foundation models. Our results indicate that well-designed UNet-based techniques outperform fully automatic foundation models for LGE MR segmentation. While the best methods achieve high-quality and stable delineations of the left ventricle and myocardium under various conditions, they remain improvable in accurately segmenting infarct regions.
Figures & tables
| Territory | D8 | M1 | M12 | ||
| Clinical metrics | |||||
| Transmurality | LAD | ||||
| (Extent of a lesion across) | (%) | LCX | |||
| the thickness of the muscle) | RCA | ||||
| LAD | |||||
| Endocardial Surface | (%) | LCX |
| \Block 2-1 Method | \Block 1-3 LV | \Block 1-3 MYO | \Block 1-3 MI | \Block 1-5 MVO | ||||||||||
| Dice | ASSD (mm) | HD (mm) | Dice | ASSD (mm) | HD (mm) | Dice | ASSD (mm) | HD (mm) | Dice | ASSD (mm) | HD (mm) | TP | TN | |
| P1 | 0.928 0.031 | 0.5 0.2 | 7.2 2.4 | 0.840 0.036 | 0.4 0.2 | 9.0 2.8 | 0.669 0.177 | 1.0 1.7 | 19.8 14.4 | 0.537 0.190 | 1.2 1.0 | 19.5 14.6 | 10/10 | 15/15 |
| \arrayrulecolor gray!50 \arrayrulecolor black P2 | 0.933 0.029 | 0.4 0.2 | 6.8 2.1 | 0.840 0.038 | 0.4 0.1 | 9.0 3.0 | 0.700 0.145 | 1.0 1.7 | 16.8 13.2 | 0.681 0.119 | 0.7 0.4 | 14.4 4.7 | 9/10 | 15/15 |
| \arrayrulecolor gray!50 \arrayrulecolor black P3 | 0.930 0.034 | 0.5 0.4 | 7.7 5.8 | 0.841 0.041 | 0.4 0.2 | 9.9 6.1 | 0.688 0.168 | 1.0 1.3 | 17.3 12.4 | 0.648 0.140 | 0.8 0.4 | 14.1 5.0 | 8/10 | 15/15 |
| \arrayrulecolor gray!50 \arrayrulecolor black P4 | 0.926 0.031 | 0.5 0.2 | 7.3 2.4 | 0.827 0.043 | 0.4 0.1 | 9.5 3.3 | 0.660 0.159 | 1.0 1.5 | 19.5 13.5 | 0.647 0.126 | 0.7 0.3 | 13.5 5.0 | 8/10 | 15/15 |
| \arrayrulecolor gray!50 \arrayrulecolor black P5 | 0.914 0.034 | 0.6 0.3 | 7.9 3.2 | 0.806 0.043 | 0.4 0.2 | 10.6 3.4 | 0.628 0.170 | 1.2 1.8 | 23.9 14.9 | 0.552 0.186 | 1.0 0.5 | 16.2 3.4 | 9/10 | 15/15 |
| \Block 2-1 Method | \Block 1-3 LV | \Block 1-3 MYO | \Block 1-3 MI | \Block 1-3 MVO | ||||||||
| Dice | ASSD (mm) | HD (mm) | Dice | ASSD (mm) | HD (mm) | Dice | ASSD (mm) | HD (mm) | Dice | ASSD (mm) | HD (mm) | |
| P1 | 0.928; 0.004 [0.921, 0.935] | 0.5; 0.0 [0.4, 0] | 7.2; 0.3 [6.7, 7.7] | 0.840; 0.004 [0.832, 0.848 ] | 0.4; 0.0 [0.3, 0 ] | 9.0; 0.3 [8.4, 9.6] | 0.669; 0.02 [0.627, 0.706] | 1.0; 0.2 [0.7, 0] | 19.8; 1.6 [16.8, 23.2] | 0.538; 0.063 [0.404, 0.652] | 1.2; 0.3 [0.7, 0] | 19.6; 4.6 [12.3, 29.5] |
| \arrayrulecolor gray!50 \arrayrulecolor black P2 | 0.933; 0.003 [0.926, 0.939] | 0.4; 0.0 [0.4, 0] | 6.8; 0.2 [6.4, 7.3] | 0.840; 0.004 [0.831, 0.848 ] | 0.4; 0.0 [0.3, 0 ] | 9.0; 0.3 [8.4, 9.7] | 0.700; 0.016 [0.668, 0.73] | 1.0; 0.2 [0.7, 0] | 16.9; 1.5 [14.1, 19.9] | 0.681; 0.039 [0.601, 0.756] | 0.7; 0.1 [0.5, 0] | 14.4; 1.5 [11.6, 17.6] |
| \arrayrulecolor gray!50 \arrayrulecolor black P3 | 0.930; 0.004 [0.922, 0.937] | 0.5; 0.0 [0.4, 0] | 7.7; 0.7 [6.7, 9.2] | 0.841; 0.005 [0.832, 0.85 ] | 0.4; 0.0 [0.3, 0 ] | 9.9; 0.7 [8.7, 11.4] | 0.688; 0.019 [0.648, 0.723] | 1.0; 0.2 [0.7, 0] | 17.3; 1.4 [14.7, 20.2] | 0.647; 0.05 [0.545, 0.741] | 0.8; 0.2 [0.5, 0] | 14.1; 1.8 [10.9, 17.8] |
| \arrayrulecolor gray!50 \arrayrulecolor black P4 | 0.926; 0.003 [0.919, 0.933] | 0.5; 0.0 [0.4, 0] | 7.3; 0.3 [6.8, 7.8] | 0.827; 0.005 [0.818, 0.836 ] | 0.4; 0.0 [0.4, 0 ] | 9.5; 0.4 [8.8, 10.3] | 0.659; 0.018 [0.623, 0.693] | 1.0; 0.2 [0.7, 0] | 19.5; 1.5 [16.6, 22.7] | 0.647; 0.045 [0.555, 0.73] | 0.7; 0.1 [0.5, 0] | 13.5; 1.8 [9.9, 17.0] |
| \arrayrulecolor gray!50 \arrayrulecolor black P5 | 0.914; 0.004 [0.906, 0.921] | 0.6; 0.0 [0.5, 0] | 7.9; 0.4 [7.3, 8.6] | 0.806; 0.005 [0.797, 0.816 ] | 0.4; 0.0 [0.4, 0 ] | 10.6; 0.4 [9.9, 11.4] | 0.627; 0.019 [0.59, 0.663] | 1.2; 0.2 [0.9, 0] | 23.9; 1.7 [20.7, 27.2] | 0.553; 0.062 [0.431, 0.674] | 1.0; 0.2 [0.7, 0] | 16.2; 1.1 [14.0, 18.3] |
| \Block 2-1 Method | \Block 1-4 LV | \Block 1-4 MYO | \Block 1-4 MI | \Block 1-4 MVO | ||||||||||||
| CC | MAE (mL) | LOA (mL) | CRPS | CC | MAE (mL) | LOA (mL) | CRPS | CC | MAE (mL) | LOA (mL) | CRPS | CC | MAE (mL) | LOA (mL) | CRPS | |
| P1 | 0.969 | 8.2 | 23.8 | 0.002 | 0.967 | 8.6 | 24.5 | 0.004 | 0.882 | 3.4 | 12.1 | 0.001 | 0.972 | 2.7 | 6.5 | 0.000 |
| \arrayrulecolor gray!50 \arrayrulecolor black P2 | 0.980 | 6.7 | 18.0 | 0.011 | 0.946 | 11.2 | 31.1 | 0.018 | 0.892 | 3.4 | 12.3 | 0.005 | 0.989 | 1.1 | 3.0 | 0.003 |
| \arrayrulecolor gray!50 \arrayrulecolor black P3 | 0.978 | 7.4 | 19.6 | 0.012 | 0.924 | 13.1 | 38.1 | 0.022 | 0.888 | 3.2 | 11.7 | 0.006 | 0.990 | 2.6 | 4.9 | 0.006 |
| \arrayrulecolor gray!50 \arrayrulecolor black P4 | 0.977 | 7.5 | 20.4 | 0.012 | 0.918 | 12.3 | 38.2 | 0.021 | 0.868 | 3.3 | 12.6 | 0.006 | 0.997 | 1.4 | 2.9 | 0.004 |
| \arrayrulecolor gray!50 \arrayrulecolor black P5 | 0.976 | 8.1 | 19.9 | 0.014 | 0.923 | 17.5 | 52.5 | 0.030 | 0.902 | 3.5 | 11.6 | 0.007 | 0.970 | 2.5 | 5.6 | 0.006 |
| Method | LV | MYO | MI | MVO |
| P2 | 0.5 0.2 | 0.4 0.1 | 1.0 1.7 | 0.7 0.3 |
| \arrayrulecolor gray!50 \arrayrulecolor black P5 | 0.6 0.2 | 0.5 0.2 | 1.2 1.8 | 1.0 0.5 |
| mcMedSAM auto | 0.4 0.1 | 0.4 0.1 | 1.3 1.2 | 1.9 1.1 |
| Inter-obs | 0.3 0.3 | 0.2 0.2 | 0.2 0.3 | 0.1 0.1 |
| Method | LV | MYO | MI | MVO |
| P2 | 0.3 0.2 | 0.2 0.1 | 1.4 2.1 | – – |
| \arrayrulecolor gray!50 \arrayrulecolor black P5 | 0.3 0.2 | 0.4 0.1 | 0.8 0.4 | – – |
| mcMedSAM auto | 0.3 0.1 | 0.4 0.1 | 2.2 1.1 | – – |
| Inter-obs | 0.2 0.2 | 0.2 0.1 | 0.1 0.1 | – – |