The MAMA-MIA Challenge: Advancing Generalizability and Fairness in Breast MRI Tumor Segmentation and Treatment Response Prediction

Lidia Garrucho; Smriti Joshi; Kaisar Kushibar; Richard Osuala; Maciej Bobowicz; Xavier Bargalló; Paulius Jaruševičius; Kai Geissler

cs.CVMar 1, 2026

The MAMA-MIA Challenge: Advancing Generalizability and Fairness in Breast MRI Tumor Segmentation and Treatment Response Prediction

Authors: Lidia Garrucho, Smriti Joshi, Kaisar Kushibar, Richard Osuala, Maciej Bobowicz, Xavier Bargalló, Paulius Jaruševičius, Kai Geissler, Raphael Schäfer, Muhammad Alberb, Tony Xu, Anne Martel, Daniel Sleiman, Navchetan Awasthi, Hadeel Awwad, Joan C. Vilanova, Robert Martí, Daan Schouten, Jeong Hoon Lee, Mirabela Rusu, Eleonora Poeta, Luisa Vargas, Eliana Pastor, Maria A. Zuluaga, Jessica Kächele, Dimitrios Bounias, Alexandra Ertl, Katarzyna Gwoździewicz, Maria-Laura Cosaka, Pasant M. Abo-Elhoda, Sara W. Tantawy, Shorouq S. Sakrana, Norhan O. Shawky-Abdelfatah, Amr Muhammad Abdo-Salem, Androniki Kozana, Eugen Divjak, Gordana Ivanac, Katerina Nikiforaki, Michail E. Klontzas, Rosa García-Dosdá, Meltem Gulsun-Akpinar, Oğuz Lafcı, Carlos Martín-Isla, Oliver Díaz, Laura Igual, Karim Lekadir

Abstract

Breast cancer is the most frequently diagnosed malignancy among women worldwide and a leading cause of cancer-related mortality. Dynamic contrast-enhanced magnetic resonance imaging plays a central role in tumor characterization and treatment monitoring, particularly in patients receiving neoadjuvant chemotherapy. However, existing artificial intelligence models for breast magnetic resonance imaging are typically developed and evaluated using heterogeneous datasets, study populations, and assessment protocols, making direct comparison difficult and limiting understanding of model robustness across institutions and clinically relevant patient subgroups. The MAMA-MIA Challenge was designed to address these challenges by providing a standardized benchmark for the joint evaluation of primary tumor segmentation and prediction of pathologic complete response using pre-treatment magnetic resonance imaging only. The training cohort comprised 1,506 patients from multiple institutions in the United States, while evaluation was conducted on an external test set of 574 patients from three independent European centers to assess cross-continental and cross-institutional generalization. A unified scoring framework combined predictive performance with subgroup consistency across age, menopausal status, and breast density. Twenty-six international teams participated in the final evaluation phase. Results demonstrate substantial performance variability under a common external evaluation framework and reveal trade-offs between overall accuracy and subgroup fairness. The challenge provides standardized datasets, evaluation protocols, and public resources to promote the development of robust and equitable artificial intelligence systems for breast cancer imaging.

Figures & tables

Clinical variables
MAMA-MIA Dataset	Training					Validation / test				Total
Collection	NACT	ISPY1	ISPY2	DUKE	Sum	GUM	KAU	HCB	Sum	Train+Val+Test
Country	USA	USA	USA	USA		Poland	Lithuania	Spain
Year	2002–06	2010–16	2000–14	1995–02		2010–20	2010–20	2010–20
pCR Yes / No / N.A.	11 / 53 / 0	49 / 118 / 4	316 / 664 / 0	64 / 216 / 11	440 / 1051 / 15	11 / 19 / –	71 / 161 / –	78 / 234 / –	160 / 414 / –	600 / 1465 / 15
Tumor subtype Lum / TN / HER2enr HER2pure / N.A.	21 / 11 / 8 6 / 18	67 / 45 / 25 29 / 5	536 / 358 / 86 0 / 0	123 / 85 / 50 30 / 3	747 / 499 / 169 65 / 26	11 / 9 / 3 7 / –	95 / 71 / 39 27 / –	209 / 43 / 49 11 / –	315 / 123 / 91 45 / –	1062 / 622 / 260 110 / 26

TABLE I : Clinical and imaging variables across the MAMA-MIA benchmark datasets [ 31 ] . Four cohorts are used for training (NACT, ISPY1, ISPY2, DUKE) and three for validation/test (GUM, KAU, HCB). Data cells with more than 3 items are split into two rows to maximize readability. Best viewed in color.

	❶ MIC	❷ FME	❸ ViCOROB	❹ Martel Lab	❺ AIH-MAMA
Preprocessing Strategy
resample	third-order spline $1\times 1\times 1$	nnUNet pre $0.7\times 0.7\times 2.0$	b-spline $1\times 1\times 1$	trilinear $1\times 1\times 1$	bilinear $1\times 1\times 1$
crop or pad	×	×	breast bbox	breast bbox	non-zero voxels $320\times 320\times 128$
reorient	×	×	×	RAS	RAS
intensity clipping	×	×	percentile $[0.1,99.9]$	percentile $[0.05,99.95]$	×
normalize	z-score	z-score	z-score	min-max	z-score (non-zero pixels only)

TABLE II : Technical summary of top-performing primary tumor segmentation methods (Task 1). Abbreviations used in the table- nnUNet pre: default nnUNet [ 13 ] preprocessing strategy; RAS: Right-Anterior-Superior orientation; p0/p1/p2: pre-contrast, first post-contrast and second post-contrast phases; px: last available post-contrast phase; s1: subtraction of first post-contrast phase with pre-contrast phase; bbox: Bounding Box; CE: Cross Entropy; CV: Cross Validation; n/a: not applicable/available.

	① pimed-lab	② FME	③ AI Strollers
Preprocessing
resample	$1\times 1\times 1$	varying per model	nnUNet pre [ 13 ] $0.7\times 0.7\times 2.0$
crop / pad	lesion bbox	lesion bbox	×
reorient	×	×	×
clip	×	×	×
normalize	z-score	z-score	z-score

TABLE III : Technical summary of top-performing pathologic complete response (pCR) prediction methods (Task 2). Abbreviations used in the table - nnUNet pre: default nnUNet [ 13 ] preprocessing strategy; bbox: Bounding Box; p0/p1/p2: pre-contrast, first post-contrast and second post-contrast phases; px: last available post-contrast phase; BCE: Binary Cross Entropy; CE: Cross Entropy; CV: Cross Validation; n/a: not applicable/available.

Panel of Figure 4 — Fig. 1 : Fairness plotted against performance across the two MAMA-MIA benchmark tasks. Teams with fairness-performance tradeoffs within the area indicated in green improve upon the baseline tradeoff. Note that the axis ranges are restricted to improve readability; consequently, teams falling outside these bounds are not shown.

Fig. 2 : Qualitative results for challenging cases. The columns correspond to the MRI image, the zoomed-in tumor region (Tumor Zoom), and the top-5 team predictions (predicted segmentations) overlaid with the expert annotated tumor segmentation (ground truth). Results are shown for: (a) clip artifact (successful), (b) breast implant (failure), (c) low contrast (failure), (d) small tumor (high variance between teams), and (e) non-mass enhancement (high variance between teams). Artifacts, low discriminability, and atypical morphology remain key challenges.

Rank	Team	Combined Score [95% CI] $\uparrow$	Fairness Score [95% CI] $\uparrow$	Performance Score [95% CI] $\uparrow$	DSC [95% CI] $\uparrow$	NormHD [95% CI] $\downarrow$
1	MIC	0.8858 [0.8577, 0.8969]	0.9531 [0.9051, 0.9651]	0.8185 [0.8018, 0.8350]	0.7360 [0.7160, 0.7539]	0.0990 [0.0816, 0.1172]
2	FME	0.8820 [0.8464, 0.8913]	0.9574 [0.8976, 0.9659]	0.8066 [0.7873, 0.8248]	0.7125 [0.6895, 0.7330]	0.0993 [0.0815, 0.1182]
3	ViCOROB	0.8782 [0.8485, 0.8891]	0.9482 [0.8963, 0.9643]	0.8083 [0.7913, 0.8248]	0.7182 [0.6976, 0.7381]	0.1017 [0.0859, 0.1189]
4	Martel Lab	0.8735 [0.8471, 0.8856]	0.9449 [0.8999, 0.9619]	0.8021 [0.7863, 0.8182]	0.7121 [0.6918, 0.7315]	0.1078 [0.0915, 0.1244]
5	AIH-Mama	0.8677 [0.8379, 0.8781]	0.9532 [0.8984, 0.9661]	0.7823 [0.7634, 0.8012]	0.6914 [0.6701, 0.7139] ∗	0.1268 [0.1081, 0.1452]
6	HWT@YCH	0.8655 [0.8365, 0.8789]	0.9339 [0.8843, 0.9514]	0.7971 [0.7798, 0.8143]	0.7080 [0.6872, 0.7273]	0.1138 [0.0967, 0.1324]

TABLE IV : Final leaderboard for Task 1: Primary Tumor Segmentation. Confidence intervals (95%) for all Task 1 metrics were estimated using a non-parametric percentile bootstrap (

B=1000

, seed=42). DSC, NormHD, and Performance Score CIs were obtained by resampling patients with replacement and recomputing the mean metric on each bootstrap sample. Fairness Score CIs were computed using patient-level bootstrapping with repeated group-level disparity estimation across demographic groups. Combined Score CIs were derived jointly from the same bootstrap resamples for both performance and fairness components.

Rank	Team	Combined Score [95% CI] $\uparrow$	Fairness Score [95% CI] $\downarrow$	Performance Score [95% CI] $\uparrow$	AUC [95% CI] $\uparrow$
1	pimed-lab	0.6907 [0.6364, 0.7272]	0.1150 [0.0399, 0.2292]	0.4964 [0.4853, 0.5102]	0.4556 [0.4035, 0.5121]
2	FME	0.6642 [0.4893, 0.6602]	0.2121 [0.2232, 0.5561]	0.5405 [0.4992, 0.5841] +	0.5722 [0.5195, 0.6305]
3	AI Strollers	0.6625 [0.4778, 0.6569]	0.2013 [0.2148, 0.5663]	0.5263 [0.4870, 0.5658]	0.5218 [0.4629, 0.5771]
4	Baseline	0.6431 [0.4600, 0.6360]	0.2179 [0.2418, 0.5701]	0.5042 [0.4551, 0.5498]	0.5031 [0.4455, 0.5551]
5	BWS-KNU	0.6355 [0.4792, 0.6325]	0.2177 [0.2205, 0.5324]	0.4886 [0.4478, 0.5314]	0.5075 [0.4513, 0.5656]
6	Martel Lab	0.6310 [0.4802, 0.6316]	0.2272 [0.2397, 0.5164]	0.4892 [0.4477, 0.5316]	0.4740 [0.4195, 0.5330]

TABLE V : Final leaderboard for Task 2: Treatment Response Prediction to Neoadjuvant Chemotherapy. Note that AUC is reported as an additional result and is excluded from the ranking scheme. Confidence intervals (95% CI) were estimated using a non-parametric percentile bootstrap (

B=1000

, seed=42). Balanced Accuracy and AUC were bootstrapped from prediction–label pairs, while Fairness and Combined Scores used patient-level resampling with repeated equalized-odds disparity computation across demographic groups. The Combined Score CI was obtained by repeating the resampling jointly for performance and fairness components.

Fig. 3 : Impact of tumor size on segmentation performance. (a) Histogram of tumor volumes ( $mm^{3}$ ) on a logarithmic scale, with dashed lines indicating the 20th and 80th percentile thresholds used to categorize tumors as Small, Medium, or Large, (b) Percentage distribution of tumor size categories across the three participating centers (GUM, KAU, HCB), with boxed values showing the average Dice Similarity Coefficient (DSC) for the top 5 teams in each category. (c) Comparison of DSC distributions between the top 5 and bottom 5 performing teams across tumor size categories, with gap bars ( $\Delta$ ) indicating the difference in mean DSC. Brackets give two-sided Wilcoxon signed-rank p-values comparing the Top 5 against the Bottom 5 teams. For each patient, the mean DSC across the five top-ranked teams was compared with the mean DSC across the five bottom-ranked teams, so patients are the unit of analysis and the two arms are paired on identical cases.

Fig. 4 : Distribution of DSC across the three participating centers (GUM, KAU, HCB) for the top 5 performing teams in Task 1. Each boxplot represents the per-case DSC distribution for a team, in order of their ranking. Horizontal dashed lines indicate the average DSC across all top 5 teams for each center.

Panel of Figure 10 — Fig. 5 : Subgroup analysis of segmentation performance. Boxplots show DSC distributions for the top 5 (blue) and bottom 5 (red) performing teams, stratified by (a) breast density, (b) age group, (c) menopausal status, and (d) Tumor Subtype. Brackets give two-sided Wilcoxon signed-rank p-values comparing the Top 5 against the Bottom 5 teams. For each patient, the mean DSC across the five top-ranked teams was compared with the mean DSC across the five bottom-ranked teams, so patients are the unit of analysis and the two arms are paired on identical cases. The performance of high-ranking methods consistently improve in comparison to the low-ranking methods. There is no substantial differences between subgroups in the challenge fairness categories. Across tumor subtype categories, the triple_negative subtype has the highest overall median DSC, while luminal category is especially challenging for the weaker models.

Panel of Figure 11 — Fig. 6 : Fairness-versus-performance evaluation tradeoffs: Effect of $\lambda$ on the ranking across tasks and participating teams.

Fig. 7 : Recall comparison between pCR (minority) and not pCR (majority) classes across participating teams. $\Delta$ denotes the recall gap, reflecting class-wise prediction bias.

Fig. 8 : Reliability diagrams for top 3 teams in pCR prediction. Predicted probabilities are grouped into quantile bins; the fraction of true positives per bin is plotted against mean predicted probability. Dashed line: perfect calibration; dotted line: class prevalence. Annotated values indicate bin sample sizes.

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 768 papers

Subtraction-Based Tumor Segmentation and Lesion-Centered pCR Prediction for the MAMA-MIA Challenge
Aug 29, 2026Kai Geissler, Raphael SchäferClinical Outcome Prediction Medical Image Segmentation
Fraunhofer Institute for Digital Medicine MEVIS Bremen, Germany
BenchX: Benchmarking AI Models for Cancer Detection and Localization with Demographic and Protocol Biases
Jun 23, 2026Qi Chen, Wenxuan Li, Pedro R. A. S. Bassi +14Distribution Shift Medical Image Analysis
Johns Hopkins University · German Cancer Research Center · University Hospital Basel +10
Cross-Dataset Generalization in Breast MRI Tumor Classification via Class-Wise Dataset Mixing
Jul 21, 2026Mohammad Ali Dadrast, Hamid UsefiAlgorithmic Bias Medical Image Classification
Data Science Program Memorial University of Newfoundland St. John’s, Canada · Department of Mathematics and Statistics Memorial University of Newfoundland St. John’s, Canada

Aug 29, 2026J/K move · Enter open · S save

Subtraction-Based Tumor Segmentation and Lesion-Centered pCR Prediction for the MAMA-MIA Challenge

Kai Geissler, Raphael Schäfer

Fraunhofer Institute for Digital Medicine MEVIS Bremen, Germany

Clinical Outcome Prediction Medical Image Segmentation 3D Medical Image Segmentation Magnetic Resonance Imaging

Open paper AskPDF arXiv

We describe the submission of team FME to the MAMA-MIA Challenge, which evaluated primary tumor segmentation and prediction of pathological complete response (pCR) from pretreatment dynamic contrast-enhanced breast MRI on an external multi-country cohort. For segmentation, we trained a five-fold residual-encoder nnU-Net ensemble using only the first post-contrast minus pre-contrast image, combined with mirroring test-time augmentation and largest-connected-component filtering. For pCR prediction, we ensembled 25 pretrained 3D video classifiers trained on lesion-centred crops from the pre-contrast and first two post-contrast volumes. FME ranked second in both tasks. The segmentation method achieved a combined performance-fairness score of 0.882, with Dice 0.713 and normalized Hausdorff distance 0.099. The pCR method achieved a combined score of 0.664, balanced accuracy of 0.541, and equalized-odds disparity of 0.212. The results indicate that subtraction-based input and ensembling support robust tumor segmentation under cross-site domain shift, whereas pCR prediction from baseline DCE-MRI alone remains limited. For the submission repository, see https://github.com/FraunhoferMEVIS/MAMA-MIA-Challenge-FME

Abstract

Medical Image Segmentation

Explore similar work

Subtraction-Based Tumor Segmentation and Lesion-Centered pCR Prediction for the MAMA-MIA Challenge

BenchX: Benchmarking AI Models for Cancer Detection and Localization with Demographic and Protocol Biases

Cross-Dataset Generalization in Breast MRI Tumor Classification via Class-Wise Dataset Mixing