MID: Mask-Image Distributional Divergence for Evaluating Medical Image Segmentation

Vincenzo Marcianò; Xiaoming Zhang; Gianluca Guglielmo; Michela Antonelli; Sebastien Ourselin; Maria A. Zuluaga

cs.CVOct 12, 2026

MID: Mask-Image Distributional Divergence for Evaluating Medical Image Segmentation

Authors: Vincenzo Marcianò, Xiaoming Zhang, Gianluca Guglielmo, Michela Antonelli, Sebastien Ourselin, Maria A. Zuluaga

Organizations: EURECOM · Sorbonne Université · King’s College London · Technische Universität Graz

Abstract

Current evaluation protocols for medical image segmentation remain largely anchored in sample-wise comparisons, e.g., Dice and Hausdorff distance, which require each prediction to be paired with a ground-truth mask. Yet two limitations arise in practice: they are inapplicable when predictions are made on unlabeled data; and even when annotations are available, they cannot assess whether a collection of predictions matches the distributional structure of the ground-truth population. We introduce MID (Mask-Image Distributional Divergence), a distribution-level evaluation metric for medical image segmentation. MID builds upon the observation that segmentation quality is a property of the joint image-mask distribution: a mask may be anatomically plausible in isolation yet incorrect for the image it accompanies. MID compares the joint distribution of images and masks between a reference set and a set of predictions, without requiring sample-wise correspondence. This captures both mask realism and image-mask consistency. We evaluate MID across 13 publicly available datasets spanning CT, MRI, and MRA. Under anatomically realistic corruptions across 56 organ groups, MID tracks mean Dice and Hausdorff distance without per-sample ground-truth pairing. MID also detects distributional shifts across modalities and anatomies, identifies image-mask misalignment through controlled shuffling experiments, and produces reliable model rankings across real segmentation models. Official implementation available at github.com/erc-caravel/pytorch-mid

Figures & tables

Table 1

Figure 1: Representative examples from the controlled corruption sweep (Experiment 1). The four panels show Task10 Colon with blob-shaped false positives, Task09 Spleen with smooth dilation, ACDC left ventricle with blob-shaped false positives, and Task07 Pancreas with smooth erosion. Overlays show the ground-truth mask and corrupted masks at mild, moderate, and severe levels. Line plots show $\mathrm{MID}$ , Dice, and HD95 as a function of degradation severity. $\mathrm{MID}$ is normalised by the panel-wise maximum for visualisation only; all reported correlations use raw values.

Figure 2: ROOD-MRI prostate on Task05_Prostate using SegResNet-300. For each artefact type, representative T2 slices across severity levels are shown above, with population metrics below.

Figure 3: a) $\mathrm{MID}$ vs. Dice across real segmentation model outputs. Points are coloured by model; y-axis is log-scaled. b) Ground-truth and predicted-mask distributions for three IXI MRA vessel segmentation methods.

Figure 4: $\mathrm{MID}$ decomposition across corruption types on TotalSegmentator v2 CT. (a) Stacked bars show the image, mask, and residual correspondence terms for each corruption type and severity level (m = mild, M = moderate, S = severe). (b) Component trends as severity increases.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Modality	Structures	$N_{\text{repr}}$
AbdomenCT-1K ∙ [ 28 ]	CT	$4$	$2888$
ACDC [ 4 ]	MRI	$3$	$900$
AMOS CT ∙ [ 20 ]	CT	$15$	$3000$
AMOS MRI ∙	MRI	$13$	$520$
IXI MRA (VesselVerse [ 6 , 12 ] )	MRA	$1$	$49$
Task03 Liver (Decathlon [ 1 ] )	CT	$2$	$262$

Appendix

Table 3: Datasets used in the experiments. Multimodal datasets are listed once per modality, giving

13

entries in total. Datasets marked ∙ are held out from the reference pool and used only as candidate sets for cross-dataset evaluation; the remaining

10

datasets form the reference pool described in Section 4.1 . ∗ 56 foreground classes are used across CT and MRI; the full 104-class annotation is available. ‡ Peripheral zone (PZ) and transition zone (TZ).

Figure 5: Anatomically realistic SDT-based mask corruptions applied to five organs. Columns show the original mask and five corruption types: elastic deformation, smooth erosion, smooth dilation, blob-shaped false positives, and partial missing slices. Overlay colours: green = preserved region, red = false positive, blue = false negative.

Figure 6: Within-group $\mathrm{MID}$ distribution across datasets. Each curve shows the kernel density estimate of clean-vs-clean $\mathrm{MID}$ values over organ groups within a dataset.

Figure 7: Per-anatomy

\mathrm{MID}

reference index across the organ groups: CT/MRI structures from TotalSegmentator and overlapping MSD/AbdomenCT-1K annotations, plus cerebral arteries from IXI MRA. Each horizontal bar spans the within-group clean-baseline

\mathrm{MID}

(diamond) to the severe-corruption

\mathrm{MID}

(triangle), with mild (circle) and moderate (square) markers at intermediate levels. Values are averaged over five SDT-based corruption types. Solid lines indicate MRI-derived organs; dotted lines indicate CT organs.

Category	Metric	Encoding	vs. Dice		vs. HD95
Category	Metric	Encoding	$\rho$	$95\text{\,}\mathrm{\%}$ CI	$\rho$	$95\text{\,}\mathrm{\%}$ CI
Ours	$\mathrm{MID}$	MedSigLIP	$-0.55$	$[-0.61,\,-0.49]$	$+0.72$	$[+0.66,\,+0.77]$
Baselines	FID-mask, 3-D	MedSigLIP	$-0.47$	$[-0.53,\,-0.40]$	$+0.61$	$[+0.54,\,+0.67]$
	FID-img, 3-D	MedSigLIP	$-0.03$	$[-0.10,\,+0.04]$	$+0.04$	$[-0.04,\,+0.12]$
	KID	MedSigLIP	$-0.28$	$[-0.36,\,-0.20]$	$+0.36$	$[+0.28,\,+0.44]$
	FID-Inception, 2-D	Inception-v3	$-0.38$	$[-0.56,\,-0.18]$	$+0.49$	$[+0.31,\,+0.65]$

Appendix

Table 4: Metric comparison on the controlled corruption sweep. Spearman

\rho

is computed cell-by-cell against mean Dice and against HD95 across

840

cells (

56

organ groups

\times

5 corruption types

\times

3 severity levels); bootstrap CIs use 1000 replicates. The two reference targets are inversely related (Dice goes up, HD95 goes down for better segmentations), so a metric that tracks segmentation quality should produce a negative

\rho

against Dice and a positive

\rho

against HD95 of comparable magnitude.

Organ	Candidate	Universal	Organ-PCA
Liver	TotalSeg CT held-out (in-distribution)	0.050	0.049
	Task03 liver, class 1 (healthy, cross-dataset)	0.154	0.152
	Task03 liver tumour, class 2 (pathological)	0.165	0.162
Pancreas	TotalSeg CT held-out (in-distribution)	0.038	0.037
	Task07 pancreas, class 1 (healthy, cross-dataset)	0.238	0.236
	Task07 pancreas tumour, class 2 (pathological)	0.216	0.214

Appendix

Table 5:

\mathrm{MID}

between TotalSegmentator CT reference and three candidate sets, under universal and organ-specific projections (

k=128

). Reference: TotalSegmentator CT training split; organ-specific PCA is fitted on the same reference.

Statistic	Candidate set size $n$
Statistic	10	25	50	100	250	500	1000
Mean $\mathrm{MID}$	1.85	0.92	0.58	0.42	0.34	0.31	0.30
CV	32%	20%	14%	10%	6.3%	4.5%	3.2%

Appendix

Table 6: Sample-size sensitivity of

\mathrm{MID}

in a cross-dataset setting (TotalSeg CT reference vs. TotalSeg MRI candidate; asymptotic

\mathrm{MID}

\approx 0.30

). The reference set is fixed at

n=500

; candidate sets are subsampled at different sizes. Mean and coefficient of variation (CV) are computed over 50 bootstrap replicates. The decreasing mean reflects the well-known finite-sample positive bias of the squared 2-Wasserstein distance between Gaussian fits.

Figure 9: t-SNE visualisation of MedSigLIP embeddings for eleven organ types. Left: mask-only embeddings. Right: joint image-mask embeddings. Points are coloured by organ, with circles denoting CT samples and triangles denoting MRI samples. Anatomical structures form coherent clusters in both spaces, whereas MRI samples separate from their CT counterparts mainly in the joint embedding space, suggesting that modality-related variation is primarily captured by the image component.

Figure 10: Mean per-organ $|\rho(\mathrm{MID}{},\mathrm{Dice})|$ for five encoder backbones on the TotalSegmentator v2 CT corruption sweep. Each bar reports the mean over per-organ rank correlations.

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 796 papers

MedSegBenchmarker: A Raw-Count-First Framework for Controlled 2D Medical Image Segmentation Benchmarks
Aug 30, 2026Vanessa Borst, Lukas Horn, Daniel Grillmeyer +2Medical Image Segmentation Medical Image Benchmarks
University of Würzburg, Germany
Few-Shot Distribution-Aligned Flow Matching for Data Synthesis in Medical Image Segmentation
Apr 3, 2026Jie Yang, Ziqi Ye, Aihua Ke +3Synthetic Data Augmentation Medical Image Synthesis
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University, Wuhan, China · Shanghai Innovation Institute, Shanghai, China · Shanghai AI Laboratory, Shanghai, China
A Decision-Aware Evaluation Protocol for Uncertainty in Medical Image Segmentation
Oct 15, 2026Saket Maganti, Shreyank N GowdaUncertainty Quantification Segmentation Uncertainty Estimation
Independent Researcher · School of Computer Science, University of Nottingham, Nottingham, UK

Aug 30, 2026J/K move · Enter open · S save

MedSegBenchmarker: A Raw-Count-First Framework for Controlled 2D Medical Image Segmentation Benchmarks

Vanessa Borst, Lukas Horn, Daniel Grillmeyer, Thomas Prantl+1

University of Würzburg, Germany

Medical Image Segmentation Medical Image Benchmarks

Open paper AskPDF arXiv

Despite rapid advances in MIS, fair and reproducible comparisons of segmentation models remain challenging due to heterogeneous datasets, inconsistent evaluation protocols, and rapidly evolving architectures. In particular, comparisons often implicitly assume that model rankings are invariant to data partitioning, preprocessing, metric aggregation, uncertainty estimation, and computational constraints. The lack of extensible and unified evaluation frameworks further limits systematic investigation of new models, datasets, and training paradigms. We present MEDSEGBENCHMARKER (MSB), a configuration-driven framework for controlled benchmarking of 2D MIS. It integrates duplicate and near-duplicate image detection, group-aware data splitting, YAML study specifications, resumable training, hyperparameter optimization, cross-validation, and checkpoint-based evaluation. Rather than retaining only aggregate performance measures, MSB exports sample- and class-level pixel counts and predictions together with the evaluation context. These elementary artifacts enable post-hoc analyses without repeated inference. We demonstrate MSB in a case study involving three heterogeneous 2D datasets and multiple MIS and general-purpose vision models evaluated at 256- and 512-pixel input resolutions. Reaggregation of identical predictions changes the top-ranked architecture in three of six dataset-resolution settings, despite high rank correlations between aggregation strategies. Increasing input resolution produces model- and dataset-dependent performance gains and losses that must be considered alongside empirically measured inference complexity. These results show that seemingly minor choices in evaluation and experimental setup can affect benchmark conclusions. MSB, available at GitHub, provides a practical and extensible basis for making benchmark conditions and evaluation choices explicit and reproducible.

Abstract

Medical Image Segmentation

Explore similar work

MedSegBenchmarker: A Raw-Count-First Framework for Controlled 2D Medical Image Segmentation Benchmarks

Few-Shot Distribution-Aligned Flow Matching for Data Synthesis in Medical Image Segmentation

A Decision-Aware Evaluation Protocol for Uncertainty in Medical Image Segmentation