FL-MedSegBench: A Comprehensive Benchmark for Federated Learning on Medical Image Segmentation

Meilu Zhu; Zhiwei Wang; Axiu Mao; Yuxing Li; Xiaohan Xing; Yixuan Yuan; Edmund Y. Lam

cs.CVMar 12, 2026

FL-MedSegBench: A Comprehensive Benchmark for Federated Learning on Medical Image Segmentation

Authors: Meilu Zhu, Zhiwei Wang, Axiu Mao, Yuxing Li, Xiaohan Xing, Yixuan Yuan, Edmund Y. Lam

Organizations: Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong, China · School of Communication Engineering, Hangzhou Dianzi University, Hang Zhou, China · Department of Radiation Oncology, Stanford University, California, USA · Department of Electronic Engineering, Chinese University of Hong Kong, Hong Kong, China

Abstract

Federated learning (FL) offers a privacy-preserving paradigm for collaborative medical image analysis without sharing raw data. However, the absence of standardized benchmarks for medical image segmentation hinders fair and comprehensive evaluation of FL methods. To address this gap, we introduce FL-MedSegBench, the first comprehensive benchmark for federated learning on medical image segmentation. Our benchmark encompasses nine segmentation tasks across ten imaging modalities, covering both 2D and 3D formats with realistic clinical heterogeneity. We systematically evaluate eight generic FL (gFL) and five personalized FL (pFL) methods across multiple dimensions: segmentation accuracy, fairness, communication efficiency, convergence behavior, and generalization to unseen domains. Extensive experiments reveal several key insights: (i) pFL methods, particularly those with client-specific batch normalization (e.g., FedBN), consistently outperform generic approaches; (ii) No single method universally dominates, with performance being dataset-dependent; (iii) Communication frequency analysis shows normalization-based personalization methods exhibit remarkable robustness to reduced communication frequency; (iv) Fairness evaluation identifies methods like Ditto and FedRDN that protect underperforming clients; (v) A method's generalization to unseen domains is strongly tied to its ability to perform well across participating clients. We will release an open-source toolkit to foster reproducible research and accelerate clinically applicable FL solutions, providing empirically grounded guidelines for real-world clinical deployment. The source code is available at https://github.com/meiluzhu/FL-MedSegBench.

Figures & tables

Figure 1: FL-MedSegBench: a comprehensive benchmark for federated learning on medical image segmentation.

Dataset	Index	Client Name	Sample size	Devices/Institutions
Fed-Vessel (Fundus)	C1	chaseDB [ 16 ]	28	Nidek NM-200-D
	C2	DR-Hagis [ 22 ]	40	Canon CR DGi/Topcon TRC-NW6s/Topcon TRC-NW8
	C3	DRIVE [ 51 ]	40	Canon CR5 nonmydriatic 3CCD/TopCon TRV-50
	C4	HRF [ 42 ]	45	Canon CF–60 UVi
	C5	LES-AV [ 44 ]	22	N/A
	C6	ORVS [ 47 ]	49	Zeiss, Viscuam 200

TABLE I: Summary of the 2D medical datasets in FL-MedSegBench

Dataset	Index	Client name	Sample size	Devices/Institutions
Fed-Pancreas (MRI-T1)	C1	NYU	162	New York University Medical Center
	C2	MCF	151	Mayo Clinic Florida
	C3	NU	30	Northwestern University
	C4	AHN	17	Allegheny Health Network
	C5	MCA	25	Mayo Clinic Arizona
Fed-M&Ms (CMR)	C1	Centre 1	95	Vall d’Hebron Hospital, Spain / Siemens

TABLE II: Summary of the 3D medical datasets in FL-MedSegBench

Methods	C1	C2	C3	C4	C5	C6
Local	$77.96\pm 0.76$	$70.43\pm 0.24$	$77.00\pm 0.55$	$74.75\pm 0.25$	$78.41\pm 0.55$	$71.34\pm 0.39$
Generic FL methods
FedAvg	$74.97\pm 0.90$	$68.50\pm 0.32$	$70.43\pm 0.70$	$74.98\pm 0.27$	$78.13\pm 0.08$	$71.02\pm 0.19$
FedAWA	$77.12\pm 0.57$	$68.45\pm 0.65$	$72.45\pm 0.61$	$75.17\pm 0.31$	$79.17\pm 0.19$	$70.41\pm 0.33$
FedLWS	$\textbf{79.96}\pm 0.37$	$\textbf{70.24}\pm 0.32$	$\textbf{74.29}\pm 0.67$	$\textbf{76.25}\pm 0.16$	$\textbf{80.35}\pm 0.21$	$70.61\pm 0.57$
FedNova	$77.23\pm 0.93$	$68.86\pm 0.25$	$72.65\pm 0.36$	$75.38\pm 0.38$	$79.34\pm 0.37$	$70.46\pm 0.11$

TABLE III : The segmentation performance (Dice) of different methods on Fed-Vessel dataset (%).

Methods	C1	C2	C3	C4	C5	C6
Local	$76.37\pm 2.19$	$86.41\pm 1.94$	$80.30\pm 1.64$	$84.06\pm 2.56$	$86.74\pm 0.76$	$62.32\pm 3.33$
Generic FL methods
FedAvg	$\textbf{76.35}\pm 2.39$	$86.29\pm 0.84$	$87.06\pm 0.36$	$87.15\pm 2.31$	$88.99\pm 1.22$	$78.48\pm 3.51$
FedAWA	$74.84\pm 7.54$	$86.99\pm 0.59$	$87.83\pm 1.30$	$86.47\pm 0.48$	$89.11\pm 0.54$	$78.84\pm 3.47$
FedLWS	$74.49\pm 3.96$	$\textbf{88.33}\pm 0.28$	$\textbf{89.17}\pm 0.70$	$87.71\pm 1.53$	$\textbf{89.37}\pm 0.71$	$\textbf{80.45}\pm 1.66$
FedNova	$74.94\pm 8.94$	$86.45\pm 0.32$	$87.99\pm 0.82$	$\textbf{88.22}\pm 0.97$	$88.67\pm 0.24$	$78.03\pm 3.20$

TABLE IV : The segmentation performance (Dice) of different methods on Fed-Prostate dataset (%).

TABLE V : The segmentation performance (Dice) of FL methods on Fed-COSAS and Fed-BUS datasets (%).

TABLE VI : The segmentation performance (Dice) of FL methods on Fed-MG and Fed-Polyp datasets (%).

Methods	C1	C2	C3	C4	C5
Local	$22.58_{\pm 11.84}$	$68.14_{\pm 1.78}$	$16.23_{\pm 8.95}$	$58.57_{\pm 5.38}$	$72.00_{\pm 2.24}$
Generic FL methods
FedAvg	$54.71_{\pm 19.44}$	$72.11_{\pm 1.52}$	$33.97_{\pm 6.16}$	$71.35_{\pm 0.30}$	$72.81_{\pm 5.03}$
FedAWA	$55.73_{\pm 17.01}$	$71.77_{\pm 1.67}$	$36.68_{\pm 7.00}$	$71.99_{\pm 1.39}$	$70.97_{\pm 4.43}$
FedLWS	$51.26_{\pm 12.99}$	$68.76_{\pm 1.21}$	$26.44_{\pm 6.15}$	$70.33_{\pm 4.08}$	$67.24_{\pm 4.48}$
FedNova	$53.40_{\pm 15.76}$	$72.93_{\pm 1.67}$	$\textbf{41.83}_{\pm 15.96}$	$71.99_{\pm 2.44}$	$72.61_{\pm 4.94}$

TABLE VII : The performance (Dice) of different methods on Fed-Pancreas dataset.

Methods	C1	C2	C3	C4	C5
Local	$81.18_{\pm 1.90}$	$82.32_{\pm 2.27}$	$82.85_{\pm 1.25}$	$72.00_{\pm 1.33}$	$76.70_{\pm 3.06}$
Generic FL methods
FedAvg	$80.67_{\pm 0.91}$	$85.35_{\pm 0.86}$	$\textbf{87.85}_{\pm 0.76}$	$79.28_{\pm 1.60}$	$79.00_{\pm 0.63}$
FedAWA	$78.75_{\pm 1.83}$	$\textbf{85.66}_{\pm 1.08}$	$87.52_{\pm 0.75}$	$79.57_{\pm 3.97}$	$\textbf{79.76}_{\pm 1.41}$
FedLWS	$79.52_{\pm 0.83}$	$85.32_{\pm 0.51}$	$87.52_{\pm 0.66}$	$79.71_{\pm 0.81}$	$79.55_{\pm 2.79}$
FedNova	$79.10_{\pm 1.30}$	$85.39_{\pm 0.65}$	$87.44_{\pm 0.96}$	$\textbf{81.78}_{\pm 1.16}$	$79.01_{\pm 1.40}$

TABLE VIII : The performance (Dice) of different methods on Fed-M&Ms dataset.

Methods	C1	C2	C3	C4	C5	C6	C7	C8	C9
Local	$78.01_{\pm 1.80}$	$87.60_{\pm 1.57}$	$88.06_{\pm 1.37}$	$77.01_{\pm 2.55}$	$82.45_{\pm 0.45}$	$76.55_{\pm 2.88}$	$85.80_{\pm 1.31}$	$90.14_{\pm 1.22}$	$79.63_{\pm 5.53}$
Generic FL methods
FedAvg	$78.40_{\pm 2.86}$	$88.19_{\pm 2.13}$	$87.90_{\pm 1.35}$	$82.43_{\pm 5.70}$	$\textbf{88.97}_{\pm 3.17}$	$79.77_{\pm 1.57}$	$85.86_{\pm 1.55}$	$90.25_{\pm 1.26}$	$88.58_{\pm 2.37}$
FedAWA	$77.60_{\pm 2.26}$	$88.21_{\pm 1.06}$	$87.53_{\pm 0.90}$	$83.04_{\pm 5.08}$	$88.32_{\pm 2.70}$	$79.71_{\pm 1.44}$	$85.02_{\pm 1.55}$	$90.43_{\pm 0.60}$	$88.62_{\pm 2.62}$
FedLWS	$77.68_{\pm 3.07}$	$88.38_{\pm 1.22}$	$87.29_{\pm 1.10}$	$82.48_{\pm 5.30}$	$88.92_{\pm 2.86}$	$80.15_{\pm 0.98}$	$85.44_{\pm 1.65}$	$89.76_{\pm 1.33}$	$88.55_{\pm 2.28}$
FedNova	$78.11_{\pm 1.85}$	$88.33_{\pm 1.03}$	$88.06_{\pm 0.87}$	$\textbf{83.17}_{\pm 4.75}$	$88.90_{\pm 2.91}$	$79.83_{\pm 1.15}$	$85.39_{\pm 1.95}$	$90.29_{\pm 0.48}$	$\textbf{88.79}_{\pm 2.22}$

TABLE IX : The segmentation performance (Dice) of different methods on FeTS2022 dataset (%).

Methods	ARIA	IOSTAR	MSD	Kvasir-SEG	BUS
FedAvg	$63.37\pm 0.85$	$63.96\pm 0.31$	$82.20\pm 1.36$	$72.11\pm 0.75$	$74.43\pm 0.74$
FedAWA	$64.79\pm 0.60$	$65.27\pm 0.79$	$82.13\pm 1.01$	$71.45\pm 0.71$	$\textbf{74.54}\pm 1.32$
FedLWS	$\textbf{66.41}\pm 0.58$	$\textbf{72.40}\pm 2.21$	$\textbf{84.27}\pm 1.36$	$20.69\pm 9.68$	$74.20\pm 0.37$
FedNova	$65.13\pm 0.61$	$65.79\pm 0.45$	$82.55\pm 0.72$	$73.43\pm 0.85$	$73.97\pm 2.41$
FedProx	$63.63\pm 0.62$	$63.74\pm 0.24$	$82.25\pm 1.42$	$72.99\pm 0.34$	$73.86\pm 1.38$
MOON	$62.94\pm 0.71$	$63.87\pm 0.86$	$82.18\pm 0.89$	$72.40\pm 0.56$	$73.52\pm 0.35$

TABLE X : The generalization performance (Dice) of different methods (%).

Panel of Figure 2 — Figure 2: The segmentation performance of different gFL methods with different communication rounds on 2D and 3D datasets.

Panel of Figure 3 — Figure 3: The segmentation performance of different pFL method with different communication rounds on 2D and 3D datasets.

Panel of Figure 4 — Figure 4: The convergence behaviors of different FL methods 2D and 3D datasets.

Methods	Fed-Vessel			Fed-Prostate			Fed-Polyp			Fed-COSAS
Methods	aDice $\uparrow$	Std $\downarrow$	wDice $\uparrow$	aDice $\uparrow$	Std $\downarrow$	wDice $\uparrow$	aDice $\uparrow$	Std $\downarrow$	wDice $\uparrow$	aDice $\uparrow$	Std $\downarrow$	wDice $\uparrow$
Generic FL methods
FedAvg	73.01	3.28	68.50	84.05	4.80	76.35	74.36	14.23	49.68	66.76	4.84	60.14
FedAWA	73.79	3.73	68.45	84.01	5.26	74.84	74.49	12.32	53.55	67.39	3.96	61.82
FedLWS	75.28	4.01	70.24	84.92	5.56	74.49	73.58	13.32	49.38	65.90	6.38	57.11
FedNova	73.98	3.68	68.86	84.05	5.46	74.94	75.92	10.60	58.79	66.76	4.84	60.14

TABLE XI : Fairness study of different FL methods on Fed-Vessel, Fed-Prostate, Fed-Polyp and Fed-COSAS (%).

Methods	Fed-BUS			Fed-Pancreas			Fed-M&Ms			FeTS2022
Methods	aDice $\uparrow$	Std $\downarrow$	wDice $\uparrow$	aDice $\uparrow$	Std $\downarrow$	wDice $\uparrow$	aDice $\uparrow$	Std $\downarrow$	wDice $\uparrow$	aDice $\uparrow$	Std $\downarrow$	wDice $\uparrow$
Generic FL methods
FedAvg	80.62	7.98	71.90	60.98	15.10	33.97	82.42	3.54	79.00	85.59	4.07	78.40
FedAWA	81.81	6.02	74.89	61.42	13.81	36.68	82.25	3.60	78.75	85.38	4.15	77.60
FedLWS	77.80	8.55	68.59	56.80	16.65	26.44	82.32	3.41	79.52	85.40	4.07	77.68
FedNova	81.66	6.53	73.44	62.55	12.73	41.83	82.54	3.37	79.01	85.65	4.10	78.11

TABLE XII : Fairness study of different FL methods on Fed-BUS, Fed-Pancreas, Fed-M&Ms and FeTS2022 (%).

Explore similar work

Federated Medical Image Segmentation under Real-World Label Noise: A Benchmark Suite for Noisy Label Learning Method Selection
Jun 15, 2026Markus Bujotzek, Dimitrios Bounias, Stefan Denner +4Medical Image Segmentation Medical Image Benchmarks
Division of Medical Image Computing, Germany Cancer Research2026 Center, Heidelberg, 69120, Germany. · Medical3HeidelbergFaculty,InstituteUniversityof Radiationof Heidelberg,OncologyHeidelberg,(HIRO), National69120, Germany. · 3HeidelbergFaculty,InstituteUniversityof Radiationof Heidelberg,OncologyHeidelberg,(HIRO), National69120, Germany. +2
ADP-FL-MedSeg: Adaptive Differential Privacy for Federated Medical Segmentation Across Diverse Modalities
Apr 7, 2026Puja Saha, Eranga UkwattaDifferentially Private Federated Learning Medical Image Segmentation
College of Engineering, University of Guelph, 50 Stone Road E, Guelph, ON, Canada
MuCALD-SplitFed: Causal-Latent Diffusion for Privacy-Preserving Multi-Task Split-Federated Medical Image Segmentation
May 4, 2026Chamani Shiranthika, Hadi Hadizadeh, Parvaneh SaeediLatent Diffusion Models Split Federated Learning
School of Engineering Science, Simon Fraser University, Burnaby, BC, Canada

Jun 15, 2026J/K move · Enter open · S save

Federated Medical Image Segmentation under Real-World Label Noise: A Benchmark Suite for Noisy Label Learning Method Selection

Markus Bujotzek, Dimitrios Bounias, Stefan Denner, Ralf Floca+3

Division of Medical Image Computing, Germany Cancer Research2026 Center, Heidelberg, 69120, Germany. · Medical3HeidelbergFaculty,InstituteUniversityof Radiationof Heidelberg,OncologyHeidelberg,(HIRO), National69120, Germany. · 3HeidelbergFaculty,InstituteUniversityof Radiationof Heidelberg,OncologyHeidelberg,(HIRO), National69120, Germany.+2

Medical Image Segmentation Medical Image Benchmarks Federated Learning

Open paper AskPDF arXiv

While federated learning (FL) enables collaborative medical image segmentation without centralizing sensitive data, real-world deployment is frequently complicated by cross-site label imperfections such as contour disagreement, missing or additional structures, and confused labels. Federated noisy label learning (FNLL) aims to mitigate these effects, yet remains underused in practice as existing evidence is largely based on synthetic noise, simplified settings, and limited real-world noisy evaluation. We address this gap by introducing a benchmark suite that combines diverse real-world noisy datasets, deployment-relevant client-noise scenarios, and label-noise-targeted evaluation to support systematic FNLL assessment and informed method selection. The suite combines curated real-world noisy medical image segmentation datasets from diverse sources with a comprehensive federated segmentation framework including various client-noise scenarios and noise-targeted evaluation. The presented suite provides a realistic and discriminative basis for FNLL evaluation in medical image segmentation and establishes a reusable foundation for fair benchmarking, dataset-specific label-noise characterization, and future method development under realistic federated settings. Code is available at https://github.com/MIC-DKFZ/FedSegNoiseBench.

Abstract

Explore similar work

Federated Medical Image Segmentation under Real-World Label Noise: A Benchmark Suite for Noisy Label Learning Method Selection

ADP-FL-MedSeg: Adaptive Differential Privacy for Federated Medical Segmentation Across Diverse Modalities

MuCALD-SplitFed: Causal-Latent Diffusion for Privacy-Preserving Multi-Task Split-Federated Medical Image Segmentation