Fed-ADApt: Federated Anytime Depth Adaptation for Resource-Aware Medical Image Segmentation

Abhijeet Parida; Zhifan Jiang; Pooneh Roshanitabrizi; Austin Tapp; Maria J. Ledesma-Carbayo; Syed Muhammad Anwar; Ziyue Xu; Marius George Linguraru

cs.CVOct 2, 2026

Fed-ADApt: Federated Anytime Depth Adaptation for Resource-Aware Medical Image Segmentation

Authors: Abhijeet Parida, Zhifan Jiang, Pooneh Roshanitabrizi, Austin Tapp, Maria J. Ledesma-Carbayo, Syed Muhammad Anwar, Ziyue Xu, Marius George Linguraru, +1 more

Organizations: Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC, United States · Escuela Técnica Superior de Ingenieros de Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain · CIBER-BBN, Instituto de Salud Carlos III, Madrid, Spain · Departments of Radiology and Pediatrics, The George Washington University, Washington, DC, United States · NVIDIA Corporation, Santa Clara, CA, United States

Abstract

Federated learning (FL) enables collaborative training of medical image segmentation models without sharing raw patient data, yet existing approaches assume a homogeneous compute budget across institutions, limiting participation of low-resource sites. We propose Fed-ADApt, a depth-adaptive federated framework for UNet-based segmentation that jointly addresses low-compute training and inference. Fed-ADApt integrates multi-depth supervision with hierarchical depth-wise aggregation, allowing each site to train according to its local compute budget while contributing to a global model that supports dynamic depth selection at deployment. We evaluated Fed-ADApt on multi-site 2D retinal fundus disc segmentation and 3D brain tumor segmentation. Across both tasks, federated collaboration substantially improves robustness under domain shift. Fed-ADApt matched the full-resource FedAvg performance in 3D and achieved competitive 2D performance with a 4.7% average Dice reduction, while reducing average inference cost by 19.5% in 3D and 34.5% in 2D and substantially reducing training cost by 98% at the most constrained sites. Importantly, Fed-ADApt enables low-resource institutions that cannot train full-capacity models to participate in federations while maintaining competitive global performance under a favorable accuracy to efficiency trade-off. By considering training and inference compute budgets, Fed-ADApt provides a practical and equitable solution for federated medical image segmentation across heterogeneous clinical and edge-enabled imaging environments.

Figures & tables

Fig. 1: Fed-ADApt for resource-aware federated segmentation. (a) The proposed multi-depth UNet produces full-resolution predictions from multiple encoder depths, enabling different compute paths within a single architecture. (b) DepthFL supports heterogeneous training by allowing clients to train depth-restricted models, but all clients perform full-depth inference. (c) Federated anytime supports adaptive inference by training all depths at every client, but requires full-model training at all sites. (d) Fed-ADApt jointly addresses both constraints by allowing each client to train only the depths permitted by its compute budget while retaining dynamic depth selection at inference.

	Retina Disc Segmentation (2D)					Brain Tumor Segmentation (3D) [ 3 ]
	Site A	Site B	Site C	Site D	Site E	Site P	Site Q	Site R	Site S	Site T
	Drishti [ 23 ]	BinRushed4 [ 1 ]	Magrabia [ 1 ]	Messidor [ 1 ]	REFUGE [ 20 ]	UCSF	Duke	UCSD	Missouri	Indiana
$C_{s}^{\mathrm{train}}$	1	2	1	4	5	5	4	3	3	1
Train	50	98	47	230	400	357	284	120	192	18
Val	25	49	24	115	200	119	95	40	64	6
Test	25	49	23	115	200	120	95	40	64	7

TABLE I: Multi-site datasets and simulated compute budgets. Summary of the 2D retinal fundus and 3D brain tumor segmentation datasets used for federated evaluation. Each site is assigned a normalized compute budget

C_{s}^{\mathrm{train}}\in{1,\ldots,5}

, where lower values represent resource-constrained clients and

C_{s}^{\mathrm{train}}=5

denotes full-capacity training.

	Segmentation						Training			Inference			Comm.
Method	(Dice)						GMAC / VRAM(GB)			GMAC / VRAM(GB)			Cost
	Site A	Site B	Site C	Site D	Site E	Avg	Max	Min	Avg	Max	Min	Avg
Local Model	0.27 $\pm$ 0.11	0.70 $\pm$ 0.18	0.20 $\pm$ 0.08	0.80 $\pm$ 0.06	0.87 $\pm$ 0.13	0.55 $\pm 0.11$ ∗	16.5 / 0.04	0.7 / 0.03	6.6 / 0.03	5.5 / 0.04	0.2 / 0.02	2.2 / 0.02	-
High Resource Model	0.41 $\pm$ 0.26	0.06 $\pm$ 0.03	0.21 $\pm$ 0.13	0.07 $\pm$ 0.04	0.87 $\pm$ 0.13	0.32 $\pm$ 0.12 ∗	16.5 / 0.04	0.0 / 0.00	3.3 / 0.01	5.5 / 0.04	5.5 / 0.04	5.5 / 0.04	-
FedAvg	0.90 $\pm$ 0.10	0.80 $\pm$ 0.17	0.91 $\pm$ 0.05	0.80 $\pm$ 0.13	0.82 $\pm$ 0.14	0.85 $\pm$ 0.12 ∗	16.5 / 0.04	16.5 / 0.04	16.5 / 0.04	5.5 / 0.04	5.5 / 0.04	5.5 / 0.04	100%
Federated Anytime	0.88 $\pm$ 0.13	0.75 $\pm$ 0.15	0.88 $\pm$ 0.16	0.86 $\pm$ 0.08	0.83 $\pm$ 0.15	0.84 $\pm$ 0.13 ∗	33.9 / 0.12	33.9 / 0.12	33.9 / 0.12	5.5 / 0.04	2.5 / 0.03	3.9 / 0.04	100%

TABLE II: 2D retinal disc segmentation. Dice scores and compute cost for various FL methods under compute constraints. GMAC was calculated for a batch size of 10. ∗ indicates statistical significance (

p<0.05

) in Dice compared to Fed-ADApt . Training strategy: % No FL; % full-capacity FL; % compute-aware FL.

Segmentation

Training

Inference

Comm.

Method

(Dice)

GMAC / VRAM(GB)

Cost

Site P

Site Q

Site R

Site S

Site T

Avg

Max

Min

Avg

Max

Min

Avg

0.81

\pm

0.13

0.85

\pm

0.12

0.71

\pm

0.22

0.74

\pm

0.23

0.33

\pm

0.14

0.69

\pm

0.17

{}^{\ }

Local Model

0.83

\pm

0.12

0.87

\pm

0.10

0.74

\pm

0.22

0.77

\pm

0.21

0.50

\pm

0.20

0.74

\pm

0.17

{}^{\ }

477 / 0.99

133 / 0.80

357 / 0.94

159 / 0.97

44 / 0.79

119 / 0.93

0.83

\pm

0.12

0.88

\pm

0.11

0.75

\pm

0.22

0.77

\pm

0.22

0.10

\pm

0.04

0.66

\pm

0.16 ∗

0.81

\pm

0.13

0.73

\pm

0.22

0.68

\pm

0.24

0.56

\pm

0.29

0.74

\pm

0.14

0.71

\pm

0.20

{}^{\ }

TABLE III: 3D brain tumor segmentation. Dice scores for enhancing tumor (ET), tumor core (TC), and whole tumor (WT) for various FL methods. GMAC was calculated for a batch size of 10. ∗ indicates statistical significance (

p<0.05

) in WT Dice compared to Fed-ADApt . Training strategy: % No FL; % full-capacity FL; % compute-aware FL.

Fig. 2: Qualitative effect of depth-adaptive inference. Retinal fundus predictions from Fed-ADApt at increasing depth levels. Shallower prediction paths provide coarse optic disc localization with lower computation, while deeper paths progressively refine the segmentation boundary and better align with the expert annotation. Green contours indicate predictions and blue contours indicate ground truth.

	Low budget		Medium budget		High budget
	$C_{s}^{\mathrm{train}}$	Dice	$C_{s}^{\mathrm{train}}$	Dice	$C_{s}^{\mathrm{train}}$	Dice
Site A	1	0.86	3	0.86	4	0.91
Site B	2	0.68	3	0.78	5	0.73
Site C	1	0.86	1	0.87	1	0.89
Site D	4	0.75	4	0.78	4	0.81
Site E	5	0.86	5	0.84	5	0.85

TABLE IV: Sensitivity of Fed-ADApt to federation compute budget. Site-level

C_{s}^{\mathrm{train}}

and Dice scores are shown for low-, medium-, and high-budget federations. Increasing the federation budget improves mean Dice.

Explore similar work

Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization
Feb 24, 2026Sachin Dudda Nagaraju, Ashkan Moradi, Bendik Skarre Abrahamsen +1Cross-Modality Medical Image Segmentation Data Augmentation
Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway. · Central Staff, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway.
Shift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical Segmentation
Jun 7, 2026Xingyue Zhao, Wenke Huang, Linghao Zhuang +7Vision Foundation Model Adaptation Non-IID Federated Learning
Department of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences · College of Computing and Data Science, Nanyang Technological University, Singapore +4
MedFL-Stress: A Systematic Robustness Evaluation of Federated Brain Tumor Segmentation under Cross-Hospital MRI Appearance Shift
May 9, 2026Kiran Naseer, Naveed Anwer ButtDistribution Shift Robustness BraTS
University of Gujrat, Pakistan

Feb 24, 2026, cs.CVJ/K move · Enter open · S save

Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization

Sachin Dudda Nagaraju, Ashkan Moradi, Bendik Skarre Abrahamsen, Mattijs Elschot

Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway. · Central Staff, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway.

Cross-Modality Medical Image Segmentation Data Augmentation 3D Medical Image Segmentation Federated Learning Heterogeneous Federated Learning

Open paper AskPDF arXiv

Purpose: Developing generalizable medical image segmentation models is challenging because imaging data are distributed across institutions and differ in modality and acquisition protocol. Federated learning (FL) enables collaborative training without centralizing raw medical images, but cross-modality domain shifts between computed tomography (CT) and magnetic resonance imaging (MRI) can substantially reduce model performance. This study investigates augmentation-driven cross-modality FL for abdominal organ and whole-heart segmentation. Methods: We evaluate convolution-based spatial augmentation, frequency-domain argumentation, domain-specific normalization, and global intensity nonlinear (GIN) augmentation for multimodal segmentation. Abdominal organ segmentation and whole-heart segmentation are first evaluated using a 2D U-Net framework. For whole-heart segmentation, we additionally perform native 3D experiments using a self-configuring nnU-Net architecture on the CARE-WHS 2026 dataset, enabling evaluation of whether the observed cross-modality FL behavior persists when moving from slice-based 2D segmentation to volumetric 3D segmentation. Results: GIN provides the most consistent cross-modality performance among the evaluated approaches in the original 2D experiments. For pancreas segmentation, the Dice similarity coefficient (DSC) improved from 0.073 to 0.437 when CT data were incorporated through federated cross-modality training. In 3D whole-heart segmentation, FedGIN improved mean DSC over FedAvg from 0.8696 to 0.8901 on the unseen CT center and from 0.7160 to 0.7956 on the unseen MRI center. Relative to centralized GIN training, FedGIN retained 92.4% of performance on unseen CT data and achieved comparable performance on unseen MRI data (0.7956 versus 0.7937).

Abstract

Explore similar work

Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization

Shift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical Segmentation

MedFL-Stress: A Systematic Robustness Evaluation of Federated Brain Tumor Segmentation under Cross-Hospital MRI Appearance Shift