FedCoRe: Target-Adaptive Completion for Missing Modalities in Healthcare Federated Learning

Holger R. Roth; Ziyue Xu; Peter Cnudde

cs.CVAug 18, 2026

FedCoRe: Target-Adaptive Completion for Missing Modalities in Healthcare Federated Learning

Authors: Holger R. Roth, Ziyue Xu, Peter Cnudde

Organizations: NVIDIA, Santa Clara, CA, USA

Abstract

Federated multimodal models often assume every site has every modality, although hospitals differ in access to EHRs, chest radiographs, and ECGs. We study this setting on a MIMIC-derived respiratory deterioration task with simulated FL clients and introduce FedCoRe (Federated Cross-Modal Representation Completion). FedCoRe learns representation- or logit-space corrections rather than generating synthetic ECGs or CXR images. When a client observes a modality that may be missing at deployment, it evaluates the same example with and without that modality to obtain paired supervision. Only clients with such pairs update the completion module, and validation may retain the unchanged prediction. We freeze the trained multimodal predictor during evaluation so that measured differences come only from completion. Hiding ECG reduced AUROC by about 0.085; paired-example FedAvg restored 0.0415 AUROC, or 49.0% of the lost performance. We therefore report two distinct effects: paired-example FedAvg partially recovers the missing-ECG gap, while validation-selected completion is a task-specific classifier-logit correction rather than literal ECG recovery. For CXR, effect-aware completion recovers 52.8% of the loss in a controlled test where CXR is hidden. Paired-example FedAvg transfers part of this effect, but validation keeps the no-completion baseline for deployment cases whose inputs lack CXR. Thus, FedCoRe should be read as a validation-gated completion/correction framework: it can recover missing-modality signal in supported settings, but it should be deployed only when paired examples and validation evidence support that modality.

Figures & tables

Figure 1: Complete-modality evaluation contrasts with federated deployment, where sites and patients may have different subsets of EHR, CXR, and ECG.

Figure 2: FedCoRe workflow: paired observations train the completion operator, sites without valid pairs receive zero weight, and validation applies completion or leaves predictions unchanged.

Template	Clients	Full-modality clients	Missing-ECG mass	Paired-example structure	Purpose
No full-modality client	4	0	66.3%	ECG-present but no complete site	Negative control
Patient-mixed any-missing	4	Mixed	30.0%	Complete and ECG-present patients within clients	Main heterogeneous setting
Patient-mixed heavy	4	Mixed	36.3%	Heavier missing-ECG patient mix	Stress setting

Table 1: Core FL heterogeneity templates for the missing-ECG frozen-predictor completion test. Missing-ECG mass is expected test-set patient mass under the client mask; paired sources are examples in which ECG is observed and can supervise completion.

Figure 3: Observed gains with the multimodal predictor frozen. Bars report matched target-removal/recovery tests rather than deployment gains; the box separately reports CXR deployment, where validation selected

\alpha=0

and retained the baseline.

Scenario	Completion	Seeds	Before	After	Full ref	Ref.-norm.	$\Delta$ AUROC [95% CI]	AUPRC	Positive
No full-modality client	FedAvg	7,17,42	0.5781	0.5781	0.6627	0.0%	+0.0000 [0.0000, 0.0000]	0.0375 / 0.0375	0/3
No full-modality client	Val-selected	7,17,42	0.5781	0.5781	0.6627	0.0%	+0.0000 [0.0000, 0.0000]	0.0375 / 0.0375	0/3
Patient-mixed combined	FedAvg	6 rows	0.5781	0.6195	0.6627	49.0%	+0.0415 [0.0023, 0.0778]	0.0375 / 0.0368	6/6
Patient-mixed combined	Val-selected	6 rows	0.5781	0.7235	0.6627	171.8%	+0.1455 [0.0617, 0.2213]	0.0375 / 0.0495	6/6

Table 2: Frozen-predictor missing-ECG test. Both methods select scale on validation; “Val-selected” also selects source. CIs use shared patient-level resamples across the six matched patient-mixed predictions. Full ref is ECG-present replay; reference-normalized gain is

(\mathrm{After}-\mathrm{Before})/(\mathrm{Full~ref}-\mathrm{Before})

, where values above 100% denote correction beyond the reference.

Seed	Full AUROC	Missing	Completed	$\Delta$ AUROC [95% CI]	Gap rec.	AUPRC before/after	$\alpha$
7	0.6673	0.5237	0.6454	+0.1217 [0.0005, 0.2330]	84.7%	0.0204 / 0.0409	0.50
17	0.6969	0.5933	0.6228	+0.0295 [-0.0520, 0.1104]	28.5%	0.0402 / 0.0331	0.75
23	0.7303	0.6187	0.6187	+0.0000 [0.0000, 0.0000]	0.0%	0.0297 / 0.0297	0.00
31	0.7072	0.5188	0.6594	+0.1405 [0.0301, 0.2464]	74.6%	0.0255 / 0.0345	0.10
42	0.7192	0.5307	0.6739	+0.1431 [-0.0037, 0.2826]	75.9%	0.0327 / 0.0611	0.10
Mean	0.7042	0.5570	0.6440	+0.0870	52.8%	0.0297 / 0.0399	–

Table 3: Frozen-predictor missing-CXR completion test. Validation selects scale and gate threshold; only scale is shown. Delta intervals are paired patient-level bootstrap 95% CIs. Means and gap recovery are unweighted across seeds.

Missing-Modality Learning

Missing-modality learning develops multimodal models that remain useful when input streams are absent, corrupted, or available only during training.

See all 171 papers

ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
Jul 7, 2026Aavash Chhetri, Bibek Niroula, Eduard Vazquez +4Multimodal Federated Learning Missing-Modality Learning
NepAl Applied Mathematics and Informatics Institute for research, Nepal · Fogsphere (Redev.AI Ltd), UK · University of Lausanne, Switzerland +4
Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning
Apr 25, 2026Wugeng Zheng, Ziwen Kan, Katie Wang +2Multimodal Federated Learning Time Series Imputation
University of Central Florida · Independent Researcher
Multimodal Federated Learning under Dual-Axis Modality Missingness
Aug 10, 2026Adiba Orzikulova, Jaehyun Kwak, Jaemin Shin +5Multimodal Federated Learning Missing-Modality Learning
1KAIST · 2CUHK · 3HKUST

Jul 7, 2026J/K move · Enter open · S save

ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities

Aavash Chhetri, Bibek Niroula, Eduard Vazquez, Yash Raj Shrestha+3

NepAl Applied Mathematics and Informatics Institute for research, Nepal · Fogsphere (Redev.AI Ltd), UK · University of Lausanne, Switzerland+4

Multimodal Federated Learning Missing-Modality Learning Medical Imaging Prototype Learning Mixture-of-Experts Models Federated Learning

Open paper AskPDF arXiv

In this paper, we address the problem of multimodal federated learning with missing modality. Existing methods utilize an additional public dataset or perform naive feature synthesis that is based solely on the available modality. To address these limitations, we propose ProMoE-FL, a Prototype-conditioned Mixture-of-Experts framework for robust missing-modality feature synthesis in multimodal federated learning. ProMoE-FL builds a global client-aware prototype bank that captures clinically meaningful modality priors across institutions. Our Mixture of Experts is conditioned on these prototypes and modality indices to enable direction-aware expert routing for dynamically synthesizing missing features. We perform extensive quantitative and qualitative evaluations on four public chest X-ray datasets (MIMIC-CXR, NIH Open-I, PadChest, and CheXpert) and demonstrate that ProMoE-FL consistently outperforms state-of-the-art methods in both homogeneous as well as the more challenging heterogeneous settings.

Abstract

Missing-Modality Learning

Explore similar work

ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities

Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning

Multimodal Federated Learning under Dual-Axis Modality Missingness