FedDAP: Domain-Aware Prototype Learning for Federated Learning under Domain Shift

Huy Q. Le; Loc X. Nguyen; Yu Qiao; Seong Tae Kim; Eui-Nam Huh; Choong Seon Hong

cs.CVApr 8, 2026

FedDAP: Domain-Aware Prototype Learning for Federated Learning under Domain Shift

Authors: Huy Q. Le, Loc X. Nguyen, Yu Qiao, Seong Tae Kim, Eui-Nam Huh, Choong Seon Hong

Organizations: G-LAMP NEXUS Institute, Kyung Hee University · Kyung Hee University

Abstract

Federated Learning (FL) enables decentralized model training across multiple clients without exposing private data, making it ideal for privacy-sensitive applications. However, in real-world FL scenarios, clients often hold data from distinct domains, leading to severe domain shift and degraded global model performance. To address this, prototype learning has been emerged as a promising solution, which leverages class-wise feature representations. Yet, existing methods face two key limitations: (1) Existing prototype-based FL methods typically construct a $\textit{single global prototype}$ per class by aggregating local prototypes from all clients without preserving domain information. (2) Current feature-prototype alignment is $\textit{domain-agnostic}$ , forcing clients to align with global prototypes regardless of domain origin. To address these challenges, we propose Federated Domain-Aware Prototypes (FedDAP) to construct domain-specific global prototypes by aggregating local client prototypes within the same domain using a similarity-weighted fusion mechanism. These global domain-specific prototypes are then used to guide local training by aligning local features with prototypes from the same domain, while encouraging separation from prototypes of different domains. This dual alignment enhances domain-specific learning at the local level and enables the global model to generalize across diverse domains. Finally, we conduct extensive experiments on three different datasets: DomainNet, Office-10, and PACS to demonstrate the effectiveness of our proposed framework to address the domain shift challenges. The code is available at https://github.com/quanghuy6997/FedDAP.

Figures & tables

Figure 1: (a) Single label-specific prototypes ignore domain variation, leading to semantic conflict when features from different domains are forced to align to general prototype. (b) Our domain-specific prototypes disentangle the feature space along both label semantics (vertical axis) and domain characteristics (horizontal axis) and enabling the better semantic alignment to the appropriate prototype.

Figure 2: Illustration of FedDAP. (1) Clients first compute local prototypes $\mathbf{p}_{m}^{(c,d)}$ based on Eq. 3 and then upload it to server. (2) The central server performs the cosine similarity-weighted fusion mechanism for the local prototypes by calculating the attention score in Eq. 4 for each row of class-domain pair prototypes. Then we perform the Domain-Specific Global Prototype aggregation using Eq. 5 . (3) Clients download the global prototypes $\mathbf{P}^{c,d}$ from the server and perform the local training process with the dual prototype alignment strategy with $\mathcal{L}_{DPA}$ in Eq. 6 and $\mathcal{L}_{CPCL}$ in Eq. 8 .

Method	DomainNet								Office-10
Method	Clipart	Infograph	Painting	Quickdraw	Real	Sketch	Avg	$\Delta$	Caltech	Amazon	Webcam	DSLR	Avg	$\Delta$
FedAvg	67.73	31.28	58.22	71.20	70.72	58.41	59.59	–	67.95	78.63	49.31	34.00	57.47	–
FedProx	68.46	34.08	58.87	74.02	72.40	55.62	60.57	+0.98	66.32	75.77	55.42	42.33	59.96	+2.49
MOON	68.11	33.81	58.19	72.26	72.63	55.87	60.15	+0.56	64.25	77.19	55.15	43.00	59.89	+2.42
COPA	68.84	34.24	60.45	76.94	73.04	54.65	61.36	+1.77	70.82	80.21	51.72	55.33	64.50	+7.03
FedGA	67.20	33.29	60.81	74.20	72.34	58.19	61.00	+1.41	65.27	76.11	57.24	51.33	62.49	+5.02

Table 1: Comparison on DomainNet and Office-10 datasets. “Avg” denotes the mean accuracy across all domains, and

\Delta

indicates improvement over the FedAvg baseline.

Method	PACS
Method	Photo	Art	Cartoon	Sketch	Avg	$\Delta$
FedAvg	80.89	68.56	72.25	86.59	77.07	–
FedProx	81.23	66.35	74.83	87.97	77.60	+0.53
MOON	83.47	72.69	74.55	91.66	80.59	+3.52
COPA	82.46	68.55	72.95	88.16	78.03	+0.96
FedGA	84.51	67.80	73.53	90.65	79.13	+2.06

Table 2: Comparison on the PACS dataset. “Avg” denotes the mean accuracy across all domains, and

\Delta

indicates improvement over the FedAvg baseline.

Panel of Figure 3 — Figure 3: Visualization of training curves of average test accuracy on three datasets under the domain shift setting.

Panel of Figure 4 — Figure 4: t-SNE visualization of learned features on the DomainNet and PACS benchmarks. In each subfigure, the left panel shows FedProto and the right panel shows our method.

$\mathcal{L}_{\mathrm{DPA}}$	$\mathcal{L}_{\mathrm{CPCL}}$	DomainNet		Office-10		PACS
$\mathcal{L}_{\mathrm{DPA}}$	$\mathcal{L}_{\mathrm{CPCL}}$	Avg	$\Delta$	Avg	$\Delta$	Avg	$\Delta$
✗	✗	59.59	–	57.47	–	77.07	–
✗	✓	62.86	+3.27	62.18	+4.71	81.87	+4.80
✓	✗	62.61	+3.02	68.53	+11.06	78.74	+1.67
✓	✓	65.20	+5.61	72.53	+15.06	84.63	+7.56

Table 3: Ablation study on key components of FedDAP across DomainNet, Office-10, and PACS datasets.

Aggregation Method	DomainNet		Office-10		PACS
Aggregation Method	Avg	$\Delta$	Avg	$\Delta$	Avg	$\Delta$
Averaging	64.15	–	71.49	–	83.22	–
Ours	65.20	+1.05	72.53	+1.04	84.63	+1.41

Table 4: Comparison of domain-specific prototype aggregation methods. “Avg” denotes mean accuracy and “

\Delta

” the improvement over uniform averaging.

Method	DomainNet								Office-10
Method	$\rightarrow$ Clipart	$\rightarrow$ Infograph	$\rightarrow$ Painting	$\rightarrow$ Quickdraw	$\rightarrow$ Real	$\rightarrow$ Sketch	Avg	$\Delta$	$\rightarrow$ Caltech	$\rightarrow$ Amazon	$\rightarrow$ Webcam	$\rightarrow$ DSLR	Avg	$\Delta$
FedAvg	40.25	46.91	45.83	43.68	42.36	44.39	43.90	–	43.54	65.81	61.64	70.04	60.26	–
COPA	43.09	47.52	47.21	41.86	44.12	43.48	44.55	+1.01	49.75	69.36	60.35	71.05	62.63	+2.37
FedGA	44.48	46.40	45.45	43.79	44.60	43.06	44.63	+0.73	45.88	65.19	62.61	69.55	60.81	+0.55
FedPLVM	46.75	49.24	50.28	45.95	46.76	46.86	47.64	+3.74	50.92	71.95	62.91	72.33	64.53	+4.27
FedRDN	40.15	47.21	47.89	41.39	43.48	45.32	44.24	+0.34	39.61	74.63	64.71	68.94	61.97	+1.71

Table 5: Comparison on DomainNet and Office-10 datasets under domain generalization setting. “Avg” denotes the mean accuracy across all domains, and

\Delta

indicates improvement over the FedAvg baseline.

Panel of Figure 5 — Figure 5: Analysis of FedDAP across all datasets with different hyper-parameters.

Method	Prototype up (MB)	Prototype down (MB)	FLOPs $\,\times 10^{10}$	Time train (s)
FedProto	0.195	0.195	119.14	21.20
FedPLVM	0.195	0.195	119.14	37.28
FedDAP	0.195	0.781	119.14	32.84

Table 6: Comparison of communication and computation costs.

Local Prototypes	DomainNet		Office-10		PACS
Local Prototypes	Avg	$\Delta$	Avg	$\Delta$	Avg	$\Delta$
w DP	64.38	–	71.51	–	83.63	–
w/o DP	65.20	+0.82	72.53	+1.02	84.63	+1.00

Table 7: Impact of differential privacy on local prototypes across different datasets.

Method	DomainNet								Office-10
Method	Clipart	Infograph	Painting	Quickdraw	Real	Sketch	Avg	$\Delta$	Caltech	Amazon	Webcam	DSLR	Avg	$\Delta$
FedAvg	64.91	31.86	56.89	60.58	59.42	53.70	54.56	–	67.61	78.86	73.72	51.67	67.97	–
FedProx	63.47	34.05	55.76	58.68	65.49	52.97	55.07	+0.51	65.42	82.56	68.44	53.00	67.36	-0.61
MOON	65.64	30.52	54.37	57.62	67.02	51.85	54.28	–0.28	63.26	81.82	60.41	52.00	64.37	-3.60
COPA	63.72	32.36	57.93	63.85	61.86	52.62	55.39	+0.83	69.02	81.37	76.21	60.00	71.65	+3.68
FedGA	65.37	32.85	55.85	62.96	63.66	56.03	56.12	+1.56	65.56	81.05	74.14	56.33	69.27	+1.30

Table 8: Comparison on DomainNet and Office-10 datasets under equal client allocation. “Avg” denotes the mean accuracy across all domains, and

\Delta

indicates improvement over the FedAvg baseline.)

Method	PACS
Method	Photo	Art	Cartoon	Sketch	Avg	$\Delta$
FedAvg	86.39	82.98	93.97	90.06	88.35	–
FedProx	83.47	83.13	93.54	90.46	87.65	-0.70
MOON	87.58	83.96	92.04	90.50	88.52	+0.17
COPA	78.90	77.64	94.87	85.00	84.10	-4.25
FedGA	86.32	83.33	95.07	89.24	88.49	+0.14

Table 9: Comparison on the PACS dataset under equal client allocation. “Avg” denotes the mean accuracy across all domains, and

\Delta

indicates improvement over the FedAvg baseline.

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 921 papers

Mar 15, 2026

Domain-Skewed Federated Learning with Feature Decoupling and Calibration

Federated learning (FL) allows distributed clients to collaboratively train a global model in a privacy-preserving manner. However, one major challenge is domain skew, where clients' data originating from diverse domains may hinder the aggregated global model from learning a consistent representation space, resulting in poor generalizable ability in multiple domains. In this paper, we argue that the domain skew is reflected in the domain-specific biased features of each client, causing the local model's representations to collapse into a narrow low-dimensional subspace. We then propose Federated Feature Decoupling and Calibration (

F^2

DC), which liberates valuable class-relevant information by calibrating the domain-specific biased features, enabling more consistent representations across domains. A novel component, Domain Feature Decoupler (DFD), is first introduced in

F^2

DC to determine the robustness of each feature unit, thereby separating the local features into domain-robust features and domain-related features. A Domain Feature Corrector (DFC) is further proposed to calibrate these domain-related features by explicitly linking discriminative signals, capturing additional class-relevant clues that complement the domain-robust features. Finally, a domain-aware aggregation of the local models is performed to promote consensus among clients. Empirical results on three popular multi-domain datasets demonstrate the effectiveness of the proposed

F^2

DC and the contributions of its two modules. Code is available at https://github.com/mala-lab/F2DC.

Huan Wang, Jun Shen, Jun Yan +1

School of Computing and Information Technology, University of Wollongong, Australia · School of Computing and Information Systems, Singapore Management University, Singapore

Federated Learning Heterogeneous Federated Learning Domain Generalization Representation Disentanglement

Ask PDF

May 13, 2026

FedHPro: Federated Hyper-Prototype Learning via Gradient Matching

Federated Learning (FL) enables collaborative training of distributed clients while protecting privacy. To enhance generalization capability in FL, prototype-based FL is in the spotlight, since shared global prototypes offer semantic anchors for aligning client-specific local prototypes. However, existing methods update global prototypes at the prototype-level via averaging local prototypes or refining global anchors, which often leads to semantic drift across clients and subsequently yields a misaligned global signal. To alleviate this issue, we introduce hyper-prototypes, defined by a set of learnable global class-wise prototypes to preserve underlying semantic knowledge across clients. The hyper-prototypes are optimized via gradient matching to align with class-relevant characteristics distilled directly from clients' real samples, rather than prototype-level descriptors. We further propose FedHPro, a Federated Hyper-Prototype Learning framework, to leverage hyper-prototypes to promote inter-class separability via mutual-contrastive learning with client-specific margin, while encouraging intra-class uniformity through a consistency penalty. Comprehensive experiments under diverse heterogeneous scenarios confirm that 1) hyper-prototypes produce a more semantically consistent global signal, and 2) FedHPro achieves state-of-the-art performance on several benchmark datasets. Code is available at https://github.com/mala-lab/FedHPro.

Huan Wang, Jun Shen, Haoran Li +6

School of Computing and Information Technology, University of Wollongong, Wollongong, Australia · School of Computing and Information Systems, Singapore Management University, Singapore, Singapore · Monash University, Australia +3

Federated Learning Heterogeneous Federated Learning

Ask PDF

Sep 19, 2026

Joint Domain-Class Modeling for Federated Learning Under Feature Skew

Federated learning (FL) enables collaborative model training without centralizing private data, but performance often degrades under feature skew: clients share labels while the conditional input distributions

p_i(x\!\mid\!y)

vary due to latent, client-specific appearance factors. We propose Joint Domain-Class Federated Learning (JDFL), a lightweight, optimizer-agnostic extension that makes this latent domain variation usable without sharing raw data. JDFL first infers domain clusters called pseudo-domains from brief local update signals. It then expands the classifier head to output

M\times C

, joint (domain-class) logits. This allows the model to represent domain-conditioned appearance while keeping a shared backbone. To train the expanded head we introduce two complementary supervision strategies based on simple intuitions: a similarity-aware soft-labeling that transfers evidence between nearby inferred domains while allowing domain-specific specialization, and a per-sample randomized target assignment that perturbs supervision across the joint outputs and serves as a low-cost training-time regularizer. JDFL integrates with existing standard FL methods (e.g., FedAvg, SCAFFOLD) with minimal changes. Empirically, both supervision modes consistently improve global test accuracy on standard domain-shifted image benchmarks; ablations and sensitivity studies show the gains stem from the proposed supervision and parametrization rather than mere capacity increase.

Sina Najafi, Mostafa Tavassolipour, Seyed Pooya Shariatpanahi

School of Electrical and Computer Engineering College of Engineering, University of Tehran Tehran, Iran

Federated Learning Heterogeneous Federated Learning Clustered Federated Learning

Ask PDF

Abstract

Federated Learning

Explore similar work

Domain-Skewed Federated Learning with Feature Decoupling and Calibration

FedHPro: Federated Hyper-Prototype Learning via Gradient Matching

Joint Domain-Class Modeling for Federated Learning Under Feature Skew