FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements

Xiao Yang; Dengbo He; Duoxun Tang; Jiyao Wang; Kaishun Wu

cs.CVOct 14, 2025

FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements

Authors: Xiao Yang, Dengbo He, Duoxun Tang, Jiyao Wang, Kaishun Wu

Organizations: Systems Hub, the Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

Abstract

Remote physiological measurement gained wide attention, while it requires collecting users' privacy-sensitive information, and existing contactless measurements still rely on labeled client data. This presents challenges when we want to further update real-world deployed models with numerous user data lacking labels. To resolve these challenges, we instantiate a new protocol called Federated Unsupervised Domain Generalization (FUDG) in this work. Subsequently, the Federated Heterogeneous Unsupervised Generalization (FedHUG) framework is proposed and consists of: (1) Minimal Bias Aggregation module dynamically adjusts aggregation weights based on prior-driven bias evaluation to cope with heterogeneous non-IID features from multiple domains. (2) The Global Distribution-aware Learning Controller parameterizes the label distribution and dynamically manipulates client-specific training strategies, thereby mitigating the server-client label distribution skew and long-tail issue. The proposal shows superior performance across state-of-the-art techniques in estimation with either RGB video or mmWave radar. The code will be released.

Figures & tables

Figure 1: (a) Overview of FUDG. We take the video-based physiological estimation as an example. (b) Domain Skew. Significant feature shifts under varying user attributes and environmental factors.(c) Label Skew. Inherent long-tail and imbalanced distribution of phys-labels across datasets (i.e., PURE Song et al. [2021] , BUAA Xi et al. [2020] , VIPL Lu et al. [2021] , MMPD Tang et al. [2023] ).

Panel of Figure 2 — Figure 2: RGB representation construction and physiologically consistent augmentation. (a) Production of an STMap from a facial RGB video using region-wise color traces. (b) Unified spatio-temporal augmentation, illustrated with an STMap: temporal sliding introduces a phase shift $\delta_{t}$ , while spatial permutation disrupts structural and environmental noise. The same temporal and spatial operations are applied to mmWave range matrices.

Figure 3: Illustration of three different aim domains. Here, the dashed circle’s center represents the semantic features, and the closer a domain is to the center, the less interference it contains.

Figure 4: Minimal bias evaluation via SSM generation. (a) The robust model yields highly consistent translation-invariant SSMs despite the phase shift between the original ( $x_{i}$ ) and augmented ( $a_{i}$ ) views. (b) The biased model succumbs to domain-specific noise, resulting in corrupted and inconsistent SSM textures. This visual discrepancy enables the score $s$ to accurately quantify model bias.

Figure 5: Illustration of the distribution of $s$ in different datasets.

	VIPL				PURE				BUAA				MMPD
Method	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$
CHROM De Haan and Jeanne [2013]	11.44	15.19	16.97	0.28	9.79	11.96	12.76	0.37	6.09	8.03	8.29	0.51	13.63	16.23	18.75	0.08
POS Wang et al. [2016]	14.59	18.11	21.26	0.19	9.82	12.85	13.44	0.34	5.04	6.54	7.12	0.63	13.84	17.07	19.18	0.15
PhysNet + Yu et al. [2019]	13.53	15.13	16.46	0.35	18.12	22.71	24.32	0.43	7.21	7.40	8.12	0.58	19.11	24.17	26.29	0.04
Physformer++ + Yu et al. [2023]	12.07	14.34	15.38	0.32	15.48	19.50	24.63	0.46	6.52	8.47	8.51	0.55	16.37	19.65	21.82	0.07
RhythmFormer + Zou et al. [2025]	10.22	12.04	12.98	0.39	6.47	8.02	9.45	0.61	4.78	6.03	6.16	0.64	13.24	15.98	16.53	0.10

Table 1: HR estimation results under FUDG protocol in the rPPG task. ‘Base’ and ‘ + ’ refer to the pre-trained model that is applied directly to the target without unsupervised training. ‘ ∗ ’ means it uses the same unsupervised loss and backbone as FedHUG. The bold texts show the significantly (p-value

<

0.05) best result within each column with the paired-t test.

	VIPL				PURE				BUAA				MMPD
Method	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$
FedAvg McMahan et al. [2017]	6.86	8.98	9.39	0.52	5.61	7.71	8.09	0.57	3.80	4.65	4.89	0.82	9.79	11.24	11.79	0.13
FedProx ∗ Li et al. [2020]	6.78	8.73	9.26	0.54	5.51	7.66	7.94	0.59	3.35	4.81	4.81	0.84	9.69	11.07	11.60	0.14
FedMut ∗ Hu et al. [2024a]	6.82	8.52	9.41	0.48	5.88	8.25	8.96	0.58	3.68	4.74	5.05	0.83	9.77	11.28	11.93	0.11
FedU2 Liao et al. [2024]	6.89	8.48	9.20	0.53	5.78	7.41	8.15	0.59	3.49	4.61	4.72	0.82	9.85	10.88	11.59	0.14
FedGaLA Pourpanah et al. [2025]	6.61	8.38	9.13	0.55	5.61	7.29	7.74	0.56	3.29	4.60	4.77	0.85	9.70	10.69	11.55	0.16

Table 2: Results on source domains. The model is pre-trained on SCAMPS and federated unsupervised trained on four datasets.

	Flat & Unobstructed				Flat & Congested				Bumpy & Congested
Method	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$	MAE $\downarrow$	SD $\downarrow$	RMSE $\downarrow$	p $\uparrow$
Base	11.21	14.66	15.91	0.09	11.45	14.35	15.27	0.12	11.56	14.98	15.73	0.10
FedAvg	10.03	12.74	13.07	0.19	9.89	12.60	12.56	0.15	10.21	12.78	12.84	0.16
FedProx	9.79	12.60	12.85	0.23	9.54	12.42	13.66	0.21	9.95	12.83	12.90	0.19
FedMut	9.90	11.87	12.67	0.22	10.19	11.97	13.12	0.18	9.93	12.41	12.49	0.23
FedU2	9.87	11.49	11.54	0.25	9.97	10.66	10.93	0.24	10.22	11.06	12.41	0.23

Table 3: HR estimation results with mmWave under FUDG.

Figure 6: Illustration of the distribution of $s$ in PURE before and after applying the MBA method (a), and MAE at different HR intervals before and after the use of GDLC (b). MAE on MMPD across communication rounds under the FUDG protocol (c) . The MAE variation across different communication rounds of the MMPD dataset under the FUDG protocol, the impact of the $\gamma$ and $\sigma$ on the model’s performance (d). BUAA’s feature distribution on unit sphere (e).

Federated Domain Generalization

See all 10 papers

FedUHD: Unsupervised Federated Learning using In-Memory Hyperdimensional Computing
Aug 16, 2025You Hak Lee, Keming Fan, Xiaofan Yu +4Compute-in-Memory Federated Learning Aggregation
University of California San Diego, USA
FedAvg for HAR: Exploring the Tradeoff Between Personalized and Generalization Accuracy
Jul 3, 2026Andrea De Luna, Susanna Peretti, Chiara Contoli +1Federated Learning Aggregation Human Activity Recognition
DiSPeA University of Urbino
Separate Aggregation of Split Network for Personalized Federated Learning
May 26, 2026Yunseok Kang, Jaeyoung SongFederated Learning Aggregation Split Federated Learning
Department of Electronics Engineering, Pusan National University, Republic of Korea

Aug 16, 2025J/K move · Enter open · S save

FedUHD: Unsupervised Federated Learning using In-Memory Hyperdimensional Computing

You Hak Lee, Keming Fan, Xiaofan Yu, Quanling Zhao+3

University of California San Diego, USA

Compute-in-Memory Federated Learning Aggregation Communication-Efficient Distributed Training Non-IID Federated Learning Hyperdimensional Computing Federated Learning+1

Open paper AskPDF arXiv

Unsupervised federated learning (UFL) enables privacy-preserving distributed training without data labeling, yet practical deployment remains challenging due to non-IID data, high computational and communication costs at edge devices, and sensitivity to communication noise. We propose FedUHD, the first UFL framework based on Hyperdimensional Computing (HDC). On the client side, FedUHD employs kNN-based cluster hypervector removal to mitigate non-IID effects by filtering detrimental local outliers. On the server side, cluster-aware HDC aggregation leverages cluster-level statistics to stabilize learning across heterogeneous clients. To further improve efficiency, we design a compute-in-memory (CIM) accelerator based on a novel phase-change memory (PCM) device, integrated with lightweight ASIC digital modules to execute the client-side HDC pipeline within the accelerator. The intrinsic robustness of HDC to low precision and device variations enables efficient mapping onto analog PCM crossbars, exploiting massive parallelism while minimizing data movement. Experimental results show that FedUHD achieves comparable accuracy to state-of-the-art neural network-based UFL methods across all datasets. On HAR and CIFAR10/100, FedUHD delivers an average 2,239x speedup and 1,542x higher energy efficiency on GPU. In addition, FedUHD reduces communication cost by up to 176x on HAR and CIFAR10/100 and demonstrates greater robustness than Orchestra under communication noise. Compared to GPU implementation of FedUHD, the proposed PCM-based accelerator provides an additional 4.07x speedup and three orders of magnitude higher energy efficiency on average. Furthermore, the results demonstrate the benefit of PCM over RRAM as a CIM substrate.

Abstract

Federated Domain Generalization

Explore similar work

FedUHD: Unsupervised Federated Learning using In-Memory Hyperdimensional Computing

FedAvg for HAR: Exploring the Tradeoff Between Personalized and Generalization Accuracy

Separate Aggregation of Split Network for Personalized Federated Learning