Embedded Inter-Subject Variability in Adversarial Learning for Inertial Sensor-Based Human Activity Recognition

Francisco M. Calatrava-Nicolás; Shoko Miyauchi; Vitor Fortes Rey; Paul Lukowicz; Todor Stoyanov; Oscar Martinez Mozos

cs.LGMar 5, 2026

Embedded Inter-Subject Variability in Adversarial Learning for Inertial Sensor-Based Human Activity Recognition

Authors: Francisco M. Calatrava-Nicolás, Shoko Miyauchi, Vitor Fortes Rey, Paul Lukowicz, Todor Stoyanov, Oscar Martinez Mozos

Organizations: Centre for Applied Autonomous Sensor Systems (AASS), Örebro University, Örebro, Sweden · Faculty of Information Science & Electrical Engineering, Kyushu University, Fukuoka, Japan · RPTU, Kaiserslautern, Germany · DFKI, Kaiserslautern, Germany · Escuela T´ecnica Superior de Ingenier´ıa y Dise˜no Industrial, Universidad Polit´ecnica de Madrid, Madrid, Spain

Abstract

This paper addresses the problem of Human Activity Recognition (HAR) using data from wearable inertial sensors. An important challenge in HAR is the model's generalization capabilities to new unseen individuals due to inter-subject variability, i.e., the same activity is performed differently by different individuals. To address this problem, we propose a novel deep adversarial framework that integrates the concept of inter-subject variability in the adversarial task, thereby encouraging subject-invariant feature representations and enhancing the classification performance in the HAR problem. Our approach outperforms previous methods in three well-established HAR datasets using a leave-one-subject-out (LOSO) cross-validation. Further results indicate that our proposed adversarial task effectively reduces inter-subject variability among different users in the feature space, and it outperforms adversarial tasks from previous works when integrated into our framework. Code: https://github.com/FranciscoCalatrava/EmbeddedSubjectVariability.git

Figures & tables

Fig. 1 : Overview of our framework: Steps 1–2 pre-train all module weights. Step 3 introduces adversarial learning to minimize inter-subject variability. Evaluation is performed on previously unseen users. Data distribution $A$ is used for the activity classification (Eq. 1 ), while $A^{\prime}$ is the data distribution used in the adversarial task (Eq. 2 ). The circular arrow in Step 3 illustrates the iterative nature of adversarial learning.

Dataset	Model	Accuracy	$F1-Score_{M}$
	MCCNN	$0.7939\pm 0.0770$	$0.7511\pm 0.0940$
	DCLSTM	$0.7853\pm 0.1012$	$0.7266\pm 0.1113$
PAMAP2	METIER	$0.7924\pm 0.0597$	$0.7540\pm 0.0574$
	UIDFE	$0.8014\pm 0.1353$	$0.7648\pm 0.1447$
	DDLearn	$0.7188\pm 0.1561$	$0.6442\pm 0.1691$
	Ours	$\mathbf{0.8703\pm 0.1219}$	$\mathbf{0.8643\pm 0.1431}$

Table 1: Comparison with previous approaches (Best in bold).

Panel of Figure 3 — Fig. 2 : First column: $F1-Score_{M}$ for MHEALTH, PAMAP2, and REALDISP (top-down). Second column: Percentage change in Wasserstein distance from Step 2 to Step 3 between training and test set distributions. The top plot gives the distance change per dataset, while the middle (PAMAP2) and bottom (REALDISP) plots detail the distance change per activity. Positive bars (blue) mark a reduction (distance after Step 3 is smaller than after Step 2); negative bars (red) mark an increase.

Dataset	Disc	Accuracy	$F1-Score_{M}$
	Superv	$0.7472\pm 0.1067$	$0.7444\pm 0.1232$
PAMAP2	Step 2	$0.8179\pm 0.1255$	$0.8229\pm 0.1357$
	Our	$\mathbf{0.8703\pm 0.1219}$	$\mathbf{0.8643\pm 0.1431}$
	Superv	$0.9064\pm 0.0724$	$0.9220\pm 0.0566$
REALDISP	Step 2	$0.9372\pm 0.0512$	$0.9098\pm 0.0620$
	Our	$\mathbf{0.9710\pm 0.0423}$	$\mathbf{0.9651\pm 0.0368}$

Table 2: Ablation Study (Best in bold).

Dataset	Disc	Accuracy	$F1-Score_{M}$
PAMAP2	$D_{i}$	$0.8274\pm 0.1045$	$0.7869\pm 0.1102$
	$D_{b}$	$0.8130\pm 0.1453$	$0.7636\pm 0.1579$
	Ours	$\mathbf{0.8703\pm 0.1219}$	$\mathbf{0.8643\pm 0.1431}$
REALDISP	$D_{i}$	$0.9447\pm 0.0772$	$0.9394\pm 0.0566$
	$D_{b}$	$0.9511\pm 0.0619$	$0.9445\pm 0.0538$
	Ours	$\mathbf{0.9710\pm 0.0423}$	$\mathbf{0.9651\pm 0.0368}$

Table 3: Comparison to prior discrimination tasks (Best in bold).

Fig. 3 : Average accuracy and $F1-Score_{M}$ for variations in $w_{A}$ (top), $w_{R}$ (middle), and $w_{C}$ (bottom) for PAMAP2.

Explore similar work

Jul 17, 2026

Joint-Embedding Predictive Architecture for Sensor-based Activity Recognition

Sensor-based human activity recognition (HAR) has achieved significant progressed in fully supervised learning settings. However, these supervised learning models rely on large amount of labeled data, which require labor-intensive collection and meticulous annotation. To address these challenges, this paper proposes a Joint Embedding Predictive Architecture framework tailored for sensor-based HAR, designed to learn robust and generalizable representations from unlabeled datasets. The proposed framework features an encoder designed to explicitly model both the fine-grained local temporal representations within individual window and the long-term temporal sequence of adjacent windows. Furthermore, we introduce an improved Variance-Invariance-Covariance Regularization (VICReg) objective function that incorporates computationally lightweight norm term to stabilize the JEPA pre-training phase. This term balances variance, invariance and covariance constraints to prevent representation collapse. The proposed HAR-JEPA framework is evaluated using two benchmark continuously performed activity datasets. The results show that high-quality representations are successfully learned by the proposed framework. Furthermore, the representations learned by HAR-JEPA demonstrates superior generalization on minority, high variance transitional activities such as sit-to-stand and sit-to-lie where supervised learning tend to overfit due to limited support.

Mohd Halim Mohd Noor, Abdulrahman M. A. Baraka

School of Computer Sciences, Universiti Sains Malaysia, 11800 Pulau Pinang, Malaysia · Artificial Intelligence Research Center, Ajman University, Ajman, United Arab Emirates · Al-Quds Open University, Ramallah, Palestine

Human Activity Recognition Self-Supervised Time Series Representation Learning Joint-Embedding Predictive Architecture Invariant Representation Learning

Ask PDF

Sep 13, 2026

EdgeHAR: An Edge-Native Compact Sensor Foundation Model for Human Activity Recognition

Sensor-based human activity recognition (HAR) is fundamental to ubiquitous and wearable computing, yet existing foundation models are largely designed for cloud-scale deployment and struggle with real-world sensing shifts, including unseen users, devices, sampling rates, and sensor placements. We present EdgeHAR, an edge-native compact sensor foundation model designed for wearable intelligence. Unlike conventional models that entangle activity knowledge with acquisition variations, EdgeHAR learns transferable representations by factorizing sensor signals into three latent codes: an (i)Activity-Semantic Code capturing reusable activity knowledge, a (ii)Motion-Dynamics Code modeling temporal patterns, and an (iii)Acquisition-Context Code representing sensor-specific variations. This disentangled design enables efficient adaptation to new users, devices, placements, and activity classes with limited target-domain data. By incorporating lightweight adaptation modules, EdgeHAR achieves foundation-model-level transferability while satisfying edge constraints in computation, memory, latency, and privacy. Experiments across heterogeneous HAR datasets demonstrate that EdgeHAR maintains competitive recognition performance under distribution shifts with substantially reduced deployment cost. EdgeHAR establishes a practical paradigm for compact, edge-first foundation models for ubiquitous sensing systems.

He Zhang, Siyu Yuan, Siyu Liu +2

Northwestern Polytechnical University Xi’an, Shaanxi, China · RPTU University Kaiserslautern-Landau Kaiserslautern, Rheinland Pfalz, Germany · Xi’an University of Technology Xi’an, Shaanxi, China

Human Activity Recognition Wearable Sensing Edge ML Domain Generalization Disentangled Representation Learning

Ask PDF

Aug 27, 2026

HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition

Human Activity Recognition (HAR) using inertial measurement units (IMUs) enables a wide range of applications, yet the field still lacks a unified model that can generalize across diverse subjects, devices, and activities. Training such a model is difficult due to two key challenges: sensing heterogeneity -- differences in sampling rates, channel configurations, and sensor placements -- and poor generalization to unseen activities and label vocabularies. We introduce HALO (Heterogeneity-Aware Language-aligned Open-set model), a domain-specific IMU foundation model that addresses both challenges through a two-stage training framework. Stage 1 pretrains the IMU encoder with heterogeneity-aware self-supervised learning, including adaptive-pooling tokenization, channel-independent feature extraction, and contextualized sensor conditioning that injects natural-language sensor descriptions into each channel embedding. Stage 2 aligns this IMU encoder with text embeddings via synonym-aware soft contrastive learning, enabling open-set recognition via cosine-similarity retrieval without per-dataset classifiers. Trained on 10 public HAR datasets and evaluated on 7 held-out datasets, HALO outperforms five state-of-the-art baselines on all 8 aggregate metrics, and still leads on 3 of 4 settings under baseline-matched inputs. Despite using only ~35M trainable parameters -- 10x fewer than the latest foundation model MOMENT (341.2M) -- HALO improves zero-shot open-set accuracy, measured over all 87 training labels, by 13.7 percentage points. On two further datasets with severe distribution shift, every model including HALO collapses zero-shot. A video demonstration of HALO's performance in real world is available at https://youtu.be/rooVKragtFU

Zihan Ding, Liyu Zhang, Xiaomin Ouyang

Hong Kong University of Science and Technology Hong Kong, China

Human Activity Recognition Multimodal Foundation Models Latent Space Alignment

Ask PDF