Embedded Inter-Subject Variability in Adversarial Learning for Inertial Sensor-Based Human Activity Recognition

Francisco M. Calatrava-Nicolás; Shoko Miyauchi; Vitor Fortes Rey; Paul Lukowicz; Todor Stoyanov; Oscar Martinez Mozos

cs.LGMar 5, 2026

Embedded Inter-Subject Variability in Adversarial Learning for Inertial Sensor-Based Human Activity Recognition

Authors: Francisco M. Calatrava-Nicolás, Shoko Miyauchi, Vitor Fortes Rey, Paul Lukowicz, Todor Stoyanov, Oscar Martinez Mozos

Organizations: Centre for Applied Autonomous Sensor Systems (AASS), Örebro University, Örebro, Sweden · Faculty of Information Science & Electrical Engineering, Kyushu University, Fukuoka, Japan · RPTU, Kaiserslautern, Germany · DFKI, Kaiserslautern, Germany · Escuela T´ecnica Superior de Ingenier´ıa y Dise˜no Industrial, Universidad Polit´ecnica de Madrid, Madrid, Spain

Abstract

This paper addresses the problem of Human Activity Recognition (HAR) using data from wearable inertial sensors. An important challenge in HAR is the model's generalization capabilities to new unseen individuals due to inter-subject variability, i.e., the same activity is performed differently by different individuals. To address this problem, we propose a novel deep adversarial framework that integrates the concept of inter-subject variability in the adversarial task, thereby encouraging subject-invariant feature representations and enhancing the classification performance in the HAR problem. Our approach outperforms previous methods in three well-established HAR datasets using a leave-one-subject-out (LOSO) cross-validation. Further results indicate that our proposed adversarial task effectively reduces inter-subject variability among different users in the feature space, and it outperforms adversarial tasks from previous works when integrated into our framework. Code: https://github.com/FranciscoCalatrava/EmbeddedSubjectVariability.git

Figures & tables

Fig. 1 : Overview of our framework: Steps 1–2 pre-train all module weights. Step 3 introduces adversarial learning to minimize inter-subject variability. Evaluation is performed on previously unseen users. Data distribution $A$ is used for the activity classification (Eq. 1 ), while $A^{\prime}$ is the data distribution used in the adversarial task (Eq. 2 ). The circular arrow in Step 3 illustrates the iterative nature of adversarial learning.

Dataset	Model	Accuracy	$F1-Score_{M}$
	MCCNN	$0.7939\pm 0.0770$	$0.7511\pm 0.0940$
	DCLSTM	$0.7853\pm 0.1012$	$0.7266\pm 0.1113$
PAMAP2	METIER	$0.7924\pm 0.0597$	$0.7540\pm 0.0574$
	UIDFE	$0.8014\pm 0.1353$	$0.7648\pm 0.1447$
	DDLearn	$0.7188\pm 0.1561$	$0.6442\pm 0.1691$
	Ours	$\mathbf{0.8703\pm 0.1219}$	$\mathbf{0.8643\pm 0.1431}$

Table 1: Comparison with previous approaches (Best in bold).

Panel of Figure 3 — Fig. 2 : First column: $F1-Score_{M}$ for MHEALTH, PAMAP2, and REALDISP (top-down). Second column: Percentage change in Wasserstein distance from Step 2 to Step 3 between training and test set distributions. The top plot gives the distance change per dataset, while the middle (PAMAP2) and bottom (REALDISP) plots detail the distance change per activity. Positive bars (blue) mark a reduction (distance after Step 3 is smaller than after Step 2); negative bars (red) mark an increase.

Dataset	Disc	Accuracy	$F1-Score_{M}$
	Superv	$0.7472\pm 0.1067$	$0.7444\pm 0.1232$
PAMAP2	Step 2	$0.8179\pm 0.1255$	$0.8229\pm 0.1357$
	Our	$\mathbf{0.8703\pm 0.1219}$	$\mathbf{0.8643\pm 0.1431}$
	Superv	$0.9064\pm 0.0724$	$0.9220\pm 0.0566$
REALDISP	Step 2	$0.9372\pm 0.0512$	$0.9098\pm 0.0620$
	Our	$\mathbf{0.9710\pm 0.0423}$	$\mathbf{0.9651\pm 0.0368}$

Table 2: Ablation Study (Best in bold).

Dataset	Disc	Accuracy	$F1-Score_{M}$
PAMAP2	$D_{i}$	$0.8274\pm 0.1045$	$0.7869\pm 0.1102$
	$D_{b}$	$0.8130\pm 0.1453$	$0.7636\pm 0.1579$
	Ours	$\mathbf{0.8703\pm 0.1219}$	$\mathbf{0.8643\pm 0.1431}$
REALDISP	$D_{i}$	$0.9447\pm 0.0772$	$0.9394\pm 0.0566$
	$D_{b}$	$0.9511\pm 0.0619$	$0.9445\pm 0.0538$
	Ours	$\mathbf{0.9710\pm 0.0423}$	$\mathbf{0.9651\pm 0.0368}$

Table 3: Comparison to prior discrimination tasks (Best in bold).

Fig. 3 : Average accuracy and $F1-Score_{M}$ for variations in $w_{A}$ (top), $w_{R}$ (middle), and $w_{C}$ (bottom) for PAMAP2.

Explore similar work

Joint-Embedding Predictive Architecture for Sensor-based Activity Recognition
Jul 17, 2026Mohd Halim Mohd Noor, Abdulrahman M. A. BarakaHuman Activity Recognition Self-Supervised Time Series Representation Learning
School of Computer Sciences, Universiti Sains Malaysia, 11800 Pulau Pinang, Malaysia · Artificial Intelligence Research Center, Ajman University, Ajman, United Arab Emirates · Al-Quds Open University, Ramallah, Palestine
EdgeHAR: An Edge-Native Compact Sensor Foundation Model for Human Activity Recognition
Sep 13, 2026He Zhang, Siyu Yuan, Siyu Liu +2Human Activity Recognition Wearable Sensing
Northwestern Polytechnical University Xi’an, Shaanxi, China · RPTU University Kaiserslautern-Landau Kaiserslautern, Rheinland Pfalz, Germany · Xi’an University of Technology Xi’an, Shaanxi, China
HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition
Aug 27, 2026Zihan Ding, Liyu Zhang, Xiaomin OuyangHuman Activity Recognition Multimodal Foundation Models
Hong Kong University of Science and Technology Hong Kong, China

Jul 17, 2026J/K move · Enter open · S save

Joint-Embedding Predictive Architecture for Sensor-based Activity Recognition

Mohd Halim Mohd Noor, Abdulrahman M. A. Baraka

School of Computer Sciences, Universiti Sains Malaysia, 11800 Pulau Pinang, Malaysia · Artificial Intelligence Research Center, Ajman University, Ajman, United Arab Emirates · Al-Quds Open University, Ramallah, Palestine

Human Activity Recognition Self-Supervised Time Series Representation Learning Joint-Embedding Predictive Architecture Invariant Representation Learning

Open paper AskPDF arXiv

Sensor-based human activity recognition (HAR) has achieved significant progressed in fully supervised learning settings. However, these supervised learning models rely on large amount of labeled data, which require labor-intensive collection and meticulous annotation. To address these challenges, this paper proposes a Joint Embedding Predictive Architecture framework tailored for sensor-based HAR, designed to learn robust and generalizable representations from unlabeled datasets. The proposed framework features an encoder designed to explicitly model both the fine-grained local temporal representations within individual window and the long-term temporal sequence of adjacent windows. Furthermore, we introduce an improved Variance-Invariance-Covariance Regularization (VICReg) objective function that incorporates computationally lightweight norm term to stabilize the JEPA pre-training phase. This term balances variance, invariance and covariance constraints to prevent representation collapse. The proposed HAR-JEPA framework is evaluated using two benchmark continuously performed activity datasets. The results show that high-quality representations are successfully learned by the proposed framework. Furthermore, the representations learned by HAR-JEPA demonstrates superior generalization on minority, high variance transitional activities such as sit-to-stand and sit-to-lie where supervised learning tend to overfit due to limited support.

Abstract

Explore similar work

Joint-Embedding Predictive Architecture for Sensor-based Activity Recognition

EdgeHAR: An Edge-Native Compact Sensor Foundation Model for Human Activity Recognition

HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition