Collaborative Temporal Feature Generation via Critic-Free Reinforcement Learning for Cross-User Sensor-Based Activity Recognition
Authors: Xiaozhou Ye, Feng Jiang, Zihan Wang, Xiulai Wang, Yutao Zhang, Kevin I-Kai Wang
Organizations: School of Artificial Intelligence (School of Future Technology), Nanjing University of Information Science and Technology, Nanjing, China · School of Cyberspace Security, Nanjing University of Information Science and Technology, Nanjing, China · Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China · Department of Electrical, Computer, and Software Engineering, The University of Auckland, Auckland, New Zealand
Human Activity Recognition using wearable inertial sensors is foundational to healthcare monitoring, fitness analytics, and context-aware computing, yet its deployment is hindered by cross-user variability arising from heterogeneous physiological traits, motor habits, and sensor placements. Existing domain generalization approaches either neglect temporal dependencies in sensor streams or depend on impractical target-domain annotations. We propose a different paradigm: modeling generalizable feature extraction as a collaborative sequential generation process governed by reinforcement learning. Our framework, CTFG (Collaborative Temporal Feature Generation), employs a Transformer-based autoregressive generator that incrementally constructs feature token sequences, each conditioned on prior context and the encoded sensor input. The generator is optimized via Group-Relative Policy Optimization, a critic-free algorithm that evaluates each generated sequence against a cohort of alternatives sampled from the same input, deriving advantages through intra-group normalization rather than learned value estimation. This design eliminates the distribution-dependent bias inherent in critic-based methods and provides self-calibrating optimization signals that remain stable across heterogeneous user distributions. A tri-objective reward comprising class discrimination, cross-user invariance, and temporal fidelity jointly shapes the feature space to separate activities, align user distributions, and preserve fine-grained temporal content. Evaluations on the DSADS and PAMAP2 benchmarks demonstrate state-of-the-art cross-user accuracy (88.53% and 75.22%), substantial reduction in inter-task training variance, accelerated convergence, and robust generalization under varying action-space dimensionalities.
Figures & tables
Figure 1: Overview of the proposed CTFG framework. During training, the autoregressive generator produces G candidate feature sequences per input, evaluated by a tri-objective reward ( Rcls , Rinv , Rtmp ) and optimized via group-relative advantages A^(g) without a value function. During inference, a single deterministic forward pass (using predicted means μi,j only) generates features, which are flattened into z~i=vec(zi) and classified by Logistic Regression to produce the predicted label y^i .
Figure 2: Group-relative advantage in the latent space. For each input, G sampled feature sequences are compared against their group mean. Sequences achieving above-average reward receive positive advantages and are reinforced; below-average sequences are suppressed.
PAMAP2 — 6 subjects, 11 activities, 100 Hz
Groups
A=[1,2], B=[5,6], C=[7,8]
Sensors
Chest/wrist/ankle; accel.+gyro.
Window
3 s (300 samples), 50% overlap
DSADS — 8 subjects, 19 activities, 25 Hz
Groups
A=[1,2], B=[3,4], C=[5,6], D=[7,8]
Sensors
Torso/arms/legs; accel.+gyro.
Table 1: Benchmark dataset configurations.
Method
ABC → D
ACD → B
ABD → C
BCD → A
AVG
STD
ERM
76.56
77.25
81.03
75.39
77.56
2.11
RSC
78.92
83.94
82.47
80.42
81.44
1.92
ANDMask
82.36
78.75
84.87
82.22
82.05
2.18
AdaRNN
79.53
77.84
83.23
76.92
79.38
2.41
ACON
85.43
88.76
89.17
87.25
87.65
1.47
PPO-var.
87.77
85.48
91.96
87.93
88.29
2.33
Table 2: Cross-user accuracy (%) on DSADS.
Method
AB → C
AC → B
BC → A
AVG
STD
ERM
61.42
66.38
64.75
64.18
2.06
RSC
64.27
71.46
65.13
66.95
3.21
ANDMask
58.75
67.13
68.72
64.87
4.37
AdaRNN
62.34
74.47
65.95
67.59
5.09
ACON
67.31
75.97
72.58
71.95
3.56
PPO-var.
69.01
79.29
74.14
74.15
4.20
Table 3: Cross-user accuracy (%) on PAMAP2.
Figure 3: Convergence comparison on PAMAP2 (epochs 0–100). Shaded regions indicate ± 1 standard deviation across leave-one-group-out configurations.
Figure 4: Convergence comparison on DSADS (epochs 0–100). GRPO converges faster with monotonically decreasing variance, while PPO exhibits mid-training instability.
Figure 5: Token count sensitivity on DSADS. GRPO maintains stable accuracy with decreasing variance, while PPO collapses at high token counts.
Figure 6: Token count sensitivity on PAMAP2. Both methods degrade at s=20 on this smaller dataset due to reward landscape flattening.
Human Activity Recognition (HAR) has shown remarkable effectiveness in various applications, such as smart healthcare and intelligent manufacturing. However, a major challenge faced by HAR is the distribution shift across different sensor data domains, which often leads to decreased performance when deployed for real-world applications. To address this issue, this paper introduces GenHAR, a novel framework designed to mitigate the domain gap by learning domain-invariant sensor representations. GenHAR aims to enhance the generalization capabilities of HAR on target domains purely with data from the source domain. The key novelty of GenHAR lies in two aspects. Firstly, GenHAR tokenizes sensor data and learns correlations among frequency sensor channel dimensions to improve the robustness of HAR models. Secondly, GenHAR improves the efficiency via selective masking and an efficient attention mechanism. We conduct a systematic analysis of GenHAR by comparing it with state-of-the-art HAR methods on real-world human activity datasets. Results show that GenHAR outperforms state-of-the-art methods by 9.97% in accuracy, and reduces Floating Point Operations by 6.4 times. Moreover, we deploy GenHAR at a leading logistics company in 4 cities, and have detected 2.15 billion real-time activities. We release our code at: https://github.com/Sensor-FoundationModel/GenHAR.
Zhiqing Hong, Zelong Li, Xiubin Fan +5
HKUST (GZ) Guangzhou, China · JD Logistics Beijing, China · Beijing Normal–Hong Kong Baptist University Zhuhai, China +2
Sensor-based human activity recognition (HAR) has achieved significant progressed in fully supervised learning settings. However, these supervised learning models rely on large amount of labeled data, which require labor-intensive collection and meticulous annotation. To address these challenges, this paper proposes a Joint Embedding Predictive Architecture framework tailored for sensor-based HAR, designed to learn robust and generalizable representations from unlabeled datasets. The proposed framework features an encoder designed to explicitly model both the fine-grained local temporal representations within individual window and the long-term temporal sequence of adjacent windows. Furthermore, we introduce an improved Variance-Invariance-Covariance Regularization (VICReg) objective function that incorporates computationally lightweight norm term to stabilize the JEPA pre-training phase. This term balances variance, invariance and covariance constraints to prevent representation collapse. The proposed HAR-JEPA framework is evaluated using two benchmark continuously performed activity datasets. The results show that high-quality representations are successfully learned by the proposed framework. Furthermore, the representations learned by HAR-JEPA demonstrates superior generalization on minority, high variance transitional activities such as sit-to-stand and sit-to-lie where supervised learning tend to overfit due to limited support.
Mohd Halim Mohd Noor, Abdulrahman M. A. Baraka
School of Computer Sciences, Universiti Sains Malaysia, 11800 Pulau Pinang, Malaysia · Artificial Intelligence Research Center, Ajman University, Ajman, United Arab Emirates · Al-Quds Open University, Ramallah, Palestine
This paper addresses the problem of Human Activity Recognition (HAR) using data from wearable inertial sensors. An important challenge in HAR is the model's generalization capabilities to new unseen individuals due to inter-subject variability, i.e., the same activity is performed differently by different individuals. To address this problem, we propose a novel deep adversarial framework that integrates the concept of inter-subject variability in the adversarial task, thereby encouraging subject-invariant feature representations and enhancing the classification performance in the HAR problem. Our approach outperforms previous methods in three well-established HAR datasets using a leave-one-subject-out (LOSO) cross-validation. Further results indicate that our proposed adversarial task effectively reduces inter-subject variability among different users in the feature space, and it outperforms adversarial tasks from previous works when integrated into our framework. Code: https://github.com/FranciscoCalatrava/EmbeddedSubjectVariability.git
Francisco M. Calatrava-Nicolás, Shoko Miyauchi, Vitor Fortes Rey +3
Centre for Applied Autonomous Sensor Systems (AASS), Örebro University, Örebro, Sweden · Faculty of Information Science & Electrical Engineering, Kyushu University, Fukuoka, Japan · RPTU, Kaiserslautern, Germany +2