Collaborative Temporal Feature Generation via Critic-Free Reinforcement Learning for Cross-User Sensor-Based Activity Recognition
Authors: Xiaozhou Ye, Feng Jiang, Zihan Wang, Xiulai Wang, Yutao Zhang, Kevin I-Kai Wang
Organizations: School of Artificial Intelligence (School of Future Technology), Nanjing University of Information Science and Technology, Nanjing, China · School of Cyberspace Security, Nanjing University of Information Science and Technology, Nanjing, China · Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China · Department of Electrical, Computer, and Software Engineering, The University of Auckland, Auckland, New Zealand
Human Activity Recognition using wearable inertial sensors is foundational to healthcare monitoring, fitness analytics, and context-aware computing, yet its deployment is hindered by cross-user variability arising from heterogeneous physiological traits, motor habits, and sensor placements. Existing domain generalization approaches either neglect temporal dependencies in sensor streams or depend on impractical target-domain annotations. We propose a different paradigm: modeling generalizable feature extraction as a collaborative sequential generation process governed by reinforcement learning. Our framework, CTFG (Collaborative Temporal Feature Generation), employs a Transformer-based autoregressive generator that incrementally constructs feature token sequences, each conditioned on prior context and the encoded sensor input. The generator is optimized via Group-Relative Policy Optimization, a critic-free algorithm that evaluates each generated sequence against a cohort of alternatives sampled from the same input, deriving advantages through intra-group normalization rather than learned value estimation. This design eliminates the distribution-dependent bias inherent in critic-based methods and provides self-calibrating optimization signals that remain stable across heterogeneous user distributions. A tri-objective reward comprising class discrimination, cross-user invariance, and temporal fidelity jointly shapes the feature space to separate activities, align user distributions, and preserve fine-grained temporal content. Evaluations on the DSADS and PAMAP2 benchmarks demonstrate state-of-the-art cross-user accuracy (88.53% and 75.22%), substantial reduction in inter-task training variance, accelerated convergence, and robust generalization under varying action-space dimensionalities.
Figures & tables
Figure 1: Overview of the proposed CTFG framework. During training, the autoregressive generator produces G candidate feature sequences per input, evaluated by a tri-objective reward ( Rcls , Rinv , Rtmp ) and optimized via group-relative advantages A^(g) without a value function. During inference, a single deterministic forward pass (using predicted means μi,j only) generates features, which are flattened into z~i=vec(zi) and classified by Logistic Regression to produce the predicted label y^i .
Figure 2: Group-relative advantage in the latent space. For each input, G sampled feature sequences are compared against their group mean. Sequences achieving above-average reward receive positive advantages and are reinforced; below-average sequences are suppressed.
PAMAP2 — 6 subjects, 11 activities, 100 Hz
Groups
A=[1,2], B=[5,6], C=[7,8]
Sensors
Chest/wrist/ankle; accel.+gyro.
Window
3 s (300 samples), 50% overlap
DSADS — 8 subjects, 19 activities, 25 Hz
Groups
A=[1,2], B=[3,4], C=[5,6], D=[7,8]
Sensors
Torso/arms/legs; accel.+gyro.
Table 1: Benchmark dataset configurations.
Method
ABC → D
ACD → B
ABD → C
BCD → A
AVG
STD
ERM
76.56
77.25
81.03
75.39
77.56
2.11
RSC
78.92
83.94
82.47
80.42
81.44
1.92
ANDMask
82.36
78.75
84.87
82.22
82.05
2.18
AdaRNN
79.53
77.84
83.23
76.92
79.38
2.41
ACON
85.43
88.76
89.17
87.25
87.65
1.47
PPO-var.
87.77
85.48
91.96
87.93
88.29
2.33
Table 2: Cross-user accuracy (%) on DSADS.
Method
AB → C
AC → B
BC → A
AVG
STD
ERM
61.42
66.38
64.75
64.18
2.06
RSC
64.27
71.46
65.13
66.95
3.21
ANDMask
58.75
67.13
68.72
64.87
4.37
AdaRNN
62.34
74.47
65.95
67.59
5.09
ACON
67.31
75.97
72.58
71.95
3.56
PPO-var.
69.01
79.29
74.14
74.15
4.20
Table 3: Cross-user accuracy (%) on PAMAP2.
Figure 3: Convergence comparison on PAMAP2 (epochs 0–100). Shaded regions indicate ± 1 standard deviation across leave-one-group-out configurations.
Figure 4: Convergence comparison on DSADS (epochs 0–100). GRPO converges faster with monotonically decreasing variance, while PPO exhibits mid-training instability.
Figure 5: Token count sensitivity on DSADS. GRPO maintains stable accuracy with decreasing variance, while PPO collapses at high token counts.
Figure 6: Token count sensitivity on PAMAP2. Both methods degrade at s=20 on this smaller dataset due to reward landscape flattening.
School of Computer Sciences, Universiti Sains Malaysia, 11800 Pulau Pinang, Malaysia · Artificial Intelligence Research Center, Ajman University, Ajman, United Arab Emirates · Al-Quds Open University, Ramallah, Palestine
Centre for Applied Autonomous Sensor Systems (AASS), Örebro University, Örebro, Sweden · Faculty of Information Science & Electrical Engineering, Kyushu University, Fukuoka, Japan · RPTU, Kaiserslautern, Germany +2