Gated Adaptation for Continual Learning in Human Activity Recognition

Reza Rahimi Azghan; Gautham Krishna Gudur; Mohit Malu; Edison Thomaz; Giulia Pedrielli; Pavan Turaga; Hassan Ghasemzadeh

cs.LGMar 8, 2026

Gated Adaptation for Continual Learning in Human Activity Recognition

Authors: Reza Rahimi Azghan, Gautham Krishna Gudur, Mohit Malu, Edison Thomaz, Giulia Pedrielli, Pavan Turaga, Hassan Ghasemzadeh

Organizations: Arizona State University, Phoenix, AZ, USA · The University of Texas at Austin, Austin, TX, USA

Abstract

Wearable sensors in Internet of Things (IoT) ecosystems increasingly support applications such as remote health monitoring, elderly care, and smart home automation, all of which rely on robust human activity recognition (HAR). Continual learning systems must balance plasticity (learning new tasks) with stability (retaining prior knowledge), yet AI models often exhibit catastrophic forgetting, where learning new tasks degrades performance on earlier ones. This challenge is especially acute in domain-incremental HAR, where on-device models must adapt to new subjects with distinct movement patterns while maintaining accuracy on prior subjects without transmitting sensitive data to the cloud. We propose a parameter-efficient continual learning framework based on channel-wise gated modulation of frozen pretrained representations. Our key insight is that adaptation should operate through feature selection rather than feature generation: by restricting learned transformations to diagonal scaling of existing features, we preserve the geometry of pretrained representations while enabling subject-specific modulation. We provide a theoretical analysis showing that gating implements a bounded diagonal operator that limits representational drift compared to unconstrained linear transformations. Empirically, freezing the backbone substantially reduces forgetting, and lightweight gates restore lost adaptation capacity, achieving stability and plasticity simultaneously. On PAMAP2 with 8 sequential subjects, our approach reduces forgetting from 39.7% to 16.2% and improves final accuracy from 56.7% to 77.7%, while training less than 2% of parameters. Our method matches or exceeds standard continual learning baselines without replay buffers or task-specific regularization, confirming that structured diagonal operators are effective and efficient under distribution shift.

Figures & tables

Fig. 1: Catastrophic forgetting in subject-incremental HAR: accuracy on Subject 1 drops from 85% to 40% after training on just three additional subjects (PAMAP2 dataset).

Fig. 2: Overview of the proposed continual learning system for HAR, comprising data collection, preprocessing, and a classification model with a frozen pretrained backbone modulated by lightweight trainable gates.

Fig. 3: Channel-wise gating mechanism. Given an intermediate feature map $U_{\ell}$ , global average pooling (squeeze) produces a channel descriptor $z_{\ell}$ , which is transformed through bottleneck layers (excitation) to produce gate values $g_{\ell}\in(0,1)^{c_{\ell}}$ . The gates then scale each channel independently, preserving feature directions while modulating magnitudes.

Fig. 4: Cross-subject feature channel correlation matrix averaged over all subject pairs in PAMAP2. Entry $(c,c^{\prime})$ shows the Pearson correlation between channel $c$ of subject $i$ and channel $c^{\prime}$ of subject $j$ , computed using activity-class centroids as paired observations. The strong diagonal (mean $\rho=0.78$ ) indicates that channels preserve their identity across subjects, while the weak off-diagonal (mean $\rho\approx 0$ ) suggests minimal cross-channel mixing. This supports Assumption 1 : cross-subject variation is predominantly channel-wise.

Stack	Channels	Layers	Stride
Embedding	$C_{0}\to 256$	Conv1d (k=1)	1
Stack 1	256	4 conv layers (k=1,3,3,1)	1
Stack 2	$256\to 384$	4 conv layers (k=1,5,3,1)	1
Stack 3	$384\to 512$	4 conv layers (k=1,3,3,1)	2
Stack 4	512	4 conv layers (k=1,3,3,1)	2
Pooling	512	AdaptiveAvgPool1d	–

TABLE I: CNN architecture with 4 residual stacks. Each stack contains multiple convolutional layers with batch normalization and ReLU activations.

	PAMAP2			DSA			UCI-HAR			RealWorld
	8 subj., 12 act.			8 subj., 18 act.			30 subj., 6 act.			15 subj., 8 act.
	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$
EWC	$66.2{\pm}7.5$	$29.8{\pm}7.5$	$95.6{\pm}0.6$	$68.5{\pm}5.2$	$30.1{\pm}5.4$	$98.5{\pm}0.3$	$76.2{\pm}8.5$	$22.3{\pm}8.8$	$98.9{\pm}0.5$	$56.4{\pm}8.1$	$38.6{\pm}8.0$	$89.9{\pm}0.7$
LwF	$64.8{\pm}10.0$	$31.3{\pm}10.3$	$96.8{\pm}0.5$	$69.8{\pm}6.0$	$29.2{\pm}5.6$	$99.0{\pm}0.3$	$74.7{\pm}11.3$	$23.6{\pm}11.7$	$98.4{\pm}0.7$	$55.2{\pm}9.6$	$39.5{\pm}9.8$	$90.5{\pm}0.6$
HAT	$67.8{\pm}8.2$	$28.5{\pm}8.4$	$96.6{\pm}0.4$	$70.2{\pm}5.8$	$28.3{\pm}5.6$	$99.1{\pm}0.2$	$77.5{\pm}6.2$	$21.0{\pm}6.4$	$98.7{\pm}0.4$	$57.9{\pm}7.0$	$36.8{\pm}7.2$	$90.6{\pm}0.5$
Ours	${77.7{\pm}2.5}$	${16.2{\pm}2.6}$	$93.8{\pm}0.4$	${78.7{\pm}1.7}$	${19.6{\pm}1.7}$	$98.3{\pm}0.3$	${80.3{\pm}5.3}$	${14.5{\pm}5.4}$	$94.9{\pm}1.2$	${65.6{\pm}3.2}$	${26.4{\pm}3.3}$	$86.5{\pm}0.8$

TABLE II: Comparison with replay-free continual learning methods (mean

\pm

std over 10 permutations). FA: Final Accuracy (%), FM: Forgetting Measure (%), LA: Learning Accuracy (%)

	PAMAP2			DSA			UCI-HAR			RealWorld
	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$
DER	$88.5{\pm}0.6$	$8.1{\pm}0.8$	$95.6{\pm}0.4$	$92.3{\pm}1.8$	$6.9{\pm}1.8$	$98.2{\pm}0.2$	$89.5{\pm}1.9$	$9.8{\pm}2.0$	$98.4{\pm}0.2$	$75.6{\pm}1.4$	$16.3{\pm}1.5$	$90.4{\pm}0.4$
DER++	${90.1{\pm}1.6}$	${6.3{\pm}1.4}$	$95.4{\pm}0.4$	${94.1{\pm}1.7}$	${5.0{\pm}1.6}$	$98.0{\pm}0.2$	${90.5{\pm}2.6}$	${8.8{\pm}2.6}$	$99.4{\pm}0.2$	${77.1{\pm}1.9}$	${14.4{\pm}1.8}$	$90.7{\pm}0.3$
Ours	$77.7{\pm}2.5$	$16.2{\pm}2.6$	$93.8{\pm}0.4$	$78.7{\pm}1.7$	$19.6{\pm}1.7$	$98.3{\pm}0.3$	$80.3{\pm}5.3$	$14.5{\pm}5.4$	$94.9{\pm}1.2$	$65.6{\pm}3.2$	$26.4{\pm}3.3$	$86.5{\pm}0.8$

TABLE III: Comparison with replay-based methods (mean

\pm

std over 10 permutations). Params: percentage of model parameters updated per task. Buffer: number of stored samples.

	Trainable	Train step	Inf. FLOPs	Peak train	Retained storage (MB)
Method	params	(rel.)	/window	mem. (rel.)	PAMAP2	DSA	UCI	RealWorld
Ours (gates, frozen)	0.19 M (2.5%)	0.60×	1.67 G	$\sim$ 0.74×	0	0	0	0
Full-backbone (EWC, HAT)	7.44 M (100%)	1.0×	1.67 G	1.0×	0	0	0	0
Replay (DER, DER++)	7.44 M (100%)	1.0×	1.67 G	1.0×	10.3	17.2	2.3	10.3
Gates + DER	0.19 M (2.5%)	0.60×	1.67 G	$\sim$ 0.74×	10.3	17.2	2.3	10.3

TABLE IV: Efficiency, storage, and deployment cost. Trainable parameters and retained storage are exact; inference FLOPs are identical across methods (same forward pass); training-step time is reported relative to full-backbone training, and peak training memory is measured on the same GPU. Retained storage is given per dataset at a 500-sample buffer.

Trainable Backbone
	PAMAP2			DSA			UCI-HAR			RealWorld
	8 subj., 12 act.			8 subj., 18 act.			30 subj., 6 act.			15 subj., 8 act.
	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$
Base	$56.7{\pm}8.9$	$39.7{\pm}8.9$	$96.5{\pm}0.3$	$68.7{\pm}3.9$	$30.4{\pm}4.1$	$99.1{\pm}0.3$	$75.6{\pm}8.6$	$23.3{\pm}8.8$	$98.9{\pm}0.6$	$51.4{\pm}7.8$	$40.9{\pm}7.9$	$90.6{\pm}0.4$
Base + Gates	$64.9{\pm}10.0$	$31.3{\pm}10.3$	$96.2{\pm}0.5$	$69.8{\pm}6.0$	$29.2{\pm}5.9$	$99.0{\pm}0.3$	$74.7{\pm}11.3$	$23.7{\pm}11.7$	$98.4{\pm}0.7$	$56.3{\pm}9.1$	$36.6{\pm}9.3$	$90.3{\pm}0.5$
Frozen Backbone

TABLE V: Effect of backbone freezing and gating mechanisms across HAR benchmarks (mean

\pm

std over 10 permutations).

Frozen Backbone with Stacked Adaptation Layers
	PAMAP2			DSA			UCI-HAR			RealWorld
	8 subj., 12 act.			8 subj., 18 act.			30 subj., 6 act.			15 subj., 8 act.
	FA	FM	LA	FA	FM	LA	FA	FM	LA	FA	FM	LA
No stacked layers	$76.5{\pm}4.0$	$17.5{\pm}4.0$	$93.9{\pm}0.4$	$75.7{\pm}2.7$	$22.1{\pm}2.7$	$97.9{\pm}0.2$	$80.1{\pm}9.4$	$13.9{\pm}9.5$	$94.0{\pm}1.5$	$64.4{\pm}3.6$	$27.3{\pm}3.6$	$86.2{\pm}0.9$
1 stacked layer	$74.2{\pm}4.8$	$20.1{\pm}4.9$	$94.3{\pm}0.4$	$74.8{\pm}3.2$	$23.5{\pm}3.3$	$98.2{\pm}0.3$	$78.9{\pm}5.8$	$16.2{\pm}5.8$	$95.1{\pm}1.1$	$62.8{\pm}4.1$	$29.6{\pm}4.2$	$87.0{\pm}0.8$
2 stacked layers	$72.1{\pm}6.2$	$22.7{\pm}6.3$	$94.8{\pm}0.4$	$73.9{\pm}4.3$	$24.6{\pm}4.5$	$98.4{\pm}0.3$	$78.1{\pm}6.1$	$18.0{\pm}6.1$	$96.1{\pm}1.1$	$61.5{\pm}5.2$	$31.1{\pm}5.3$	$87.6{\pm}0.8$

TABLE VI: Comparison of gating (selection) versus stacked fully connected layers (generation) across benchmarks (mean

\pm

std over 10 permutations). Stacked layers are 512→512 with ReLU and dropout.

Task-Free Gates (Single Shared Gates)
	PAMAP2			DSA			UCI-HAR			RealWorld
	8 subj., 12 act.			8 subj., 18 act.			30 subj., 6 act.			15 subj., 8 act.
	FA	FM	LA	FA	FM	LA	FA	FM	LA	FA	FM	LA
Without KD (Ours)	${77.7{\pm}2.5}$	${16.2{\pm}2.6}$	$93.8{\pm}0.4$	${78.7{\pm}1.7}$	${19.6{\pm}1.7}$	$98.3{\pm}0.3$	${80.3{\pm}5.3}$	${14.5{\pm}5.4}$	$94.9{\pm}1.2$	${65.6{\pm}3.2}$	${26.4{\pm}3.3}$	$86.5{\pm}0.8$
With KD	$77.5{\pm}3.5$	$12.7{\pm}4.0$	$90.2{\pm}1.0$	$78.1{\pm}3.6$	$17.3{\pm}3.0$	$95.4{\pm}0.7$	$82.1{\pm}4.4$	$8.9{\pm}5.0$	$90.9{\pm}2.6$	$65.2{\pm}3.9$	$22.9{\pm}3.7$	$83.4{\pm}1.4$
Task-Aware Gates (Separate Gates per Task)

TABLE VII: Effect of knowledge distillation, task-aware gates, and experience replay (mean

\pm

std over permutations).

	PAMAP2			DSA			UCI-HAR			RealWorld
	8 subj., 12 act.			8 subj., 18 act.			30 subj., 6 act.			15 subj., 8 act.
	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$	FA $\uparrow$	FM $\downarrow$	LA $\uparrow$
0 gates (classifier only)	$76.5{\pm}4.0$	$17.5{\pm}4.0$	$93.9{\pm}0.4$	$75.7{\pm}2.7$	$22.1{\pm}2.7$	$97.9{\pm}0.2$	$80.1{\pm}9.4$	$13.9{\pm}9.5$	$94.0{\pm}1.5$	$64.4{\pm}3.6$	$27.3{\pm}3.6$	$86.2{\pm}0.9$
1 gate	$75.3{\pm}2.4$	$15.2{\pm}2.7$	$90.5{\pm}0.8$	$70.5{\pm}4.0$	$25.4{\pm}3.6$	$95.9{\pm}0.8$	$79.5{\pm}3.6$	$11.7{\pm}4.2$	$91.2{\pm}2.8$	$62.8{\pm}3.8$	$28.9{\pm}3.7$	$85.4{\pm}1.0$
2 gates	$72.6{\pm}3.2$	$18.2{\pm}3.0$	$90.8{\pm}0.7$	$71.5{\pm}2.7$	$24.8{\pm}2.2$	$96.3{\pm}0.8$	$78.7{\pm}6.6$	$14.0{\pm}5.7$	$92.8{\pm}2.3$	$61.9{\pm}4.5$	$29.6{\pm}4.4$	$86.0{\pm}1.0$
3 gates	$77.0{\pm}2.5$	$13.8{\pm}2.1$	$90.8{\pm}0.6$	$71.9{\pm}6.0$	$24.0{\pm}5.8$	$95.9{\pm}0.3$	$79.8{\pm}6.0$	$12.2{\pm}5.8$	$92.0{\pm}3.0$	$64.1{\pm}4.1$	$27.1{\pm}4.0$	$86.1{\pm}0.9$

TABLE VIII: Effect of the number of channel gates inserted into the frozen backbone (mean

\pm

std over 10 permutations). Gates are added to the first

N

stacks, from

0

(frozen classifier only) to

4

(all stacks, our default).

0

and

4

gates correspond to the Pretrained and Pretrained + Gates (Ours) rows of Table V .

Fig. 5: Per-subject accuracy evolution with our gating approach on PAMAP2. Unlike the baseline in Figure 1 , which dropped to 40% on Subject 1 after four subjects, our method retains 69.5% accuracy while achieving 93.8% learning accuracy on new subjects.

Explore similar work

May 12, 2026

Efficient and Adaptive Human Activity Recognition via LLM Backbones

Human Activity Recognition (HAR) is a core task in pervasive computing systems, where models must operate under strict computational constraints while remaining robust to heterogeneous and evolving deployment conditions. Recent advances based on Transformer architectures have significantly improved recognition performance, but typically rely on task-specific models trained from scratch, resulting in high training cost, large data requirements, and limited adaptability to domain shifts. In this paper, we propose a paradigm shift that reuses large pretrained language models (LLMs) as generic temporal backbones for sensor-based HAR, instead of designing domain-specific Transformers. To bridge the modality gap between inertial time series and language models, we introduce a structured convolutional projection that maps multivariate accelerometer and gyroscope signals into the latent space of the LLM. The pretrained backbone is kept frozen and adapted using parameter-efficient Low-Rank Adaptation (LoRA), drastically reducing the number of trainable parameters and the overall training cost. Through extensive experiments on standard HAR benchmarks, we show that this approach enables rapid convergence, strong data efficiency, and robust cross-dataset transfer, particularly in low-data and few-shot settings. At the same time, our results highlight the complementary roles of convolutional frontends and LLMs, where local invariances are handled at the signal level while long-range temporal dependencies are captured by the pretrained backbone. Overall, this work demonstrates that LLMs can serve as a practical, frugal, and scalable foundation for adaptive HAR systems, opening new directions for reusing foundation models beyond their original language domain.

Aleksandr Bredikhin, Philippe Lalanda, German Vega

Univ. Grenoble Alpes, France

Human Activity Recognition Domain Generalization Time-Series Foundation Models

Ask PDF

Jun 3, 2026

BayaHAR: Lightweight Bayesian Few-Shot User Adaptation for On-Device Personalized Human Activity Recognition

Sensor-based Human Activity Recognition (HAR) models often degrade on unseen users due to domain shifts caused by individual movement patterns and sensor placement. Practical wearable HAR systems therefore require personalization methods that are lightweight, applicable across diverse calibration scenarios, and robust under limited calibration data. We present BayaHAR, a gradient-free framework that repurposes pretrained HAR classifiers as Prototypical Networks using prior prototypes that preserve zero-shot performance while regularizing adaptation. For labeled calibration data, we introduce closed-form Bayesian prototype estimation and extend the same principle to weakly labeled data, requiring only knowledge of which activities were performed. With only 3 seconds of calibration (one shot) per activity, supervised adaptation improves test macro-F1 on unseen users by +2.76 to +33.44 percentage points across four datasets, while weakly supervised adaptation improves by +0.56 to +32.13 points. Since adaptation requires only closed-form prototype updates, the framework enables efficient and robust on-device personalization of preexisting HAR classifiers.

Maximilian Burzer, Till Riedel, Michael Beigl +1

Karlsruhe Institute of Technology Karlsruhe, Germany · IPAI Foundation gGmbH Heilbronn, Germany

Human Activity Recognition Domain Adaptation Few-Shot Learning On-Device Inference

Ask PDF

Sep 13, 2026

EdgeHAR: An Edge-Native Compact Sensor Foundation Model for Human Activity Recognition

Sensor-based human activity recognition (HAR) is fundamental to ubiquitous and wearable computing, yet existing foundation models are largely designed for cloud-scale deployment and struggle with real-world sensing shifts, including unseen users, devices, sampling rates, and sensor placements. We present EdgeHAR, an edge-native compact sensor foundation model designed for wearable intelligence. Unlike conventional models that entangle activity knowledge with acquisition variations, EdgeHAR learns transferable representations by factorizing sensor signals into three latent codes: an (i)Activity-Semantic Code capturing reusable activity knowledge, a (ii)Motion-Dynamics Code modeling temporal patterns, and an (iii)Acquisition-Context Code representing sensor-specific variations. This disentangled design enables efficient adaptation to new users, devices, placements, and activity classes with limited target-domain data. By incorporating lightweight adaptation modules, EdgeHAR achieves foundation-model-level transferability while satisfying edge constraints in computation, memory, latency, and privacy. Experiments across heterogeneous HAR datasets demonstrate that EdgeHAR maintains competitive recognition performance under distribution shifts with substantially reduced deployment cost. EdgeHAR establishes a practical paradigm for compact, edge-first foundation models for ubiquitous sensing systems.

He Zhang, Siyu Yuan, Siyu Liu +2

Northwestern Polytechnical University Xi’an, Shaanxi, China · RPTU University Kaiserslautern-Landau Kaiserslautern, Rheinland Pfalz, Germany · Xi’an University of Technology Xi’an, Shaanxi, China

Human Activity Recognition Wearable Sensing Edge ML Domain Generalization Disentangled Representation Learning

Ask PDF