Coverage-Aware Virtual IMU Augmentation for Low-Resource Human Activity Recognition

Jiayuan Gao; Yingwei Zhang; Ziyao Tang; Yuejia Ma; Yuanzhe Chen; Shuchao Song; Boshi Tang

cs.AISep 15, 2026

Coverage-Aware Virtual IMU Augmentation for Low-Resource Human Activity Recognition

Authors: Jiayuan Gao, Yingwei Zhang, Ziyao Tang, Yuejia Ma, Yuanzhe Chen, Shuchao Song, Boshi Tang

Organizations: Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Nanyang Technological University, Singapore · University of Science and Technology Beijing, Beijing, China · Tsinghua University, Beijing, China

Abstract

IMU-based human activity recognition (HAR) enables continuous, privacy-friendly monitoring of daily activities using wearable sensors. However, building reliable HAR models that generalize across diverse users and real-world conditions requires large amounts of labeled IMU data, which are expensive and difficult to collect. Existing approaches mainly rely on augmentation or synthesis to expand available data, but indiscriminately adding virtual samples may provide little new coverage and introduce unreliable supervision. To overcome these challenges, we propose a novel coverage-aware virtual IMU augmentation framework that decides where to supplement real data, how to generate and select virtual candidates, and how strongly to weight them during training. Specifically, we select diversity and scarcity anchors in a learned sensor embedding space, convert anchor dynamics into prompts, and generate virtual IMU candidates for each anchor. We then rank candidates by a selection cost combining anchor proximity and label consistency, and incorporate the selected candidates into HAR training with reliability-based weights. Experiments on public HAR benchmarks show that our method consistently improves recognition performance over competitive baselines, and ablation studies confirm the effectiveness of the proposed framework design.

Figures & tables

Fig. 1: Motivation for coverage-aware virtual IMU augmentation. Limited real IMU collections provide reliable but incomplete observations of activity patterns, whereas virtual IMU data can expand training coverage but may introduce shifted or unreliable samples.

Fig. 2: Framework overview. (1) Anchor-guided virtual IMU generation: select real training windows as diversity and scarcity anchors in a learned sensor embedding space, construct anchor-conditioned prompts from activity labels and anchor-level tempo, intensity, and periodicity attributes, and synthesize multiple virtual IMU candidates. (2) Virtual candidate assessment: assess candidates using their embedding-space proximity to the corresponding anchors and label consistency estimated by the seed classifier head, combine the two criteria into a selection cost, and retain the candidates with the lowest costs for each anchor. (3) Reliability-weighted HAR training: assign reliability-based sample weights using candidate selection costs and within-anchor ranks, and train the HAR model on real IMU windows and selected virtual candidates.

HAR benchmarks
Dataset	Subjects	Activities	Sensor placement	Protocol
USC-HAD	14	7	front-right hip	LOSO
PAMAP2	8	5	chest	LOSO
RealWorld	15	5	forearm	LOSO
Exploratory medical extension
Dataset	Folds	Task	Sensor placement	Protocol

TABLE I: Experimental datasets, sensor configurations, and evaluation protocols.

Aspect	Setting
Data preprocessing	2-s windows; 1-s stride; 20 Hz
Anchor selection	class-wise diversity and scarcity anchors from real training windows
Prompt conditioning	activity label; tempo, intensity, periodicity
Virtual generation	T2M-GPT followed by IMUSim
Candidate pool	20 candidates per anchor
Candidate selection	lowest-cost $K_{\mathrm{sel}}$ candidates based on embedding distance and label consistency

TABLE II: Main settings for virtual IMU construction and HAR training.

Dataset	Method	DeepConvLSTM [ 36 ]			DeepConvLSTM-Attention [ 37 ]			MLP-HAR [ 38 ]
Dataset	Method	Acc. $\uparrow$	F1 $\uparrow$	$\Delta$ F1	Acc. $\uparrow$	F1 $\uparrow$	$\Delta$ F1	Acc. $\uparrow$	F1 $\uparrow$	$\Delta$ F1
USC-HAD	Real-only	75.44 $\pm$ 3.05	74.18 $\pm$ 1.99	–	79.21 $\pm$ 1.17	78.28 $\pm$ 1.63	–	58.20 $\pm$ 0.63	54.14 $\pm$ 0.97	–
	Trad. Aug. [ 9 ]	80.23 $\pm$ 0.44	78.94 $\pm$ 0.87	+4.76	81.08 $\pm$ 0.59	80.25 $\pm$ 0.40	+1.97	62.22 $\pm$ 1.27	58.26 $\pm$ 1.22	+4.12
	TimeGAN [ 39 ]	77.47 $\pm$ 1.15	76.20 $\pm$ 1.38	+2.02	79.31 $\pm$ 1.43	78.05 $\pm$ 1.07	-0.23	60.05 $\pm$ 1.30	55.41 $\pm$ 1.48	+1.27
	Diffusion-TS [ 40 ]	80.13 $\pm$ 2.47	78.77 $\pm$ 2.29	+4.59	80.33 $\pm$ 1.05	79.22 $\pm$ 0.87	+0.94	63.18 $\pm$ 1.33	59.99 $\pm$ 1.51	+5.84
	IMUGPT [ 5 ]	75.41 $\pm$ 1.33	73.02 $\pm$ 0.98	-1.16	75.06 $\pm$ 0.36	73.22 $\pm$ 0.26	-5.06	61.43 $\pm$ 0.56	58.59 $\pm$ 0.64	+4.45

TABLE III: Main results across three downstream HAR models.

Fig. 3: Effect of anchor-source composition. Bars show the Macro-F1 gain over the corresponding Real-only baseline. Combining diversity and scarcity anchors performs best on USC-HAD and PAMAP2, whereas scarcity-only is slightly higher on RealWorld.

Dataset	Real-only	Random	Full Ours	$\Delta$ F1
USC-HAD	74.32 $\pm$ 1.93	80.18 $\pm$ 0.81	80.94 $\pm$ 0.74	+0.76
PAMAP2	62.95 $\pm$ 1.07	73.94 $\pm$ 0.50	77.35 $\pm$ 0.51	+3.41
RealWorld	66.97 $\pm$ 1.12	70.98 $\pm$ 1.74	71.68 $\pm$ 1.45	+0.70

TABLE IV: Coverage-aware versus random anchor selection.

Fig. 4: Effect of candidate ordering across the three datasets.

Dataset	Full Ours	Equal-rank	$\Delta$
USC-HAD	80.94 $\pm$ 0.68	78.71 $\pm$ 1.78	-2.23
PAMAP2	77.35 $\pm$ 0.51	73.59 $\pm$ 0.18	-3.76
RealWorld	71.68 $\pm$ 1.45	70.13 $\pm$ 1.95	-1.55

TABLE V: Comparison of rank-aware and equal-rank weighting. Values are Macro-F1 (%);

\Delta

is computed as Equal-rank minus Full Ours.

Fig. 5: Macro-F1 under different labeled real-data ratios on (a) USC-HAD, (b) PAMAP2, and (c) RealWorld.

Fig. 6: Macro-F1 with different numbers of selected virtual candidates per anchor on USC-HAD, PAMAP2, and RealWorld. Error bars indicate the standard deviation across random seeds.

Fig. 7: t-SNE visualization of low-label real training data and final selected virtual IMU samples on (a) USC-HAD, (b) PAMAP2, and (c) RealWorld. Colors denote activity classes, circles and triangles indicate real and virtual samples, respectively, and triangle size represents the effective training weight of each virtual sample.

Method	Bal. Acc. $\uparrow$	Macro-F1 $\uparrow$	$\Delta$ Macro-F1
Real-only	$63.64\pm 8.90$	$65.02\pm 11.30$	–
Trad. Aug.	$73.82\pm 10.47$	$74.94\pm 10.10$	$+9.92$
TimeGAN	$76.82\pm 9.84$	$73.29\pm 10.36$	$+8.27$
Diffusion-TS	$77.27\pm 4.31$	$77.22\pm 4.53$	$+12.19$
IMUGPT	$80.20\pm 8.76$	$78.28\pm 7.74$	$+13.26$
Ours	$\mathbf{80.52\pm 9.08}$	$\mathbf{78.94\pm 8.64}$	$\mathbf{+13.92}$

TABLE VI: PD-vs-HC classification results on PADS.

Explore similar work

Modular Retrieval-Augmented Generalization for Human Action Recognition
Apr 28, 2026Peng Liao, Shangsong Liang, Lin Chen +1Human Activity Recognition
School of Computer Science and Engineering, Sun Yat-sen University · Engineering Research Centre of Applied Technology on Machine Translation and Artificial Intelligence, Macao Polytechnic University
VSMP-IMU: Video-Grounded Semantic Motion Programs for Sensor-Aware Synthetic IMU Generation
Aug 6, 2026Lala Shakti Swarup Ray, Vitor Fortes Rey, Mengxi Liu +2Human Activity Recognition Synthetic Data Generation
DFKI, Germany · DFKI · RPTU Kaiserslautern-Landau
Rotation-Invariant Multi-IMU Activity Recognition under Independent Per-Location Orientation Shifts
Aug 16, 2026Seungyeol Baek, Yoonbyung Chai, Yonghyeon Lee +2Human Activity Recognition Wearable Sensing
Korea University Seoul, Republic of Korea · Massachusetts Institute of Technology Cambridge, Massachusetts, USA · Dept. of Artificial Intelligence Korea University Seoul, Republic of Korea

Apr 28, 2026J/K move · Enter open · S save

Modular Retrieval-Augmented Generalization for Human Action Recognition

Peng Liao, Shangsong Liang, Lin Chen, Peijia Zheng

School of Computer Science and Engineering, Sun Yat-sen University · Engineering Research Centre of Applied Technology on Machine Translation and Artificial Intelligence, Macao Polytechnic University

Human Activity Recognition

Open paper AskPDF arXiv

Inertial Measurement Unit (IMU)-based Human Activity Recognition (HAR) aims to interpret and classify user behaviors from temporal motion signals. Recently, deep learning frameworks have advanced this task by learning and extracting discriminative spatiotemporal representations, significantly improving recognition performance. However, IMU-based HAR still faces several critical challenges, particularly limited training samples and static knowledge utilization, both of which severely hinder its large-scale deployment. In this paper, we introduce MoRA, the first Retrieval-Augmented Module specifically designed for motion series. It can be flexibly integrated into any existing HAR model, enhancing recognition performance while maintaining inference efficiency. To address issues such as information redundancy in retrieval results and rigid fusion strategies, we propose an uncertainty-adaptive fusion unit within MoRA. This unit leverages previous physical knowledge from IMU signals to dynamically adjust the fusion strategy between original outputs and retrieved information, enabling more robust recognition. Extensive experiments on ten real-world datasets demonstrate that MoRA significantly improves the performance of existing IMU-based HAR models, consistently delivering stable and effective gains. The source code of MoRA is available at: https://github.com/liavonpenn/mora.

Abstract

Explore similar work

Modular Retrieval-Augmented Generalization for Human Action Recognition

VSMP-IMU: Video-Grounded Semantic Motion Programs for Sensor-Aware Synthetic IMU Generation

Rotation-Invariant Multi-IMU Activity Recognition under Independent Per-Location Orientation Shifts