Rotation-Invariant Multi-IMU Activity Recognition under Independent Per-Location Orientation Shifts

Seungyeol Baek; Yoonbyung Chai; Yonghyeon Lee; Sungjoon Choi; Sungho Suh

cs.AIAug 16, 2026

Rotation-Invariant Multi-IMU Activity Recognition under Independent Per-Location Orientation Shifts

Authors: Seungyeol Baek, Yoonbyung Chai, Yonghyeon Lee, Sungjoon Choi, Sungho Suh

Organizations: Korea University Seoul, Republic of Korea · Massachusetts Institute of Technology Cambridge, Massachusetts, USA · Dept. of Artificial Intelligence Korea University Seoul, Republic of Korea

Abstract

Human Activity Recognition (HAR) with self-administered wearables, such as at-home rehabilitation and exercise monitoring, often requires reattaching inertial measurement units (IMUs) across sessions. In multi-IMU settings, this can induce independent orientation offsets across body locations, a deployment shift that conventional scalar HAR models do not structurally handle. Existing remedies rely on rotation augmentation, whose robustness depends on sampled transformations, or calibration and orientationnormalization pipelines requiring additional reference-frame assumptions or explicit procedures. We present Truly Rotation-Invariant HAR (TRI-HAR), a rotation-invariant framework that makes robustness to independent per-location IMU orientation offsets a structural model property. TRI-HAR reshapes accelerometer and gyroscope streams into triaxial vectors, applies a shared SO(3)-equivariant backbone and invariant projection to each IMU location, and fuses the resulting invariant features for activity classification. Across four multi-IMU benchmarks, TRI-HAR preserves macro-F1 under fixed independent per-location SO(3) rotations and outperforms rotation-augmented baselines under this target shift without requiring rotational augmentation.

Figures & tables

Figure 1. Body-worn IMUs may be reattached with independent orientation offsets across uses. TRI-HAR performs per-location equivariant encoding and invariant projection before fusion for stable activity predictions. Teaser diagram for TRI-HAR. The left side shows a person wearing IMUs at the chest, wrist, thigh, and ankle during an initial placement. The middle shows the same activity during inference after sensors have been reattached, with each IMU coordinate frame rotated differently and labeled with independent rotations R1 through R4. The right side shows a simplified TRI-HAR block with equivariant encoding, invariant projection, and location-wise fusion, producing the same activity label before and after reattachment.

Figure 2. Architecture of TRI-HAR. Windowed input signals are vectorized into triaxial streams and grouped by physical IMU location. A shared location encoder

E_{\theta}

is applied to each group

X^{(l)}

to produce an invariant feature

h_{l}

; the features

h_{1},\ldots,h_{L}

are then concatenated in a fixed location order and passed to an MLP classifier. Dimensions inside

E_{\theta}

are shown for one location group, and the encoder weights are shared across locations. Diagram of the TRI-HAR architecture. A sensor input window is first vectorized from flattened accelerometer and gyroscope channels into triaxial streams, then grouped by IMU location. Each location group is passed through the same shared encoder, labeled

E_\theta

, which contains VN-EdgeConv layers, a VN invariant layer, and pooling, producing one invariant feature

h_l

per location. The invariant features from all locations are concatenated and passed through an MLP to produce activity-class predictions.

Dataset	Freq.	WL	#Subj.	#Cls.	#Locs.
PAMAP2 ( Reiss and Stricker, 2012 )	33	5.12	8	12	3
DSADS ( Altun et al., 2010 )	25	5.00	8	19	5
Opportunity ( Chavarriaga et al., 2013 )	30	1.00	4	17	7
RealDISP ( Baños et al., 2012 )	50	2.40	17	33	9

Table 1. Characteristics of the datasets used in the evaluation

Model	Train Aug.	PAMAP2 ( Reiss and Stricker, 2012 )		DSADS ( Altun et al., 2010 )		Opportunity ( Chavarriaga et al., 2013 )		RealDISP ( Baños et al., 2012 )
Model	Train Aug.	$\mathbf{I}$	$\mathbf{\mathrm{SO}(3)_{\text{loc-fix}}}$	$\mathbf{I}$	$\mathbf{\mathrm{SO}(3)_{\text{loc-fix}}}$	$\mathbf{I}$	$\mathbf{\mathrm{SO}(3)_{\text{loc-fix}}}$	$\mathbf{I}$	$\mathbf{\mathrm{SO}(3)_{\text{loc-fix}}}$
MC-CNN ( Yang et al., 2015 )	none	80.45 $\pm$ 8.18	31.52 $\pm$ 6.21	82.80 $\pm$ 8.09	30.63 $\pm$ 5.80	41.18 $\pm$ 6.62	11.82 $\pm$ 3.90	87.48 $\pm$ 10.39	48.95 $\pm$ 9.81
MC-CNN ( Yang et al., 2015 )	loc-sample	—	79.14 $\pm$ 6.75	—	81.75 $\pm$ 6.43	—	33.56 $\pm$ 8.10	—	87.19 $\pm$ 8.35
DeepConvLSTM ( Ordóñez and Roggen, 2016 )	none	71.13 $\pm$ 10.39	21.84 $\pm$ 10.05	76.05 $\pm$ 11.27	25.85 $\pm$ 6.74	39.44 $\pm$ 7.99	6.16 $\pm$ 1.85	86.99 $\pm$ 10.21	53.37 $\pm$ 16.25
DeepConvLSTM ( Ordóñez and Roggen, 2016 )	loc-sample	—	76.65 $\pm$ 8.80	—	77.97 $\pm$ 7.94	—	30.01 $\pm$ 9.06	—	87.65 $\pm$ 7.65
MLP-HAR ( Zhou et al., 2024 )	none	80.08 $\pm$ 6.73	28.52 $\pm$ 9.27	83.23 $\pm$ 7.80	22.55 $\pm$ 6.46	42.01 $\pm$ 3.93	5.45 $\pm$ 1.32	88.49 $\pm$ 6.82	39.70 $\pm$ 4.80

Table 2. Benchmark comparison under fixed independent per-location test rotations. Values are macro-F1 (%, mean

\pm

standard deviation) over cross-subject folds. Columns report performance on the original test data

I

and under

\mathrm{SO}(3)_{\text{loc-fix}}

; entries under

I

are omitted for loc-sample rows for compactness. Bold denotes the best value within each dataset/test column.

Model	none	loc-sample	$\boldsymbol{\Delta_{\mathrm{aug}}}$
MC-CNN	67.95	74.84	$+6.89$
DeepConvLSTM	67.68	72.90	$+5.22$
MLP-HAR	77.55	74.67	$-2.88$
TinyHAR	72.55	78.49	$+5.94$
SA-HAR	49.89	51.28	$+1.39$
TRI-HAR	85.29		—

Table 3. Macro-F1 (%) on RealDISP Mutual-displacement fold.

\Delta_{\mathrm{aug}}

is augmentation-induced change in baseline score. Bold marks the highest score.

Setting	Model	FP32 (MiB)	CPU (ms)	GPU (ms)
3-IMU	MLP-HAR	0.67	0.86/1.08	0.69/1.09
	MC-CNN	23.14	1.79/2.02	0.53/0.94
	TRI-HAR	13.40	67.38/67.61	2.58/2.97
5-IMU	MLP-HAR	1.84	0.97/1.21	1.14/1.48
	MC-CNN	28.46	2.61/2.84	0.88/1.25
	TRI-HAR	21.41	85.56/85.78	3.01/3.36

Table 4. Host-side live p99 latency for the 3-IMU and 5-IMU settings. CPU and GPU entries report model/window-to-label in ms. FP32 denotes parameter memory.

Explore similar work

Modular Retrieval-Augmented Generalization for Human Action Recognition
Apr 28, 2026Peng Liao, Shangsong Liang, Lin Chen +1Human Activity Recognition
School of Computer Science and Engineering, Sun Yat-sen University · Engineering Research Centre of Applied Technology on Machine Translation and Artificial Intelligence, Macao Polytechnic University
HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition
Aug 27, 2026Zihan Ding, Liyu Zhang, Xiaomin OuyangHuman Activity Recognition Multimodal Foundation Models
Hong Kong University of Science and Technology Hong Kong, China
Beyond Motion Primitives: Behavioral Activity Recognition from Head-Mounted IMU
May 26, 2026Chung-Ta Huang, Leopold Das, Jeffrey Zhou +10Human Activity Recognition Wearable Sensing
Harvard AI and Robotics Lab, Harvard University

Apr 28, 2026J/K move · Enter open · S save

Modular Retrieval-Augmented Generalization for Human Action Recognition

Peng Liao, Shangsong Liang, Lin Chen, Peijia Zheng

School of Computer Science and Engineering, Sun Yat-sen University · Engineering Research Centre of Applied Technology on Machine Translation and Artificial Intelligence, Macao Polytechnic University

Human Activity Recognition

Open paper AskPDF arXiv

Inertial Measurement Unit (IMU)-based Human Activity Recognition (HAR) aims to interpret and classify user behaviors from temporal motion signals. Recently, deep learning frameworks have advanced this task by learning and extracting discriminative spatiotemporal representations, significantly improving recognition performance. However, IMU-based HAR still faces several critical challenges, particularly limited training samples and static knowledge utilization, both of which severely hinder its large-scale deployment. In this paper, we introduce MoRA, the first Retrieval-Augmented Module specifically designed for motion series. It can be flexibly integrated into any existing HAR model, enhancing recognition performance while maintaining inference efficiency. To address issues such as information redundancy in retrieval results and rigid fusion strategies, we propose an uncertainty-adaptive fusion unit within MoRA. This unit leverages previous physical knowledge from IMU signals to dynamically adjust the fusion strategy between original outputs and retrieved information, enabling more robust recognition. Extensive experiments on ten real-world datasets demonstrate that MoRA significantly improves the performance of existing IMU-based HAR models, consistently delivering stable and effective gains. The source code of MoRA is available at: https://github.com/liavonpenn/mora.

Abstract

Explore similar work

Modular Retrieval-Augmented Generalization for Human Action Recognition

HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition

Beyond Motion Primitives: Behavioral Activity Recognition from Head-Mounted IMU