One Residual with Three Reuses: A Wristband Front End for Gesture Sensing

Mouhssine Rifaki

cs.LGAug 17, 2026

One Residual with Three Reuses: A Wristband Front End for Gesture Sensing

Authors: Mouhssine Rifaki

Organizations: Dept. of Electrical Engineering Stanford University Stanford, CA, USA

Abstract

Continuous wrist-worn hand sensing for gesture interfaces and motor symptom monitoring needs an always-on front end that fits inside a coin-cell power budget while pairing a micro-electro-mechanical-systems (MEMS) inertial measurement unit (IMU) with a 60 GHz frequency-modulated continuous-wave (FMCW) radar to stay robust under occlusion and on-body drift. We present a design study of such a wristband front end in which classifier wake-up gating, mmWave versus IMU routing, and innovation-based EKF measurement reweighting share a single on-chip residual generator. The shared generator occupies 14.4 KB of program memory and 278 B of state and runs at 110K multiply-accumulates (MACs) per frame on an Ambiq Apollo4 Blue Plus class edge microcontroller unit (MCU). Across four public sensor data corpora (IPN Hand, SHREC 2021, MiliPoint 60 GHz FMCW radar, EAT-Radar) the front end reaches detection probability $P_D = 0.72/0.80$ at a 1% false-alarm rate, sustains a 47% classifier invocation energy reduction at 90% gesture detection recall, and lowers pose tracking root-mean-square error by $4.6\times$ under measurement bias drift relative to an adaptive Kalman with $R$ -inflation baseline. Measured silicon power and on-body capture are deferred to follow-on hardware; the contribution here is a design study.

Figures & tables

Panel of Figure 1 — Fig. 1 : Signal chain for the proposed wristband architecture. The conditioned IMU and mmWave streams feed into a single recurrent forecaster located on the MCU. The forecaster produces an estimate of the next conditioned frame vector $\hat{y}_{t}$ . Its one-step prediction error $\xi_{t}$ is used to gate the heavy classifier, to rank the available modalities through the router, and to re-estimate the EKF measurement noise covariance.

Fig. 2 : Wake-up trace of IPN Hand. The average

\xi_{t}

peak levels reach

6.2

at the beginning of each gesture versus a background level of

4.3

, with rapid changes between gestures. Therefore, the

90

% recall operating point is set based on the

\tau

value representing the

82

nd percentile.

Fig. 3 : Tracking bias drift at

b_{s}=5

cm. The EKF measurement noise re-estimation block reduces tracking RMS by

4.6\times

when compared to the adaptive Kalman noise re-estimation baseline of [ 7 ] and by

2\times

when compared to the hybrid IMM-style filter of [ 6 ] .

Front end	MACs/fr.	Memory	W	R	D
This work	$110$ K	$14.4$ KB	$\bullet$	$\bullet$	$\bullet$
Scherer [ 12 ]	$\sim$ 500 K	$\sim$ 92 KB	$\bullet$	–	–
Zhang [ 2 ]	$\sim$ 5 M	n/a	$\bullet$	–	–
Safa [ 13 ]	spiking	n/a	$\bullet$	–	–

TABLE I : Comparison of wearable front ends utilizing mmWave and IMU. W : wake-up, R : routing, D : drift tracking. Baseline systems have measured silicon for wake-up alone; this paper provides a design study combining all three features with a single residual.

Wearable Sensing

Wearable sensing uses body-worn devices to measure motion, physiology, and interaction for health, activity, and assistive applications.

See all 129 papers

Efficient Sensor Fusion for Gesture Recognition on Resource-Constrained Devices
May 13, 2026Pietro Bartoli, Christian Veronesi, Tommaso Bondini +2Human Activity Recognition Tiny ML
Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB) Politecnico di Milano, Milan, Italy
A Comparative Study of EMG- and IMU-based Gesture Recognition at the Wrist and Forearm
Dec 8, 2025Soroush Baghernezhad, Elaheh Mohammadreza, Vinicius Prado da Fonseca +2Human Activity Recognition Wearable Sensing
Department of Computer Science, Memorial University of Newfoundland, St. John’s, NL A1B 3X5, Canada · Department of Mechanical and Mechatronics Engineering, Memorial University of Newfoundland, St. John’s, NL A1B 3X5, Canada
Joint Angle Estimation with Customized Wristband Based on Online Incremental Learning
May 28, 2026Shuo Wang, Xiaobin Chen, Xiaoming TaoWearable Sensing
Research Institute for Intelligent Wearable Systems, The Hong Kong Polytechnic University, Kowloon, Hong Kong.

May 13, 2026J/K move · Enter open · S save

Efficient Sensor Fusion for Gesture Recognition on Resource-Constrained Devices

Pietro Bartoli, Christian Veronesi, Tommaso Bondini, Andrea Giudici+1

Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB) Politecnico di Milano, Milan, Italy

Human Activity Recognition Tiny ML Wearable Sensing Gesture Recognition Edge ML Multimodal Sensor Fusion

Open paper AskPDF arXiv

Gesture recognition is a cornerstone of Human-Computer Interaction (HCI) for smart eyewear, enabling natural and device-free control in augmented reality environments. Traditional vision-based approaches face significant challenges regarding power consumption, computational latency, and user privacy. This paper proposes a lightweight, privacy-preserving gesture recognition system based on the fusion of low-resolution Time-of-Flight (ToF) and Infrared (IR) thermal sensors. We used an 8 times 8 multizone ToF sensor (VL53L8CH) and an 8 times 8 IR array (AMG8833) to capture complementary depth and thermal cues. A compact Convolutional Neural Network (CNN) with a specialized grouped-convolution architecture is designed to fuse these modalities efficiently on a microcontroller (MCU). Experimental results on a custom dataset of 7 static gestures, validated via k-fold cross-validation, demonstrate that the proposed fusion strategy significantly outperforms single-sensor baselines with an accuracy of 92.3% and a macro F1-score of 0.93. Finally, on-device benchmarks on STM32F4 and STM32H7 MCUs confirm the system's suitability for resource-constrained wearables, requiring only 6,343 parameters and achieving millisecond-level inference latency with a total system power of 50 mW.

Abstract

Wearable Sensing

Explore similar work

Efficient Sensor Fusion for Gesture Recognition on Resource-Constrained Devices

A Comparative Study of EMG- and IMU-based Gesture Recognition at the Wrist and Forearm

Joint Angle Estimation with Customized Wristband Based on Online Incremental Learning