EventHOI: Event-Driven Resource-Efficient Hand-Object Interaction Logging on Smartglasses
Organizations: University of Michigan · Samsung Research America
Abstract
Hand-object interactions (HOIs) can be captured by smartglasses to learn how people manipulate objects in daily life. Automatic logging of these interactions from an egocentric perspective provides time-stamped evidence of daily actions for applications like memory assistance. However, continuous RGB capture and wireless transmission exceed the energy and bandwidth budgets of smartglasses. We present EventHOI, an event-driven sensing and compression system that couples selective RGB capture with HOI-aware encoding to enable egocentric logging over Bluetooth Low Energy (BLE) while preserving downstream HOI logging performance. Our key insight is to use low-power event sensing both to determine when RGB capture is needed and to identify where visual detail must be preserved. A lightweight IMU-based egomotion cancellation pipeline suppresses egomotion-induced events so that residual HOI motion can guide RGB triggering and spatial compression. We build a smartglasses prototype and conduct an end-to-end evaluation of EventHOI on a 50-hour egocentric benchmark with 97 subjects and a 20-hour real-world dataset we collect from 10 participants, together covering a broad range of daily-life scenarios and environments. Compared to RGB-only baselines, EventHOI reduces always-on backend power consumption by 5.8-10.4x and uplink bandwidth by 2.9x, with minimal HOI recognition accuracy loss. The resulting RGB uplink of less than 130 kbps fits within commodity BLE bandwidth budgets, making stable BLE video streaming feasible for egocentric HOI logging. Demo video and project website: https://eventhoi.github.io
Figures & tables
| Workload | Power (W) | Battery Life (hrs) | Use Cases and Examples | |
|---|---|---|---|---|
| Radio | Sensor | |||
| Off | Off | 0.1 | 7.5 | No video capability, no stream, on-device only |
| BLE | Off | 0.15 ( ) | 5.0 ( ) | Low-resource modality stream (image, audio, etc.) |
| Off | RGB | 0.7 ( ) | 1.1 ( ) | No real-time video stream |
| WiFi | RGB | 1.1 ( ) | 0.7 ( ) | Video stream (Ray-Ban Meta ( Meta, 2025 ) , Project Aria ( Engel and others, 2023 ) ) |
| Baselines | EventHOI Ablations | |||||
|---|---|---|---|---|---|---|
| Uniform | YOLO ( Khanam and Hussain, 2024 ) | Hands23 ( Cheng et al., 2023 ) | Vanilla EVS | Detector only | EventHOI | |
| Reduce RGB frames? | No | Yes ∗ | No | No | Yes | Yes |
| Can compress frames? | No | No | Yes | No | No | Yes |
| On-device feasible? | Yes | Yes | No | Yes | Yes | Yes |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Location | Leisure | Working | Eating | Cooking | Housework | Recreation | Total |
|---|---|---|---|---|---|---|---|
| Home1 | 2.5 (P1, P2, P6) | 2.6 (P1, P2) | 0.8 (P1, P2, P6) | 1.6 (P1, P6) | 1.6 (P1, P2, P6) | 2.2 (P1, P2, P6) | 11.3 |
| Home2 | – | 3.4 (P4) | 0.3 (P4) | 0.8 (P4) | – | – | 4.5 |
| Workspace1 | 0.3 (P5) | 0.5 (P3) | 0.5 (P1) | – | – | – | 1.3 |
| Workspace2 | 0.3 (P9) | 0.7 (P3, P10) | – | – | – | 0.8 (P8, P9) | 1.7 |
| Office | 0.4 (P7) | 0.7 (P10) | – | – | – | – | 1.1 |
| Total | 3.5 | 7.8 | 1.6 | 2.4 | 1.6 | 2.9 | 19.8 |
Wearable Sensing
Wearable sensing uses body-worn devices to measure motion, physiology, and interaction for health, activity, and assistive applications.
See all 142 papers