HexVIO: Towards All-Day Stereo-Inertial Tracking Through Commodity DSPs

Patrick Wolf; Mateo de Mayo; Daniel Cremers

cs.ROOct 2, 2026

HexVIO: Towards All-Day Stereo-Inertial Tracking Through Commodity DSPs

Authors: Patrick Wolf, Mateo de Mayo, Daniel Cremers

Organizations: Technical University of Munich, Munich, Germany · Munich Center for Machine Learning, Munich, Germany

Abstract

The ability of a device to localize itself within its surroundings is a fundamental prerequisite for spatial computing. Visual-inertial odometry (VIO) has proven to be a cost-effective and accurate solution for this task. Robots, wearables, XR devices, and drones can benefit significantly from efficient implementations of VIO since they allow for cooler, lighter, and cheaper devices with longer battery life and a better user experience. In this work, we propose to enhance the efficiency of a VIO system by leveraging the Hexagon DSP, a commodity co-processor present in many modern smartphones and XR devices. Our approach offloads the visual frontend of a stereo-inertial odometry system to the DSP while keeping the backend on the main CPU. By optimizing the implementation for the DSP architecture, we achieve significant reductions in power consumption and latency compared to CPU-only execution. Our system, HexVIO, demonstrates a 67% reduction in power consumption or an 86% increase in throughput on a commodity smartphone, with the ability to sustain long-term real-time 30 fps tracking for 0.83 W, corresponding to ~18 hours of tracking on the testing device. These results highlight the potential of commodity DSPs for enabling all-day visual-inertial tracking in robotics and mobile devices.

Figures & tables

Fig. 1: Top left: efficiency of HexVIO (DSP) against the CPU baseline on a Samsung Galaxy S23, MOO05 sequence. Each curve spans four operating points of one implementation: the paced 30 fps configuration (CPU B , DSP B ), the frontend on two and on four mid-cluster threads, and a fourth point that is unpinned for the CPU and the maximum-performance RPC policy of Table I for the DSP. At the baseline’s maximum throughput (98 fps), HexVIO draws 67 % less power; at HexVIO’s maximum-performance point (4.5 W), it delivers 86 % higher throughput than the CPU at the same power. Filled markers are measured operating points (mean of two runs); curves are shape-preserving cubic interpolants (PCHIP), and open markers are matched points read from the interpolants. Top right: top-down trajectories of both implementations on the same sequence against ground truth, frozen when the unpaced DSP run finishes; the unpaced CPU run has processed 54 % of the sequence at the same wall-clock time, with the same ATE (1.7 vs 1.8 cm). Bottom: one frame from each of the eight evaluation sequences, labeled with camera count, resolution and frame rate.

Panel of Figure 2 — Fig. 2: The Hexagon V73 compute DSP. The scalar core, HVX vector unit and HMX matrix unit share one memory hierarchy. Each hardware thread has its own scalar registers (GRF, predicates P0–P3); each vector context has its own vector registers (VRF, predicates Q0–Q3) and the six functional units that operate on them. The gather instruction, the basis of sparse feature sampling, addresses VTCM only.

Panel of Figure 3 — Fig. 3: Two vectorization axes. Image stages use one pixel per lane: adjacent pixels map to adjacent lanes, and for FAST, ring point $k$ for all 128 candidates is row $y{+}dy_{k}$ shifted by $dx_{k}$ lanes. Tracking uses one feature per lane: at each offset of the shared 52-point pattern, every lane gathers the word holding its own feature’s pixel pair, so 32 features advance in lockstep.

Panel of Figure 4 — Fig. 4: System partition. The frontend runs on the cDSP in one fused FastRPC call per stereo frame; inertial integration and the estimator stay on the CPU. The call fetches the full-resolution stereo pair from a shared DDR buffer into VTCM by DMA; every other argument is sparse feature data, passed by shared-memory mapping rather than copied. The current and previous stereo pyramids remain resident in VTCM.

ATE [cm]

RTE [cm]

FPS

Frontend [ms]

Backend [ms]

End-to-end [ms]

p99 [ms]

Sequence

CPU

DSP

CPU

DSP

CPU

DSP

CPU

DSP

CPU

DSP

CPU

DSP

CPU

DSP

EV102

5.40

5.41

1.12

1.12

70.1

\mathbf{120.0}

7.75\pm 2.25

\mathbf{2.32\pm 0.29}

7.15\pm 2.87

\mathbf{5.98\pm 2.30}

15.00\pm 3.12

\mathbf{8.38\pm 2.48}

23.5

\mathbf{14.7}

EMH03

6.05

6.43

1.18

1.18

56.2

\mathbf{94.9}

9.05\pm 1.81

\mathbf{2.47\pm 0.32}

9.32\pm 4.32

\mathbf{7.97\pm 3.42}

18.48\pm 4.57

\mathbf{10.52\pm 3.63}

31.3

\mathbf{19.2}

TR2

7.30

7.33

0.70

0.70

105.6

\mathbf{180.1}

5.79\pm 1.20

\mathbf{2.00\pm 0.22}

3.88\pm 1.56

\mathbf{3.55\pm 1.48}

9.59\pm 1.90

\mathbf{5.62\pm 1.64}

15.6

\mathbf{10.3}

TR6

2.06

2.00

0.38

0.38

93.7

\mathbf{155.0}

6.54\pm 1.61

\mathbf{2.18\pm 0.24}

4.65\pm 1.71

\mathbf{4.30\pm 1.50}

10.87\pm 2.19

\mathbf{6.55\pm 1.64}

17.4

\mathbf{11.4}

MOO02

23.48

23.43

1.54

1.54

103.2

\mathbf{206.3}

6.68\pm 0.73

\mathbf{2.10\pm 0.13}

2.80\pm 1.81

\mathbf{2.51\pm 1.54}

9.68\pm 1.98

\mathbf{4.78\pm 1.62}

16.0

\mathbf{10.4}

TABLE I: Maximum throughput runs: accuracy and times of CPU baseline vs. HexVIO (DSP)

Fig. 5: Per-stage frontend timing on MOO05, unpaced, both arms pinned to the mid cluster with four threads. Left: per-frame stage times of the CPU baseline and of HexVIO on the DSP at its maximum-performance operating point (TURBO, polling RPC), one fused dispatch per frame. Right: mean and 99th percentile per frame in ms and share of the summed stage means. RPC overhead, the round trip of the fused dispatch, exists only on the DSP side and counts in the DSP total but not in the shares.

Fig. 6: Effect of fusing the DSP calls on MOO05 (unpaced, TURBO, host on the mid cluster). Unfused issues six calls per frame (one stereo-pyramid build, two temporal tracks, two detections, one stereo track), fused one, under the adaptive QoS of DSP B /DSP L and the polling QoS of DSP. Bars are DSP stage time plus the RPC round trip, as in Fig. 5 . Fusion saves 2.3 ms per frame under adaptive QoS and 0.7 ms under polling.

	Power [W]				E2E [ms]				ATE [cm]				RTE [cm]
Sequence	CPU	DSP	DSP B	DSP L	CPU	DSP	DSP B	DSP L	CPU	DSP	DSP B	DSP L	CPU	DSP	DSP B	DSP L
EV102	1.42	0.93	0.88	0.86	25.4	21.8	23.8	18.6	5.40	5.41	5.41	4.41	1.12	1.12	1.12	1.12
EMH03	1.63	1.12	1.02	0.92	28.1	24.4	27.8	25.3	6.05	6.43	6.43	6.01	1.18	1.18	1.18	1.17
TR2	1.00	0.82	0.72	0.66	23.5	15.1	16.7	13.5	7.30	7.33	7.33	9.54	0.70	0.70	0.70	0.80
TR6	1.07	0.86	0.77	0.66	23.6	17.8	18.9	14.2	2.06	2.00	2.00	3.30	0.38	0.38	0.38	0.44
MOO02	1.23	0.84	0.77	0.72	18.0	13.0	15.0	13.1	23.48	23.43	23.43	27.48	1.54	1.54	1.54	1.59

TABLE II: Power consumption on runs paced at sensor rate

Fig. 7: 40-minute MOO12 run under CPU, DSP, DSP B and DSP L : (a) pinned mid-cluster clock, (b) whole-device power from the sampled current and voltage (right-edge labels: run means; the charge counter quoted in the text reads 8–12 % higher), (c) SoC die temperature, the hottest on-die sensor, and (d) E2E p99 latency against the 33.3 ms frame budget.

Visual-Inertial Odometry

Visual-inertial odometry estimates camera motion by combining image geometry with inertial measurements for metric navigation.

See all 57 papers

Sep 24, 2026

VkVIO: Cross-platform GPU Acceleration for Visual-Inertial Odometry with Vulkan

Perception in robotics and XR fundamentally relies on good state estimation. Visual-inertial odometry (VIO) and Simultaneous Localization and Mapping (VI-SLAM) are proven ways of achieving this goal in a cost-effective and accurate manner. Efficiency in these systems allows for smaller, cooler, and lighter devices. GPU acceleration is a natural approach for reducing latency, thanks to their wide availability in platforms like embedded computers, mobile phones, and XR headsets. However, previous works in the literature have limited themselves to the use of CUDA for this task, significantly reducing deployment options to a single vendor. We instead leverage the vendor-agnostic Vulkan API, originally designed for the strict performance requirements of 3D graphics applications. In this work, we present VkVIO, the first, to the best of our knowledge, cross-platform GPU-accelerated VIO method. We provide state-of-the-art accuracy with causal estimates required for real-time operation. We deploy VkVIO on a diverse range of devices spanning a workstation, a laptop, and an extremely inexpensive single-board computer, while outperforming CUDA-based systems on the same hardware. VkVIO enables possibilities for low-latency, low-power, and low-cost VIO in robotics and XR.

Ole Hoffmann, Mateo de Mayo, Daniel Cremers

Technical University of Munich, Munich, Germany · Munich Center for Machine Learning, Munich, Germany

Visual-Inertial Odometry GPU Acceleration

Ask PDF

Nov 12, 2025

SMF-VO: Direct Ego-Motion Estimation via Sparse Motion Fields

Traditional Visual Odometry (VO) and Visual Inertial Odometry (VIO) methods rely on a 'pose-centric' paradigm, which computes absolute camera poses from the local map thus requires large-scale landmark maintenance and continuous map optimization. This approach is computationally expensive, limiting their real-time performance on resource-constrained devices. To overcome these limitations, we introduce Sparse Motion Field Visual Odometry (SMF-VO), a lightweight, 'motion-centric' framework. Our approach directly estimates instantaneous linear and angular velocity from sparse optical flow, bypassing the need for explicit pose estimation or expensive landmark tracking. We also employed a generalized 3D ray-based motion field formulation that works accurately with various camera models, including wide-field-of-view lenses. SMF-VO demonstrates superior efficiency and competitive accuracy on benchmark datasets, achieving over 100 FPS on a Raspberry Pi 5 using only a CPU. Our work establishes a scalable and efficient alternative to conventional methods, making it highly suitable for mobile robotics and wearable devices.

Sangheon Yang, Yeongin Yoon, Hong Mo Jung +1

Department of Artificial Intelligence, Hanyang University, Seoul, Korea · Department of Mechanical Engineering, Seoul National University, Seoul, Korea · All authors are with the Robot Vision Lab, Seoul National University.

Visual Odometry

Ask PDF

May 30, 2026

BEVIO: Efficient Bird's-Eye-View based Sparse-Update Visual-Inertial Odometry for Lunar Day-Night Navigation

Visual-Inertial Odometry (VIO) provides smooth, high-rate state estimates and has been widely used for robotic navigation in both terrestrial and planetary applications. However, its performance is typically dependent on the frequency of visual updates, which is a challenge for planetary rovers operating under extreme resource constraints and low frame rates. This work investigates enabling reliable VIO with very sparse visual updates for lunar rover applications, addressing both day and night-time operations where feature associations become especially difficult under self-illumination conditions. We propose a Bird's Eye View (BEV)-based image matching scheme that remains robust to larger inter-frame motions and more reliable feature matching despite significant visual appearance changes. We extensively evaluate our proposed approach, BEVIO, through high-fidelity photorealistic lunar and real-time robotic experiments conducted using a half-scale lunar rover, in a long-term day-night deployment at Plaster City, CA, USA. The results demonstrate that our method enables reliable day and nighttime self-illuminated traverses at visual update rates as low as 0.25 Hz, underscoring its suitability for navigation on power- and compute-limited lunar rovers.

Mohit Singh, Shehryar Khattak, Ashish Goel +3

Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA. · Autonomous Robots Lab at the Norwegian University of Science and Technology, Trondheim, Norway

Robot Navigation Visual Odometry Bird's-Eye-View Representation Visual-Inertial Odometry

Ask PDF

Abstract

Visual-Inertial Odometry

Explore similar work

VkVIO: Cross-platform GPU Acceleration for Visual-Inertial Odometry with Vulkan

SMF-VO: Direct Ego-Motion Estimation via Sparse Motion Fields

BEVIO: Efficient Bird's-Eye-View based Sparse-Update Visual-Inertial Odometry for Lunar Day-Night Navigation