DAPEVO: Deep Adaptive Patch Frame-Event Visual Odometry

Luca Gandolfi; Simone Nascivera; Roberto Pellerito; Rong Zou; Chiara Plizzari; Davide Scaramuzza

cs.CVSep 25, 2026

DAPEVO: Deep Adaptive Patch Frame-Event Visual Odometry

Authors: Luca Gandolfi, Simone Nascivera, Roberto Pellerito, Rong Zou, Chiara Plizzari, Davide Scaramuzza

Organizations: Robotics and Perception Group, University of Zurich, Switzerland · Bocconi University, Italy

Abstract

Visual odometry is essential for autonomous navigation in GPS-denied environments, yet RGB-based methods remain vulnerable to motion blur, challenging illumination, and dropped frames. Event cameras complement conventional cameras with high temporal resolution and dynamic range, but their asynchronous measurements complicate reliable correspondence estimation. We present DAPEVO, a learned visual odometry system that estimates image and event correspondences independently at shared patch locations and fuses their correlation evidence before motion refinement. Each tracked patch maintains image and event descriptors, and a learned scalar gate combines modality-specific correlation embeddings for each patch--frame edge before a shared recurrent refinement and bundle-adjustment update. DAPEVO also supports event-only observations, enabling continued tracking when RGB frames are sparse or unavailable, while modality-aware keyframe culling preserves scarce frame constraints. On UZH-FPV, when retaining only one in six RGB frames, DAPEVO's mean absolute trajectory error (ATE) increases by only 36%, from 1.00 to 1.36m, whereas the ATE of DPVO and RAMP-VO rises by factors of $3.7\times$ and $3.1\times$ , respectively. On TartanEvent, DAPEVO similarly remains below 1m ATE at 3Hz RGB input, while DPVO and RAMP-VO exceed 9m. Under degraded RGB input on TartanEvent, DAPEVO achieves an ATE of 0.60m, compared with more than 4m for both DPVO and RAMP-VO, while also outperforming event-only DEVO at 0.87m.

Figures & tables

Fig. 1: Overview of DAPEVO. Joint RGB–event and event-only observations are processed by separate image and event encoders. Patches with RGB and event descriptors are correlated independently, then adaptively fused by a learned gate before recurrent refinement and bundle adjustment. Patch colors and track lines indicate the event contribution $\alpha$ , from green ( $\alpha=0$ ) to red ( $\alpha=1$ ). A low-light augmentation is applied to the central RGB observation to illustrate reduced reliance on degraded frame information. When RGB is unavailable at either correspondence endpoint, $\alpha=1$ , enabling event-only tracking.

Fig. 2: DAPEVO architecture. Images and event voxel grids are processed by separate matching encoders. Each patch $k$ is associated with image and event matching features $\mathbf{g}_{k}^{I}$ and $\mathbf{g}_{k}^{E}$ . For each patch–frame edge $(k,j)$ , image and event correlations are computed independently and mapped to embeddings $\mathbf{e}^{I}_{kj}$ and $\mathbf{e}^{E}_{kj}$ . A learned gate predicts an edge-specific weight $\alpha_{kj}$ to fuse the two embeddings. When RGB images are unavailable, only the event embedding is used. The resulting embedding $\mathbf{e}_{kj}$ is processed by the recurrent update operator to predict correspondence corrections and confidence weights, which are then used by differentiable bundle adjustment (DBA) to jointly refine camera poses and patch depths.

		Nominal RGB		Degraded RGB
Method	Input	AUC $\uparrow$	ATE [m] $\downarrow$	AUC $\uparrow$	ATE [m] $\downarrow$
DPVO [ 6 ]	I	0.83	0.17	0.14	4.96
DEVO [ 7 ]	E	0.56	0.87	0.56	0.87
RAMP-VO [ 9 ]	I+E	0.85	0.15	0.14	7.01
Feature fusion	I+E	0.78	0.22	0.50	1.68
DAPEVO	I+E	0.85	0.15	0.63	0.60

TABLE I: TartanEvent validation results under nominal and degraded RGB input.

	VINS- Mono [ 33 ]	Ultimate SLAM [ 27 ]	PL-EVIO [ 28 ]	DEVO [ 7 ]	DPVO [ 6 ]	DAPEVO
Loop closure	✓	✓	✓	$\times$	$\times$	$\times$
fwd_3	0.65	fail	0.38	0.37	0.32	0.38
fwd_5	1.07	fail	0.90	0.40	0.24	0.25
fwd_6	0.25	fail	0.30	0.31	0.42	0.39
fwd_7	0.37	fail	0.55	0.50	0.23	0.19
fwd_9	0.51	fail	0.44	0.61	0.57	0.56

TABLE II: MPE [%] on indoor UZH-FPV. All results are from [ 28 , 7 ] , except for DPVO and DAPEVO. Best results are bold.

	Ultimate SLAM [ 27 ]	PL-EIO [ 28 ]	PL-EVIO [ 28 ]	DPVO [ 6 ]	DAPEVO
				Mean / Median
Input	I+E	E	I+E	I	I+E
Loop clos.	✓	✓	✓	$\times$	$\times$
hdr1	2.44	0.67	0.17	0.11 / 0.12	0.12 / 0.12
hdr2	1.11	0.45	0.12	0.17 / 0.16	0.15 / 0.15
hdr3	0.83	0.74	0.19	0.27 / 0.27	0.26 / 0.25

TABLE III: HKU monocular MPE [%]. DPVO/DAPEVO report mean/median over five trials; other results are from [ 28 ] and use an IMU. I: images; E: events.

	ORB- SLAM3 [ 11 ]	VINS- Fusion [ 34 ]	ESVIO [ 35 ]	Ultimate SLAM [ 27 ]	PL- EVIO [ 28 ]	ESVO [ 20 ]	EVO [ 19 ]	DPVO [ 6 ]	DEVO [ 7 ]	DAPEVO
IF1	5.31	1.50	0.94	fail	1.35	4.00	5.09	0.16	0.26	0.16
IF2	5.65	6.98	1.00	fail	1.00	3.66	fail	0.15	0.32	0.16
IF3	2.90	0.73	0.47	fail	0.64	1.71	2.58	0.08	0.19	0.08
IF4	6.99	3.62	5.55	2.77	5.31	fail	fail	0.30	1.08	0.36
Avg.	5.21	3.21	1.99	fail	2.08	fail	fail	0.17	0.46	0.19

TABLE IV: MVSEC indoor-flying MPE [%]. Best/second-best results are bold/underlined.

	ORB- SLAM3 [ 11 ]	COLMAP [ 36 ]	DROID- SLAM [ 16 ]	DSO [ 12 ]	DROID- VO [ 16 ]	DPVO [ 6 ]	RAMP- VO [ 9 ]	DAPEVO
Input	I	I	I	I	I	I	I+E	I+E
Loop clos.	✓	✓	✓	$\times$	$\times$	$\times$	$\times$	$\times$
ME00	13.61	15.20	0.17	9.65	0.22	0.16	0.20	0.14
ME01	16.86	5.58	0.06	3.84	0.15	0.11	0.04	0.08
ME02	20.57	10.86	0.36	12.20	0.24	0.11	0.10	0.14
ME03	16.00	3.93	0.87	8.17	1.27	0.66	0.46	0.41

TABLE V: ECCV 2020 SLAM Competition monocular with simulated event ATE [m]. DAPEVO: median of five trials; baselines: [ 9 ] . Checkmarks denote global optimization or loop closure. Among other methods, best /second-best results are bold/underlined.

Fig. 3: X–Y trajectories on UZH-FPV indoor_45_13 at increasing image-drop rates. As the drop rate increases, DPVO exhibits substantial trajectory deviation, whereas DAPEVO remains closely aligned with the ground truth.

Panel of Figure 9 — Fig. 4: UZH-FPV ATE averaged over five trials per sequence and all sequences. Equally spaced markers denote retaining every $k$ th RGB frame, $k=1,\ldots,6$ ; labels give the corresponding drop percentages.

RGB rate	DPVO [ 6 ]		RAMP-VO [ 9 ]		DAPEVO
[Hz]	AUC $\uparrow$	ATE $\downarrow$	AUC $\uparrow$	ATE $\downarrow$	AUC $\uparrow$	ATE $\downarrow$
30	0.83	0.17	0.85	0.15	0.85	0.15
15	0.80	0.39	0.74	0.45	0.79	0.21
10	0.77	0.45	0.66	1.32	0.77	0.29
6	0.48	1.70	0.39	3.33	0.73	0.31
5	0.34	3.64	0.27	4.45	0.69	0.35

TABLE VI: TartanEvent validation AUC and ATE [m] at reduced RGB rates. DAPEVO event voxel grids remain at 30 Hz.

Fig. 5: Temporal distribution of the event-fusion weight

\alpha

on TartanEvent japanesealley/Hard/P005 . Columns show normalized edges fractions in

0.05

-wide bins; we exclude edges whose reprojections fall out-of-frame. Dashed curves show the mean. Larger

\alpha

favors events, while smaller

\alpha

favors RGB.

	Nominal RGB		Degraded RGB
$\alpha$	AUC $\uparrow$	ATE [m] $\downarrow$	AUC $\uparrow$	ATE [m] $\downarrow$
0.0 (RGB only)	0.87	0.13	0.19	5.04
Dataset Mean	0.84	0.16	0.57	0.99
0.5	0.82	0.19	0.57	0.99
1.0 (events only)	0.58	0.96	0.59	0.73
Adaptive	0.85	0.15	0.63	0.60

TABLE VII: Fusion-weight ablation on TartanEvent validation. Dataset Mean fixes

\alpha

to the adaptive model’s mean over the considered dataset.

Method	Threshold (joint, evt-only)	AUC $\uparrow$	ATE [m] $\downarrow$	Joint frames
Original [ 6 , 7 ]	(15, 15)	0.57	0.70	0.20
Original [ 6 , 7 ]	(15, 25)	0.64	0.71	0.57
Subgraph	(15, 15)	0.66	0.39	0.54
Subgraph	(15, 25)	0.69	0.35	0.74

TABLE VIII: Culling ablation on TartanEvent validation at 5 Hz RGB. Joint frames: fraction of retained keyframes with both modalities.

Explore similar work

Feb 22, 2026

OpenVO: Open-World Visual Odometry with Temporal Dynamics Awareness

We introduce OpenVO, a novel framework for Open-world Visual Odometry (VO) with temporal awareness under limited input conditions. OpenVO effectively estimates real-world-scale ego-motion from monocular dashcam footage with varying observation rates and uncalibrated cameras, enabling robust trajectory dataset construction from rare driving events recorded in dashcam. Existing VO methods are trained on fixed observation frequency (e.g., 10Hz or 12Hz), completely overlooking temporal dynamics information. Many prior methods also require calibrated cameras with known intrinsic parameters. Consequently, their performance degrades when (1) deployed under unseen observation frequencies or (2) applied to uncalibrated cameras. These significantly limit their generalizability to many downstream tasks, such as extracting trajectories from dashcam footage. To address these challenges, OpenVO (1) explicitly encodes temporal dynamics information within a two-frame pose regression framework and (2) leverages 3D geometric priors derived from foundation models. We validate our method on three major autonomous-driving benchmarks - KITTI, nuScenes, and Argoverse 2 - achieving more than 20 performance improvement over state-of-the-art approaches. Under varying observation rate settings, our method is significantly more robust, achieving 46%-92% lower errors across all metrics. These results demonstrate the versatility of OpenVO for real-world 3D reconstruction and diverse downstream applications.

Phuc D. A. Nguyen, Anh N. Nhu, Ming C. Lin

University of Maryland, College Park, USA

Visual Odometry

Ask PDF

Oct 1, 2026

Real-time Event-camera Stereo Visual Odometry via Keytime Gaussian Process Regression

Event cameras have microsecond-level temporal resolution and high dynamic range which make them more resilient to motion blur and poor illumination than standard frame-based cameras. Event-camera visual odometry (VO) pipelines maximize these benefits when they process the asynchronous event stream at the native temporal resolution. Continuous-time Gaussian process (GP) regression and a white-noise-on-acceleration (WNOA) prior can handle asynchronous measurements but result in a prohibitively large estimation state when applied naively. This paper presents a continuous-time event-camera stereo VO pipeline that maintains the native measurement times of asynchronous events while also running in real time. It reduces the estimation states to keytimes while maintaining full temporal resolution by interpolating measurements to their exact timestamps with a physically founded WNOA prior. This decouples the state size from the dense number of measurements without discarding their asynchronous nature. The real-time continuous-time VO pipeline is evaluated on the MVSEC and DSEC datasets. It provides estimates in real time that are more accurate than ES-PTAM, a state-of-the-art discrete estimator, in all but one of the tested sequences. The pipeline respectively provides estimates at 22 Hz and 6 Hz on MVSEC and DSEC and RMS relative errors of 0.46 cm and 0.038 degrees across all valid sequences, which were 11 and 15 times better than ES-PTAM, respectively.

Nikan Nobari, Jonathan D. Gammell

Estimation, Search, and Planning (ESP) Research Group, Queen’s University, Kingston ON, Canada.

Visual Odometry Stereo Vision Gaussian Process Regression

Ask PDF

Aug 10, 2026

ROEVO: Robust Organized Edge Feature-based Visual Odometry Using RGB-D Cameras

This work presents a visual odometry (VO) system that leverages image edge features. Edges are spatially expressive cues commonly present across diverse environments, offering rich textural and structural information. However, existing edge-based VO methods often fail to fully exploit this potential. To this end, we introduce a novel feature representation termed organized edges, which transforms disjoint edge pixels into sequentialized clusters, enabling more effective retention and utilization of the underlying textural and structural information. Another nice property of this formulation is that organized edges can perform edge-level association across multiple frames, enabling the establishment of a co-visibility graph. To achieve precise and efficient pose estimation, we propose a range of particularly designed tracking and joint optimization methods based on the characteristics of organized edges. For tracking, we formulate edge-wise rather than pixel-wise residuals to achieve robust and accurate inter-frame registration. For joint optimization, we introduce a novel shape-preserving edge-fitting method and an organized edge-based Bundle Adjustment (BA) approach, which decomposes the traditional BA problem into fitting and registration to preserve the structural integrity. Based on these novel techniques, we develop a complete VO system that exclusively employs organized edge features, achieving efficient tracking and precise local mapping. Extensive experiments demonstrate its accuracy and robustness in indoor environments, outperforming or achieving comparable performance to state-of-the-art methods. The source code is publicly available at https://github.com/liumingrui814/ROEVO

Mingrui Liu, Xingxing Zuo, Renlang Huang +3

College of Control Science and Engineering, Zhejiang University, Hangzhou, 310027, China. · Department of Robotics, Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, PO Box 7909, UAE.

Bundle Adjustment Visual Odometry

Ask PDF