DINO-VO: Learning Where to Focus for Enhanced State Estimation

Qi Chen; Guanghao Li; Sijia Hu; Xin Gao; Junpeng Ma; Xiangyang Xue; Jian Pu

cs.CVApr 5, 2026

DINO-VO: Learning Where to Focus for Enhanced State Estimation

Authors: Qi Chen, Guanghao Li, Sijia Hu, Xin Gao, Junpeng Ma, Xiangyang Xue, Jian Pu

Organizations: Fudan University · Shanghai Innovation Institute

Abstract

We present DINO Patch Visual Odometry (DINO-VO), an end-to-end monocular visual odometry system with strong scene generalization. Current Visual Odometry (VO) systems often rely on heuristic feature extraction strategies, which can degrade accuracy and robustness, particularly in large-scale outdoor environments. DINO-VO addresses these limitations by incorporating a differentiable adaptive patch selector into the end-to-end pipeline, improving the quality of extracted patches and enhancing generalization across diverse datasets. Additionally, our system integrates a multi-task feature extraction module with a differentiable bundle adjustment (BA) module that leverages inverse depth priors, enabling the system to learn and utilize appearance and geometric information effectively. This integration bridges the gap between feature learning and state estimation. Extensive experiments on the TartanAir, KITTI, Euroc, and TUM datasets demonstrate that DINO-VO exhibits strong generalization across synthetic, indoor, and outdoor environments, achieving state-of-the-art tracking accuracy.

Figures & tables

Figure 1: The overview of the system. Three modules establish our system from left to right: Multi-task Feature Extractor, Adaptive Patch Selector, and Sparse Bundle Adjustment Layer. The Multi-task Feature Extractor extracts the corresponding features for matching, selecting, and adjusting the bundle. The Adaptive Patch Selector selects high-weight patch features for bundle adjustment. The Sparse Bundle Adjustment Layer performs bundle adjustment to optimize the pose in the factor graph.

Figure 2: Architecture of the Multi-task Feature Extractor. The module predicts context features, matching features, an inverse depth map, and a prior weight map from a single RGB image.

Figure 3: Pipeline for the adaptive patch selector. We utilize the prior weight and depth maps to uniformly select high-weight patches, which is more useful for the next sparse bundle adjustment.

Figure 4: Comparison of our adaptive patch selector with existing systems. The first row is the input image from different datasets. The second row presents the random patch selection strategy proposed in [ 60 ] and [ 40 ] , while the third row illustrates our patch selection strategy, highlighting its focus on areas that contribute significantly to bundle adjustment.

Figure 5: Comparison of reconstruction results on TartanAir [ 64 ] . Our sparse map is more informative than DPVO’s, particularly in the texture-rich area, as shown in the three small images on the right side.

Methods	ME 000	ME 001	ME 002	ME 003	ME 004	ME 005	ME 006	ME 007	MH 000	MH 001	MH 002	MH 003	MH 004	MH 005	MH 006	MH 007	Avg
ORB-SLAM3* [ 4 ]	13.61	16.86	20.57	16.00	22.27	9.82	21.61	7.74	15.44	2.92	13.51	8.18	2.59	21.91	11.70	25.88	14.38
DROID-SLAM* [ 59 ]	0.17	0.06	0.36	0.87	1.14	0.13	1.13	0.06	0.08	0.05	0.04	0.02	0.01	0.68	0.30	0.07	0.33
DPVO [ 60 ]	0.14	0.11	0.18	0.50	0.39	0.12	0.34	0.14	0.26	0.04	0.04	0.06	0.63	0.20	0.10	0.09	0.21
Ours	0.22	0.05	0.17	0.17	0.51	0.04	0.33	0.08	0.24	0.04	0.07	0.06	0.32	0.29	0.18	0.09	0.18

Table 1: Results on the TartanAir monocular test split. Results are reported as the ATE with scale alignment, in which bold and underline represents the best and second, seperately. For our framework, we report the mean of 5 runs. Methods marked with (*) use global optimization/loop closure.

Methods	00	01	02	03	04	05	06	07	08	09	10	Avg
ORB-SLAM3* [ 4 ]	8.27	x	26.86	1.21	0.77	7.91	12.54	3.44	46.81	76.54	6.61	x
LDSO* [ 18 ]	9.32	11.68	31.98	2.85	1.22	5.10	13.55	2.96	129.02	21.64	17.36	22.42
DROID-SLAM* [ 59 ]	92.10	344.60	x	2.38	1.00	118.50	62.47	21.78	161.60	x	118.70	x
DPV-SLAM++* [ 40 ]	7.79	12.11	43.01	2.51	0.79	5.41	11.07	1.69	109.97	76.64	13.33	25.85
DPVO [ 60 ]	113.21	12.69	123.4	2.09	0.68	58.96	54.78	19.26	115.90	75.10	13.63	53.61
Ours	100.39	7.50	111.48	2.07	1.41	48.30	50.18	17.80	91.47	68.62	10.53	46.32

Table 2: Results on the KITTI Odometry dataset. Results are reported as ATE with scale alignment. For our system, we report the mean of 5 runs. Methods marked with (*) use global optimization/loop closure. x indicates that the system does not converge on the specified scene.

Figure 6: Percentage of effective patch projections in bundle adjustment. Our method selects more high-weight patches than the random selection method.

Methods	360	desk	desk2	floor	plant	room	rpy	teddy	xyz	Avg
ORB-SLAM3* [ 4 ]	x	0.017	0.210	x	0.034	x	x	x	0.009	x
DSO [ 14 ]	0.173	0.567	0.916	0.080	0.121	0.379	0.058	x	0.036	x
DROID-VO [ 59 ]	0.161	0.028	0.099	0.033	0.028	0.327	0.028	0.169	0.013	0.098
DPVO [ 60 ]	0.179	0.026	0.113	0.061	0.040	0.469	0.040	0.083	0.011	0.114
Ours	0.136	0.021	0.052	0.053	0.038	0.260	0.025	0.131	0.010	0.081

Table 3: Results on the TUM-RGBD dataset. Results are reported as ATE with scale alignment. Results are reported as ATE with scale alignment. For our system, we report the mean of 5 runs. Methods marked with (*) use global optimization/loop closure. x indicates that the system does not converge on the specified scene.

Methods	MH01	MH02	MH03	MH04	MH05	V101	V102	V103	V201	V202	V203	Avg
Tartan VO [ 63 ]	0.639	0.325	0.550	1.153	1.021	0.447	0.389	0.622	0.433	0.749	1.152	0.680
SVO [ 16 ]	0.100	0.120	0.410	0.430	0.300	0.070	0.210	x	0.110	0.110	1.080	0.294
DSO [ 14 ]	0.046	0.046	0.172	3.810	0.110	0.089	0.107	0.903	0.044	0.132	1.152	0.601
DROID-VO [ 59 ]	0.163	0.121	0.242	0.399	0.270	0.103	0.165	0.158	0.102	0.115	0.204	0.186
DPVO [ 60 ]	0.087	0.078	0.147	0.147	0.135	0.048	0.137	0.086	0.060	0.045	0.423	0.127
Ours(random)	0.102	0.024	0.116	0.143	0.166	0.047	0.116	0.036	0.060	0.059	1.054	0.175

Table 4: Results on the EuRoC dataset. Results are reported as ATE with scale alignment. For our system, we report the mean of 5 runs. x indicates that the system does not converge on the specified scene. random indicates using the random patch selection mecth as DPV-SLAM [ 40 ] does, and w/o depth indocates we do not use the inverse depth map from our Multi-task Feature Extractor.

Figure 7: Trajector comparison between DPV-SLAM [ 40 ] and our system with loop closure mechanism in KITTI Sequence 00

Visual Odometry

Visual odometry estimates camera or robot motion from successive images, enabling navigation and mapping when external positioning is unavailable.

See all 55 papers

MVOFormer: Flow-Semantic Transformer for Robust Monocular Visual Odometry
Jun 15, 2026Jituo Li, Shunwang Sun, Jialu Zhang +5Visual Odometry
State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University and with Zhejiang Key Laboratory of Industrial Big Data and Robot Intelligent Systems. They are also with the School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China, and with Robotics Institute, Zhejiang University, Hangzhou 310027, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha, 410012, China · Rural Health Research Institute, Charles Sturt University, Sydney, Australia +1
DAPEVO: Deep Adaptive Patch Frame-Event Visual Odometry
Sep 25, 2026Luca Gandolfi, Simone Nascivera, Roberto Pellerito +3Visual Odometry RGB-Event Sensor Fusion
Robotics and Perception Group, University of Zurich, Switzerland · Bocconi University, Italy
ROFT-VINS: Robust Feature Tracking-based Visual-Inertial State Estimation for Harsh Environment
Mar 19, 2026Sanghyun Park, Soohee HanVisual-Inertial Odometry Visual Object Tracking
Department of Convergence IT Engineering, POSTECH, Pohang, 37673, Korea · Department of Electrical Engineering and of Convergence IT Engineering, POSTECH, Pohang, 37673, Korea

Jun 15, 2026J/K move · Enter open · S save

MVOFormer: Flow-Semantic Transformer for Robust Monocular Visual Odometry

Jituo Li, Shunwang Sun, Jialu Zhang, Xinqi Liu+4

State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University and with Zhejiang Key Laboratory of Industrial Big Data and Robot Intelligent Systems. They are also with the School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China, and with Robotics Institute, Zhejiang University, Hangzhou 310027, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha, 410012, China · Rural Health Research Institute, Charles Sturt University, Sydney, Australia+1

Visual Odometry

Open paper AskPDF arXiv

Monocular visual odometry (MVO) is foundational to autonomous navigation and robotic localization. However, existing learning-based MVO approaches often struggle with either a lack of interpretable, complementary features or overly complex multi-stage architectures. These limitations inherently restrict their robustness and cross-domain generalization. In this work, we propose MVOFormer, a novel transformer framework for robust monocular visual odometry. Our architecture features a Flow-Semantic Dual Branch Encoder that synergizes dense geometric motion cues with object-centric semantic priors, explicitly distinguishing static structures from dynamic distractors. These representations are then fused by an Iterative Multimodal Decoder, enabling coarse-to-fine pose refinement while dynamically suppressing attention on unreliable regions. Extensive evaluations demonstrate that, without any target-domain fine-tuning, MVOFormer achieves superior zero-shot generalization and robustness, significantly outperforming prior learning-based frame-to-frame methods across diverse benchmarks including TartanAir, KITTI, TUM-RGBD, and ETH3D-SLAM.

Abstract

Visual Odometry

Explore similar work

MVOFormer: Flow-Semantic Transformer for Robust Monocular Visual Odometry

DAPEVO: Deep Adaptive Patch Frame-Event Visual Odometry

ROFT-VINS: Robust Feature Tracking-based Visual-Inertial State Estimation for Harsh Environment