Omnidirectional (360∘) cameras capture the full surrounding scene in every frame, providing a strong advantage for visual odometry (VO) over pinhole sensors with a limited field of view. However, their equirectangular projection (ERP) introduces latitude-dependent distortion, an artificial seam at the image border, and a sampling bias toward the poles. Existing 360∘ VO methods address these challenges only partially. DEFDVO closes this gap with a patch-based deep VO method that operates directly on ERP imagery, combining spherically initialized deformable feature extraction, end-to-end seam-aware estimation, and spherical-gradient patch selection. The deformable feature extractor combines spherical geometric priors with learned sampling offsets, while seam-aware processing preserves ERP continuity, and spherical-gradient selection reduces polar sampling bias. Trained exclusively on synthetic TartanAirV2 ERP imagery, DEFDVO improves the best prior average absolute trajectory error (ATE) by 4.8× (0.201 m to 0.042 m) on the TartanAirV2 evaluation set defined by 360DVO and achieves the lowest average ATE on both splits of the real-world 360DVO benchmark. Code is available at https://github.com/BurakSevsay/DEFDVO
Figures & tables
Figure 1: DEFDVO overview. Given an equirectangular (ERP) stream, a spherically initialized deformable encoder extracts distortion-aware features, spherical-gradient selection samples informative patches on the viewing sphere, and seam-aware tracking with spherical bundle adjustment estimates the camera trajectory.
Figure 2: Overview of DEFDVO on TartanAirV2 ERP imagery. (a) The six 640×640 rig views are combined into a single 2048×1024 equirectangular (ERP) frame. (b) ERP image with spherical-gradient patch centroids and the ±75∘ latitude cap (Sec. 3.6 ). The green half-boxes represent continuity on the ERP seam. (c) On the sphere, the seam is an ordinary meridian, and the pipeline treats it consistently end-to-end through circular feature extraction and correlation and azimuth-wrapped bundle-adjustment residuals. (d,e) 3×3 deformable-kernel taps at the two crops marked A/B in (b): the regular sampling grid (gray squares), analytic spherical prior (blue circles), and prior plus the learned residual of the trained encoder (orange stars).
Methods
Input
CountryHouse
House
OldTownNight
VictorianStreet
Avg.
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
DROID-SLAM [ 22 ]
Pin.
0.002 /0.000/0.013
0.003 /0.001/0.015
0.015 /0.001/0.021
1.893/0.125/0.316
3.951/0.025/0.057
3.422/0.139/2.004
0.009 /0.001/0.027
0.017 /0.002/0.015
1.164/0.037/0.308
DPVO [ 23 ]
Pin.
0.012/0.001/0.015
0.011/0.001/0.021
0.030 /0.001/0.022
2.000/0.131/0.884
2.893/0.020/0.105
8.960/0.289/3.378
0.052/0.002/0.034
0.090/0.003/0.020
1.756/0.056/0.560
MASt3R-SLAM [ 16 ]
Pin.
0.041/0.010/0.256
0.043/0.017/0.425
0.231/0.014/0.274
0.476/0.068/2.047
8.659/0.193/2.083
10.465/0.934/6.869
0.397/0.040/0.813
0.703/0.065/1.622
2.627/0.168/1.798
VGGT-SLAM [ 14 ]
Pin.
0.145/0.048/1.181
0.165/0.070/1.136
0.594/0.157/1.547
0.333/0.100/2.139
1.375/0.345/1.057
7.208/1.845/10.315
0.729/0.163/1.520
0.683/0.177/1.306
1.404/0.363/2.525
OpenVSLAM [ 20 ]
ERP
0.783/0.266/3.223
0.621/0.373/57.328
2.025/1.096/43.676
1.169/0.607/10.045
3.848/0.651/16.078
3.204/0.493/8.473
3.371/2.223/8.151
5.347/3.110/36.991
2.546/1.102/22.996
Table 1: ATE/RPE(t)/RPE(r) on TartanAirV2 Set 1 (360DVO evaluation scenes). Input: Pin. = pinhole, ERP = equirectangular. Best ATE per sequence in bold , second-best underlined .
Methods
Input
Antiquity3D
DesertGasStation
HongKong
HQWesternSaloon
IndustrialHangar
OldTownNight
Avg.
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
DROID-SLAM [ 22 ]
Pin.
12.460
4.267
0.103
–
0.034
0.164
0.008
2.446
–
–
3.951
0.053
–
DPVO [ 23 ]
Pin.
0.119
2.892
0.128
9.236
0.065
1.558
0.033
3.060
0.634
7.002
2.893
0.200
2.318
MASt3R-SLAM [ 16 ]
Pin.
17.682
17.491
9.196
9.281
–
0.491
0.205
6.035
19.972
20.843
8.659
0.663
–
VGGT-SLAM [ 14 ]
Pin.
1.825
1.874
1.999
2.324
1.183
2.990
1.383
1.893
6.854
7.984
1.375
3.154
2.903
OpenVSLAM [ 20 ]
ERP
1.621
1.273
7.715
9.446
1.285
0.115
4.844
5.684
15.293
4.822
3.848
0.862
4.734
Table 2: ATE on TartanAirV2 Set 2 (CubeDVO evaluation scenes). Input: Pin. = pinhole, ERP = equirectangular, Cube. = cubemap. “–” denotes tracking failure. Best ATE per sequence in bold , second-best underlined .
Methods
Input
Easy Sequences
Avg.
00
01
02
03
04
05
06
07
08
09
DROID-SLAM [ 22 ]
Pin.
2.28/0.40/0.35
0.61/0.13/0.04
–/–/–
22.7/1.08/0.69
0.57/0.02/0.03
5.27/0.15/0.07
0.37 /0.22/0.41
–/–/–
23.7/0.72/1.58
2.70/0.04/0.09
–/–/–
DPVO [ 23 ]
Pin.
1.37 /0.32/0.11
0.13 /0.44/0.11
37.5/1.36/0.13
6.91/1.33/0.46
0.57/0.23/0.03
4.89/0.24/0.07
0.35 /0.21/0.52
16.4/1.68/0.15
7.93/0.88/0.44
2.23 /0.41/0.10
7.83/0.71/0.21
MASt3R-SLAM [ 16 ]
Pin.
8.23/0.44/0.40
3.99/0.34/0.63
–/–/–
–/–/–
–/–/–
–/–/–
–/–/–
–/–/–
–/–/–
4.25/0.34/0.33
–/–/–
VGGT-SLAM [ 14 ]
Pin.
23.14/9.35/35.41
26.45/4.36/11.79
–/–/–
36.32/69.75/4.18
1.78/10.11/0.89
–/–/–
6.13/3.61/7.73
16.04/8.65/3.05
–/–/–
8.56/5.30/7.68
–/–/–
OpenVSLAM [ 20 ]
ERP
10.59/6.61/22.27
2.45/1.42/2.30
17.19 /1.83/0.08
–/–/–
0.52 /0.28/0.04
33.21/18.22/19.78
23.25/5.01/21.28
0.10 /0.22/0.33
0.25 /0.43/0.12
24.91/11.68/0.18
–/–/–
Table 3: ATE/RPE(t)/RPE(r) on the 360DVO real-world dataset ( Easy and Hard ). Input: Pin. = pinhole, ERP = equirectangular. “–” denotes failure. Best ATE per sequence in bold , second-best underlined .
Figure 3: Qualitative trajectory comparison. ERP frames (top) and estimated trajectories (bottom) for two evaluation sequences. All trajectories are Sim(3)-aligned to the ground truth.
Method
360DVO dataset
TartanAirV2
FPS
Easy
Hard
Avg.
Set 1
Set 2
Avg.
OpenVSLAM [ 20 ]
–
–
–
2.546
4.734
3.640
7.63
CubeDVO [ 1 ]
na
na
na
na
1.02
na
na
360DVO (def.) [ 10 ]
3.60
6.06
4.83
0.202
0.142
0.172
5.96
360DVO (fast)
5.94
3.45
4.70
0.201
0.421
0.311
16.62
DEFDVO (ours)
3.26
3.29
3.27
0.042
0.088
0.065
7.13
Table 4: Summary ATE comparison of omnidirectional methods on the 360DVO dataset splits and the two TartanAirV2 evaluation sets. Avg. is the mean of the two split (set) averages. “na” denotes not available. “–” denotes tracking failure. Best ATE in bold , second-best underlined .
Method
Feat. Extr.
Seam Awar.
Patch Sel.
ATE
Equirect. DPVO
ResNet
✗
Grad.
4.36
+ Sph. ResNet
S-ResNet
✗
Grad.
4.02
+ Seam Awareness
S-ResNet
✓
Grad.
3.95
+ DCNv2 (zero)
DCNv2
✓
Grad.
3.82
+ DCNv2 (geom.)
DCNv2
✓
Grad.
3.69
+ Patch Selection
DCNv2
✓
S-Grad.
3.27
Table 5: Ablation of the proposed adaptations on the 360DVO dataset in ATE. Each row builds on the preceding variant, with the final row corresponding to DEFDVO.
ID
Scale
Patch Selection
ATE
Train
Eval
Easy
Hard
Avg.
1
0.5
Grad.
Grad.
3.27
4.10
3.69
2
S-Grad.
3.21
4.06
3.63
3
S-Grad.
Grad.
7.59
3.86
5.72
4
S-Grad.
3.26
3.29
3.27
5
1.0
Grad.
Grad.
4.28
3.93
4.11
Table 6: Ablation of image scale and patch selection on the 360DVO dataset in ATE. Grad. refers to gradient, while S-Grad. denotes spherical-gradient. Each training setting uses a separately trained model, while patch selection is varied at evaluation. Best per column in bold , second-best underlined .
Figure 4: Per-module runtime decomposition for the default and full image scales, measured on an NVIDIA RTX A5000.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Optimizer
AdamW
Learning rate
8×10−5 , one-cycle (linear, 1% warmup)
Weight decay
10−6
Gradient clipping
10 (global norm)
Training steps
120k
Frames per sequence
15
Patches per frame
96 (spherical-gradient)
Appendix
Table 7: Training hyperparameters.
ID
Settings
ATE
FPS
Resolution
N
Sel.
Easy
Hard
Avg.
#D
1920 × 960
192
S-Grad.
3.26
3.29
3.27
7.13
#R0
3840 × 1920
192
S-Grad.
3.10
4.85
3.98
2.23
#R1
960 × 480
192
S-Grad.
4.32
3.50
3.91
18.04
#P0
1920 × 960
384
S-Grad.
3.40
3.30
3.35
5.64
#P1
1920 × 960
96
S-Grad.
3.30
3.72
3.51
8.08
Appendix
Table 8: Ablation of DEFDVO inference-time hyperparameters on the 360DVO dataset in ATE. All rows use the same default-trained model. #D denotes the default setting used in all other experiments, and #R, #P, and #S vary image resolution, patch count ( N ), and patch selection, respectively.
ϕmax
Sphere
ERP rows
1/cosϕmax
65∘
9.4%
27.8%
2.37
75∘ (default)
3.4%
16.7%
3.86
85∘
0.4%
5.6%
11.47
Appendix
Table 9: Regions excluded by the polar cap ϕmax of Eq. (9) in the main paper: shares of the sphere and of the ERP image rows with ∣ϕ∣>ϕmax (both poles together), and the horizontal reprojection sensitivity 1/cosϕmax relative to the equator (Sec. 3.4 of the main paper).
Sel.
Cap
Candidates
Selected patches
>75∘
>65∘
>75∘
>85∘
360DVO
Random
none
16.0%
26.9%
16.0%
5.0%
Grad.
none
16.0%
3.6%
0.91%
0.06%
S-Grad.
none
3.4%
< 0.01%
0
0
S-Grad.
75∘
0
< 0.01%
0
0
Appendix
Table 10: Shares of candidates and of selected patch centers with ∣ϕ∣ above the given latitude, for random (#S1), planar gradient (#S0), and spherical-gradient selection without a cap and with the default cap (#D); IDs as in Table 8 . Measured at the default resolution on every frame of the 20 360DVO sequences (20,008 frames, 1920×960 ) and of the 19 TartanAirV2 evaluation trajectories (12,179 frames, 1024×512 ), with one draw per frame as in tracking, i.e., 3.84 M and 2.34 M selected patches per row. Random selection does not rank candidates, so its candidates are its patches.
Visual odometry is essential for autonomous navigation in GPS-denied environments, yet RGB-based methods remain vulnerable to motion blur, challenging illumination, and dropped frames. Event cameras complement conventional cameras with high temporal resolution and dynamic range, but their asynchronous measurements complicate reliable correspondence estimation. We present DAPEVO, a learned visual odometry system that estimates image and event correspondences independently at shared patch locations and fuses their correlation evidence before motion refinement. Each tracked patch maintains image and event descriptors, and a learned scalar gate combines modality-specific correlation embeddings for each patch--frame edge before a shared recurrent refinement and bundle-adjustment update. DAPEVO also supports event-only observations, enabling continued tracking when RGB frames are sparse or unavailable, while modality-aware keyframe culling preserves scarce frame constraints. On UZH-FPV, when retaining only one in six RGB frames, DAPEVO's mean absolute trajectory error (ATE) increases by only 36%, from 1.00 to 1.36m, whereas the ATE of DPVO and RAMP-VO rises by factors of 3.7× and 3.1×, respectively. On TartanEvent, DAPEVO similarly remains below 1m ATE at 3Hz RGB input, while DPVO and RAMP-VO exceed 9m. Under degraded RGB input on TartanEvent, DAPEVO achieves an ATE of 0.60m, compared with more than 4m for both DPVO and RAMP-VO, while also outperforming event-only DEVO at 0.87m.
Deep learning-based visual odometry (VO) has achieved significant progress, yet most existing methods focus on a monocular approach, which suffers from scale ambiguity. Stereo VO provides real metric by its nature, but remains less studied in deep learning VO due to its high computational cost and modeling complexity. Recent advances in stereo matching and optical flow estimation have made dense visual correspondence increasingly accurate and reliable, but their complementary geometric information has not been fully exploited for VO. In this paper, we present SFVO, a correspondence-driven stereo VO framework that directly builds upon pretrained stereo matching and optical flow models. SFVO exploits pretrained stereo matching and optical flow models to estimate stereo and temporal correspondences. Instead of learning pose directly from images, SFVO maps learned correspondences into geometric constraints and predicts which points are trustworthy. To improve the reliability of visual correspondence-based geometric constraints, we introduce decoupled confidence maps for rotation and translation. This design better aligns the characteristics of visual correspondence and 6-DoF transformations. Extensive experiments on outdoor and indoor datasets demonstrate that SFVO achieves robust and accurate pose estimation with strong generalization capability. The code will be released.
Kai Zhang, Guoyang Zhao, Jun Ma
Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology, Hong Kong SAR, China
We introduce OpenVO, a novel framework for Open-world Visual Odometry (VO) with temporal awareness under limited input conditions. OpenVO effectively estimates real-world-scale ego-motion from monocular dashcam footage with varying observation rates and uncalibrated cameras, enabling robust trajectory dataset construction from rare driving events recorded in dashcam. Existing VO methods are trained on fixed observation frequency (e.g., 10Hz or 12Hz), completely overlooking temporal dynamics information. Many prior methods also require calibrated cameras with known intrinsic parameters. Consequently, their performance degrades when (1) deployed under unseen observation frequencies or (2) applied to uncalibrated cameras. These significantly limit their generalizability to many downstream tasks, such as extracting trajectories from dashcam footage. To address these challenges, OpenVO (1) explicitly encodes temporal dynamics information within a two-frame pose regression framework and (2) leverages 3D geometric priors derived from foundation models. We validate our method on three major autonomous-driving benchmarks - KITTI, nuScenes, and Argoverse 2 - achieving more than 20 performance improvement over state-of-the-art approaches. Under varying observation rate settings, our method is significantly more robust, achieving 46%-92% lower errors across all metrics. These results demonstrate the versatility of OpenVO for real-world 3D reconstruction and diverse downstream applications.