SFVO: Decoupled Confidence-Guided Stereo-Flow Visual Odometry with Bidirectional PnP
Authors: Kai Zhang, Guoyang Zhao, Jun Ma
Organizations: Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology, Hong Kong SAR, China
Deep learning-based visual odometry (VO) has achieved significant progress, yet most existing methods focus on a monocular approach, which suffers from scale ambiguity. Stereo VO provides real metric by its nature, but remains less studied in deep learning VO due to its high computational cost and modeling complexity. Recent advances in stereo matching and optical flow estimation have made dense visual correspondence increasingly accurate and reliable, but their complementary geometric information has not been fully exploited for VO. In this paper, we present SFVO, a correspondence-driven stereo VO framework that directly builds upon pretrained stereo matching and optical flow models. SFVO exploits pretrained stereo matching and optical flow models to estimate stereo and temporal correspondences. Instead of learning pose directly from images, SFVO maps learned correspondences into geometric constraints and predicts which points are trustworthy. To improve the reliability of visual correspondence-based geometric constraints, we introduce decoupled confidence maps for rotation and translation. This design better aligns the characteristics of visual correspondence and 6-DoF transformations. Extensive experiments on outdoor and indoor datasets demonstrate that SFVO achieves robust and accurate pose estimation with strong generalization capability. The code will be released.
Figures & tables
Fig. 1: Overview of SFVO and its application on an unmanned ground vehicle (UGV). SFVO uses pretrained stereo disparity and optical flow models to establish bidirectional 3D–2D correspondences. Decoupled rotation and translation confidence maps guide a bidirectional PnP solver for frame-to-frame pose estimation. The bottom panel shows the UGV platform and a representative trajectory, demonstrating accurate motion estimation in a real-world environment.
Fig. 2: SFVO structure. The input consists of two stereo image frames, stereo disparities are estimated by stereo matching model, forward and backward optical flow are estimated by optical flow model. The confidence estimator predicts translation-rotation-decoupled confidence maps from features extracted by optical flow and disparity models, which serve as weighting factors in the differentiable PnP pose solver.
Method
Sensor
MH02 (E)
MH04 (D)
V103 (D)
ATE (m)
RPE t (m)
RPE r (deg)
ATE (m)
RPE t (m)
RPE r (deg)
ATE (m)
RPE t (m)
RPE r (deg)
DPVO
Mono
0.105
0.007
0.586
0.177
0.007
0.685
0.126
0.017
1.312
BotVIO
Mono+IMU
0.190
-
-
0.150
-
-
0.200
-
-
ORB-SLAM3
Stereo
0.037
0.024
0.548
0.112
0.054
0.701
0.199
0.040
2.326
SFVO (Ours)
Stereo
0.087
0.002
0.033
0.217
0.006
0.044
0.112
0.009
0.160
TABLE I: Quantitative comparison on the EuRoC MAV dataset.
Fig. 3: Trajectory comparisons on the EuRoC MAV dataset. Results are shown for sequences MH04 (D) (left) and V103 (D) (right).
Fig. 4: Trajectory comparisons on the KITTI odometry dataset. Results are shown for sequences 09 (left) and 10 (right).
Method
Sequence
ATE (m)
RPE t (m)
RPE r (deg)
DPVO
09
74.868
0.531
0.037
BotVIO
9.942
0.082
0.407
ORB-SLAM3
2.077
0.017
0.044
SFVO (Ours)
9.830
0.016
0.039
DPVO
10
13.602
0.166
0.044
BotVIO
13.417
0.084
0.364
TABLE II: Quantitative comparison on KITTI sequences 09 and 10.
Fig. 5: UGV platform and sample images from the campus sequence. The data-collection platform is shown on the left, and representative image samples are shown on the right.
Fig. 6: Trajectory comparison on the unseen UGV campus sequence. This sequence is used to evaluate generalization to a new platform, motion pattern, and environment.
Method
Sensor
ATE (m)
RPE t (m)
RPE r (deg)
DPVO
Mono
12.798
0.052
0.247
ORB-SLAM3
Stereo
12.282
0.090
0.784
SFVO (Ours)
Stereo
4.906
0.043
0.312
TABLE III: Quantitative comparison on the unseen UGV campus sequence.
Fig. 7: Visualization of rotation–translation decoupled confidence maps. Distant regions tend to receive higher rotation confidence, while nearby regions are assigned higher translation confidence, consistent with their different geometric contributions to pose estimation.
Fig. 8: Confidence prediction in dynamic regions. SFVO assigns low confidence to dynamic objects, including pedestrians (top) and moving vehicles (bottom), reducing their influence on pose estimation.
Confidence
Sequence
ATE (m)
RPE t (m)
RPE r (deg)
Coupled
09
15.006
0.053
0.111
Decoupled
9.830
0.016
0.039
Coupled
10
6.284
0.046
0.105
Decoupled
2.835
0.017
0.046
TABLE IV: Ablation of Coupled and Decoupled Confidence on KITTI Sequences 09 and 10
Fig. 9: Visualization of coupled confidence maps. Sharing a single confidence map between rotation and translation produces highly polarized predictions and tends to suppress distant correspondences, despite their usefulness for rotational estimation.
Fig. 10: Effect of pose solver iterations on KITTI 09. With identity initialization, the pose estimate progressively converges and becomes stable after approximately 12 iterations.
Guangzhou Purpleriver Electronic Technology Co., Ltd, Guangzhou, China · State Key Laboratory of Intelligent Manufacturing Equipment and Technology, School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China