Organizations: GNSS Research Center, Wuhan University, Wuhan 430079, China · Electronic Information School, Wuhan University, Wuhan 430079, China · Hubei Technology Innovation Center for Spatiotemporal Information and Positioning Navigation · Hubei Luojia Laboratory
Most existing image keypoint detection and description methods rely on datasets with accurate pose and depth annotations, limiting scalability and generalization, and often degrading navigation and localization performance. We propose ViBA, a sustainable learning framework that integrates geometric optimization with feature learning for continuous online training on unconstrained video streams. Embedded in a standard visual odometry pipeline, it consists of an implicitly differentiable geometric residual framework: (i) an initial tracking network for inter-frame correspondences, (ii) depth-based outlier filtering, and (iii) differentiable global bundle adjustment that jointly refines camera poses and feature positions by minimizing reprojection errors. By combining geometric consistency from BA with long-term temporal consistency across frames, ViBA enforces stable and accurate feature representations. We evaluate ViBA on EuRoC and UMA datasets. Compared with state-of-the-art methods such as SuperPoint+SuperGlue, ALIKED, and LightGlue, ViBA reduces mean absolute translation error (ATE) by 12-18% and absolute rotation error (ARE) by 5-10% across sequences, while maintaining real-time inference speeds (FPS 36-91). When evaluated on unseen sequences, it retains over 90% localization accuracy, demonstrating robust generalization. These results show that ViBA supports continuous online learning with geometric and temporal consistency, consistently improving navigation and localization in real-world scenarios.
Figures & tables
Fig. 1: The pipeline of the proposed method . The snowflake denotes a frozen network, the flame indicates an activated network, and the spark associated with the loss marks the transition from frozen to activated.
Fig. 2: Architecture of the point matching network. It comprises a shared encoder and a feature matching module. Blocks labeled 1×1 and 3×3 represent residual units with the corresponding convolution kernel sizes. N denotes the number of local image patches.
Fig. 3: Feature tracking and geometric initialization. The tracking network θ predicts feature observations from input images to form multi-frame feature tracks, which are then used to initialize the geometric state (camera poses and feature depths) for subsequent optimization.
Fig. 4: Overall training loss construction pipeline integrating differentiable bundle adjustment with multi-frame temporal consistency at both trajectory and descriptor levels.
Method
R
P
AUC - RANSAC
AUC - DLT
@1px
@5px
@1px
@5px
Features + Matcher
SuperPoint
NN+mutual
71.3
62.4
32.1
75.3
0.0
2.0
SuperGlue
89.7
83.4
35.2
77.1
31.6
73.6
LightGlue
92.6
84.9
35.7
78.5
32.3
74.0
ViBA( fθ )
90.5
86.8
40.6
84.7
35.8
78.5
TABLE I: Homography estimation on HPatches. ViBA produces highly reliable correspondences, achieving top precision (P) and high recall (R), enabling accurate downstream tasks such as homography estimation with RANSAC or direct linear transform (DLT).
Method
Pose estimation AUC
P
time
@2°
@5°
@10°
Features + Matcher
SuperPoint
NN + mutual
23.2
43.6
67.2
60.6
4.6
SuperGlue
35.5
71.1
85.6
76.9
68.3
LightGlue
36.8
75.0
86.3
77.4
44.5
ViBA( fθ )
42.9
78.8
89.1
81.6
11.5
TABLE II: Wide-baseline indoor pose estimation. We report the AUC of pose errors, GPU time (ms), and matching precision (P), all in percent except for time. When applied to the SuperPoint, ViBA outperforms other learning-based matchers. Moreover, in end-to-end evaluations, ViBA achieves the highest efficiency, despite a modest increase in runtime.
Method
MH_03
MH_05
V1_02
V1_03
V2_02
V2_03
Mean
t_ape (m)
r_ape (°)
t_ape (m)
r_ape (°)
t_ape (m)
r_ape (°)
t_ape (m)
r_ape (°)
t_ape (m)
r_ape (°)
t_ape (m)
r_ape (°)
t_ape (m)
r_ape (°)
OpenVINS [ 53 ]
0.1509
1.8826
0.4936
2.2101
0.0583
2.3035
0.0601
3.0555
0.0578
2.0987
0.2001
2.7169
0.1701
2.3779
SuperPoint
NN + mutual
0.3125
3.2147
0.6821
3.8425
0.1184
3.9562
0.1567
4.2158
0.1432
3.4876
0.3985
3.9021
0.3019
3.7698
SuperGlue [ 19 ]
0.1826
2.1458
0.4013
1.8642
0.0725
2.6453
0.0896
2.7431
0.0814
2.4187
0.2367
2.3314
0.1774
2.3581
LightGlue [ 49 ]
0.1647
2.0279
0.4339
1.2753
0.0709
2.8605
0.0920
2.5322
0.0737
2.3279
0.2077
2.2267
0.1738
2.2084
ViBA( fθ )
0.1392
1.8571
0.3038
1.1031
0.0537
2.3767
0.0597
1.9230
0.0464
2.3165
0.1357
1.7220
0.1231
1.8831
TABLE III: Absolute Trajectory Error (ATE) and Absolute Rotation Error (ARE). Translation errors are reported in meters, and rotation errors are reported in degrees. Red denotes the best, while blue denotes the second best.
Method
Params/M
ATE (m) ↓
ARE (°) ↓
FPS ↑
0.5x
origin(1x)
2x
0.5x
origin(1x)
2x
0.5x
origin(1x)
2x
SuperPoint
NN + mutual
1.30
0.3285
0.3019
0.2937
3.4215
3.7698
3.2054
102.3
98.7
96.3
SuperGlue
13.32
0.1897
0.1774
0.1689
2.4821
2.3581
2.1053
29.6
24.3
24.1
LightGlue
13.15
0.1592
0.1738
0.1769
2.0212
2.2084
1.8216
44.3
42.6
43.7
ViBA( fθ )
1.34
0.1315
0.1231
0.1207
1.9543
1.8831
1.8320
91.4
54.5
36.9
End-to-End
ALIKED
0.68
0.1406
0.1246
0.1312
2.4801
2.2727
2.2510
78.8
33.4
16.7
TABLE IV: Impact of input resolution on VIO performance. We report the mean ATE (m) , ARE(°) and runtime FPS on EuRoC sequences using OpenVINS.
Fig. 5: Trade-offs between accuracy and runtime at varying input resolutions.
Fig. 6: Overview of the geometric initialization process. An anchor frame and a terminal frame are selected to estimate the relative pose and triangulate initial 3D landmarks. Inverse depths are refined through Bayesian updates, after which the poses of remaining frames are initialized via PnP and refined within a sliding window.
Most existing visual odometry (VO) systems treat feature correspondences as deterministic measurements or assign uniform uncertainty, ignoring the inherent localization ambiguity of different observations. However, correspondence uncertainty is often anisotropic due to image structures such as edges, repetitive patterns, and motion blur, which can significantly affect geometric optimization. In this work, we propose MomentBA, a geometry-aware bundle adjustment framework that derives anisotropic correspondence uncertainty from second-order spatial moments of local similarity responses. Instead of introducing additional covariance prediction networks, the proposed method directly converts matching response distributions into interpretable covariance estimates and incorporates them into bundle adjustment as correspondence-specific information matrices for uncertainty-aware residual weighting. Furthermore, the proposed formulation is integrated into a differentiable optimization framework, establishing a direct connection between correspondence uncertainty and geometric estimation. Experiments on the EuRoC MAV and TartanAir v1 Hard datasets demonstrate that MomentBA improves monocular visual odometry accuracy compared with existing feature-based and learning-based approaches. The proposed anisotropic covariance model achieves lower rotational errors and more robust trajectory estimation than fixed and isotropic uncertainty models, validating the effectiveness of geometry-induced uncertainty modeling for challenging visual environments.
Yuqing Wang, Xiaoji Niu, Yan Wang +3
GNSS Research Center, Wuhan University, Wuhan 430079, China · Electronic Information School, Wuhan University, Wuhan 430079, China · Hubei Technology Innovation Center for Spatiotemporal Information and Positioning Navigation, Wuhan 430079, China +1
This work presents ViGeo, a feed-forward foundation model for recovering spatially dense and temporally consistent geometry from video sequences. Built upon a plain transformer architecture without task-specific architectural modifications, ViGeo supports streaming, full-sequence, and long-video inference within a unified model. The key design is dynamic chunking attention, which exposes the model to both bidirectional and causal temporal contexts during training and allows it to adapt its attention pattern at test time without retraining. To improve supervision quality, we further introduce a completion-based data refinement framework. This framework trains a video depth completion teacher that conditions on sparse and noisy annotations and exploits video/multi-view context to produce dense, temporally coherent, and geometrically reliable training targets. Beyond depth and point maps, ViGeo also predicts surface normals within the same framework. Trained solely on public datasets, ViGeo achieves state-of-the-art performance across online, offline, and long-video depth estimation, surface normal estimation, and video point map estimation.
Zhu Yu, Jingnan Gao, Runmin Zhang +9
Zhejiang University · Shanghai Jiao Tong University · Tongyi Lab, Alibaba Group +1
Accurately recovering the camera's calibration and metric poses for any unconstrained video would unlock large-scale training data for navigation and scene understanding. The dominant approaches to this problem are severely limited: Simultaneous Localization and Mapping (SLAM) is sensitive to initialization and transient failures due to its causal, incremental nature; it is often over-optimized for real-time operation and generally requires known camera calibration; while Structure-from-Motion (SfM) typically forgoes any image ordering, enabling optimal initialization and global optimization, but lacks robustness to visual symmetries and extreme motions. To bridge this gap, we introduce a system that combines the strong sequential constraints of SLAM with the flexibility and global optimization of offline SfM, enabling the metric reconstruction of arbitrary, long, uncalibrated videos. This system leverages recent advances in wide-baseline dense image matching, treats temporal ordering as a first-class citizen for reliable loop closure, and augments global optimization with metric monocular depth priors. As a result, thorough evaluations on diverse, challenging datasets that exhibit extreme motion and visual symmetries reveal that our approach is significantly more robust and accurate than both state-of-the-art SLAM and SfM, classical or learned, with given or unknown camera calibration. The code is publicly available at https://github.com/cvg/vidmap.