Authors: Sicheng Li, Zaiwang Gu, Jie Zhang, Qing Guo, Xudong Jiang, Jun Cheng
Organizations: S. Li and X. Jiang are with the School of Electrical and Electronic Engineering, Nanyang Technological University (NTU), Singapore. · J. Zhang and S. Li are with Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore. · J. Cheng and Z. Gu are with the Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR), Singapore. · Q. Guo is with the College of Computer Science, Nankai University, Tianjin, China.
Establishing reliable image correspondences is essential for many robotic vision problems. However, existing methods often struggle in challenging scenarios with large viewpoint changes or textureless regions, where incorrect cor- respondences may still receive high similarity scores. This is mainly because conventional models rely solely on fea- ture similarity, lacking an explicit mechanism to estimate the reliability of predicted matches, leading to overconfident errors. To address this issue, we propose SURE, a Semi- dense Uncertainty-REfined matching framework that jointly predicts correspondences and their confidence by modeling both aleatoric and epistemic uncertainties. Our approach in- troduces a novel evidential head for trustworthy coordinate regression, along with a lightweight spatial fusion module that enhances local feature precision with minimal overhead. We evaluated our method on multiple standard benchmarks, where it consistently outperforms existing state-of-the-art semi-dense matching models in both accuracy and efficiency. our code will be available on https://github.com/LSC-ALAN/SURE.
Figures & tables
Fig. 1 : Comparison of E-LoFTR and our method on the MegaDepth dataset. Lines highlighted in green and red correspond to points with an epipolar error less than or greater than 10−4 respectively.
Fig. 2 : Overview of the proposed SURE framework. (1) A backbone extracts coarse features Fc and a Spatial Fusion Module provides fine features Ff . (2) The coarse matching module produces initial correspondences Mc , which are used to sample Ff for fine-level refinement. (3) Trustworthy Regression produces precise offsets (Δx,Δy) along with uncertainty estimates. Specifically, an Evidential Head predicts the parameters (ψ,η,κ,ρ) of a Normal-Inverse-Gamma distribution, which jointly encode the offset and its associated aleatoric and epistemic uncertainties.
Fig. 3 : Qualitative comparisons of SURE against Light Glue and E-LoFTR on both indoor and outdoor scenes. SURE achieves a higher number of correct matches and reduces mismatches, demonstrating robustness in low-texture areas as well as under significant viewpoint and lighting variations. Red regions denote points with epipolar error exceeding 5×10−4 for indoor and 1×10−4 for outdoor scenes.
Fig. 4 : We selected large viewpoint changes and weak-texture scenarios. Among 2048 correspondences, the 50 pairs with the highest model uncertainty and data uncertainty were chosen. The lighter the line color, the higher the uncertainty.
Category
Method
ScanNet
MegaDepth
Time(ms)
AUC@5°
AUC@10°
AUC@20°
AUC@5°
AUC@10°
AUC@20°
Sparse
SP + SG CVPR’20
16.2
32.8
49.7
57.6
72.6
83.5
96.9
SP + LG ICCV’23
14.8
30.8
47.5
58.8
73.6
84.1
84.2
XFeat CVPR’24
16.7
32.6
47.8
44.2
58.2
69.2
14.2
Dense
DKM CVPR’23
26.6
47.1
64.1
67.3
79.7
88.1
554.2
ROMA CVPR’24
28.9
50.4
68.3
68.5
80.6
88.8
824.9
TABLE I : Results of Relative Pose Estimation on ScanNet and MegaDepth Dataset. We report the pose estimation AUC under three error thresholds, along with the overall runtime efficiency of the models.
Category
Method
AUC
@3px
@5px
@10px
Sparse
SP + NN
41.6
55.8
71.7
R2D2 + NN
50.6
63.9
76.8
DISK + NN
52.3
64.9
78.9
SP + SG
53.9
68.3
81.7
SP + LG
54.2
68.3
81.5
TABLE II : Homography estimation results on the HPatches dataset. We report the AUC of corner reprojection error.
Setup ↓
AUC@10° ↑
Time (ms) ↓
ScanNet
MegaDepth
I (Baseline): E-LoFTR
37.0
77.0
69.6
II : Direct Regression
35.9
75.0
58.3
III : Coord. Regression
36.3
75.8
59.0
IV : Spatial Fusion
37.3
76.7
62.5
V : KL Loss
37.2
76.5
62.5
TABLE III : Ablation study on MegaDepth and ScanNet datasets. We report AUC@10° and inference time. Each component is examined to assess its individual and combined impact on performance.
Fig. 5 : Uncertainty analysis. (a) and (b) are the heat maps of the Spearman rank correlation analysis between the EPE and the uncertainties.
University of Engineering and Technology, Vietnam National University, Hanoi · Optoelectronics Center, Viettel Aerospace Institute, Viettel Group · National Yang Ming Chiao Tung University, Taiwan
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, P. R. China · National Key Laboratory of Space Target Awareness, Space Engineering University, Beijing, P. R. China