Authors: Sicheng Li, Zaiwang Gu, Jie Zhang, Qing Guo, Xudong Jiang, Jun Cheng
Organizations: S. Li and X. Jiang are with the School of Electrical and Electronic Engineering, Nanyang Technological University (NTU), Singapore. · J. Zhang and S. Li are with Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore. · J. Cheng and Z. Gu are with the Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR), Singapore. · Q. Guo is with the College of Computer Science, Nankai University, Tianjin, China.
Establishing reliable image correspondences is essential for many robotic vision problems. However, existing methods often struggle in challenging scenarios with large viewpoint changes or textureless regions, where incorrect cor- respondences may still receive high similarity scores. This is mainly because conventional models rely solely on fea- ture similarity, lacking an explicit mechanism to estimate the reliability of predicted matches, leading to overconfident errors. To address this issue, we propose SURE, a Semi- dense Uncertainty-REfined matching framework that jointly predicts correspondences and their confidence by modeling both aleatoric and epistemic uncertainties. Our approach in- troduces a novel evidential head for trustworthy coordinate regression, along with a lightweight spatial fusion module that enhances local feature precision with minimal overhead. We evaluated our method on multiple standard benchmarks, where it consistently outperforms existing state-of-the-art semi-dense matching models in both accuracy and efficiency. our code will be available on https://github.com/LSC-ALAN/SURE.
Figures & tables
Fig. 1 : Comparison of E-LoFTR and our method on the MegaDepth dataset. Lines highlighted in green and red correspond to points with an epipolar error less than or greater than 10−4 respectively.
Fig. 2 : Overview of the proposed SURE framework. (1) A backbone extracts coarse features Fc and a Spatial Fusion Module provides fine features Ff . (2) The coarse matching module produces initial correspondences Mc , which are used to sample Ff for fine-level refinement. (3) Trustworthy Regression produces precise offsets (Δx,Δy) along with uncertainty estimates. Specifically, an Evidential Head predicts the parameters (ψ,η,κ,ρ) of a Normal-Inverse-Gamma distribution, which jointly encode the offset and its associated aleatoric and epistemic uncertainties.
Fig. 3 : Qualitative comparisons of SURE against Light Glue and E-LoFTR on both indoor and outdoor scenes. SURE achieves a higher number of correct matches and reduces mismatches, demonstrating robustness in low-texture areas as well as under significant viewpoint and lighting variations. Red regions denote points with epipolar error exceeding 5×10−4 for indoor and 1×10−4 for outdoor scenes.
Fig. 4 : We selected large viewpoint changes and weak-texture scenarios. Among 2048 correspondences, the 50 pairs with the highest model uncertainty and data uncertainty were chosen. The lighter the line color, the higher the uncertainty.
Category
Method
ScanNet
MegaDepth
Time(ms)
AUC@5°
AUC@10°
AUC@20°
AUC@5°
AUC@10°
AUC@20°
Sparse
SP + SG CVPR’20
16.2
32.8
49.7
57.6
72.6
83.5
96.9
SP + LG ICCV’23
14.8
30.8
47.5
58.8
73.6
84.1
84.2
XFeat CVPR’24
16.7
32.6
47.8
44.2
58.2
69.2
14.2
Dense
DKM CVPR’23
26.6
47.1
64.1
67.3
79.7
88.1
554.2
ROMA CVPR’24
28.9
50.4
68.3
68.5
80.6
88.8
824.9
TABLE I : Results of Relative Pose Estimation on ScanNet and MegaDepth Dataset. We report the pose estimation AUC under three error thresholds, along with the overall runtime efficiency of the models.
Category
Method
AUC
@3px
@5px
@10px
Sparse
SP + NN
41.6
55.8
71.7
R2D2 + NN
50.6
63.9
76.8
DISK + NN
52.3
64.9
78.9
SP + SG
53.9
68.3
81.7
SP + LG
54.2
68.3
81.5
TABLE II : Homography estimation results on the HPatches dataset. We report the AUC of corner reprojection error.
Setup ↓
AUC@10° ↑
Time (ms) ↓
ScanNet
MegaDepth
I (Baseline): E-LoFTR
37.0
77.0
69.6
II : Direct Regression
35.9
75.0
58.3
III : Coord. Regression
36.3
75.8
59.0
IV : Spatial Fusion
37.3
76.7
62.5
V : KL Loss
37.2
76.5
62.5
TABLE III : Ablation study on MegaDepth and ScanNet datasets. We report AUC@10° and inference time. Each component is examined to assess its individual and combined impact on performance.
Fig. 5 : Uncertainty analysis. (a) and (b) are the heat maps of the Spearman rank correlation analysis between the EPE and the uncertainties.
Reliable semi-dense matching is essential for modern geometric vision systems. Designed under a coarse-to-fine paradigm, it achieves an optimal balance between performance and computational cost. However, existing methods often struggle to provide well-quantified uncertainties, where catastrophic coarse-assignment failures are ignored, leading to truncated error distributions and severely misjudged geometric estimations. In this paper, we propose a lightweight, post-hoc overall uncertainty estimation framework that introduces a two-component calibrated Laplace mixture model with only 9 learnable parameters. The objective is to explicitly capture both the sharp local refinement noise and the broader tail of coarse-assignment failures. We introduce the Coarse-success posterior Refit (CoRe) method, a geometric refitting module that utilizes the posterior probability of coarse-assignment success as soft correspondence weights. Extensive experiments show that our method consistently improves downstream geometric accuracy across various pretrained-only matchers and robust estimators with minimal computational overhead. Our code is available at https://github.com/khoavpt/Probabilistic-matching.
Khoa Hoang, Hoang-Tuan Nguyen, Huong Ninh +2
University of Engineering and Technology, Vietnam National University, Hanoi · Optoelectronics Center, Viettel Aerospace Institute, Viettel Group · National Yang Ming Chiao Tung University, Taiwan
Reliable correspondence estimation is a fundamental problem in image processing, underpinning applications such as Structure from Motion, visual localization, and image registration. Existing learning-based methods have significantly improved local feature representations, yet most still operate at the pixel or patch level and lack explicit modeling of regions that are jointly visible across views. We propose SAMatcher, a feature matching framework that formulates correspondence estimation through co-visibility modeling. Instead of directly matching local features, SAMatcher first predicts co-visible region masks and bounding boxes as structured priors for correspondence estimation. Built upon the Segment Anything Model (SAM), it introduces a symmetric cross-view interaction mechanism that enables bidirectional feature exchange and cross-view semantic alignment. We further develop a unified supervision scheme that jointly optimizes mask prediction and box localization through mask learning, box regression, and mask-box consistency constraints. Extensive experiments on challenging benchmarks demonstrate substantial improvements over existing matching pipelines, particularly under large viewpoint and scale variations. Our results show that foundation models originally designed for monocular segmentation can be effectively extended to multi-view correspondence reasoning through explicit co-visibility modeling, offering a new perspective on structured representation learning for image matching. Code and project page: https://xupan.top/Projects/samatcher
Xu Pan, Qiyuan Ma, Mingyue Dong +3
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, P. R. China · National Key Laboratory of Space Target Awareness, Space Engineering University, Beijing, P. R. China
Dense feature matching aims to estimate all correspondences between two images of a 3D scene and has recently been established as the gold standard due to its high accuracy and robustness. However, existing dense matchers still fail or perform poorly for many hard real-world scenarios, and high-precision models are often slow, limiting their applicability. In this paper, we attack these weaknesses on a wide front through a series of systematic improvements that together yield a significantly better model. In particular, we construct a novel matching architecture and loss, which, combined with a curated diverse training distribution, enables our model to solve many complex matching tasks. We further make training faster through a decoupled two-stage matching-then-refinement pipeline, and at the same time, significantly reduce refinement memory usage through a custom CUDA kernel. Finally, we leverage the recent DINOv3 foundation model along with multiple other insights to make the model more robust and unbiased. In our extensive set of experiments, we show that the resulting novel matcher sets a new state-of-the-art, being significantly more accurate than its predecessors. Code is available at https://github.com/Parskatt/romav2
Johan Edstedt, David Nordström, Yushan Zhang +7
Linköping University · Chalmers University of Technology · University of Amsterdam +1