Visual relocalization is a fundamental task in the field of 3D computer vision, estimating a camera's pose when it revisits a previously known scene. While point-based hierarchical relocalization methods have shown strong scalability and efficiency, they are often limited by sparse image observations and weak feature matching. In this work, we propose SplatHLoc, a novel hierarchical visual relocalization framework that uses Feature Gaussian Splatting as the scene representation. To address the sparsity of database images, we propose an adaptive viewpoint retrieval method that synthesizes virtual candidates with viewpoints more closely aligned with the query, thereby improving the accuracy of initial pose estimation. For feature matching, we observe that Gaussian-rendered features and those extracted directly from images exhibit different strengths across the two-stage matching process: the former performs better in the coarse stage, while the latter proves more effective in the fine stage. Therefore, we introduce a hybrid feature matching strategy, enabling more accurate and efficient pose estimation. Extensive experiments on both indoor and outdoor datasets show that SplatHLoc enhances the robustness of visual relocalization, setting a new state-of-the-art.
Figures & tables
Figure 1: SplatHLoc: a novel hierarchical visual relocalization framework based on Feature Gaussian Splatting (FGS). FGS renders color , depth , and feature maps from novel views, which our method exploits to improve the image retrieval and feature matching process. Upon retrieving a reference image, we match it to the query to estimate an initial pose (initial relocalization). We then render views from the estimated pose and iteratively match them to the query to refine the pose (refined relocalization).
Figure 2: An overview of the proposed SplatHLoc framework. Starting from a database of reference images, we build an SfM model to initialize the Gaussian primitives, and then training the FGS map, see Section 3.1 . SplatHLoc follows a hierarchical relocalization pipeline. (a) In the retrieval stage, we propose an adaptive coarse-to-fine viewpoint retrieval strategy. We first perform the coarse retrieval to obtain retrieved images and then use a lightweight feature matcher to perform geometric verification for each query–retrieved image pair. If geometric verification yields fewer inliers than a threshold, we perform the fine viewpoint retrieval to get a fine retrieved pose, see Section 3.2 . (b) In the matching stage, we apply the proposed hybrid feature matching strategy to establish 2D–2D correspondences between the query and the retrieved image, see Section 3.3 . Next, the rendered depth map lifts the 2D–2D matches to 2D–3D. We then estimate an initial pose using RANSAC-PnP. Rendering from the estimated pose and repeating the matching stage enables pose refinement. The overall relocalization process is further discussed in Section 3.4 .
Figure 3: Illustration of the FGS training process. Feature decoder d is introduced to reduce the dimensionality of the rendered feature Frlow for improved efficiency and reduced map size.
Methods
Chess
Fire
Heads
Office
Pumpkin
RedKitchen
Stairs
Avg.↓
Str.
AS (SIFT)
3/0.87
2/1.01
1/0.82
4/1.15
7/1.69
5/1.72
4/1.01
3.71/1.18
HLoc (SP+SG)
2.39/0.84
2.29/0.91
1.13/0.77
3.14/0.92
4.92/1.30
4.22/1.39
5.05/1.41
3.31/1.08
DVLAD+R2D2
2.56/0.88
2.21/0.86
0.98/0.75
3.48/1.00
4.79/1.28
4.21/1.44
4.60/1.27
3.26/1.07
SplatHLoc init (Ours)
1.14/0.36
1.50/0.53
0.98/0.69
3.33/0.66
2.67/0.53
2.51/0.57
4.40/1.14
2.36/0.64
Reg.
DSAC*
0.50/0.17
0.78/0.29
0.50/0.34
1.19/0.35
1.19/0.29
0.72/0.21
2.65/0.78
1.07/0.35
ACE
0.55/0.18
0.83/0.33
0.53/0.33
1.05/0.29
1.06/0.22
0.77/0.21
2.89/0.81
1.10/0.34
Table 1: Relocalization results on 7-Scenes dataset. We report the median translation (cm) and rotation (°) errors for each scene. The best and second-best results are bolded and underlined , respectively.
Figure 4: Visualization of the relocalization errors. Each subfigure is divided by a diagonal: the top-right part shows the query image in grayscale, while the bottom-left part shows the rendered image from the estimated pose. The red dashed boxes highlight regions with pronounced visual differences in each column. More visualizations and details are in the supplementary material.
Figure 5: Qualitative comparison of camera pose estimation errors between HLoc [ 48 ] and our proposed SplatHLoc across five scenes from the 7-Scenes dataset. Visualizations of the remaining two scenes and more details are provided in the supplementary material. For each scene, we visualize the reconstructed point cloud map together with the trajectory of query images. Trajectory colors denote position error, while the color bar below shows rotation errors, with numbers indicating image indices.
Methods
Avg. Err [cm/°] ↓
R @[2cm, 2°] ↑
DSAC* ( T-PAMI’21 )
0.5 /0.25
96.7
Marepo ( CVPR’24 )
2.1/1.04
50.4
Marepo+GS-CPR ( ICLR’25 )
0.7/0.28
90.9
ACE ( CVPR’23 )
0.7/0.26
97.2
ACE+GS-CPR ( ICLR’25 )
0.5/0.21
98.7
SplatHLoc init (Ours)
1.34/0.37
71.3
Table 2: Relocalization results on the 12-Scenes dataset. We report the median translation (cm) and rotation errors (°), and the percentage of query images below 2 cm, 2° pose error.
Methods
Court
College
Hospital
Shop
Church
Avg.↓
Str.
AS (SIFT)
24/0.13
13/0.22
20/0.36
4 /0.21
8/0.25
14/0.23
HLoc (SP+SG)
18/0.11
11 /0.20
15/0.31
4 /0.20
7/0.22
11/0.21
SplatHLoc init (Ours)
31/0.15
19/0.26
24/0.44
6/0.19
6/0.21
17/0.25
Reg.
DSAC*
33/0.21
18/0.31
21/0.40
5/0.24
15/0.51
19/0.33
ACE
43/0.2
28/0.4
31/0.6
5/0.3
18/0.6
25/0.4
GLACE
19/0.12
19/0.32
18/0.42
4 /0.23
8/0.29
14/0.28
Table 3: Relocalization results on Cambridge Landmarks dataset. We report the median translation (cm) and rotation (°) errors of different methods.
Figure 6: Runtime analysis. We report the average relocalization time per query on the 7-Scenes dataset for STDLoc versus our proposed SplatHLoc. The runtime is divided into two parts: initial pose estimation and iterative pose refinement. Both methods perform four rounds of pose refinement.
Methods
Map Size (MB)
Mapping Time (Min)
GPU Mem. (GB)
STDLoc
904.02
∼ 146
∼ 12
SplatHLoc (Ours)
353.21
∼ 46
∼ 4
Table 4: Mapping analysis. We report the FGS map size, mapping time and peak GPU memory on the Chess scene.
Setups on Stairs
Avg. Err [cm/°] ↓
R @[5cm, 5°] ↑
I (Baseline): MixVPR + M : SP+LG
1.82/0.49
75.5
II: I + Adaptive Retrieval
1.57/0.45
80.5
III: I + M : Hybrid Matcher
1.14/0.33
84.0
IV (Ours): II + M : Hybrid Matcher
1.03/0.30
91.9
Table 5: Ablation study I. We investigate the impact of the proposed Adaptive C2F Retrieval and Hybrid C2F Matching.
Datasets
7-Scenes
Cambridge
Methods
R @[2cm, 2 ∘ ] ↑
Avg. Err [cm/°] ↓
R @[5m, 10 ∘ ] ↑
M : FGS-Matcher
91.46
0.65/0.22
98.28
E : ELoFTR
90.30
0.57/0.19
97.86
M+Ecoarse
89.98 (−0.32)
0.64/0.21
96.72 (−1.14)
M+Efine
93.55 (+3.25)
0.56/0.18
98.73 (+0.87)
J : JamMa
90.93
0.60/0.19
97.78
Table 6: Ablation study II. We ablate the hybrid matcher by isolating rendered features’ coarse and fine contributions. FGS-Matcher is adopted from STDLoc, which uses rendered features in both coarse and fine matching stages.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Scene Type
k1
k2
k3
a
b
I
Indoor
10
150
5
5
0.5
150
Outdoor
10
100
5
5
0.8
300
Appendix
Table I : Hyperparameters in the adaptive retrieval process.
Distributions
Avg. Err [cm/°] ↓
R @[5cm, 5°] ↑
Time (s) ↓
Uniform
1.11/0.33
85.3
0.60
Normal
1.03/0.30
92.8
0.60
Random
1.03/0.30
91.9
0.57
Appendix
Table II: Comparison of Uniform, Normal, and Random perturbation strategies. Time denotes the average runtime required for the initial localization.
Figure I : The R@[5cm, 5°] accuracy under different perturbation angle values a , with the perturbation distance b fixed to 0.5 m .
Figure II : The R@[5cm, 5°] accuracy under different perturbation distance values b , with the perturbation angle a fixed to 5°.
Figure III : The R@[5cm, 5°] accuracy and the average initial relocalization time under different values of k2 .
Figure IV : Qualitative visualization of the Adaptive Viewpoint Retrieval process. Green lines denote the inlier correspondences that pass geometric verification, whereas the other matches are visualized in blue. (a) Coarse retrieval often yields candidates with limited co-visibility and low inlier counts. (b) By synthesizing pose-perturbed virtual views and performing fine retrieval, our method significantly increases co-visibility, resulting in substantially more inliers.
Table 19
Figure V : Additional comparison of camera pose estimation errors for the 7-Scenes dataset.
Figure VI : Visualization of the failure cases. (a) The first row illustrates a failure case caused by poor rendering quality of the Gaussian map near sparsely observed viewpoints. (b) The second row shows a failure induced by repetitive structures in the scene.
Figure VII : Qualitative visualization of the relocalization errors on the Cambridge Landmarks dataset. For each image pair, the upper-left triangle shows the query image, while the lower-right triangle displays the rendered image obtained from the final estimated pose. Better visual alignment between the two indicates that our SplatHLoc achieves more accurate relocalization.
Figure VIII : Qualitative visualization of the relocalization errors on the 7-Scenes dataset. For each image pair, the upper-left triangle shows the query image, while the lower-right triangle displays the rendered image obtained from the final estimated pose. Better visual alignment between the two indicates that our SplatHLoc achieves more accurate relocalization.
We introduce GSVisLoc, a visual localization method designed for 3D Gaussian Splatting (3DGS) scene representations. Given a 3DGS model of a scene and a query image, our goal is to estimate the camera's position and orientation. We accomplish this by robustly matching scene features to image features. Scene features are produced by downsampling and encoding the 3D Gaussians while image features are obtained by encoding image patches. Our algorithm proceeds in three steps, starting with coarse matching, then fine matching, and finally by applying pose refinement for an accurate final estimate. Importantly, our method leverages the explicit 3DGS scene representation for visual localization without requiring modifications, retraining, or additional reference images. We evaluate GSVisLoc on both indoor and outdoor scenes, demonstrating competitive localization performance on standard benchmarks while outperforming existing 3DGS-based baselines. Moreover, our approach generalizes effectively to novel scenes without additional training.
Visual localization becomes extremely challenging in planetary-like terrains characterized by low texture, perceptual aliasing, harsh illumination, and sparse, weakly overlapping viewpoints induced by forward rover motion and unconstrained driving directions. Under these conditions, state-of-the-art image-to-image and image-to-map matching pipelines suffer significant performance degradation. In this work, we propose a visual relocalization method that departs from classical correspondence-based pipelines by directly estimating camera poses against a differentiable map representation built with 3D Gaussian Splatting (3DGS). Our key contribution is a geometry-aware training strategy that combines photometric and geometric losses, where the geometric supervision is provided for the first time by combining multi-view stereo (MVS) and LiDAR depths. We show that this joint optimization produces a 3DGS model that better fits the underlying scene geometry, leading to improved photometric and geometric consistency and more robust, accurate single-image 6-DoF pose estimation. Extensive experiments on data acquired in planetary-analog environments validate the effectiveness of our approach, showing substantial gains in relocalization accuracy under challenging conditions. Code is available at https://github.com/DLR-RM/multimodal-gsplat-relocalization.
Maria Peribañez, Javier Civera, Rudolph Triebel +1
University of Zaragoza, Spain · German Aerospace Center (DLR), Institute of Robotics and Mechatronics · Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
Indoor visual relocalization plays a critical role in emerging spatial and embodied AI applications. However, prior research was predominantly devoted to low-level vision schemes, struggling to perceive scene semantics and compositions, which limits both interpretability and applicability. In this paper, we explore the issue of how to organize rich object information in a scene, including semantics, layout, and geometry, into a structured map representation, thereby utilizing object units exclusively to drive the camera relocalization task. To this end, we propose OpenReLoc, a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities. Leveraging recent foundation models, we first introduce a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching. Additionally, we design object-oriented reference frames as position priors, paired with a reference frame selection strategy based on the Distance-IoU (DIOU), enabling extension to scalable scenes. Moreover, to ensure stable and accurate pose optimization, we also propose a dual-path 2D Iterative Closest Pixel loss guided by object shape. Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets. Our source code will be released upon acceptance.
Zhaopeng Cui, Jiarui Hu, Jingbo Liu +7
1State Key Lab of CAD&CG, Zhejiang University · 2Ant Group