Hierarchical Visual Relocalization with Nearest View Synthesis from Feature Gaussian Splatting

Huaqi Tao; Bingxi Liu; Guangcheng Chen; Fulin Tang; Li He; Hong Zhang

cs.CVMar 31, 2026

Hierarchical Visual Relocalization with Nearest View Synthesis from Feature Gaussian Splatting

Authors: Huaqi Tao, Bingxi Liu, Guangcheng Chen, Fulin Tang, Li He, Hong Zhang

Organizations: Southern University of Science and Technology, Shenzhen, China. · Institute of Automation, Chinese Academy of Sciences, Beijing, China.

Abstract

Visual relocalization is a fundamental task in the field of 3D computer vision, estimating a camera's pose when it revisits a previously known scene. While point-based hierarchical relocalization methods have shown strong scalability and efficiency, they are often limited by sparse image observations and weak feature matching. In this work, we propose SplatHLoc, a novel hierarchical visual relocalization framework that uses Feature Gaussian Splatting as the scene representation. To address the sparsity of database images, we propose an adaptive viewpoint retrieval method that synthesizes virtual candidates with viewpoints more closely aligned with the query, thereby improving the accuracy of initial pose estimation. For feature matching, we observe that Gaussian-rendered features and those extracted directly from images exhibit different strengths across the two-stage matching process: the former performs better in the coarse stage, while the latter proves more effective in the fine stage. Therefore, we introduce a hybrid feature matching strategy, enabling more accurate and efficient pose estimation. Extensive experiments on both indoor and outdoor datasets show that SplatHLoc enhances the robustness of visual relocalization, setting a new state-of-the-art.

Figures & tables

Figure 1: SplatHLoc: a novel hierarchical visual relocalization framework based on Feature Gaussian Splatting (FGS). FGS renders color , depth , and feature maps from novel views, which our method exploits to improve the image retrieval and feature matching process. Upon retrieving a reference image, we match it to the query to estimate an initial pose (initial relocalization). We then render views from the estimated pose and iteratively match them to the query to refine the pose (refined relocalization).

Figure 2: An overview of the proposed SplatHLoc framework. Starting from a database of reference images, we build an SfM model to initialize the Gaussian primitives, and then training the FGS map, see Section 3.1 . SplatHLoc follows a hierarchical relocalization pipeline. (a) In the retrieval stage, we propose an adaptive coarse-to-fine viewpoint retrieval strategy. We first perform the coarse retrieval to obtain retrieved images and then use a lightweight feature matcher to perform geometric verification for each query–retrieved image pair. If geometric verification yields fewer inliers than a threshold, we perform the fine viewpoint retrieval to get a fine retrieved pose, see Section 3.2 . (b) In the matching stage, we apply the proposed hybrid feature matching strategy to establish 2D–2D correspondences between the query and the retrieved image, see Section 3.3 . Next, the rendered depth map lifts the 2D–2D matches to 2D–3D. We then estimate an initial pose using RANSAC-PnP. Rendering from the estimated pose and repeating the matching stage enables pose refinement. The overall relocalization process is further discussed in Section 3.4 .

Figure 3: Illustration of the FGS training process. Feature decoder $d$ is introduced to reduce the dimensionality of the rendered feature $F_{r}^{\text{low}}$ for improved efficiency and reduced map size.

	Methods	Chess	Fire	Heads	Office	Pumpkin	RedKitchen	Stairs	Avg.↓
Str.	AS (SIFT)	3/0.87	2/1.01	1/0.82	4/1.15	7/1.69	5/1.72	4/1.01	3.71/1.18
	HLoc (SP+SG)	2.39/0.84	2.29/0.91	1.13/0.77	3.14/0.92	4.92/1.30	4.22/1.39	5.05/1.41	3.31/1.08
	DVLAD+R2D2	2.56/0.88	2.21/0.86	0.98/0.75	3.48/1.00	4.79/1.28	4.21/1.44	4.60/1.27	3.26/1.07
	SplatHLoc ${}_{\text{init}}$ (Ours)	1.14/0.36	1.50/0.53	0.98/0.69	3.33/0.66	2.67/0.53	2.51/0.57	4.40/1.14	2.36/0.64
Reg.	DSAC*	0.50/0.17	0.78/0.29	0.50/0.34	1.19/0.35	1.19/0.29	0.72/0.21	2.65/0.78	1.07/0.35
	ACE	0.55/0.18	0.83/0.33	0.53/0.33	1.05/0.29	1.06/0.22	0.77/0.21	2.89/0.81	1.10/0.34

Table 1: Relocalization results on 7-Scenes dataset. We report the median translation (cm) and rotation (°) errors for each scene. The best and second-best results are bolded and underlined , respectively.

Figure 4: Visualization of the relocalization errors. Each subfigure is divided by a diagonal: the top-right part shows the query image in grayscale, while the bottom-left part shows the rendered image from the estimated pose. The red dashed boxes highlight regions with pronounced visual differences in each column. More visualizations and details are in the supplementary material.

Figure 5: Qualitative comparison of camera pose estimation errors between HLoc [ 48 ] and our proposed SplatHLoc across five scenes from the 7-Scenes dataset. Visualizations of the remaining two scenes and more details are provided in the supplementary material. For each scene, we visualize the reconstructed point cloud map together with the trajectory of query images. Trajectory colors denote position error, while the color bar below shows rotation errors, with numbers indicating image indices.

Methods	Avg. Err [cm/°] $\downarrow$	$R$ @[2cm, 2°] $\uparrow$
DSAC* ( T-PAMI’21 )	0.5 /0.25	96.7
Marepo ( CVPR’24 )	2.1/1.04	50.4
Marepo+GS-CPR ( ICLR’25 )	0.7/0.28	90.9
ACE ( CVPR’23 )	0.7/0.26	97.2
ACE+GS-CPR ( ICLR’25 )	0.5/0.21	98.7
SplatHLoc ${}_{\text{init}}$ (Ours)	1.34/0.37	71.3

Table 2: Relocalization results on the 12-Scenes dataset. We report the median translation (cm) and rotation errors (°), and the percentage of query images below 2 cm, 2° pose error.

	Methods	Court	College	Hospital	Shop	Church	Avg.↓
Str.	AS (SIFT)	24/0.13	13/0.22	20/0.36	4 /0.21	8/0.25	14/0.23
	HLoc (SP+SG)	18/0.11	11 /0.20	15/0.31	4 /0.20	7/0.22	11/0.21
	SplatHLoc ${}_{\text{init}}$ (Ours)	31/0.15	19/0.26	24/0.44	6/0.19	6/0.21	17/0.25
Reg.	DSAC*	33/0.21	18/0.31	21/0.40	5/0.24	15/0.51	19/0.33
	ACE	43/0.2	28/0.4	31/0.6	5/0.3	18/0.6	25/0.4
	GLACE	19/0.12	19/0.32	18/0.42	4 /0.23	8/0.29	14/0.28

Table 3: Relocalization results on Cambridge Landmarks dataset. We report the median translation (cm) and rotation (°) errors of different methods.

Figure 6: Runtime analysis. We report the average relocalization time per query on the 7-Scenes dataset for STDLoc versus our proposed SplatHLoc. The runtime is divided into two parts: initial pose estimation and iterative pose refinement. Both methods perform four rounds of pose refinement.

Methods	Map Size (MB)	Mapping Time (Min)	GPU Mem. (GB)
STDLoc	904.02	$\sim$ 146	$\sim$ 12
SplatHLoc (Ours)	353.21	$\sim$ 46	$\sim$ 4

Table 4: Mapping analysis. We report the FGS map size, mapping time and peak GPU memory on the Chess scene.

Setups on Stairs	Avg. Err [cm/°] $\downarrow$	$R$ @[5cm, 5°] $\uparrow$
I (Baseline): MixVPR + $\mathcal{M}$ : SP+LG	1.82/0.49	75.5
II: I + Adaptive Retrieval	1.57/0.45	80.5
III: I + $\mathcal{M}$ : Hybrid Matcher	1.14/0.33	84.0
IV (Ours): II + $\mathcal{M}$ : Hybrid Matcher	1.03/0.30	91.9

Table 5: Ablation study I. We investigate the impact of the proposed Adaptive C2F Retrieval and Hybrid C2F Matching.

Datasets	7-Scenes		Cambridge
Methods	$R$ @[2cm, 2 ∘ ] $\uparrow$	Avg. Err [cm/°] $\downarrow$	$R$ @[5m, 10 ∘ ] $\uparrow$
$\mathcal{M}$ : FGS-Matcher	91.46	0.65/0.22	98.28
$\mathcal{E}$ : ELoFTR	90.30	0.57/0.19	97.86
$\mathcal{M}+\mathcal{E_{\text{coarse}}}$	89.98 ${}_{\,(-0.32)}$	0.64/0.21	96.72 ${}_{\,(-1.14)}$
$\mathcal{M}+\mathcal{E_{\text{fine}}}$	93.55 ${}_{\,(+3.25)}$	0.56/0.18	98.73 ${}_{\,(+0.87)}$
$\mathcal{J}$ : JamMa	90.93	0.60/0.19	97.78

Table 6: Ablation study II. We ablate the hybrid matcher by isolating rendered features’ coarse and fine contributions. FGS-Matcher is adopted from STDLoc, which uses rendered features in both coarse and fine matching stages.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Scene Type	$k_{1}$	$k_{2}$	$k_{3}$	$a$	$b$	$\mathcal{I}$
Indoor	10	150	5	5	0.5	150
Outdoor	10	100	5	5	0.8	300

Appendix

Table I : Hyperparameters in the adaptive retrieval process.

Distributions	Avg. Err [cm/°] $\downarrow$	$R$ @[5cm, 5°] $\uparrow$	Time (s) $\downarrow$
Uniform	1.11/0.33	85.3	0.60
Normal	1.03/0.30	92.8	0.60
Random	1.03/0.30	91.9	0.57

Appendix

Table II: Comparison of Uniform, Normal, and Random perturbation strategies. Time denotes the average runtime required for the initial localization.

Figure I : The R@[5cm, 5°] accuracy under different perturbation angle values

a

, with the perturbation distance

b

fixed to

0.5\text{ m}

Figure II : The R@[5cm, 5°] accuracy under different perturbation distance values

b

, with the perturbation angle

a

fixed to 5°.

Figure III : The R@[5cm, 5°] accuracy and the average initial relocalization time under different values of

k_{2}

Figure IV : Qualitative visualization of the Adaptive Viewpoint Retrieval process. Green lines denote the inlier correspondences that pass geometric verification, whereas the other matches are visualized in blue. (a) Coarse retrieval often yields candidates with limited co-visibility and low inlier counts. (b) By synthesizing pose-perturbed virtual views and performing fine retrieval, our method significantly increases co-visibility, resulting in substantially more inliers.

Table 19

Figure V : Additional comparison of camera pose estimation errors for the 7-Scenes dataset.

Figure VI : Visualization of the failure cases. (a) The first row illustrates a failure case caused by poor rendering quality of the Gaussian map near sparsely observed viewpoints. (b) The second row shows a failure induced by repetitive structures in the scene.

Figure VII : Qualitative visualization of the relocalization errors on the Cambridge Landmarks dataset. For each image pair, the upper-left triangle shows the query image, while the lower-right triangle displays the rendered image obtained from the final estimated pose. Better visual alignment between the two indicates that our SplatHLoc achieves more accurate relocalization.

Figure VIII : Qualitative visualization of the relocalization errors on the 7-Scenes dataset. For each image pair, the upper-left triangle shows the query image, while the lower-right triangle displays the rendered image obtained from the final estimated pose. Better visual alignment between the two indicates that our SplatHLoc achieves more accurate relocalization.

Camera Pose Estimation

Camera pose estimation recovers a camera’s position and orientation from images, visual features, scene geometry, and sometimes sensor measurements.

See all 241 papers

GSVisLoc: Generalizable Visual Localization for Gaussian Splatting Scene Representations
Aug 25, 2025Fadi Khatib, Dror Moran, Guy Trostianetsky +3Visual Place Recognition Camera Pose Estimation
Weizmann Institute of Science · NVIDIA
Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis
Jul 24, 2026Maria Peribañez, Javier Civera, Rudolph Triebel +13D Gaussian Splatting Camera Pose Estimation
University of Zaragoza, Spain · German Aerospace Center (DLR), Institute of Robotics and Mechatronics · Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization
Jun 23, 2026Zhaopeng Cui, Jiarui Hu, Jingbo Liu +7Camera Pose Estimation 3D Scene Understanding
1State Key Lab of CAD&CG, Zhejiang University · 2Ant Group

Aug 25, 2025J/K move · Enter open · S save

GSVisLoc: Generalizable Visual Localization for Gaussian Splatting Scene Representations

Fadi Khatib, Dror Moran, Guy Trostianetsky, Yoni Kasten+2

Weizmann Institute of Science · NVIDIA

Visual Place Recognition Camera Pose Estimation Map-Based Localization Robot Localization

Open paper AskPDF arXiv

We introduce GSVisLoc, a visual localization method designed for 3D Gaussian Splatting (3DGS) scene representations. Given a 3DGS model of a scene and a query image, our goal is to estimate the camera's position and orientation. We accomplish this by robustly matching scene features to image features. Scene features are produced by downsampling and encoding the 3D Gaussians while image features are obtained by encoding image patches. Our algorithm proceeds in three steps, starting with coarse matching, then fine matching, and finally by applying pose refinement for an accurate final estimate. Importantly, our method leverages the explicit 3DGS scene representation for visual localization without requiring modifications, retraining, or additional reference images. We evaluate GSVisLoc on both indoor and outdoor scenes, demonstrating competitive localization performance on standard benchmarks while outperforming existing 3DGS-based baselines. Moreover, our approach generalizes effectively to novel scenes without additional training.

Abstract

Camera Pose Estimation

Explore similar work

GSVisLoc: Generalizable Visual Localization for Gaussian Splatting Scene Representations

Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis

Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization