$L^3$:Scene-agnostic Visual Localization in the Wild

Yu Zhang; Muhua Zhu; Yifei Xue; Tie Ji; Yizhen Lao

cs.CVMar 9, 2026

$L^3$ :Scene-agnostic Visual Localization in the Wild

Authors: Yu Zhang, Muhua Zhu, Yifei Xue, Tie Ji, Yizhen Lao

Organizations: Hunan University, Changsha, Hunan 410082, China

Abstract

Standard visual localization methods typically require offline pre-processing of scenes to obtain 3D structural information for better performance. This inevitably introduces additional computational and time costs, as well as the overhead of storing scene representations. Can we visually localize in a wild scene without any off-line preprocessing step? In this paper, we leverage the online inference capabilities of feed-forward 3D reconstruction networks to propose a novel map-free visual localization framework $L^3$ . Specifically, by performing direct online 3D reconstruction on RGB images, followed by two-stage metric scale recovery and pose refinement based on 2D-3D correspondences, $L^3$ achieves high accuracy without the need to pre-build or store any offline scene representations. Extensive experiments demonstrate $L^3$ not only that the performance is comparable to state-of-the-art solutions on various benchmarks, but also that it exhibits significantly superior robustness in sparse scenes (fewer reference images per scene).

Figures & tables

Figure 2: Comparison of dense and sparse scenes. Green denotes ground truth query poses and references are cyan. The predicted poses by different methods are shown in different colors: red for ACE [ 4 ] , magenta for ACE+GS-CPR [ 27 ] , and blue for our proposed $L^{3}$ . Dense Scene: Localization using all 1000 reference images. Sparse Scene: We sampled the 1000 reference images and retain only 20. Our $L^{3}$ significantly outperforms other baselines in this challenging case.

Figure 3: Overview of $L^{3}$ . Coarse Localization: Given a query image $I_{q}$ and retrieved references $\{I_{r,i}\}_{i=1}^{K}$ , we first perform feed-forward 3D reconstruction to predict local point clouds $\mathcal{P}_{r}^{local}$ , query pose $\mathbf{P}_{q}^{local}$ and reference poses $\mathbf{P}_{r}^{local}$ . A scale estimation module then computes the scale factor $S$ via a two-stage strategy to initialize the pose $\mathbf{P}_{q}^{\text{init}}$ . Pose Refinement: The pose is refined via structure optimization and PnP to yield the final 6-DoF pose $\mathbf{P}_{q}$ . Output: We compare the query image (top-left) with its rendering under the predicted pose (bottom-right), separated by a diagonal line.

Figure 4: Scale Estimation Strategy. Stage 1: Absolute depths are triangulated using ground truth (GT) reference poses. If the median ratio $S_{tri}$ between GT and local depths (from $\mathcal{P}_{r}^{local}$ ) has a deviation $d_{tri}$ below a threshold, $S_{tri}$ is adopted. Stage 2: Otherwise, we align the predicted trajectory $\textbf{P}^{local}_{r}$ with the GT trajectory $\textbf{P}^{GT}_{r}$ via rotation $R_{\text{align}}$ . A RANSAC scheme then yields the scale $S_{traj}$ by minimizing Euclidean distance error. $S_{traj}$ is accepted if $d_{traj}$ < $d_{tri}$ (fallback to $S_{tri}$ otherwise).

	Methods	Chess	Fire	Heads	Office	Pumpkin	Redkitchen	Stairs	Avg. $\downarrow$ [cm/ ∘ ]
APR	PoseNet [ 21 ] 1 1 1 These data are cited from NeFeS [ 11 ] .	10/4.02	27/10.0	18/13.0	17/5.97	19/4.67	22/5.91	35/10.5	21/7.74
	DFNet [ 13 ] 1 1 1 These data are cited from NeFeS [ 11 ] .	3/1.12	6/2.30	4/2.29	6/1.54	7/1.92	7/1.74	12/2.63	6/1.93
	Marepo [ 12 ] 2 2 2 These data are cited from GS-CPR [ 27 ] .	1.9/0.83	2.3/0.92	2.1/1.24	2.9/0.93	2.5/0.88	2.9/0.98	5.9/1.48	2.9/1.04
SCR	DSAC* [ 7 ] 2 2 2 These data are cited from GS-CPR [ 27 ] .	0.5 /0.17	0.8/0.28	0.5/0.34	1.2/0.34	1.2/0.28	0.7 /0.21	2.7/0.78	1.1/0.34
	ACE [ 4 ]	0.5 /0.19	0.8/0.33	0.5/0.33	1.1/0.29	1.1/ 0.22	0.8/0.20	2.9/0.80	1.1/0.34
	GLACE [ 43 ]	0.6/0.18	0.9/0.34	0.6/0.35	1.0/0.29	0.9 /0.23	0.8/0.20	3.1/0.95	1.1/0.36

Table 1: Results on 7Scenes dataset. We compare the median translation and rotation errors (cm/ ∘ ) of different methods, lower is better. Best results are in bold .

APR	Marepo [ 12 ] 2 2 2 These data are cited from GS-CPR [ 27 ] .	2.1/1.04	95	$-$
	Methods	Avg. $\downarrow$ [cm/ ∘ ]	Avg. $\uparrow$ [5cm, 5 ∘ ]	Avg. $\uparrow$ [1cm, 1 ∘ ]
SCR	ACE [ 4 ]	0.7/0.26	100	75.1
SCR	GLACE [ 43 ]	0.7/0.25	100	78.7
NVS	ACE + GS-CPR [ 27 ]	0.5/0.21	100	86.7
	$L^{3}$ (ours)	0.4/0.19	99.5	86.7

Table 2: Results on 12Scenes dataset. We compare the median translation and rotation errors (cm/ ∘ ) of different methods, and the average accuracy of frames under a specified threshold. Best results are in bold .

	Methods	Kings	Hospital	Shop	Church	Avg. $\downarrow$ [cm/ ∘ ]
FM	AS (SIFT) [ 36 ]	13/ 0.2	20/0.4	4/0.2	8/0.3	11/0.28
	HLoc (SP+SG) [ 15 , 35 ]	12/ 0.2	15/ 0.3	4/0.2	7/0.2	10/0.23
	ImLoc [ 19 ]	11/0.2	14/0.3	4/0.2	7/0.2	9/0.23
APR	LENS [ 32 ]	33/0.5	44/0.9	27/1.6	53/1.6	39/1.15
	DFNet [ 13 ]	73/2.37	200/2.98	67/2.21	137/4.03	119/2.90
	PMNet [ 25 ]	68/1.97	103/1.31	58/2.10	133/3.73	90/2.27

Table 3: Results on Cambridge Landmarks dataset . We compare the median translation and rotation errors (cm/ ∘ ) of different methods. Best results are in bold .

Datasets	Methods	Dense	Sparse
Datasets	Methods	Dense	N=30	N=20	N=10	N=5
7Scenes	ACE [ 4 ]	1.1/0.34	2.7/0.65	3.2/0.79	9.2/1.53	348.7/4.78
	ACE + GS-CPR [ 27 ]	0.8/0.25	1.5/0.42	1.6/0.46	5.2/1.33	$-$
	${L^{3}}_{\text{coarse}}$ (ours)	2.5/0.71	3.7/1.02	4.3/1.17	5.7/1.52	6.4/1.65
	$L^{3}_{\text{w/o. Optim}}$ (ours)	2.5/0.67	2.8/0.79	3.1/0.84	5.1/1.42	6.5/1.66
	$L^{3}$ (ours)	1.3/0.41	1.9/0.56	2.3/0.63	4.5/1.26	5.9/1.61

Table 4: Sparse view localization on the 7Scenes and 12Scenes datasets. We report the mean of median translation and rotation errors (cm/ ∘ ) across all subscenes.

N

is the number of sampled reference images. ”-” indicates localization failure in at least one subscene. Best results are in bold .

Methods	Dense	Sparse
Methods	Dense	N=60	N=50	N=40	N=30
ACE [ 4 ]	21/0.47	83/1.70	122/2.85	111/2.43	261/5.65
ACE + GS-CPR [ 27 ]	15/0.33	41/1.10	72/2.13	88/5.41	350/8.63
${L^{3}}_{\text{coarse}}$ (ours)	36/0.54	42/0.64	40/0.64	56/0.68	54/0.76
$L^{3}_{\text{w/o. Optim}}$ (ours)	15/0.33	23/0.46	20/0.53	23/0.47	34/0.66
$L^{3}$ (ours)	11/0.27	18/0.39	17/0.37	20/0.42	25/0.53

Table 5: Results of sparse view localization on the Cambridge dataset. We compare the mean of the median translation and rotation errors (cm/ ∘ ) for different methods across all subscenes. Best results are in bold .

Panel of Figure 5 — Figure 5: Robustness analysis under increasing sparsity. We plot the log-scale translation error growth relative to the dense setting on three datasets. ” ✕ ” indicates localization failure beyond that sparsity level.

Panel of Figure 6 — Figure 6: Ablation of scale estimation strategies on 12Scenes dataset. We compare our full method in Sec. 3.1 ( blue line ) against variants using only stage 1 (local geometric consistency, red line ) or stage 2 (global trajectory constraints, yellow line ).

Panel of Figure 7 — Figure 7: Comparison of localization errors. Position (trajectories) and rotation (bottom bars, x-axis denotes frame indices) errors are colored by magnitude for ACE [ 4 ] , GS-CPR [ 27 ] , and our $L^{3}$ . (a) Outdoor StMarysChurch (Cambridge Landmarks). (b) Indoor Office2_5b (12Scenes). Both panels compare dense (top) and sparse (bottom, $N=30$ ) settings.

Metrics	ACE [ 4 ]	GS-CPR [ 27 ]	$L^{3}$ (ours)
Pre-processing time (min) $\downarrow$	2	31	0.6
Inference time (s) $\downarrow$	0.02	0.26	2.1
Storage (MB) $\downarrow$	4	203	0
Avg. Error [cm/ ∘ ] $\downarrow$	21/0.47	15/0.33	11/0.27

Table 6: Time comparison on the Cambridge Landmarks dataset. We only need top retrieval. Best results are in bold .

Robot Localization

Robot localization estimates a robot’s position and orientation from onboard sensors, maps, and observations of its surroundings.

See all 173 papers

Memory Over Maps: 3D Object Localization Without Reconstruction
Mar 20, 2026Rui Zhou, Xander Yap, Jianwen Cao +33D Visual Grounding Robot Localization
University of Zurich · ETH Zurich · Microsoft
MultiLoc: Look Around As You Localize for Fast And Robust Visual Re-localization
Mar 28, 2026Nobel Dang, Bing LiRelative Pose Estimation Camera Pose Estimation
Clemson University, SC, USA
VGG-T $^3$ : Offline Feed-Forward 3D Reconstruction at Scale
Feb 26, 2026Sven Elflein, Ruilong Li, Sérgio Agostinho +43D Reconstruction Feed-Forward 3D Reconstruction
NVIDIA · Vector Institute · University of Toronto

Mar 20, 2026J/K move · Enter open · S save

Memory Over Maps: 3D Object Localization Without Reconstruction

Rui Zhou, Xander Yap, Jianwen Cao, Allison Lau+2

University of Zurich · ETH Zurich · Microsoft

3D Visual Grounding Robot Localization Object-Goal Navigation

Open paper AskPDF arXiv

Target localization is a prerequisite for embodied tasks such as navigation and manipulation. Conventional approaches rely on constructing explicit 3D scene representations to enable target localization, such as point clouds, voxel grids, or scene graphs. While effective, these pipelines incur substantial mapping time, storage overhead, and scalability limitations. Recent advances in vision-language models suggest that rich semantic reasoning can be performed directly on 2D observations, raising a fundamental question: is a complete 3D scene reconstruction necessary for object localization? In this work, we revisit object localization and propose a map-free pipeline that stores only posed RGB-D keyframes as a lightweight visual memory--without constructing any global 3D representation of the scene. At query time, our method retrieves candidate views, re-ranks them with a vision-language model, and constructs a sparse, on-demand 3D estimate of the queried target through depth backprojection and multi-view fusion. Compared to reconstruction-based pipelines, this design drastically reduces preprocessing cost, enabling scene indexing that is over two orders of magnitude faster to build while using substantially less storage. We further validate the localized targets on downstream object-goal navigation tasks. Despite requiring no task-specific training, our approach achieves strong performance across multiple benchmarks, demonstrating that direct reasoning over image-based scene memory can effectively replace dense 3D reconstruction for object-centric robot navigation. Project page: https://ruizhou-cn.github.io/memory-over-maps/

Abstract

Robot Localization

Explore similar work

Memory Over Maps: 3D Object Localization Without Reconstruction

MultiLoc: Look Around As You Localize for Fast And Robust Visual Re-localization

VGG-T3^33: Offline Feed-Forward 3D Reconstruction at Scale

VGG-T $^3$ : Offline Feed-Forward 3D Reconstruction at Scale