While structure-based relocalizers have long strived for point correspondences when establishing or regressing query-map associations, in this paper, we pioneer the use of planar primitives and 3D planar maps for lightweight 6-DoF camera relocalization in structured environments. Planar primitives, beyond being fundamental entities in projective geometry, also serve as region-based representations that encapsulate both structural and semantic richness. This motivates us to introduce PlanaReLoc, a streamlined plane-centric paradigm where a deep matcher associates planar primitives across the query image and the map within a learned unified embedding space, after which the 6-DoF pose is solved and refined under a robust framework. Through comprehensive experiments on the ScanNet and 12Scenes datasets across hundreds of scenes, our method demonstrates the superiority of planar primitives in facilitating reliable cross-modal structural correspondences and achieving effective camera relocalization without requiring realistically textured/colored maps, pose priors, or per-scene training. The code and data are available at https://github.com/3dv-casia/PlanaReLoc .
Figures & tables
Figure 2: Overview of the planar primitive embedding ( Sec. 3.1 ) and matching ( Sec. 3.2 ) pipeline between the query and the map. (a) The query image is first reconstructed into a set of 3D planar primitives via a frozen monocular plane recovery module, with each primitive further encoded into a plane embedding by aggregating visual features within its corresponding 2D segment. (b) Each map primitive is encoded by an object encoder and a scene encoder, capturing both its shape and pose features. (c) The two sets of embeddings are fed into a stack of transformer layers to produce a soft assignment matrix, from which plane correspondences are inferred.
Figure 3: Pose refinement via per-primitive depth alignment. Given the optimization variables—the offset seed δi and Ttr —the query primitive Πiq is warped onto the depth rendering D via Eq. 8 . Then, the depth alignment error is computed in Eq. 9 .
Map trunc.
Coarse init.
Map ap- pearance
ΔR ( \SIUnitSymbolDegree ) ↓
Δt ( m ) ↓
Pose Recall ( % ) ↑
Time ( s−1 )
Mean
Med.
Mean
Med.
( 0.2m , 10\SIUnitSymbolDegree )
( 0.5m , 15\SIUnitSymbolDegree )
( 1.0m , 30\SIUnitSymbolDegree )
Coarse Init.
✓
32.7
28.7
1.00
0.94
0.4
5.3
33.9
-
Table 1: Relocalization results on ScanNet. For each method, a “✓” denotes the use of auxiliary map truncation, coarse pose initialization, or realistic map appearance. We report rotation and translation errors, along with pose recalls—the ratio of successfully localized queries across thresholds. Top-3 results are highlighted as the first , second , and third . We also provide the average runtime per query.
Feature type
ScanNet
12Scenes
Prec. ↑
Rec. ↑
F 1↑
AP ↑
# TP
# GT
Prec. ↑
Rec. ↑
F 1↑
AP ↑
# TP
# GT
GeoTransformer-T [ 86 ]
Point
30.8
22.8
26.2
38.5
13026
57253
20.3
16.8
18.4
28.0
1565
9319
FreeReg [ 118 ]
Point
21.7
19.2
20.4
34.1
7837
40857
20.2
14.1
16.6
21.5
1077
7647
MASt3R [ 55 ]
Point ∙
61.7
45.0
52.0
84.1
18372
40857
59.8
42.9
50.0
81.6
3283
7647
MatchAnything [ 37 ]
Point
42.1
48.2
45.0
67.7
19698
40857
51.2
56.1
53.5
77.2
4289
7647
NOPE-SAC [ 110 ]
Plane ∙
51.4
35.4
41.9
79.1
14462
40857
43.8
22.0
29.3
71.4
1684
7647
Table 2: Matching evaluation with IoU score ≥0.3 . Point correspondences are first lifted to plane matches via majority voting. # TP and # GT denote the total number of true positives and ground-truth correspondences, respectively. ∙ indicates reliance on visual appearance.
Med. Err. ↓
Pose Recall ( % ) ↑
ΔR ( \SIUnitSymbolDegree )
Δt ( m )
( 0.2m , 10\SIUnitSymbolDegree )
( 0.5m , 15\SIUnitSymbolDegree )
( 1.0m , 30\SIUnitSymbolDegree )
Coarse Init.
22.5
0.47
0.5
18.0
69.8
GeoTr.-T [ 86 ]
33.2
0.80
22.2
37.8
43.5
FreeReg [ 118 ]
23.7
0.49
1.7
17.5
64.2
SP + LG [ 24 , 61 ]
43.9
0.96
10.4
17.4
34.0
LoFTR [ 107 ]
31.4
0.62
31.9
40.2
48.8
Table 3: Relocalization results on 12Scenes.
Figure 6
Figure 5: Impact of plane richness. Results with post-refinement. Three thresholds from coarse to fine are referred to as: T 1 , T 2 , and T 3 .
Figure 6: Qualitative examples. (c) Plane correspondences are color-coded, with true positives outlined in green and false ones in red. (d) Camera poses relocalized by different methods are compared from two viewpoints. Legend: the ground truth , PlanaReLoc (Ours) , GeoTransformer-T , Coarse Init. , MASt3R , NOPE-SAC . See the appendix for additional visualizations on both datasets.
F 1 ( % ) ↑
Med. Err. ↓
Time ( ms−1 )
ΔR ( \SIUnitSymbolDegree )
Δt ( m )
PlaneTR [ 109 ]
65.8
6.2
0.42
∼52.659
PlaneRecTR [ 100 ]
63.6
4.9
0.31
∼58.033
ZeroPlane [ 67 ]
66.0
4.0
0.41
∼293.71000000000004
Plana3R [ 63 ]
61.4
3.7
0.28
∼2781.82
MoGe-2 [ 121 ] +RANSAC
64.3
3.9
0.27
∼59.854
Table 5: Results ( w/o refine. ) with plane recovery alternatives on ScanNet. MoGe-2+RANSAC is used in our default pipeline.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Statistics of 1513 3D planar maps constructed from ScanNet. Left y-axis: distribution of 3D planar maps grouped by their number of planar primitives. Right y-axis: average storage footprint of the colored and simplified map versions in each bin.
Figure 8: Visual overlap analysis for the map truncation in (a) and the oracle coarse initialization strategy in (b) and (c).
Figure 9: Architecture of the front-end network on the query side. We retrofit the monocular geometry estimation model MoGe-2 [ 121 ] with an additional head to encode dense features for 2D plane embedding.
Figure 10: Camera relocalization results on ScanNet and 12Scenes datasets. We plot cumulative accuracy curves that show the proportion of correctly localized frames as a function of varying translation and rotation error thresholds.
Plane Matching ( % ) ↑
Pose Recall ↑
Time ( ms−1 )
Prec.
Rec.
F 1
(1) ResNet50 [ 36 ]
62.3
56.9
59.5
34.2
∼63.86899999999999
(2) DINOv2 [ 80 ]
65.5
59.6
62.4
35.5
∼94.979
↪ (3) w/ Conv. head
66.3
60.3
63.1
36.2
∼97.079
Ours
67.6
61.3
64.3
37.1
∼59.854
Appendix
Table 6: Ablation study on 2D encoder variants. Experiments are conducted on ScanNet with no post-refinement.
Figure 11: Impact of the Query Primitive Size on Matching Performance. Left y-axis: the size distribution of all detected query primitives, where size is defined as the percentage of the image area occupied. Right y-axis: the matching precision for each size bin.
Methods
Map Type
Map Size ↓
Mapping Time ↓
Visual Cues
Pose Prior
Chess
Fire
Heads
Office
Pumpkin
Kitchen
Stairs
Avg. ↓ ( cm / \SIUnitSymbolDegree )
PoseNet17 [ 50 ]
Network
50MB
4h24h
✓
13/4.5
27/11.3
17/13.0
19/5.6
26/4.8
23/5.4
35/12.4
23/8.1
HLoc(SP+SG) [ 93 ]
SfM points
∼2GB
∼1.5h
✓
✓
2/0.8
2/0.9
1/0.8
3/0.9
5/1.3
4/1.4
5/1.5
3/1.1
GoMatch(SP) [ 140 ]
SfM points
∼56MB
∼1.5h
✓
4/1.6
12/3.7
5/3.4
7/1.8
8/5.7
14/3.0
58/13.1
18/4.6
ACE [ 9 ]
Network
5MB
5min
✓
0.6/0.2
0.8/0.3
0.5/0.3
1/0.3
1/0.2
0.8/0.2
3/0.8
1/0.3
Reloc3r [ 25 ]
Ref. images
∼1.6GB
7s
✓
✓
3/0.9
3/0.8
1/1.0
4/0.9
6/1.1
4/1.3
7/1.3
4/1.0
STDLoc [ 43 ]
3DGS
∼0.8GB
∼2h
✓
0.5/0.2
0.6/0.2
0.4/0.3
1/0.2
1/0.2
0.6/0.2
1/0.4
0.8/0.2
Appendix
Table 7: Camera relocalization results on the 7Scenes dataset. We report median position and rotation errors in centimeters ( cm ) and degrees ( \SIUnitSymbolDegree ), respectively. We summarize the map type, map size, the time needed for mapping, and whether mapping and localization rely on visual appearance or pose priors ( e.g . , image retrieval). Results of other methods are taken from the literature.
Figure 12: Qualitative examples on 12Scenes. Correspondences in (c) are color-coded, with true positives outlined in green and false ones in red. Legend for different relocalizers in (d): the ground truth , PlanaReLoc (Ours) , GeoTransformer-T , Coarse Init. , MASt3R , NOPE-SAC .
Figure 13: More qualitative examples on ScanNet. Correspondences in (c) are color-coded, with true positives outlined in green and false ones in red. Legend for different relocalizers in (d): the ground truth , PlanaReLoc (Ours) , GeoTransformer-T , Coarse Init. , MASt3R , NOPE-SAC .
Figure 14: A representative case where PlanaReLoc underperforms. Despite four out of six primitives being correctly matched (colored in green ), the pose estimation framework fails to reject matching outliers (colored in orange ) due to the perfectly repeated pattern (compare the query image with the colored rendering from the predicted pose).
Figure 15: Relocalization results on Integrated Rooms. Following [ 11 ] , we arrange scenes in 12Scenes inside a 2D grid with a cell size of 5m and integrate varying numbers of adjacent scenes to form larger maps. As the integration size increases, PlanaReLoc’s performance degrades due to increased ambiguities and the scene encoder’s limited capacity for larger maps.
Indoor visual relocalization plays a critical role in emerging spatial and embodied AI applications. However, prior research was predominantly devoted to low-level vision schemes, struggling to perceive scene semantics and compositions, which limits both interpretability and applicability. In this paper, we explore the issue of how to organize rich object information in a scene, including semantics, layout, and geometry, into a structured map representation, thereby utilizing object units exclusively to drive the camera relocalization task. To this end, we propose OpenReLoc, a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities. Leveraging recent foundation models, we first introduce a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching. Additionally, we design object-oriented reference frames as position priors, paired with a reference frame selection strategy based on the Distance-IoU (DIOU), enabling extension to scalable scenes. Moreover, to ensure stable and accurate pose optimization, we also propose a dual-path 2D Iterative Closest Pixel loss guided by object shape. Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets. Our source code will be released upon acceptance.
Zhaopeng Cui, Jiarui Hu, Jingbo Liu +7
1State Key Lab of CAD&CG, Zhejiang University · 2Ant Group
Relative camera pose is a basic geometric cue for visual re-localization and scene understanding. When using images alone, visual evidence may not sufficiently support reliable estimation of 3D-consistent motion. The challenge is especially acute in unfamiliar and ever-changing environments for real-time applications, where a system must infer spatial structure under tight time constraints from what it sees rather than rely on scene-specific reconstruction or training. This raises a fundamental question: how can live 3D camera motion be estimated accurately for re-localization in unseen environments while drawing on enough scene context to resolve pose ambiguity? We introduce MultiLoc, a multi-view-guided relative pose regressor trained at scale to achieve spatial and geometric consistent representations for robust visual re-localization in unseen scenarios with high inference speed. Specifically, MultiLoc creates a minimal 3D-sub-scene representation of the environment and efficiently transforms it into 3D spatially and geometrically consistent features in a single forward pass, enabling precise pose estimates with high inference speed gains. Across diverse indoor, outdoor, and in-the-wild visual re-localization benchmarks---Indoor6, Cambridge Landmarks and WaySpots---MultiLoc consistently outperforms state-of-the-art relative pose regression methods. We also highlight that MultiLoc, a pose regressor, also performs competitively with inference-heavy and structure-based approaches while retaining sub-second inference and generalizing to unseen scenes. Given a small posed support set, it also surpasses relative pose regression, feature-matching, and non-regression methods on several relative camera pose benchmarks. Code will be released.
Visual localization -- estimating a camera pose within a pre-existing map -- is a fundamental problem in computer vision. Floorplans are an attractive map representation: they are readily available for most buildings, compact, and inherently invariant to visual appearance changes. However, bridging the severe domain gap between camera observations and floorplan geometry remains challenging. Existing methods address this gap through data-driven learning, yet they require large-scale training data and environment-specific retraining, limiting their practical deployment. We propose a zero-shot floorplan localization method that generalizes to novel environments without any retraining. Our key insight is that dominant geometric primitives -- lines and circles -- are ubiquitous in human-made environments and provide appearance-invariant structural constraints. We extract these primitives from a bird's-eye-view (BEV) projection of monocular 3D reconstructions and match them to the floorplan via dedicated minimal solvers within a robust estimation framework. Experiments on both simulated and real-world datasets show that our approach outperforms state-of-the-art learning-based methods on unseen environments, while using a single fixed set of hyperparameters across all experiments. The source code will be made publicly available.
Ayumi Umemura, Toshinori Kuwahara, Marc Pollefeys +1