PlanaReLoc: Camera Relocalization in 3D Planar Primitives via Region-Based Structure Matching

Hanqiao Ye; Yuzhou Liu; Yangdong Liu; Shuhan Shen

cs.CVMar 21, 2026

PlanaReLoc: Camera Relocalization in 3D Planar Primitives via Region-Based Structure Matching

Authors: Hanqiao Ye, Yuzhou Liu, Yangdong Liu, Shuhan Shen

Organizations: School of Artificial Intelligence, University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences

Abstract

While structure-based relocalizers have long strived for point correspondences when establishing or regressing query-map associations, in this paper, we pioneer the use of planar primitives and 3D planar maps for lightweight 6-DoF camera relocalization in structured environments. Planar primitives, beyond being fundamental entities in projective geometry, also serve as region-based representations that encapsulate both structural and semantic richness. This motivates us to introduce PlanaReLoc, a streamlined plane-centric paradigm where a deep matcher associates planar primitives across the query image and the map within a learned unified embedding space, after which the 6-DoF pose is solved and refined under a robust framework. Through comprehensive experiments on the ScanNet and 12Scenes datasets across hundreds of scenes, our method demonstrates the superiority of planar primitives in facilitating reliable cross-modal structural correspondences and achieving effective camera relocalization without requiring realistically textured/colored maps, pose priors, or per-scene training. The code and data are available at https://github.com/3dv-casia/PlanaReLoc .

Figures & tables

Figure 2: Overview of the planar primitive embedding ( Sec. 3.1 ) and matching ( Sec. 3.2 ) pipeline between the query and the map. (a) The query image is first reconstructed into a set of 3D planar primitives via a frozen monocular plane recovery module, with each primitive further encoded into a plane embedding by aggregating visual features within its corresponding 2D segment. (b) Each map primitive is encoded by an object encoder and a scene encoder, capturing both its shape and pose features. (c) The two sets of embeddings are fed into a stack of transformer layers to produce a soft assignment matrix, from which plane correspondences are inferred.

Figure 3: Pose refinement via per-primitive depth alignment. Given the optimization variables—the offset seed $\delta_{i}$ and $\mathbf{T}_{\mathrm{tr}}$ —the query primitive $\Pi^{q}_{i}$ is warped onto the depth rendering ${\bm{D}}$ via Eq. 8 . Then, the depth alignment error is computed in Eq. 9 .

Map trunc.

Coarse init.

Map ap- pearance

\Delta{\mathbf{R}}

(

\mathrm{\SIUnitSymbolDegree}

)

\downarrow

\Delta{\mathbf{t}}

(

\mathrm{m}

)

\downarrow

Pose Recall (

\mathrm{\%}

)

\uparrow

Time (

\mathrm{s}\text{\,}{\mathrm{}}^{-1}

)

Mean

Med.

Mean

Med.

(

0.2\text{\,}\mathrm{m}

10\text{\,}\mathrm{\SIUnitSymbolDegree}

)

(

0.5\text{\,}\mathrm{m}

15\text{\,}\mathrm{\SIUnitSymbolDegree}

)

(

1.0\text{\,}\mathrm{m}

30\text{\,}\mathrm{\SIUnitSymbolDegree}

)

Coarse Init.

✓

32.7

28.7

1.00

0.94

0.4

5.3

33.9

Table 1: Relocalization results on ScanNet. For each method, a “✓” denotes the use of auxiliary map truncation, coarse pose initialization, or realistic map appearance. We report rotation and translation errors, along with pose recalls—the ratio of successfully localized queries across thresholds. Top-3 results are highlighted as the first , second , and third . We also provide the average runtime per query.

	Feature type	ScanNet						12Scenes
		Prec. $\uparrow$	Rec. $\uparrow$	F ${}_{\text{1}}$ $\uparrow$	AP $\uparrow$	# TP	# GT	Prec. $\uparrow$	Rec. $\uparrow$	F ${}_{\text{1}}$ $\uparrow$	AP $\uparrow$	# TP	# GT
GeoTransformer-T [ 86 ]	Point	$30.8$	$22.8$	$26.2$	$38.5$	$13\,026$	$57\,253$	$20.3$	$16.8$	$18.4$	$28.0$	$1565$	$9319$
FreeReg [ 118 ]	Point	$21.7$	$19.2$	$20.4$	$34.1$	$7837$	$40\,857$	$20.2$	$14.1$	$16.6$	$21.5$	$1077$	$7647$
MASt3R [ 55 ]	Point ∙	$61.7$	$45.0$	$52.0$	$84.1$	$18\,372$	$40\,857$	$59.8$	$42.9$	$50.0$	$81.6$	$3283$	$7647$
MatchAnything [ 37 ]	Point	$42.1$	$48.2$	$45.0$	$67.7$	$19\,698$	$40\,857$	$51.2$	$56.1$	$53.5$	$77.2$	$4289$	$7647$
NOPE-SAC [ 110 ]	Plane ∙	$51.4$	$35.4$	$41.9$	$79.1$	$14\,462$	$40\,857$	$43.8$	$22.0$	$29.3$	$71.4$	$1684$	$7647$

Table 2: Matching evaluation with IoU score

\geq

0.3

. Point correspondences are first lifted to plane matches via majority voting. # TP and # GT denote the total number of true positives and ground-truth correspondences, respectively. ∙ indicates reliance on visual appearance.

	Med. Err. $\downarrow$		Pose Recall ( $\mathrm{\%}$ ) $\uparrow$
	$\Delta{\mathbf{R}}$ ( $\mathrm{\SIUnitSymbolDegree}$ )	$\Delta{\mathbf{t}}$ ( $\mathrm{m}$ )	( $0.2\text{\,}\mathrm{m}$ , $10\text{\,}\mathrm{\SIUnitSymbolDegree}$ )	( $0.5\text{\,}\mathrm{m}$ , $15\text{\,}\mathrm{\SIUnitSymbolDegree}$ )	( $1.0\text{\,}\mathrm{m}$ , $30\text{\,}\mathrm{\SIUnitSymbolDegree}$ )
Coarse Init.	$22.5$	$0.47$	$0.5$	$18.0$	$69.8$
GeoTr.-T [ 86 ]	$33.2$	$0.80$	$22.2$	$37.8$	$43.5$
FreeReg [ 118 ]	$23.7$	$0.49$	$1.7$	$17.5$	$64.2$
SP + LG [ 24 , 61 ]	$43.9$	$0.96$	$10.4$	$17.4$	$34.0$
LoFTR [ 107 ]	$31.4$	$0.62$	$31.9$	$40.2$	$48.8$

Table 3: Relocalization results on 12Scenes.

Panel of Figure 5 — Figure 5: Impact of plane richness. Results with post-refinement. Three thresholds from coarse to fine are referred to as: T ${}_{\text{1}}$ , T ${}_{\text{2}}$ , and T ${}_{\text{3}}$ .

Figure 6: Qualitative examples. (c) Plane correspondences are color-coded, with true positives outlined in green and false ones in red. (d) Camera poses relocalized by different methods are compared from two viewpoints. Legend: the ground truth , PlanaReLoc (Ours) , GeoTransformer-T , Coarse Init. , MASt3R , NOPE-SAC . See the appendix for additional visualizations on both datasets.

	F ${}_{\text{1}}$ ( $\mathrm{\%}$ ) $\uparrow$	Med. Err. $\downarrow$		Time ( $\mathrm{ms}\text{\,}{\mathrm{}}^{-1}$ )
		$\Delta{\mathbf{R}}$ ( $\mathrm{\SIUnitSymbolDegree}$ )	$\Delta{\mathbf{t}}$ ( $\mathrm{m}$ )
PlaneTR [ 109 ]	$65.8$	$6.2$	$0.42$	$\sim$ $52.659$
PlaneRecTR [ 100 ]	$63.6$	$4.9$	$0.31$	$\sim$ $58.033$
ZeroPlane [ 67 ]	$66.0$	$4.0$	$0.41$	$\sim$ $293.710\,000\,000\,000\,04$
Plana3R [ 63 ]	$61.4$	$3.7$	$0.28$	$\sim$ $2781.82$
MoGe-2 [ 121 ] +RANSAC	$64.3$	$3.9$	$0.27$	$\sim$ $59.854$

Table 5: Results ( w/o refine. ) with plane recovery alternatives on ScanNet. MoGe-2+RANSAC is used in our default pipeline.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Statistics of

1513

3D planar maps constructed from ScanNet. Left y-axis: distribution of 3D planar maps grouped by their number of planar primitives. Right y-axis: average storage footprint of the colored and simplified map versions in each bin.

Figure 8: Visual overlap analysis for the map truncation in (a) and the oracle coarse initialization strategy in (b) and (c).

Figure 9: Architecture of the front-end network on the query side. We retrofit the monocular geometry estimation model MoGe-2 [ 121 ] with an additional head to encode dense features for 2D plane embedding.

Figure 10: Camera relocalization results on ScanNet and 12Scenes datasets. We plot cumulative accuracy curves that show the proportion of correctly localized frames as a function of varying translation and rotation error thresholds.

	Plane Matching ( $\mathrm{\%}$ ) $\uparrow$			Pose Recall $\uparrow$	Time ( $\mathrm{ms}\text{\,}{\mathrm{}}^{-1}$ )
	Prec.	Rec.	F ${}_{\text{1}}$
(1) ResNet50 [ 36 ]	$62.3$	$56.9$	$59.5$	$34.2$	$\sim$ $63.868\,999\,999\,999\,99$
(2) DINOv2 [ 80 ]	$65.5$	$59.6$	$62.4$	$35.5$	$\sim$ $94.979$
$\hookrightarrow$ (3) w/ Conv. head	$66.3$	$60.3$	$63.1$	$36.2$	$\sim$ $97.079$
Ours	$67.6$	$61.3$	$64.3$	$37.1$	$\sim$ $59.854$

Appendix

Table 6: Ablation study on 2D encoder variants. Experiments are conducted on ScanNet with no post-refinement.

Figure 11: Impact of the Query Primitive Size on Matching Performance. Left y-axis: the size distribution of all detected query primitives, where size is defined as the percentage of the image area occupied. Right y-axis: the matching precision for each size bin.

Methods

Map Type

Map Size

\downarrow

Mapping Time

\downarrow

Visual Cues

Pose Prior

Chess

Fire

Heads

Office

Pumpkin

Kitchen

Stairs

Avg.

\downarrow

(

\mathrm{cm}

\mathrm{\SIUnitSymbolDegree}

)

PoseNet17 [ 50 ]

Network

50\text{\,}\mathrm{MB}

4\text{\,}\mathrm{h}24\text{\,}\mathrm{h}

✓

13/4.5

27/11.3

17/13.0

19/5.6

26/4.8

23/5.4

35/12.4

23/8.1

HLoc(SP+SG) [ 93 ]

SfM points

\sim

2\text{\,}\mathrm{GB}

\sim

1.5\text{\,}\mathrm{h}

✓

2/0.8

2/0.9

1/0.8

3/0.9

5/1.3

4/1.4

5/1.5

3/1.1

GoMatch(SP) [ 140 ]

SfM points

\sim

56\text{\,}\mathrm{MB}

\sim

1.5\text{\,}\mathrm{h}

✓

4/1.6

12/3.7

5/3.4

7/1.8

8/5.7

14/3.0

58/13.1

18/4.6

ACE [ 9 ]

Network

5\text{\,}\mathrm{MB}

5\text{\,}\mathrm{min}

✓

0.6/0.2

0.8/0.3

0.5/0.3

1/0.3

1/0.2

0.8/0.2

3/0.8

1/0.3

Reloc3r [ 25 ]

Ref. images

\sim

1.6\text{\,}\mathrm{GB}

7\text{\,}\mathrm{s}

✓

3/0.9

3/0.8

1/1.0

4/0.9

6/1.1

4/1.3

7/1.3

4/1.0

STDLoc [ 43 ]

3DGS

\sim

0.8\text{\,}\mathrm{GB}

\sim

2\text{\,}\mathrm{h}

✓

0.5/0.2

0.6/0.2

0.4/0.3

1/0.2

0.6/0.2

1/0.4

0.8/0.2

Appendix

Table 7: Camera relocalization results on the 7Scenes dataset. We report median position and rotation errors in centimeters (

\mathrm{cm}

) and degrees (

\mathrm{\SIUnitSymbolDegree}

), respectively. We summarize the map type, map size, the time needed for mapping, and whether mapping and localization rely on visual appearance or pose priors ( e.g . , image retrieval). Results of other methods are taken from the literature.

Figure 12: Qualitative examples on 12Scenes. Correspondences in (c) are color-coded, with true positives outlined in green and false ones in red. Legend for different relocalizers in (d): the ground truth , PlanaReLoc (Ours) , GeoTransformer-T , Coarse Init. , MASt3R , NOPE-SAC .

Figure 13: More qualitative examples on ScanNet. Correspondences in (c) are color-coded, with true positives outlined in green and false ones in red. Legend for different relocalizers in (d): the ground truth , PlanaReLoc (Ours) , GeoTransformer-T , Coarse Init. , MASt3R , NOPE-SAC .

Figure 14: A representative case where PlanaReLoc underperforms. Despite four out of six primitives being correctly matched (colored in green ), the pose estimation framework fails to reject matching outliers (colored in orange ) due to the perfectly repeated pattern (compare the query image with the colored rendering from the predicted pose).

Figure 15: Relocalization results on Integrated Rooms. Following [ 11 ] , we arrange scenes in 12Scenes inside a 2D grid with a cell size of

5\text{\,}\mathrm{m}

and integrate varying numbers of adjacent scenes to form larger maps. As the integration size increases, PlanaReLoc’s performance degrades due to increased ambiguities and the scene encoder’s limited capacity for larger maps.

Camera Pose Estimation

Camera pose estimation recovers a camera’s position and orientation from images, visual features, scene geometry, and sometimes sensor measurements.

See all 241 papers

Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization
Jun 23, 2026Zhaopeng Cui, Jiarui Hu, Jingbo Liu +7Camera Pose Estimation 3D Scene Understanding
1State Key Lab of CAD&CG, Zhejiang University · 2Ant Group
MultiLoc: Look Around As You Localize for Fast And Robust Visual Re-localization
Mar 28, 2026Nobel Dang, Bing LiRelative Pose Estimation Camera Pose Estimation
Clemson University, SC, USA
Z-FLoc: Zero-Shot Floorplan Localization via Geometric Primitives
Jun 3, 2026Ayumi Umemura, Toshinori Kuwahara, Marc Pollefeys +1Floor-Plan-Based Visual Localization Camera Pose Estimation
ETH Zurich · Tohoku University

Jun 23, 2026J/K move · Enter open · S save

Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization

Zhaopeng Cui, Jiarui Hu, Jingbo Liu, Boming Zhao+6

1State Key Lab of CAD&CG, Zhejiang University · 2Ant Group

Camera Pose Estimation 3D Scene Understanding

Open paper AskPDF arXiv

Indoor visual relocalization plays a critical role in emerging spatial and embodied AI applications. However, prior research was predominantly devoted to low-level vision schemes, struggling to perceive scene semantics and compositions, which limits both interpretability and applicability. In this paper, we explore the issue of how to organize rich object information in a scene, including semantics, layout, and geometry, into a structured map representation, thereby utilizing object units exclusively to drive the camera relocalization task. To this end, we propose OpenReLoc, a camera relocalization system designed to provide scene understanding and accurate pose estimation capabilities. Leveraging recent foundation models, we first introduce a multi-modal mechanism to integrate open-vocabulary semantic knowledge for effective 2D-3D object matching. Additionally, we design object-oriented reference frames as position priors, paired with a reference frame selection strategy based on the Distance-IoU (DIOU), enabling extension to scalable scenes. Moreover, to ensure stable and accurate pose optimization, we also propose a dual-path 2D Iterative Closest Pixel loss guided by object shape. Experimental results demonstrate that OpenReLoc achieves superior relocalization recall and accuracy across various datasets. Our source code will be released upon acceptance.

Abstract

Camera Pose Estimation

Explore similar work

Compact Object-Level Representations with Open-Vocabulary Understanding for Indoor Visual Relocalization

MultiLoc: Look Around As You Localize for Fast And Robust Visual Re-localization

Z-FLoc: Zero-Shot Floorplan Localization via Geometric Primitives