PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splatting
Authors: Changkun Liu, Bin Tan, Zeran Ke, Shangzhan Zhang, Jiachen Liu, Ming Qian, Nan Xue, Yujun Shen, Tristan Braud
Organizations: The Hong Kong University of Science and Technology · Ant Group · Wuhan University · Zhejiang University · The Pennsylvania State University
This paper addresses metric 3D reconstruction of indoor scenes by exploiting their inherent geometric regularities with compact representations. Using planar 3D primitives - a well-suited representation for man-made environments - we introduce PLANA3R, a pose-free framework for metric Planar 3D Reconstruction from unposed two-view images. Our approach employs Vision Transformers to extract a set of sparse planar primitives, estimate relative camera poses, and supervise geometry learning via planar splatting, where gradients are propagated through high-resolution rendered depth and normal maps of primitives. Unlike prior feedforward methods that require 3D plane annotations during training, PLANA3R learns planar 3D structures without explicit plane supervision, enabling scalable training on large-scale stereo datasets using only depth and normal annotations. We validate PLANA3R on multiple indoor-scene datasets with metric supervision and demonstrate strong generalization to out-of-domain indoor environments across diverse tasks under metric evaluation protocols, including 3D surface reconstruction, depth estimation, and relative pose estimation. Furthermore, by formulating with planar 3D representation, our method emerges with the ability for accurate plane segmentation. The project page is available at https://lck666666.github.io/plana3r
Figures & tables
Figure 1: Our proposed Plana3R learns to predict planar 3D primitives and metric-scale relative poses, providing a compact 3D representation of two-view input images with accurate pose estimation, surface geometry, and semantically meaningful 2D & 3D planar segmentation—all in one pass. As a side output, our model also performs well on single-view planar 3D reconstruction.
Figure 2: Overview of our Plana3R . Given two images captured from the same scene, Plana3R outputs a set of 3D planar primitives and 6-DoF relative camera pose Prel in metric scale. Plana3R does not perform per-pixel primitive prediction. Instead, it employs a deconvolution network to predict primitives at two distinct resolutions, based on the patch divisions from the ViT encoder. During both training and inference phases, we selectively integrate low-resolution and high-resolution primitives, resulting in a highly compact and expressive representation. Note that even for high-resolution primitives, their total number is only 8H×8W , which remains highly sparse.
Figure 3: Comparisons of two-view 3D planar reconstruction on the ScanNetV2 [ 5 ] (the first row) and the Matterport3D [ 3 ] (the last two rows) datasets.
Method
Translation (m)
Rotation ( ∘ )
Chamfer ↓
F-score ↑
Med.
Mean
<=1
<=0.5
<=0.1
Med.
Mean
<=30
<=15
<=5
ScanNetV2 dataset
SparsePlanes [ 11 ]
0.56
0.81
73.7
44.6
–
15.46
33.38
70.5
48.7
–
–
–
PlaneFormers [ 1 ]
0.55
0.81
75.3
45.5
–
14.34
32.08
73.2
52.1
–
–
–
NOPE-SAC [ 28 ]
0.41
0.65
82
59.1
5.01
8.27
22.12
82.6
73.2
25.03
0.26
61.86
MASt3R [ 14 ]
0.11
0.19
97.65
93.98
47.37
2.17
6.67
95.04
94.08
84.37
0.21
74.92
Table 1: Quantitative comparison of two-view planar reconstruction and relative camera pose estimation. The best results are in bold .
Method
PlaneNet [ 15 ]
PlaneAE [ 40 ]
PlaneRCNN [ 16 ]
PlaneTR [ 27 ]
PlaneRecTR [ 23 ]
MASt3R [ 14 ]
Ours
Rel ↓
0.239
0.205
0.183
0.195
0.157
0.152
0.132
log10↓
0.124
0.097
0.076
0.095
0.073
0.058
0.059
RMSE ↓
0.913
0.820
0.619
0.803
0.547
0.51
0.463
δ1↑
53.0
61.3
71.8
63.3
74.2
83.0
86.4
δ2↑
78.3
87.2
93.1
88.2
94.2
95.6
96.3
δ3↑
90.4
95.8
98.3
96.1
99.0
98.7
98.4
Table 2: Quantitative comparison of metric monocular depth estimation on the NYUv2 dataset. The best results are in bold .
Figure 4: Comparisons of single-view plane segmentation and 3D reconstruction on the Replica [ 26 ] .
Figure 5: Comparisons of two-view 3D plane segmentation on the Matterport3D [ 3 ] (the first two rows) and the ScanNetV2 [ 5 ] (the last row) datasets.
Method
Plane Segmentation
Plane Recall (depth)
Plane Recall (normal)
RI ↑
VOI ↓
SC ↑
@ 0.1m
@ 0.6m
@5∘
@30∘
PlaneRecTR [ 23 ]
0.85
1.81
0.58
1.78
13.46
3.79
18.92
Ours
0.89
1.62
0.63
7.79
30.74
28.52
36.31
Table 3: Quantitative comparison of single-view planar reconstruction on the Replica dataset [ 26 ] . The best results are in bold .
Method
RRA@5
RTA@5
RRA@10
RTA@10
RRA@15
RTA@15
Ours
0.9000
0.3935
0.9985
0.7442
1.0000
0.8614
MASt3R
0.9828
0.2657
0.9964
0.5371
1.0000
0.6878
Table 4: Comparison with MASt3R on multi-view RRA (Relative Rotation Accuracy) and RTA (Relative Translation Accuracy).
Figure 6: Reconstruction example from 8 frames of a ScanNet sequence. We show the input 8 frames and planar 3D reconstruction.
ScanNetV2 Reconstruction
NYUv2 Depth Estimation
Metrics
Chamfer ↓
F-score ↑
Avg. # primitives
RMSE ↓
δ1↑
Avg. # primitives
Ours (0)
0.10
93.10
3072
0.45
86.8
3072
Ours (0.5)
0.11
92.52
1417
0.46
86.4
1565
Ours (10)
0.11
92.32
768
0.49
85.3
768
Table 5: Ablation study on the gradient threshold ( gth ). We show the relationship between the number of per-view primitives and performance.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Supplement comparisons of single-view plane segmentation and 3D reconstruction on the 7-Scenes [ 24 ] (the first four rows) and Replica [ 26 ] (the last four rows).
Figure 8: Supplement comparisons of two-view 3D plane segmentation on the ScanNetV2 [ 5 ] (the first two rows) and Matterport3D [ 3 ] (the last three rows).
Figure 9: Supplement comparisons of two-view 3D plane reconstruction on the ScanNetV2 [ 5 ] (the first two rows) and Matterport3D [ 3 ] (the last three rows).
Figure 10: Reconstruction example from 8 frames of a ScanNet sequence. We show the input 8 frames and planar 3D reconstruction.
Overlap
Training Data
Translation
Rotation
Med. (m) ↓
Mean (m) ↓
≤ 0.2m ↑
Med. ( ∘ ) ↓
Mean ( ∘ ) ↓
≤ 10 ∘↑
ScanNetV2
Easy ( > 50%)
Full
0.05
0.07
97.40
1.53
1.80
99.72
w/o Ext. Data
0.05
0.09
95.73
1.77
2.77
99.16
Medium (15–50%)
Full
0.06
0.10
93.56
2.02
2.68
98.88
w/o Ext. Data
0.07
0.15
89.68
2.25
4.24
97.60
Appendix
Table 6: Results on ScanNetV2 and MP3D (w/o Ext. Data means training without 0.57M non-overlapping image pairs).
Figure 11: Test-time performance versus overlap degree.
Datasets
Type
N Pairs
ScanNetV2 [ 5 ]
Indoor/Real
610K
ScanNet++ [ 39 ]
Indoor/Real
810K
ARKitScenes [ 2 ]
Indoor/Real
2400K
Habitat [ 22 ]
Indoor/Synthetic
120k
Appendix
Table 7: Training data and sample sizes for Plana3R .
Figure 12: (a) A 3D planar primitive using learnable shape parameters. Vπx and Vπy are the positive direction of the X-axis and Y-axis, respectively. (b) We compute the gradient magnitude for each pixel in 16H×16W low-resolution predicted normal patches Nlowpatch . To combine the low- and high-resolution primitives, we use binary masks (with white indicating valid regions) to merge only the valid patches from both resolutions, rather than directly using all predicted primitives.