Reconstructing accurate camera poses and floor-plan layouts from sparse, unposed, wide-baseline RGB panoramas remains a challenging open problem. Direct multi-room floor-plan prediction is especially difficult in this setting: pose, geometry, semantics, and global layout structure are tightly coupled, yet a complete floor-plan is hard to learn as a single global output. Our key insight is to keep the model's predictions strictly local. Multi-view Ordered Wall Instance Segmentation (MOIS) is designed around this concept. Each panorama column predicts only the wall it observes plus the next two adjacent walls in clockwise order. The global wall partition, room layouts, and shared coordinates emerge from deterministic aggregation of overlapping local predictions. Building on MOIS, we propose Multi-view floor-plan Feed-Forward Network (MvFFN), which jointly predicts coarse camera poses, dense global geometry, and per-wall semantic outputs. A downstream Layout Chaining (LC) post processing pipeline then assembles these predictions into multi-room floor-plans with shared global coordinates, consistent wall identities, and connected room structure. The dense MOIS predictions recover cross-view wall identity and local within-room connectivity more accurately than composed modular baselines, and the assembled layouts substantially outperform prior methods on wall and junction level accuracy. We also introduce the ZInD-CrossView dataset, which augments ZInD with globally unique wall instances and cross-view segmentation labels for this task.
Figures & tables
Figure 1: MvFFN reformulates floor plan reconstruction from direct global prediction to local MOIS prediction: each anchor column, shown as a vertical line in the panorama, predicts cross-view masks for the anchor wall and two clockwise adjacent walls. Colors indicate corresponding wall slots/instances across panoramas and in the assembled layout. Layout Chaining aggregates these local predictions into global wall identities, room-level wall sequences, and metric floor plan layouts, enabling downstream refinement.
Figure 2: Architecture of MvFFN with two-stage training for MOIS prediction.
Methods
5v Pos (cm) ↓
5v Rot ( ∘ ) ↓
8v Pos (cm) ↓
8v Rot ( ∘ ) ↓
Mn
Std
Med
Mn
Std
Med
Mn
Std
Med
Mn
Std
Med
CovisPose + GST
12.6
23.3
9.3
1.3
0.9
0.7
17.3
37.3
11.8
1.9
1.6
0.8
VGGT
30.7
39.4
13.3
11.1
16.0
4.0
53.4
63.1
27.8
13.3
17.7
4.9
MvFFN
12.5
10.8
10.2
1.2
0.6
1.1
17.0
15.6
13.8
1.3
0.8
1.2
Table 1: Camera pose evaluation on ZInD-XV dataset.
Modular Pipeline
5-view wall
5-view junction
8-view wall
8-view junction
pose
geom.
layouts
Prec. ↑
Rec. ↑
F1 ↑
Dist ↓
Prec. ↑
Rec. ↑
F1 ↑
Dist ↓
Prec. ↑
Rec. ↑
F1 ↑
Dist ↓
Prec. ↑
Rec. ↑
F1 ↑
Dist ↓
MvFFN
88.5
87.7
88.1
13.9
79.2
68.2
73.3
13.8
89.4
87.5
88.4
19.0
81.0
69.8
75.0
19.1
CP
RF
31.9
44.2
37.1
84.9
15.4
20.9
17.7
137.8
35.0
42.4
38.3
79.0
15.7
18.6
17.0
136.8
CP
PR
33.6
29.3
31.3
86.6
17.4
14.9
16.1
129.7
31.8
25.0
28.0
86.2
13.8
10.5
11.9
133.4
CP
LGT
LGT-PP
75.6
71.4
73.4
21.3
60.0
54.8
57.3
21.3
76.3
72.7
74.5
24.4
61.7
56.7
59.1
24.4
GT
LGT
LGT-PP
78.8
70.5
74.4
19.2
62.4
54.3
58.1
19.7
80.5
70.8
75.3
22.7
65.1
55.5
59.9
22.7
Table 2: Floor-layout evaluation on ZInD-XV . Distance is in cm. Geom. denotes the intermediate geometric input and Layout denotes the vectorization backend. GT-pose denotes a control group.
View
Method
All samples
Improved samples
Mean dist
Med. dist
Improved (%)
Mean dist
Med. dist
before
after
before
after
before
after
before
after
5-view
MvFFN + BA
13.9
13.7
6.9
6.7
68.0%
14.8
12.3
7.5
5.9
MvFFN + BADGR
13.9
17.7
6.9
8.1
45.9%
16.2
14.1
8.2
6.2
8-view
MvFFN + BA
19.0
18.8
8.8
8.8
66.0%
20.6
17.8
9.7
8.2
MvFFN + BADGR
19.0
21.0
10.1
10.4
48.5%
22.9
20.0
8.8
8.8
Table 3: Matched-wall distance before and after refinement on ZInD-XV . Distances are in cm.
Method
Geom Enc
Sem Enc
Geom Fusion
Train Stages
5v pos (cm) ↓
5v rot ( ∘ ) ↓
5v wall
5v junction
Mn
Std
Med
Mn
Std
Med
Prec ↑
Rec ↑
F1↑
Dist ↓
Prec ↑
Rec ↑
F1↑
Dist ↓
Shared-Enc
✓
✗
✗
2
35.8
50.1
26.4
2.1
1.3
2.0
86.4
69.8
77.2
31.2
72.9
50.4
59.6
39.9
No-Fusion
✓
✓
✗
2
–
–
–
–
–
–
92.6
22.7
36.5
28.0
84.3
14.4
24.6
21.4
1-stage
✓
✓
✓
1
46.8
50.5
30.1
2.5
1.5
1.9
87.7
71.9
79.0
33.9
75.8
53.1
62.5
44.1
MvFFN
✓
✓
✓
2
12.5
10.8
10.2
1.2
0.6
1.1
88.5
87.7
88.1
13.9
79.2
68.2
73.3
13.8
Table 4: Ablation study of MvFFN on ZInD-XV , comparing encoder design, geometry fusion, and staged training. – denotes pose metrics identical to MvFFN .
Figure 3: Qualitative comparisons on ZInD-XV test set. We show the strongest modular baseline, LGT-Net with CovisPose .
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
5-view inputs
8-view inputs
Method
wall 1
wall 2
wall 3
wall 1
wall 2
wall 3
MvFFN
81.9
71.1
57.8
75.4
63.8
47.7
LGT + CP
63.7
37.8
20.3
60.4
34.5
18.5
Appendix
Table C.1: Pooled Ordered Wall-Slot IoU ↑ (%) for raw MOIS predictions before Layout Chaining . Wall 1 denotes the anchor wall, and 2-3 denote the next ordered neighboring walls.
Methods
5v Pos (cm) ↓
5v Rot ( ∘ ) ↓
8v Pos (cm) ↓
8v Rot ( ∘ ) ↓
Mn
Std
Med
Mn
Std
Med
Mn
Std
Med
Mn
Std
Med
VGGT
35.2
109.3
10.4
6.1
16.3
2.1
62.3
71.8
37.4
13.7
6.4
12.3
CovisPose + GST
30.1
24.7
19.3
2.2
3.3
1.1
44.0
59.7
25.9
2.9
4.4
1.4
MvFFN
26.8
21.2
21.5
1.8
0.6
1.6
32.0
23.5
26.6
1.9
0.5
1.7
Appendix
Table D.1: Camera pose evaluation on the CovisPose -feasible subset of HM3D-XV . Mn/Std/Med denote mean, standard deviation, and median errors.
pose
pts
layouts
5-view wall
5-view junction
8-view wall
8-view junction
prec
recall
f1
dist
prec
recall
f1
dist
prec
recall
f1
dist
prec
recall
f1
dist
CP
LGT
LGT
57.0
62.6
59.7
51.8
18.2
33.3
23.5
53.1
59.4
60.2
59.8
52.1
19.0
30.5
23.4
53.8
GT
LGT
LGT
69.1
66.8
67.9
36.9
25.5
44.3
32.4
35.7
69.7
64.6
67.1
39.9
25.5
38.9
30.8
38.6
MvFFN
77.8
69.6
73.5
22.1
55.9
38.8
45.8
24.1
75.0
71.6
73.3
28.9
52.9
35.0
42.1
29.3
Appendix
Table D.2: Floor-layout evaluation on HM3D-XV . Distance is in cm. Geom. denotes the intermediate geometric input and Layout denotes the vectorization backend. GT-pose denotes a control group.
Figure E.1: Effect of inductive bias across input views. Box plots show translation (left) and rotation (right) errors under different inductive-bias variants. Dashed lines and diamond markers indicate mean errors. Stronger inductive bias improves translation accuracy and suppresses extreme rotation outliers.
# walls
5-view wall
5-view junction
8-view wall
8-view junction
Prec ↑
Rec ↑
F1↑
Dist ↓
Prec ↑
Rec ↑
F1↑
Dist ↓
Prec ↑
Rec ↑
F1↑
Dist ↓
Prec ↑
Rec ↑
F1↑
Dist ↓
3-wall
88.5
87.7
88.1
13.9
79.2
68.2
73.3
13.8
89.4
87.5
88.4
19.0
81.0
69.8
75.0
19.1
5-wall
88.7
82.8
85.6
14.7
78.8
64.2
70.8
17.0
88.4
83.7
86.0
19.6
79.4
64.7
71.3
23.2
7-wall
87.6
81.4
84.4
16.2
76.3
58.7
66.4
18.7
87.7
78.9
83.1
19.2
77.9
59.0
67.1
21.7
Appendix
Table E.1: MvFFN layout accuracy reported on 5- and 8-view test set from ZInD-XV . The model is trained to predict different numbers of adjacent walls in MOIS prediction.
Figure H.1: Failure case caused by incorrect MOIS wall-instance prediction, where MvFFN fails to distinguish different wall instances across views.
Figure H.2: Failure case caused by geometric errors in the initial MvFFN predictions, where inaccurate poses and point maps lead to distorted layout geometry.
Figure J.1: Qualitative comparisons on ZInD-XV test set.
Figure J.2: More Qualitative comparisons on ZInD-XV test set.
Figure J.3: Qualitative comparisons on HM3D-XV test set.
Asset
Role in this work
Version / access date
License or terms
Release treatment
ZInD-XV annotations
New derived annotation asset
v1.0
CC BY-NC 4.0, subject to original ZInD terms
We release derived wall-instance labels, splits, metadata, conversion scripts, and evaluation files only.
HM3D-XV annotations
New derived annotation asset
v1.0
CC BY-NC 4.0, subject to original HM3D and HM3DSem terms
We release derived annotations, splits, metadata, conversion scripts, and evaluation files only.
Appendix
Table K.1: Existing and newly derived assets used in this work. We release only assets created by this work under the stated terms and do not redistribute raw restricted source data.
Estimating room layouts from multi-view imagery is a core task for indoor scene understanding. Existing methods are typically limited either by poor generalization to new datasets or restrictive geometric assumptions of the room shape or camera configuration. Most also estimate rooms independently, failing to exploit shared building structure such as dominant directions, ground plane or ceiling height. We propose PolyLayout, a multi-room layout estimation method that parameterizes room layouts as Manhattan 3D polygons and optimizes them jointly across multiple rooms. The optimization objective is predicted by a neural network on top of robust pre-trained visual features and trained end-to-end with supervision only on output room layouts. At the same time, camera projection and polygon updates remain explicit and model-based. This separation between learned scoring and geometry improves generalization to new datasets and camera parameters. During optimization, PolyLayout adaptively refines the polygon topology through iterative wall split and merge operations while jointly utilizing structural cues across rooms. We introduce two new multi-view multi-room layout benchmarks by providing layout annotations to existing datasets, and experiments show that PolyLayout outperforms prior approaches, both in terms of accuracy and robustness. Project page: https://ghanning.github.io/PolyLayout
Gustav Hanning, Shaohui Liu, Rémi Pautrat +3
Lund University · ETH Zurich · Microsoft Spatial AI Lab
Many public buildings provide floorplans with a "you are here" indicator to help visitors orient themselves. Floorplan localization seeks to computationally replicate this capability by determining where visual observations were captured within a floorplan. However, existing methods typically assume controlled small-scale environments and precise vectorized floorplans, limiting their ability to operate in large-scale buildings and rasterized floorplans. In this work, we present an approach for performing floorplan localization in the wild by grounding the task in a reconstructed 3D representation of the scene. Given an unconstrained image collection, our method reconstructs a gravity-aligned 3D scene and projects it into a 2D density map that serves as a floorplan proxy. Floorplan localization is then formulated as aligning this proxy with the input floorplan via a 2D similarity transform. To bridge the appearance gap between density maps and architectural floorplans, we adapt a 2D foundation model to learn cross-modal correspondences, introducing a fine-tuning scheme that encourages semantically aligned matches while preserving structural consistency. Extensive experiments demonstrate substantial improvements over prior methods, including in extremely sparse settings with as little as a single input image. Our code and data will be publicly available.
Junhyeong Cho, Ruojin Cai, Hadar Averbuch-Elor
Cornell University · Kempner Institute, Harvard University
The rise of home-deployed embodied AI systems is driving a growing need for fast, metric 3D reconstruction of residential spaces to support navigation, interaction, and long-horizon task execution. However, the commonly used pinhole-camera 3D reconstruction pipelines struggle to model large indoor residences efficiently due to their limited field of view, to which achieving full coverage across multiple rooms often requires thousands of images and incurs drift from long chains of incremental alignment. In this work, we present CasaMaestro (Spanish words meaning house'' and master''), a feedforward model that can take only twenty to fifty sparse multi-view indoor panoramas as input and directly predicts metric depth along with camera poses, allowing fast point-cloud reconstruction of the entire house with full coverage. CasaMaestro is the first model that supports house-scale reconstruction with multi-view panoramas. Experiments show that CasaMaestro can robustly provide high quality results in both real-world and synthetic scenes, which can serve as a strong foundation for acquiring house-scale 3D indoor assets to be applied in close-loop simulation.
Yuzhou Ji, Xiaotian Yang, Zhipeng Zhang
AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University, China