The growing demand for Embodied AI and VR applications has highlighted the need for synthesizing high-quality 3D indoor scenes from sparse inputs. However, existing approaches struggle to infer massive amounts of missing geometry in large unseen areas while maintaining global consistency, often producing locally plausible but globally inconsistent reconstructions. We present Rein3D, a framework that reconstructs full 360-degree indoor environments by coupling explicit 3D Gaussian Splatting (3DGS) with temporally coherent priors from video diffusion models. Our approach follows a "restore-and-refine" paradigm: we employ a radial exploration strategy to render imperfect panoramic videos along trajectories starting from the origin, effectively uncovering occluded regions from a coarse 3DGS initialization. These sequences are restored by a panoramic video-to-video diffusion model and further enhanced via video super-resolution to synthesize high-fidelity geometry and textures. Finally, these refined videos serve as pseudo-ground truths to update the global 3D Gaussian field. To support this task, we construct PanoV2V-15K, a dataset of over 15K paired clean and degraded panoramic videos for diffusion-based scene restoration. Experiments demonstrate that Rein3D produces photorealistic and globally consistent 3D scenes and significantly improves long-range camera exploration compared with existing baselines.
Figures & tables
Figure 1: Overview of Rein3D. Starting from a single RGB-D panorama, Rein3D initializes a coarse 3DGS scene and renders degraded panoramic videos along radial trajectories to expose unseen regions. A panoramic video diffusion model restores these videos with temporally consistent priors, and the restored views are fused to refine the global 3DGS. This restore-and-refine paradigm enables photorealistic and globally consistent indoor scene reconstruction under large viewpoint changes.
Figure 2: Illustration of PanoV2V-15K construction. For each scene, we render a clean panoramic video along the longest traversable trajectory and generate its paired degraded RGB-alpha video by rendering a coarse 3DGS initialized from the first panoramic RGB-D observation.
Figure 3: Overview of the Rein3D pipeline. (a) Given a single panoramic RGB-D observation, we lift it into an initial 3D point cloud / coarse 3DGS representation. Rendering this coarse representation from novel viewpoints exposes holes, missing geometry, and opacity artifacts caused by single-view initialization. (b) The resulting low-quality RGB-alpha panoramic videos are restored by our panoramic video diffusion model. The RGB-alpha inputs are decomposed into observed and missing-region streams, combined with a context anchor, and processed with latitude-aware noise sampling to produce temporally consistent restored panoramas. (c) The restored videos are further enhanced by super-resolution and fused back as pseudo observations to refine the global 3DGS, yielding a more complete and view-consistent indoor scene.
Dataset
Method
PSNR ↑
SSIM ↑
WS-PSNR ↑
WS-SSIM ↑
FVD ↓
PanoV2V-15K
ProPainter
18.46
0.760
17.79
0.740
14.56
VACE
21.17
0.787
20.80
0.771
11.67
VACE-FT
22.18
0.798
21.89
0.790
9.27
Ours
23.77
0.823
23.45
0.820
5.26
3D-FRONT
ProPainter
19.12
0.674
18.68
0.656
29.27
VACE
21.91
0.701
21.00
0.702
25.50
Table 1: Quantitative comparison on panoramic video restoration. VACE-FT denotes VACE fine-tuned on PanoV2V-15K.
Figure 4: Qualitative comparison of rendered perspective views on Structured3D. The top row shows the input panoramas, and the following rows show novel views rendered from reconstructed 3D scenes. Rein3D maintains more stable geometry and fewer artifacts under large viewpoint changes.
Method
Q-Align ↑
CLIP ↑
NIQE ↓
BRISQUE ↓
EmbodiedGen
2.2914
0.2216
7.9231
58.3813
DreamScene360
1.8342
0.2356
8.5295
61.4567
Ours
3.1033
0.2609
6.4925
46.5411
Table 2: Quantitative comparison on single-panorama 3D scene reconstruction on 50 Structured3D test scenes. We report no-reference visual quality and semantic consistency metrics on novel views.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
EmbodiedGen
15.05
0.517
0.678
Ours
17.64
0.624
0.540
Table 3: Novel view synthesis comparison on 30 3D-FRONT scenes. Each method takes a single panoramic RGB-D image as input and is evaluated on 100 held-out target views per scene.
Setting
PSNR ↑
SSIM ↑
WS-PSNR ↑
WS-SSIM ↑
FVD ↓
w/o Context Anchor
22.84
0.810
22.40
0.802
7.75
w/o Latitude-Decay Loss
23.28
0.819
22.97
0.816
5.66
w/o Latitude-Aware Noise Sampling
22.96
0.816
22.65
0.812
6.24
Full
23.77
0.823
23.45
0.820
5.26
Table 4: Ablation study of key components in the panoramic video restoration model. Context Anchor stabilizes long-range restoration, while latitude-aware sampling and latitude-decay loss adapt the model to equirectangular geometry.
Setting
PSNR ↑
SSIM ↑
WS-PSNR ↑
WS-SSIM ↑
FVD ↓
λ=1.0
23.28
0.819
22.97
0.816
5.66
λ=0.5
23.80
0.822
23.38
0.818
5.66
λ=0.3
23.42
0.819
23.05
0.816
5.45
λ=0
23.44
0.824
22.99
0.819
5.62
λ=0.1 (Full)
23.77
0.823
23.45
0.820
5.26
Table 5: Sensitivity analysis of the latitude-decay coefficient λ . The default setting λ=0.1 provides the best overall balance between spherical fidelity and temporal consistency.
Setting
PSNR ↑
SSIM ↑
LPIPS ↓
w/o Radial Exploration
14.84
0.448
0.672
w/o VSR
17.42
0.601
0.549
Full ( τ=4 )
17.36
0.595
0.553
Full ( τ=8 )
17.64
0.624
0.540
Table 6: Ablation study on 3D-FRONT. w/o Radial Exploration uses local exploration around the input panorama center. w/o VSR removes super-resolution before 3DGS refinement. τ denotes the number of radial trajectories.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Overview of the PanoV2V-15K dataset. (Left) A pie chart illustrating the distribution of the 15,500 indoor scenes across different room types. (Right) Consecutive frames extracted from a high-quality panoramic video within the dataset, accompanied by its corresponding highly descriptive text prompt generated by Qwen2.5-VL.
Setting
Value
Batch size
4
Training steps
15,000
Evaluation / save steps
5,000 / 10,000 / 15,000
Test split
Every 16th view
SH degree
3
Near / far plane
0.01 / 100.0
Appendix
Table 7: Key hyperparameters for final 3DGS refinement.
Figure 7: Visualizations of our camera exploration and evaluation sampling strategies. (Top) The depth-optimized radial trajectory generation, illustrating the safe profile, the cross-score optimization, and the selected top-down trajectories. (Bottom) The Farthest Point Sampling (FPS) process for evaluation, showing the safe navigable area, the Monte Carlo candidate pool, and the selected 5 globally distributed camera locations.
Figure 8: Qualitative results of panoramic video restoration. The odd-numbered rows (1st, 3rd, and 5th) show sequences of degraded panoramic frames rendered from the initial coarse 3DGS, which suffer from severe missing regions (black holes) and structural artifacts. Directly below them, the even-numbered rows (2nd, 4th, and 6th) display the corresponding high-fidelity frames restored by our panoramic video diffusion model, demonstrating remarkable spatial completeness and temporal coherence across the trajectory.
Figure 9: Additional qualitative comparisons on novel view synthesis against EmbodiedGen and DreamScene360. The views are rendered from camera locations far from the initialization point to evaluate long-range exploration consistency. While baselines exhibit severe distortions, floating artifacts, and missing structures, our method preserves global geometry and fine textural details.
We address the problem of reconstructing a high-fidelity, freely navigable 3D scene from a single 360∘ panorama, without per-scene optimization or multi-view capture. Existing methods either lack metric trajectory control, which hinders reliable downstream 3D reconstruction, or struggle with large disocclusions under long-range camera motion while requiring high-end multi-GPU servers.We present Genie Sim PanoWorld, a two-stage feed-forward pipeline that bridges generation and reconstruction via an explicit, trajectory-controllable panoramic video. A NavMesh-planned SE(3) roaming trajectory is injected into a latent video diffusion model through dense geometry-warped conditioning; long--short trajectory mixed training and a self-consistency objective based on shortcut models together yield high-fidelity video in four CFG-free denoising steps. A feed-forward panoramic reconstructor then lifts the generated video into a high-fidelity 3D Gaussian scene that supports real-time, free-viewpoint roaming and can be directly used as a simulation-ready asset for embodied AI applications. Experiments show that Genie Sim PanoWorld outperforms geometry-conditioned baselines in both panoramic video generation and downstream 3D reconstruction, while generalizing zero-shot to unseen indoor scenes.
We present GuidedSceneGen, a text-to-3D generation framework that produces metrically accurate, globally consistent, and semantically interpretable indoor scenes. Unlike prior text-driven methods that often suffer from geometric drift or scale ambiguity, our approach maintains an absolute world coordinate frame throughout the entire generation process. Starting from a textual scene description, we predict a global 3D layout encoding both semantic and geometric structure, which serves as a guiding proxy for downstream stages. A semantics- and depth-conditioned panoramic diffusion model then synthesizes 360° imagery aligned with the global layout, substantially improving spatial coherence. To explore unobserved regions, we employ a video diffusion model guided by optimized camera trajectories that balances coverage and collision avoidance, achieving up to 10x faster sampling compared to exhaustive path exploration. The generated views are fused using 3D Gaussian Splatting, yielding a consistent and fully navigable 3D scene in absolute scale. GuidedSceneGen enables accurate transfer of object poses and semantic labels from layout to reconstruction, and supports progressive scene expansion without re-alignment. Quantitative results and a user study demonstrate greater 3D consistency and layout plausibility compared to recent panoramic text-to-3D baselines.
Stefan Ainetter, Thomas Deixelberger, Edoardo A. Dominici +3
Huawei Technologies, Austria · Graz University of Technology, Austria · Huawei Technologies, Switzerland
Recent advances in 3D scene reconstruction have highlighted the intricate trade-offs among rendering quality, inference efficiency, and data dependency. To address the challenge of rapidly reconstructing detailed 3D indoor scenes from minimal input, we introduce FastPano3D, an end-to-end framework that directly generates renderable 3D Gaussian representations from a single panoramic image. Unlike perspective-based methods, panoramic images inherently suffer from equirectangular projection distortions and spatially non-uniform feature distributions, making direct feed-forward Gaussian generation particularly challenging. In contrast to existing Gaussian Splatting based methods that rely on multi-view supervision or per-scene optimization, FastPano3D employs a lightweight feature encoder, adaptive Gaussian sampling, and a point-cloud-guided refinement strategy to achieve efficient and accurate scene generation without any test-time optimization. Our approach reconstructs high-fidelity 3D scenes within seconds, achieving up to 156 times faster inference than prior state-of-the-art methods such as Pano2Room, while using only half the parameters. Extensive experiments demonstrate that FastPano3D delivers rendering quality comparable to NeRF- and 3DGS-based reconstructions, establishing a new benchmark for rapid, single-view 3D scene inference.
Jianqiang Li, Liumei Zhang, Wenjia Guo +5
Xi’an Shiyou University, Shaanxi, China · Xidian University, Shaanxi, China · Swansea University, Swansea, United Kingdom +1