The synthesis of immersive 3D scenes from text is rapidly maturing, driven by novel video generative models and feed-forward 3D reconstruction, with vast potential in AR/VR and world modeling. While panoramic images have proven effective for scene initialization, existing approaches suffer from a trade-off between visual fidelity and explorability: autoregressive expansion suffers from context drift, while panoramic video generation is limited to low resolution. We present Stepper, a unified framework for text-driven immersive 3D scene synthesis that circumvents these limitations via stepwise panoramic scene expansion. Stepper leverages a novel multi-view 360° diffusion model that enables consistent, high-resolution expansion, coupled with a geometry reconstruction pipeline that enforces geometric coherence. Trained on a new large-scale, multi-view panorama dataset, Stepper achieves state-of-the-art fidelity and structural consistency, outperforming prior approaches, thereby setting a new standard for immersive scene generation.
Figures & tables
Figure 2: Method overview. a) Our model generates a new panoramic image from a previously unobserved viewpoint based on a given input panorama. To ensure high quality, we utilize a pre-trained diffusion model with expanded multi-view attention that is instrumental for jointly denoising the high-resolution cubefaces of the newly generated novel-view panorama. b) Our ability to generate novel-view panoramas enables auto-regressive scene generation in all directions of the scene yielding a set of high quality, consistent panoramas that effectively complete the representation of the 3D scene. c) The generated panoramas are processed with the feed-forward reconstruction model, i.e. MapAnything. The output pointcloud serves as the initialization of a custom 3D Gaussian Splatting reconstruction enabling high quality novel view synthesis of the generated 3D scene.
Figure 3: Dataset Samples. The dataset generated with Infinigen consists of a diverse set of high quality synthetic panoramas of indoor and outdoor scenes. For every panorama, we rendered a pair from a novel viewpoint enabling the training of the multi-view panorama generation model. All panoramas are aligned to the horizontal line.
Figure 4: 3D Scene Generation. We provide visual example of generated novel-view panoramas on the left side. The details of the initial panorama are well preserved and previously unseen regions are filled in. On the right side we show novel-view renderings of the reconstructed scenes indicating the 3D consistency of the generated panoramas.
Figure 5: Comparison with Baselines. Given a high quality input panorama, we observe that our approach achieves consistent scene generation while showing significantly more details and sharpness in the rendered novel view images in comparison to the baselines.
Infinigen Indoors
PSNR ↑
SSIM ↑
LPIPS ↓
WorldExplorer
11.864
0.674
0.739
LayerPano3D
18.305
0.783
0.509
Matrix-3D
18.532
0.753
0.502
Ours
21.775
0.797
0.430
Infinigen Outdoors
PSNR ↑
SSIM ↑
LPIPS ↓
WorldExplorer
13.912
0.561
0.594
Table 1: Quantitative Evaluation. We compare our approach to the baseline methods for three different datasets on common image metrics. We observe that our approach yields significant improvements on all metrics and datasets.
Figure 6: Single vs multiple panoramas to 3DGS. The multi panorama input to the 3DGS reconstruction consistently fills in the unobserved regions in the initial panorama without sacrificing the quality of the input panorama.
Figure 7: Effect of step size. Novel panos. generated by a model with a) adjustable step direction, b) a larger step size d=0.5m .
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Table 2: User study results. n=10 participants cast votes on the most visually appealing , most accurate geometry and most details categories for 15 video comparisons, yielding 150 votes per category in total.
Our stages (single step = 30s)
Ours vs Baselines
1. Gen. n=8 novel panos
n× 30s
WorldExplorer
7h
2. MapAnything
20s
Matrix-3D
40min
3. Filtering
2min
LayerPano3D
32min
4. 3DGS optimization
8min
Ours
15min
Appendix
Table 3: Inference time - Panorama to 3D scene.
Figure 8: Impact of Multi-View Panoramas We depict the effect of using various options of panorama input for MapAnything and found that the auto-regressive expansion and sampling yield the most complete and accurate outputs.
Figure 9: Auto-regressive steps. We perform a number of auto-regressive steps to generate novel-view panoramas from the initial panorama to the left and right for several examples. From the generated panoramas, we visualize the forward-facing view to highlight the generated details. A single step corresponds to a step length of 0.25m .
Figure 10: Multi-view panoramas pairs generated with Infinigen.
Figure 11: Input panoramas used for testing.
Figure 12: User study form. Participants fill out this randomized form and cast a single vote per category.