We introduce a novel, training-free system for reconstructing, understanding, and rendering 3D indoor scenes from a sparse set of unposed RGB images. Unlike traditional radiance field approaches that require dense views and per-scene optimization, our pipeline achieves high-fidelity results without any training or pose preprocessing. The system integrates three key innovations: (1) A robust point cloud reconstruction module that filters unreliable geometry using a warping-based anomaly removal strategy; (2) A warping-guided 2D-to-3D instance lifting mechanism that propagates 2D segmentation masks into a consistent, instance-aware 3D representation; and (3) A novel rendering approach that projects the point cloud into new views and refines the renderings with a 3D-aware diffusion model. Our method leverages the generative power of diffusion to compensate for missing geometry and enhances realism, especially under sparse input conditions. We further demonstrate that object-level scene editing such as instance removal can be naturally supported in our pipeline by modifying only the point cloud, enabling the synthesis of consistent, edited views without retraining. Our results establish a new direction for efficient, editable 3D content generation without relying on scene-specific optimization. Project page: https://jiatongxia.github.io/TID3R/
Figures & tables
Figure 1. We present a novel, unified pipeline that transforms sparse image inputs into a clean, instance-aware point cloud without requiring any pose pre-processing or scene-specific learning, and can directly synthesizes photorealistic novel views through a tailored diffusion-based rendering approach. Beyond reconstruction and rendering, our method also showing good potential supports essential downstream tasks such as scene-level editing.
Figure 2. Framework . We apply a feed-forward a feed-forward model (e.g., MV-DUSt3R) to predict the initial point cloud from the input unposed sparse images, followed by a novel warping-based anomaly points removal strategy to eliminate unreliable points in the initial point cloud, forming a clean and accurate scene point cloud. For 2D-to-3D instance segmentation, we first utilize a foundational segmentation model (e.g., SAM) to generate initial segmentation masks. We then introduce a warping-based instance unification strategy to align instances across frames. By associating the instance masks with the corresponding pointmaps, we obtain an instance-segmented 3D point cloud. The instance-segmented 3D point cloud facilitates downstream tasks such as object-level editing. Finally, the point cloud can be directly rendered into high-quality 2D images through our designed diffusion-based rendering pipeline.
Figure 3. Anomaly points . The red box highlights anomaly points, which exhibit abnormal depth values.
Figure 4. Main results . We present the main experimental results, including instance segmentation, point cloud reconstruction, novel view synthesis, and scene edits through object removal. Each row corresponds to a test scene. The left column shows the instance segmentation results. The center section labeled “Original” displays the reconstructed point cloud (top) and novel view synthesis results (bottom, with ground truth images on the top and rendered results on the bottom) of original input images. The two columns on the right illustrate two different object removal results, with the removed objects are highlighted with black dashed boxes in the point clouds
Method
AP ↑
AP50 ↑
AP75 ↑
DEVA ( Cheng et al., 2023 )
13.9
25.9
11.7
PE3R ( Hu et al., 2025 )
20.2
32.6
22.3
w/o Warping-based unifi.
17.0
26.8
18.1
Ours
25.0
41.8
25.4
Table 1. Instance segmentation evaluation . We show overall instance segmentation performance using AP, AP50, and AP75 metrics on the selected scenes.
Method
RMSE ↓
δ<1.25 ↑
DepthAnythingV2 ( Yang et al., 2024b )
0.577
0.689
PE3R ( Hu et al., 2025 )
0.378
0.905
MV-DUSt3R+ ( Tang et al., 2025 )
0.219
0.977
Ours
0.209
0.981
Table 2. Depth evaluation . We show the overall video depth evaluation results on selected reconstruction scenes.
Figure 5. Novel views rendering . We show the visualization comparisons on novel view synthesis.
Poses
Training
Novel View Synthesis
Method
Free
Free
PSNR ↑
SSIM ↑
LPIPS ↓
3DGS ( Kerbl et al., 2023 )
✗
✗
15.61
0.602
0.541
2DGS ( Huang et al., 2024 )
✗
✗
15.55
0.591
0.544
GaussianGroup. ( Ye et al., 2024 )
✗
✗
15.72
0.610
0.532
DNGaussian ( Li et al., 2024 )
✗
✗
16.82
0.627
0.549
CoR-GS ( Zhang et al., 2024 )
✗
✗
17.12
0.631
0.496
Table 3. Rendering evaluation on novel views . We show the overall novel views synthesis performance of methods on selected reconstruction scenes. We indicate whether each method requires camera parameter input or per-scene training. We apply our rendering pipeline to the reconstructed point clouds produced by PE3R and MV-DUSt3R for a comparison.
Figure 6. Object removal comparison . We present the visualization results of removing the same object under two different views, with the original ground-truth image on the left as reference. For each method, we display the rendered results before and after object removal. The removed object is highlighted with a red box in our results.
Figure 7. Instance segmentation . We present a visual comparison between our 2D-to-3D instance segmentation results and those of PE3R, with the reference raw 3D point cloud on the left. While PE3R’s reconstructed point cloud appears to be of lower quality than ours, it is worth noting that their segmentation results are produced independently and are not affected by the reconstruction quality.
Method
Poses
Reconstruct
Inference
Total
NeRF ( Mildenhall et al., 2020 )
2 mins
146 mins
1.5 mins
149.5 mins
3DGS ( Kerbl et al., 2023 )
2 mins
7 mins
-
9 mins
Ours
-
1 min
2.5 mins
3.5 mins
Table 4. Runtime costs . We show a breakdown of processing cost comparison including camera poses processing, scene reconstruction and views inference between our overall pipeline and conventional approaches.
Figure 8. Anomaly points elimination . We show the reconstruction, warping, and rendering visualization results before and after anomaly point elimination. On the left, the baseline method without anomaly point removal shows a large number of outlier points in the reconstructed point cloud, which significantly degrades the warping and final rendering results. In contrast, our proposed method effectively eliminates these anomaly points, resulting in cleaner reconstructions and better rendering quality.
Figure 9. Rendering masks .We show the object removal results with and without applying rendering masks to the source images. Without rendering masks, the rendered outputs tend to preserve the full geometry from the source images, even after the object has been removed from the point cloud. This leads to a failure in reflecting the intended object removal in the final rendered image.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 10. Editing tasks example . We show the original image of the view on the left. We apply a object point-cloud translation (red arrow) in the first row, followed by warping and rendering. We modify the object (red box) color of the input images before reconstruction, and the final rendered results reflect the applied color changes.
Sparse-view 3D reconstruction is essential for modeling scenes from casual captures, but remain challenging for non-generative reconstruction. Existing diffusion-based approaches mitigates this issues by synthesizing novel views, but they often condition on only one or two capture frames, which restricts geometric consistency and limits scalability to large or diverse scenes. We propose AnyRecon, a scalable framework for reconstruction from arbitrary and unordered sparse inputs that preserves explicit geometric control while supporting flexible conditioning cardinality. To support long-range conditioning, our method constructs a persistent global scene memory via a prepended capture view cache, and removes temporal compression to maintain frame-level correspondence under large viewpoint changes. Beyond better generative model, we also find that the interplay between generation and reconstruction is crucial for large-scale 3D scenes. Thus, we introduce a geometry-aware conditioning strategy that couples generation and reconstruction through an explicit 3D geometric memory and geometry-driven capture-view retrieval. To ensure efficiency, we combine 4-step diffusion distillation with context-window sparse attention to reduce quadratic complexity. Extensive experiments demonstrate robust and scalable reconstruction across irregular inputs, large viewpoint gaps, and long trajectories.
Yutian Chen, Shi Guo, Renbiao Jin +7
Shanghai AI Lab · Shanghai Jiao Tong University · The University of Hong Kong +2
Reconstructing 3D scenes from sparse, unposed images remains challenging under real-world conditions with varying illumination and transient occlusions. Existing methods rely on scene-specific optimization using appearance embeddings or dynamic masks, which requires extensive per-scene training and fails under sparse views. Moreover, evaluations on limited scenes raise questions about generalization. We present GenWildSplat, a feed-forward framework for sparse-view outdoor reconstruction that requires no per-scene optimization. Given unposed internet images, GenWildSplat predicts depth, camera parameters, and 3D Gaussians in a canonical space using learned geometric priors. An appearance adapter modulates appearance for target lighting conditions, while semantic segmentation handles transient objects. Through curriculum learning on synthetic and real data, GenWildSplat generalizes across diverse illumination and occlusion patterns. Evaluations on PhotoTourism and MegaScenes benchmark demonstrate state-of-the-art feed-forward rendering quality, achieving real-time inference without test-time optimization
Vinayak Gupta, Chih-Hao Lin, Shenlong Wang +2
University of Maryland, College Park · University of Illinois Urbana-Champaign · Johns Hopkins University
Generating complete 3D scenes from sparse, unconstrained views is a fundamental challenge in 3D vision which requires reasoning beyond observed content while remaining computationally tractable. Existing feed-forward reconstruction methods are inherently limited to content visible in the input images, while 3D generative modeling is hindered by the high computational cost of dense volumetric representations and the scarcity of large-scale 3D supervision. We introduce SPAR3S, a sparse voxel-aligned 3D latent generative model for conditional scene completion without requiring ground-truth 3D data for supervision. Our key insight is to formulate 3D scene generation in a structured, compact, voxel-aligned 3D latent space where only occupied voxels are represented. We learn this sparse latent space directly from multi-view images using photometric supervision via differentiable 3D Gaussian Splatting. Given a partial set of observed voxels encoded from sparse input views, scene completion reduces to predicting the missing latent tokens and their spatial support within the voxel grid. To this end, we train a masked autoregressive transformer that jointly models voxel occupancy and latent token values, enabling efficient and spatially consistent generation of unseen regions. We demonstrate the effectiveness of our method on synthetic indoor scenes, achieving higher novel-view quality than prior work. We further validate its generalization on RealEstate10k, highlighting its applicability to real-world data.
Thomas Lucas, Maxime Pietrantoni, Philippe Weinzaepfel +4
1NAVER LABS Europe, France · 2NAVER LABS, South Korea · 3Carnegie Mellon University, USA