WorldRecon: Generative Multi-View Mesh Reconstruction for Large-Scale 3D World
Organizations: Zhejiang University · Tencent Hunyuan · Fudan University
Abstract
Reconstructing complete, high-quality meshes of large-scale scenes from unposed multi-view images requires faithful geometric grounding while plausibly completing surfaces that are unobserved or only sparsely captured. We present WorldRecon, a generative multi-view reconstruction framework for recovering complete, large-scale 3D scene meshes from unposed RGB images. WorldRecon uses a coarse, noisy global point cloud and camera poses estimated from the input images by a reconstruction model as a global scaffold for whole-scene generation and partitions it into overlapping spatial blocks. To robustly leverage these imperfect geometric priors, we equip the native 3D generative model TRELLIS.2 with a dedicated reconstruction-based conditioning architecture that generates each block from its local point-cloud crop and block-relevant image observations, producing sparse structure, detailed geometry, and PBR materials. To maintain scene-level coherence, we introduce neighbour-aware latent inpainting and a causal world-generation procedure that propagates structural, geometric, and material context across overlapping blocks. Together, these designs scale multi-view generative reconstruction to large-scale scenes, yielding coherent, unified meshes. Extensive experiments demonstrate that WorldRecon achieves state-of-the-art reconstruction performance, reducing Chamfer distance by 14.9% and 43.1% while improving F-score by 17.3% and 12.0% over the strongest baselines on synthetic and real-world indoor scenes, respectively, thereby demonstrating faithful and complete generative reconstruction as well as strong sim-to-real generalization. Project page: https://kszpxxzmc.github.io/WorldRecon-website/.
Figures & tables
| Method | 3D Geometry | Coverage | Depth | Normal | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CD | F-score | NC | Complete. | RMSE | AbsRel | LPIPS | CLIP | Angular | LPIPS | CLIP | |
| VGGT- +TSDF | 0.2934 | 0.4596 | 0.5433 | 0.7363 | 0.5906 | 0.1548 | 0.4873 | 0.8679 | 20.9899 | 0.6188 | 0.8568 |
| Murre | 0.5295 | 0.2426 | 0.4729 | 0.4572 | 0.8847 | 0.2397 | 0.5273 | 0.8332 | 22.6896 | 0.6671 | 0.8542 |
| MonoSDF | 0.2558 | 0.4936 | 0.5730 | 0.9056 | 0.6052 | 0.1616 | 0.2566 | 0.9161 | 18.2223 | 0.3560 | 0.9062 |
| FineRecon | 0.2448 | 0.5301 | 0.6252 | 0.8126 | 0.5578 | 0.1490 | 0.3398 | 0.8906 | 17.6099 | 0.4419 | 0.8845 |
| GenRecon | 0.1861 | 0.6196 | 0.6128 | 0.9587 | 0.5348 | 0.1666 | 0.2548 | 0.9490 | 17.9822 | 0.3139 | 0.9543 |
| Method | 3D Geometry | Coverage | Depth | Normal | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CD | F-score | NC | Complete. | RMSE | AbsRel | LPIPS | CLIP | Angular | LPIPS | CLIP | |
| VGGT- +TSDF | 0.1121 | 0.7467 | 0.7036 | 0.8737 | 0.3984 | 0.1433 | 0.3907 | 0.8991 | 25.4852 | 0.5075 | 0.8888 |
| Murre | 0.1756 | 0.5640 | 0.6222 | 0.5567 | 0.6619 | 0.1863 | 0.5587 | 0.8690 | 28.4193 | 0.5642 | 0.8761 |
| MonoSDF | 0.1665 | 0.6224 | 0.6913 | 0.9670 | 0.6293 | 0.3043 | 0.2705 | 0.9273 | 23.7527 | 0.3861 | 0.9016 |
| FineRecon | 0.1259 | 0.6861 | 0.7226 | 0.9388 | 0.4405 | 0.1673 | 0.2707 | 0.9183 | 22.6830 | 0.4043 | 0.8886 |
| GenRecon | 0.1227 | 0.5901 | 0.5969 | 0.9722 | 0.6837 | 0.3048 | 0.3862 | 0.9291 | 31.5722 | 0.4711 | 0.9172 |
| Components | 3D Metrics | 2D Metrics | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| SS | Geo&Mat | Inpaint | Stitch | CD | F-score | NC | Complete. | MAE | AbsRel | Angular | RGB CLIP |
| 0.4003 | 0.2013 | 0.1682 | 0.7377 | 1.1847 | 0.6160 | 48.6471 | 0.7255 | ||||
| ✓ | 0.0996 | 0.7438 | 0.7173 | 0.9722 | 0.1659 | 0.0980 | 19.6434 | 0.8244 | |||
| ✓ | ✓ | 0.0954 | 0.7624 | 0.7349 | 0.9858 | 0.1589 | 0.0890 | 17.7610 | 0.8609 | ||
| ✓ | ✓ | ✓ | 0.1023 | 0.7482 | 0.7202 | 0.9868 | 0.1439 | 0.0830 | 17.2462 | 0.8609 | |
| ✓ | ✓ | ✓ | ✓ | 0.0830 | 0.8226 | 0.7640 | 0.9894 | 0.1394 | 0.0815 | 16.5789 | 0.8671 |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Stage | Image | Latent | Points | GPUs | Global Batch | LR | EMA | Adaptation |
|---|---|---|---|---|---|---|---|---|
| SS generation | full | |||||||
| SS inpainting | full | |||||||
| Geometry generation | – | full | ||||||
| Geometry inpainting | – | full | ||||||
| Material generation | – | LoRA | ||||||
| Material inpainting | – | LoRA |
3D Reconstruction
3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.
See all 921 papers