Easy-3Dfy: Object-Level Geometric Composition for Single-Image 3D Scene Construction

Ziang Zhang; Wenyou Yi; Zehan Wang; Jiachen Xu; Xintong Han; Chunchao Guo; Zhou Zhao

cs.CVOct 23, 2026

Easy-3Dfy: Object-Level Geometric Composition for Single-Image 3D Scene Construction

Authors: Ziang Zhang, Wenyou Yi, Zehan Wang, Jiachen Xu, Xintong Han, Chunchao Guo, Zhou Zhao

Organizations: Zhejiang University · Tencent

Abstract

Single-image 3D scene construction is a fundamental problem in 3D vision. Recent methods such as SAM-3Dfy enable end-to-end reconstruction of 3D assets and their scene layouts, but still struggle with fine-grained scene geometry and asset quality. Meanwhile, specialized 3D object generators provide detailed object geometry, while 3D foundation models offer strong and generalizable spatial understanding. We propose Easy-3Dfy, a compositional workflow that decouples object generation from scene layout estimation, allowing specialized models to fully exploit their respective strengths. An RPF-based aligner then bridges these capabilities by establishing geometric correspondence between object-local meshes and partial scene pointmaps, recovering object layout for scene composition. Without requiring complete scene-level supervision, the proposed workflow consistently outperforms SAM-3Dfy across multiple geometric metrics on Scene-Maker and 3D-FUTURE200. Our results demonstrate that explicit geometric composition can effectively combine high-quality object generation with strong spatial understanding, while enabling the workflow to benefit independently from advances in both 3D object generation and foundation models.

Figures & tables

Figure 2: Overview of Easy-3Dfy. Given object masks, a spatial model predicts a scene pointmap. For each object, an image editor produces a completed object image and a replaceable image-to-3D provider generates a local mesh. The RPF-based aligner matches its anchor point cloud to the object-specific partial query extracted from the pointmap, predicts correspondence positions, and recovers a uniform-scale similarity transform via Umeyama fitting. Transforming all complete meshes assembles the final scene.

Figure 3: Object-level data construction and RPF training. Aligned RGB–depth renders yield a monocular partial query and pixel-matched ground-truth correspondence positions. Independent anchor- and query-side geometric augmentations define the registration task, on which RPF learns the conditional velocity field.

Dataset	Method	Scene-level		Object-level
Dataset	Method	CD $\downarrow$	F-Score $\uparrow$	CD $\downarrow$	F-Score $\uparrow$	BBox IoU $\uparrow$
Scene-Maker	MIDI	0.442	0.329	0.379	0.384	0.065
	SceneGen	0.455	0.368	0.264	0.543	0.085
	SAM-3Dfy	0.261	0.615	0.188	0.695	0.366
	Easy-3Dfy	0.242	0.649	0.188	0.695	0.399
3D-FUTURE200	MIDI	0.275	0.627	0.171	0.736	0.319

Table 1: Main results on Scene-Maker and 3D-FUTURE200. Easy-3Dfy and SAM-3Dfy share identical meshes, isolating layout prediction. Bold marks the best result per dataset.

Figure 4: How mesh quality affects visual and layout quality. Matched input–panorama–close-up comparisons. Each case consists of two columns, with the first showing the reconstructed full 3D scene and the second showing close-up views of the same asset regions.

Table 5

Figure 5: What direct alignment improves. Matched comparisons of MIDI, SceneGen, SAM-3Dfy, and Easy-3Dfy under a common rendering protocol. The multi-view results show that Easy-3Dfy reduces floating and interpenetrating objects while improving object ordering and placement.

Mesh	Sce. CD $\downarrow$	Sce. F $\uparrow$	Obj. CD $\downarrow$	Obj. F $\uparrow$	BBox IoU $\uparrow$
GT	0.242	0.660	0.144	0.789	0.415
SAM-3Dfy	0.242	0.649	0.188	0.695	0.399
TRELLIS-2.0	0.284	0.557	0.273	0.548	0.317
HY3D-2.0	0.295	0.523	0.278	0.514	0.291
HY3D-3.1	0.256	0.629	0.196	0.678	0.385
Meshy-T2	0.253	0.632	0.201	0.667	0.381

Table 4: Mesh-provider comparison on the Scene-Maker dataset with all other Easy-3Dfy components fixed. GT provides an upper-bound reference under the current layout workflow.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Variant	Scene CD $\downarrow$	Scene F $\uparrow$	BBox IoU $\uparrow$
Scene-Maker	Standard	0.242	0.649	0.399
	Uncleaned Point	0.242	0.646	0.399
	XYZ only	0.245	0.641	0.394
3D-FUTURE200	Standard	0.140	0.836	0.554
	Uncleaned Point	0.148	0.824	0.544
	XYZ only	0.150	0.812	0.535

Appendix

Table 5: Complete alignment-design ablation. “Uncleaned Point” omits point-cloud cleaning, while “XYZ only” removes surface-normal conditioning.

Dataset	Variant	Scene CD $\downarrow$	Scene F $\uparrow$	Obj. CD $\downarrow$	Obj. F $\uparrow$	BBox IoU $\uparrow$
Scene-Maker	MogeV2 + FLUX-2	0.253	0.632	0.201	0.667	0.381
	MogeV2 + GPT-Image-2	0.254	0.630	0.208	0.656	0.381
	DAv3 + FLUX-2	0.267	0.609	0.215	0.640	0.356
3D-FUTURE200	MogeV2 + FLUX-2	0.146	0.814	0.151	0.782	0.543
	MogeV2 + GPT-Image-2	0.142	0.822	0.156	0.775	0.543
	DAv3 + FLUX-2	0.169	0.772	0.173	0.743	0.476

Appendix

Table 6: Complete scene-geometry and object-completion ablation with Meshy-T2 fixed as the mesh provider and six-channel XYZ–normal RPF input.

Dataset	Mesh source	Sce. CD $\downarrow$	Sce. F $\uparrow$	Obj. CD $\downarrow$	Obj. F $\uparrow$	BBox IoU $\uparrow$
Scene- Maker	GT	0.242	0.660	0.144	0.789	0.415
	SAM-3Dfy	0.242	0.649	0.188	0.695	0.399
	TRELLIS-2.0	0.284	0.557	0.273	0.548	0.317
	HY3D-2.0	0.295	0.523	0.278	0.514	0.291
	HY3D-3.1	0.256	0.629	0.196	0.678	0.385
	Meshy-T2	0.253	0.632	0.201	0.667	0.381

Appendix

Table 7: Complete Mesh-provider comparison with all other Easy-3Dfy components fixed. GT provides an upper-bound reference under the current layout workflow.

Dataset	Metric	9ch: XYZ + normal + RGB	6ch: XYZ + normal	$\Delta$ (6ch $-$ 9ch)
3D-FUTURE200	Scene CD $\downarrow$	0.1534	0.1455	$-0.0079$
	Scene F $\uparrow$	0.7973	0.8136	$+0.0163$
	BBox IoU $\uparrow$	0.5231	0.5428	$+0.0197$
Scene-Maker	Scene CD $\downarrow$	0.2617	0.2527	$-0.0090$
	Scene F $\uparrow$	0.6204	0.6320	$+0.0116$
	BBox IoU $\uparrow$	0.3709	0.3812	$+0.0103$

Appendix

Table 8: Effect of RGB conditioning under fixed FLUX-2-pro completion and Meshy-T2 assets. The 9-channel checkpoint is retrained on training pairs that include RGB features.

Figure A1: Additional visualization results on 3D-FUTURE.

Figure A2: Failure mode comparison under geometrically challenged conditions.

Image-to-3D Scene Generation

See all 27 papers

Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstruction
Sep 20, 2026Yang-Tian Sun, Tianjia Liu, Zehuan Huang +73D Reconstruction 3D Scene Layout Generation
The University of Hong Kong · VAST
3D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Image
Apr 6, 2026Ze-Xin Yin, Liu Liu, Xinjie Wang +43D Scene Completion 3D Asset Generation
College of Computer Science, Nankai University · Horizon Robotics · D-Robotics +1
Pose-Aware Diffusion for 3D Generation
May 1, 2026Zihan Zhou, Luxi Chen, Jingzhi Zhou +43D Asset Generation 3D Scene Generation
Gaoling School of AI, Renmin University of China · VCIP, School of Computer Science, Nankai University · THU-Bosch MLCenter, Tsinghua University +1

Sep 20, 2026J/K move · Enter open · S save

Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstruction

Yang-Tian Sun, Tianjia Liu, Zehuan Huang, Yi-Hua Huang+6

The University of Hong Kong · VAST

3D Reconstruction 3D Scene Layout Generation Single-View 3D Reconstruction Image-to-3D Scene Generation

Open paper AskPDF arXiv

Single-image 3D object generation can now produce high-fidelity assets, yet accurately placing them into a coherent scene layout remains an open challenge. A central difficulty lies in how object layout is represented. Holistic methods absorb placement into a scene-level generation process, sacrificing object-level detail. Compositional methods preserve object fidelity by decoupling geometry from layout, but typically parameterize layout as sparse, unbounded pose variables that are difficult to learn and generalize poorly under scarce scene-level supervision. We present Mira-Scene, a compositional 3D scene reconstruction framework that replaces sparse pose regression with dense, bounded correspondence recovery. At its core is the Canonical Coordinate Map (CCM), a pixel-aligned field that maps each visible object pixel to a surface coordinate in the object's bounded canonical space. When paired with a scene-space Point Cloud Map (PCM) from monocular geometry estimation, CCM induces dense canonical-to-scene correspondences from which object transformations are recovered through robust geometric alignment. Because CCM operates in bounded canonical space, it provides a stable prediction target that can be trained from scalable object-level 3D data without requiring scene-level layout annotations. Mira-Scene further introduces a multimodal diffusion transformer that jointly generates object geometry and CCMs, using modality-specific expert streams with shared attention and positional encoding to promote geometry-layout consistency. Experiments on indoor, outdoor, synthetic, and in-the-wild scenes show that Mira-Scene substantially outperforms strong baselines in layout accuracy, achieving relative gains of 39.8% in 3D-IoU and 16.5% in 2D-IoU over SAM3D, using limited open-source training data.

Abstract

Image-to-3D Scene Generation

Explore similar work

Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstruction

3D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Image

Pose-Aware Diffusion for 3D Generation