Align3D: Image-Structure Co-Aligned 3D Scene Generation with Observability-Aware Fidelity-Preserving Guidance

Chenliang Zhou; Yifan Wang; Andrzej Banburski-Fahey

cs.CVNov 6, 2026

Align3D: Image-Structure Co-Aligned 3D Scene Generation with Observability-Aware Fidelity-Preserving Guidance

Authors: Chenliang Zhou, Yifan Wang, Andrzej Banburski-Fahey

Abstract

We present Align3D, an image-conditioned diffusion model that generates 3D indoor layouts that are both faithful to a single input photograph and physically plausible beyond the observed view. From the photograph we infer a confidence-calibrated scene graph and train a set-diffusion layout denoiser with image-structure co-alignment where confidence-weighted modality fusion balances image and scene-graph alignment using predicted confidences, while observability-aware fidelity-preserving guidance projects conflicting scene-quality gradients and reweighs them with an object-level confidence mask so under-observed degrees of freedom absorb physical priors without undoing the image-graph match. On 3D-FRONT, Align3D improves image-alignment and scene-quality metrics over recent baselines; once trained, it maps previously unseen photographs to layouts in a single feed-forward pass.

Figures & tables

Figure 1: Align3D couples an image signal and a confidence-calibrated structure signal via image-structure co-alignment; Observability-aware fidelity-preserving guidance completes under-observed regions under scene-quality guidance while protecting well-supported layout variables, yielding a coherent 3D scene.

Figure 2: Align3D pipeline, including scene graph inference, fused image-graph conditioning, and set-diffusion layout prediction. Training uses image-structure co-alignment and observability-aware fidelity-preserving guidance, and inference retrieves shapes for rendering.

Method	CD $\downarrow$	F-Score $\uparrow$	IoU $\uparrow$	CLIP $\uparrow$	DINO $\uparrow$	FID $\downarrow$	HP-Align $\uparrow$	HP-Scene $\uparrow$
DiffuScene ( Tang et al., 2024 )	0.292	36.1	0.289	0.784	0.758	29.8	42.8	44.1
Gen3DSR ( Ardelean et al., 2025 )	0.058	59.6	0.286	0.808	0.468	52.1	46.2	47.8
PartCrafter ( Lin et al., 2025 )	0.194	42.0	0.236	0.814	0.552	33.8	46.8	48.1
MIDI ( Huang et al., 2025 )	0.076	52.4	0.328	0.866	0.698	23.6	51.5	50.8
SceneGen ( Meng et al., 2026 )	0.016	87.8	0.541	0.909	0.824	20.3	54.2	54.8
Align3D (ours)	0.019	88.7	0.558	0.925	0.843	20.8	57.2	56.1

Table 1: Main comparison on 3D-FRONT ( Fu et al., 2021a ) . Align3D is best on image-alignment metrics (CLIP, DINO, HP-Align) and is best or second-best across scene-quality metrics.

Figure 3: Qualitative image-to-scene comparison on real-world indoor scenes. Each row shows the input photograph and matched-view renderings from different methods. Align3D better matches the observed furniture arrangement and reduces collisions and floating objects relative to the baselines.

Variant	CD $\downarrow$	F-Score $\uparrow$	IoU $\uparrow$	CLIP $\uparrow$	DINO $\uparrow$	FID $\downarrow$
Quality loss: $\lambda\mathcal{L}_{\text{quality}}$	0.286	36.8	0.312	0.786	0.754	29.1
Image alignment: $\mathcal{L}_{\text{image}}+\lambda\mathcal{L}_{\text{quality}}$	0.068	76.4	0.438	0.896	0.816	23.9
SG alignment: $\mathcal{L}_{\text{sg}}+\lambda\mathcal{L}_{\text{quality}}$	0.044	81.2	0.516	0.838	0.782	25.4
Co-alignment: $\mathcal{L}_{\text{image}}+\mathcal{L}_{\text{sg}}+\lambda\mathcal{L}_{\text{quality}}$	0.027	86.4	0.542	0.898	0.818	22.1
CWMF: $\mathcal{L}_{\text{co-align}}+\lambda\mathcal{L}_{\text{quality}}$	0.025	86.8	0.554	0.901	0.822	21.2

Table 2: Image-structure co-alignment ablation. Quality loss alone cannot ground layouts in the input; image and scene-graph terms are each necessary for complementary visual and structural gains, and confidence-weighted CWMF introduces further improvements over fixed-weight fusion.

Variant	CD $\downarrow$	F-Score $\uparrow$	IoU $\uparrow$	CLIP $\uparrow$	DINO $\uparrow$	FID $\downarrow$
Sum: $\mathbf{g}_{\text{co-align}}+\lambda\mathbf{g}_{\text{quality}}$	0.025	86.8	0.554	0.901	0.822	21.2
Projection: $\mathbf{g}_{\text{co-align}}+\lambda\tilde{\mathbf{g}}_{\text{quality}}$	0.020	88.1	0.555	0.921	0.840	21.0
Reversed projection: $\tilde{\mathbf{g}}_{\text{co-align}}+\lambda\mathbf{g}_{\text{quality}}$	0.031	84.6	0.571	0.866	0.798	20.5
OFG: $\mathbf{g}_{\text{co-align}}+\lambda\,C\tilde{\mathbf{g}}_{\text{quality}}$	0.019	88.7	0.558	0.925	0.843	20.8

Table 3: OFG ablation. Unconstrained summation hurts alignment; projection is necessary to avoid gradient conflict, and confidence mask

C

introduces further improvements.

Method	Training (h)	Inference (s/scene)
DiffuScene ( Tang et al., 2024 )	120	14
Gen3DSR ( Ardelean et al., 2025 )	48	250
PartCrafter ( Lin et al., 2025 )	380	34
MIDI ( Huang et al., 2025 )	120	42
SceneGen ( Meng et al., 2026 )	600	125
Align3D (ours)	60	10

Table 4: Training and inference cost. Our method is fast while producing high quality results.

Figure 4: Quality vs. efficiency plots. Our method achieves both high quality and fast runtime.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Qualitative comparison on nine real-world indoor scenes.

3D Scene Layout Generation

3D scene layout generation arranges objects and structures into coherent, controllable scenes from text, images, plans, or functional goals.

See all 85 papers

Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstruction
Sep 20, 2026Yang-Tian Sun, Tianjia Liu, Zehuan Huang +73D Reconstruction 3D Scene Layout Generation
The University of Hong Kong · VAST
Pose-Aware Diffusion for 3D Generation
May 1, 2026Zihan Zhou, Luxi Chen, Jingzhi Zhou +43D Asset Generation 3D Scene Generation
Gaoling School of AI, Renmin University of China · VCIP, School of Computer Science, Nankai University · THU-Bosch MLCenter, Tsinghua University +1
Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Image
Mar 6, 2026Zidian Qiu, Ancong Wu3D Scene Generation Image-to-3D Scene Generation
Sun Yat-sen University

Sep 20, 2026J/K move · Enter open · S save

Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstruction

Yang-Tian Sun, Tianjia Liu, Zehuan Huang, Yi-Hua Huang+6

The University of Hong Kong · VAST

3D Reconstruction 3D Scene Layout Generation Single-View 3D Reconstruction Image-to-3D Scene Generation

Open paper AskPDF arXiv

Single-image 3D object generation can now produce high-fidelity assets, yet accurately placing them into a coherent scene layout remains an open challenge. A central difficulty lies in how object layout is represented. Holistic methods absorb placement into a scene-level generation process, sacrificing object-level detail. Compositional methods preserve object fidelity by decoupling geometry from layout, but typically parameterize layout as sparse, unbounded pose variables that are difficult to learn and generalize poorly under scarce scene-level supervision. We present Mira-Scene, a compositional 3D scene reconstruction framework that replaces sparse pose regression with dense, bounded correspondence recovery. At its core is the Canonical Coordinate Map (CCM), a pixel-aligned field that maps each visible object pixel to a surface coordinate in the object's bounded canonical space. When paired with a scene-space Point Cloud Map (PCM) from monocular geometry estimation, CCM induces dense canonical-to-scene correspondences from which object transformations are recovered through robust geometric alignment. Because CCM operates in bounded canonical space, it provides a stable prediction target that can be trained from scalable object-level 3D data without requiring scene-level layout annotations. Mira-Scene further introduces a multimodal diffusion transformer that jointly generates object geometry and CCMs, using modality-specific expert streams with shared attention and positional encoding to promote geometry-layout consistency. Experiments on indoor, outdoor, synthetic, and in-the-wild scenes show that Mira-Scene substantially outperforms strong baselines in layout accuracy, achieving relative gains of 39.8% in 3D-IoU and 16.5% in 2D-IoU over SAM3D, using limited open-source training data.

Abstract

3D Scene Layout Generation

Explore similar work

Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstruction

Pose-Aware Diffusion for 3D Generation

Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Image