We present Align3D, an image-conditioned diffusion model that generates 3D indoor layouts that are both faithful to a single input photograph and physically plausible beyond the observed view. From the photograph we infer a confidence-calibrated scene graph and train a set-diffusion layout denoiser with image-structure co-alignment where confidence-weighted modality fusion balances image and scene-graph alignment using predicted confidences, while observability-aware fidelity-preserving guidance projects conflicting scene-quality gradients and reweighs them with an object-level confidence mask so under-observed degrees of freedom absorb physical priors without undoing the image-graph match. On 3D-FRONT, Align3D improves image-alignment and scene-quality metrics over recent baselines; once trained, it maps previously unseen photographs to layouts in a single feed-forward pass.
Figures & tables
Figure 1: Align3D couples an image signal and a confidence-calibrated structure signal via image-structure co-alignment; Observability-aware fidelity-preserving guidance completes under-observed regions under scene-quality guidance while protecting well-supported layout variables, yielding a coherent 3D scene.
Figure 2: Align3D pipeline, including scene graph inference, fused image-graph conditioning, and set-diffusion layout prediction. Training uses image-structure co-alignment and observability-aware fidelity-preserving guidance, and inference retrieves shapes for rendering.
Method
CD ↓
F-Score ↑
IoU ↑
CLIP ↑
DINO ↑
FID ↓
HP-Align ↑
HP-Scene ↑
DiffuScene ( Tang et al., 2024 )
0.292
36.1
0.289
0.784
0.758
29.8
42.8
44.1
Gen3DSR ( Ardelean et al., 2025 )
0.058
59.6
0.286
0.808
0.468
52.1
46.2
47.8
PartCrafter ( Lin et al., 2025 )
0.194
42.0
0.236
0.814
0.552
33.8
46.8
48.1
MIDI ( Huang et al., 2025 )
0.076
52.4
0.328
0.866
0.698
23.6
51.5
50.8
SceneGen ( Meng et al., 2026 )
0.016
87.8
0.541
0.909
0.824
20.3
54.2
54.8
Align3D (ours)
0.019
88.7
0.558
0.925
0.843
20.8
57.2
56.1
Table 1: Main comparison on 3D-FRONT ( Fu et al., 2021a ) . Align3D is best on image-alignment metrics (CLIP, DINO, HP-Align) and is best or second-best across scene-quality metrics.
Figure 3: Qualitative image-to-scene comparison on real-world indoor scenes. Each row shows the input photograph and matched-view renderings from different methods. Align3D better matches the observed furniture arrangement and reduces collisions and floating objects relative to the baselines.
Variant
CD ↓
F-Score ↑
IoU ↑
CLIP ↑
DINO ↑
FID ↓
Quality loss: λLquality
0.286
36.8
0.312
0.786
0.754
29.1
Image alignment: Limage+λLquality
0.068
76.4
0.438
0.896
0.816
23.9
SG alignment: Lsg+λLquality
0.044
81.2
0.516
0.838
0.782
25.4
Co-alignment: Limage+Lsg+λLquality
0.027
86.4
0.542
0.898
0.818
22.1
CWMF: Lco-align+λLquality
0.025
86.8
0.554
0.901
0.822
21.2
Table 2: Image-structure co-alignment ablation. Quality loss alone cannot ground layouts in the input; image and scene-graph terms are each necessary for complementary visual and structural gains, and confidence-weighted CWMF introduces further improvements over fixed-weight fusion.
Variant
CD ↓
F-Score ↑
IoU ↑
CLIP ↑
DINO ↑
FID ↓
Sum: gco-align+λgquality
0.025
86.8
0.554
0.901
0.822
21.2
Projection: gco-align+λg~quality
0.020
88.1
0.555
0.921
0.840
21.0
Reversed projection: g~co-align+λgquality
0.031
84.6
0.571
0.866
0.798
20.5
OFG: gco-align+λCg~quality
0.019
88.7
0.558
0.925
0.843
20.8
Table 3: OFG ablation. Unconstrained summation hurts alignment; projection is necessary to avoid gradient conflict, and confidence mask C introduces further improvements.
Method
Training (h)
Inference (s/scene)
DiffuScene ( Tang et al., 2024 )
120
14
Gen3DSR ( Ardelean et al., 2025 )
48
250
PartCrafter ( Lin et al., 2025 )
380
34
MIDI ( Huang et al., 2025 )
120
42
SceneGen ( Meng et al., 2026 )
600
125
Align3D (ours)
60
10
Table 4: Training and inference cost. Our method is fast while producing high quality results.
Figure 4: Quality vs. efficiency plots. Our method achieves both high quality and fast runtime.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Qualitative comparison on nine real-world indoor scenes.
Single-image 3D object generation can now produce high-fidelity assets, yet accurately placing them into a coherent scene layout remains an open challenge. A central difficulty lies in how object layout is represented. Holistic methods absorb placement into a scene-level generation process, sacrificing object-level detail. Compositional methods preserve object fidelity by decoupling geometry from layout, but typically parameterize layout as sparse, unbounded pose variables that are difficult to learn and generalize poorly under scarce scene-level supervision. We present Mira-Scene, a compositional 3D scene reconstruction framework that replaces sparse pose regression with dense, bounded correspondence recovery. At its core is the Canonical Coordinate Map (CCM), a pixel-aligned field that maps each visible object pixel to a surface coordinate in the object's bounded canonical space. When paired with a scene-space Point Cloud Map (PCM) from monocular geometry estimation, CCM induces dense canonical-to-scene correspondences from which object transformations are recovered through robust geometric alignment. Because CCM operates in bounded canonical space, it provides a stable prediction target that can be trained from scalable object-level 3D data without requiring scene-level layout annotations. Mira-Scene further introduces a multimodal diffusion transformer that jointly generates object geometry and CCMs, using modality-specific expert streams with shared attention and positional encoding to promote geometry-layout consistency. Experiments on indoor, outdoor, synthetic, and in-the-wild scenes show that Mira-Scene substantially outperforms strong baselines in layout accuracy, achieving relative gains of 39.8% in 3D-IoU and 16.5% in 2D-IoU over SAM3D, using limited open-source training data.
Generating pose-aligned 3D objects is challenging due to the spatial mismatches and transformation ambiguities inherent in decoupled canonical-then-rotate paradigms. To this end, we introduce Pose-Aware Diffusion (PAD), a novel end-to-end diffusion framework that synthesizes 3D geometry directly within the observation space. By unprojecting monocular depth into a partial point cloud and explicitly injecting it as a 3D geometric anchor, PAD abandons canonical assumptions to enforce rigorous spatial supervision. This native generation intrinsically resolves pose ambiguity, producing high-fidelity pose-aligned assets. Extensive experiments demonstrate that PAD achieves superior geometric alignment and image-to-3D correspondence compared to state-of-the-art methods. Additionally, PAD naturally extends to compositional 3D scene reconstruction via a simple union of independently generated objects, highlighting its robust ability to preserve precise spatial layouts.
Zihan Zhou, Luxi Chen, Jingzhi Zhou +4
Gaoling School of AI, Renmin University of China · VCIP, School of Computer Science, Nankai University · THU-Bosch MLCenter, Tsinghua University +1
Current compositional image-to-3D scene generation approaches construct 3D scenes by time-consuming iterative layout optimization or inflexible joint object-layout generation. Moreover, most methods rely on limited field-of-view perspective images, hindering the creation of complete 360-degree environments. To address these limitations, we design Pano3DComposer, an efficient feed-forward framework for panoramic images. To decouple object generation from layout estimation, we propose a plug-and-play Object-World Transformation Predictor. This module converts the 3D objects generated by off-the-shelf image-to-3D models from local to world coordinates. To achieve this, we adapt the VGGT architecture to Alignment-VGGT by using target object crop, multi-view object renderings and camera parameters to predict the transformation. The predictor is trained using pseudo-geometric supervision to address the shape discrepancy between generated and ground-truth objects. For input images from unseen domains, we further introduce a Coarse-to-Fine (C2F) alignment mechanism for Pano3DComposer that iteratively refines geometric consistency with feedback of scene rendering. Our method achieves superior geometric accuracy for image/text-to-3D tasks on synthetic and real-world datasets. It can generate a high-fidelity 3D scene in approximately 20 seconds on an RTX 4090 GPU. Project page: https://qiuzidian.github.io/pano3dcomposer-page/.