We identify occlusion reasoning as a fundamental yet overlooked aspect for 3D layout-conditioned generation. It is essential for synthesizing partially occluded objects with depth-consistent geometry and scale. While existing methods can generate realistic scenes that follow input layouts, they often fail to model precise inter-object occlusions. We propose SeeThrough3D, a model for 3D layout conditioned generation that explicitly models occlusions. We introduce an occlusion-aware 3D scene representation (OSCR), where objects are depicted as translucent 3D boxes placed within a virtual environment and rendered from desired camera viewpoint. The transparency encodes hidden object regions, enabling the model to reason about occlusions, while the rendered viewpoint provides explicit camera control during generation. We condition a pretrained flow based text-to-image image generation model by introducing a set of visual tokens derived from our rendered 3D representation. Furthermore, we apply masked self-attention to accurately bind each object bounding box to its corresponding textual description, enabling accurate generation of multiple objects without object attribute mixing. To train the model, we construct a synthetic dataset with diverse multi-object scenes with strong inter-object occlusions. SeeThrough3D generalizes effectively to unseen object categories and enables precise 3D layout control with realistic occlusions and consistent camera control.
Figures & tables
Figure 2: OSCR: We propose O cclusion-Aware Sc ene R epresentation (OSCR) for 3D layout control in text-to-image generation. OSCR describes objects as translucent 3D boxes, which exposes occluded regions, enabling the generative model to reason about occlusions. Further, each box face is color-coded with a mapping to encode its 3D orientation. (a) A user specifies the object bounding boxes ( b0 and b1 ) and sets desired viewpoint C in an interactive graphic environment. (b) These boxes are rendered to obtain our OSCR representation, (c) which is used to condition the generation for occlusion aware 3D control.
Figure 3: Towards occlusion aware 3D scene layouts: existing methods represent scenes as (a) 3D layout depth maps [ 4 , 19 , 65 ] , which fail to represent occluded objects (see dashed red box), or (b) object layers [ 76 , 37 ] , which are not 3D aware, hence fail to capture camera viewpoint and perspective. (c) Therefore, we propose OSCR, where objects are described using translucent 3D bounding boxes. The transparency exposes occluded regions (red box), providing cues for occlusion reasoning, while enabling 3D layout control.
Figure 4: SeeThrough3D: We encode the rendered OSCR condition map r using the VAE to obtain OSCR tokens. These are concatenated with text prompt tokens p and noisy image tokens xt . The concatenated result is passed through the DiT based text-to-image model where they are jointly processed using self attention modules. We inject LoRA [ 25 ] onto the attention projections corresponding to OSCR tokens; this enables control while preserving prior of the base model [ 79 , 61 , 62 ] .
Figure 5: (a) Inside the mmDiT block, text tokens p , image tokens xt and OSCR tokens z are jointly processed using self attention, conditioning the generation on our OSCR representation. To bind objects to corresponding boxes, we mask the attention to enable OSCR tokens within each box {bi} to attend to corresponding object tokens {pi} using a mask M (b) For this, we require spatial extent for each object box bi , which we obtain we use its amodal segmentation mask si . When multiple boxes overlap, their region of intersection (green) attends to multiple objects.
Figure 6: Visualizing object disentanglement in latent space: Given a layout with heavy occlusion like (a), our model’s outputs show precise occlusion boundaries (b). To understand this, we visualize attention from image-tokens to object tokens in prompt (bicycle and van). Interestingly, the attention maps themselves reveal occlusion boundaries: inside the empty regions of the bicycle structure, attention on the van remains visible, accurately reflecting its presence behind the bicycle. This suggests that object-specific features remain distinct in the model’s latent space, indicating strong priors for occlusion reasoning.
Figure 7: Dataset creation: We place 3D assets in controlled configurations in Blender [ 12 ] . Object placements and camera viewpoint are controlled to ensure strong occlusions, while ensuring adequate visibility for each object. To generate realistic augmentations, we estimate image depth, and pass it through a depth-to-image model [ 35 ] with diverse background prompts.
Figure 8: Qualitative results: Our method is able to precisely follow 3D scene layouts, with high occlusion consistency. Our approach preserves the prior of text-to-image model, as evident from capabilities like see-through transparent objects (A,B,G,J), text rendering (G) and inter-object interactions (E,F). Additionally, our method enables control over viewpoint of generated image (C,D). Despite being trained on layouts with only upto 4 objects, our method is able to generalize to complex scenes with many objects (G,H,I,J).
Figure 9: Qualitative comparison: We compare against works on 3D layout control : LooseControl [ 4 ] and Build-A-Scene [ 19 ] , and on occlusion control : LaRender [ 76 ] and VODiff [ 37 ] .
Baselines
depth ord. ↑
obj. score ↑
angular err. ↓
text align. ↑
KID (×10−3)↓
VODiff [ 37 ]
0.68
19.70
92.73
29.51
15.40
LooseControl [ 4 ]
0.82
20.02
89.88
28.43
14.32
Build-A-Scene [ 19 ]
0.89
21.0
91.62
28.05
20.12
LaRender [ 76 ]
1.02
21.83
89.63
30.20
13.46
Ours
1.46
22.86
47.92
31.87
5.43
Table 1: Quantitative comparison: We compute (a) depth ordering, which reflects 3D location and occlusion consistency, (b) CLIP objectness score, which indicates layout adherence and object fidelity (c) angular error, which indicates orientation correctness (d) image-text prompt alignment using CLIP [ 56 ] , and (e) KID [ 5 ] , which measures image fidelity.
Figure 10: User study: Each bar indicates the % of times our method’s output was preferred over the baseline, for each category.
Figure 11: Personalization: Our method can be extended for personalized 3D control using reference image of an object.
Figure 12: Ablations: We ablate upon key aspects of OSCR representation, our binding mechanism and data preparation strategy.
Ablations
depth ord. ↑
obj. score ↑
angular err. ↓
text align. ↑
KID (×10−3)↓
w/o transparency
1.20
21.67
46.15
31.39
5.90
w/o color-coding
1.36
22.23
88.77
31.57
5.93
w/o binding
0.98
20.45
57.44
31.61
6.35
w/o hard data
1.24
21.89
49.73
31.32
6.34
Ours
1.46
22.86
47.92
31.87
5.43
Table 2: Quantitative results of ablative experiments.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 13: CLIP filtering on augmentations: We use a depth-to-image model (FLUX.1-Depth-dev [ 35 ] ) to generate realistic augmentations of the rendered images (a). However, the depth-to-image model occasionally misaligns objects with their intended depth regions, causing incorrect placements. For example, on the left pane, the depth-to-image model incorrectly generates a pigeon in place of a crow according to original layout. We mitigate such issues by applying object-level CLIP-based filtering [ 56 ] to retain only those augmentations that adhere to the original layout. For this, we first obtain the object segmentation masks using Blender [ 12 ] , and use these to obtain cropped object segments in the augmented image (b). Next, we compute CLIP similarity between these object segments and corresponding text description ( e.g . cat, pigeon, etc.), as shown in (c). If any object has a CLIP score less than the threshold value of 0.25 , the augmentation is filtered out. High CLIP scores for all object segments (as shown on the right pane) indicates accurate layout adherence, and these images are included in the training dataset.
Figure 14: Statistics of training dataset: (a) We plot the distribution of minimum visibility ratio for any object in the scene. Since our filtering strategy favors heavy occlusion scenarios, we observe that there is a bias towards cases with low visibility ratios. (b) Next, we observe that the distribution of orientation values is roughly uniform, thus avoiding any unwanted biases. (c) Interestingly, we observe that the frequency of examples with large 2D bounding box dimension shows a decreasing trend. This is because smaller object sizes enable placement of multiple objects in a scene, while ensuring all of them are visible. (d) High camera shots tend to have weaker occlusions compared to low camera shots; for instance, there are very little inter-object occlusions in bird’s-eye-view of a scene (high camera elevation). Since our data selection process favors high occlusion scenarios, renders with low camera are usually favored by the rendering pipeline algorithm, explaining the observed trend.
Figure 15: Statistics of 3DOcBench evaluation benchmark: Similar to the training dataset, we observe that the 3DOcBench evaluation benchmark contains (a) heavy occlusion scenarios, (b) roughly uniform distribution of orientations, (c) higher frequency of smaller objects, measured using 2D bounding box dimension, and (d) large number of cases with low camera elevation and consequently high inter-object occlusion.
Figure 16: Samples from our training dataset: We create scenes in Blender [ 12 ] by placing 3D assets in controlled configurations and defining the rendering camera viewpoint. The object arrangements and camera viewpoint are controlled to ensure strong inter-object occlusions, while ensuring that each object is sufficiently visible in the image. Along with the main image, we render the corresponding OSCR representation, which consists of color-coded translucent 3D bounding boxes of the objects. The rendered images are further augmented using a depth-to-image pipeline to obtain realistic images that follow the same layout as shown in Fig: 13 .
Figure 17: Visualizing object disentanglement in latent space: We use the layouts (shown in first frame) to condition our model, the outputs are shown in second frame. We store the intermediate attention maps from image tokens to object tokens in the text prompt, visualized in third and fourth images. Evidently, the attention maps reveal occlusion boundaries, and show some interesting patterns. Notably, in cases involving transparent objects like water (b) and chemical flask (g), the physically hidden regions of sparrow and flask respectively are visible in attention map. Even in case of semantically similar categories, such as cow and horse (a), flasks with differently colored chemicals (g), motorbike and bicycle (d) the attention is highly localized, with only minimal leakage.
Figure 18: Measuring spatial alignment of image to object attention using correlation coefficient (CC): We create a dataset of 8 complex layouts containing strong occlusion scenarios (see Fig. 17 ). We obtain SeeThrough3D’s outputs on these layouts, and store intermediate attention maps from image tokens to object tokens in the text prompt. Next, we run Segment Anything [ 30 ] on the generated outputs to obtain object-level segmentation masks. Finally, we use correlation coefficient (CC) to measure alignment between the ground truth object segment and corresponding image to object attention maps. We compute the CC across space (layers) and time (denoising timesteps), and the obtained heatmap reveals interesting insights. For a given layer, timestep combination, a high CC value indicates strong spatial awareness. We observe that very specific layers in the DiT are spatially aware; early layers from 8 to 25, after which the spatial awareness decreases sharply. Secondly, the spatial properties in attention emerge after 5th denoising step (out of 25 steps) in these layers. The pattern of spatially aware layers is very irregular, indicating that different layers in the DiT contribute very differently to the generated image, consistent with findings from [ 1 ] .
Figure 19: Taking control with SeeThrough3D: We demonstrate the individual controls that our approach offers over the scene composition, which includes 3D attributes such as (a) object orientation, (b) object size, (c) object location, (d) scene camera elevation, as well as (e) text prompt and object semantics, all while ensuring occlusion consistency. Notably, all the images in this figure were generated using the same random seed , highlighting the effectiveness of control. Disentangled control: In (a), (b) and (c), we are able to control the 3D attributes of one object (Rolls Royce), without altering the other object, indicating disentangled control. Notice how occlusion consistency is preserved even in case of heavy overlap (b4), when the white car has become very big, and in (d1), where the camera elevation is very low. The model is able to model interesting controls such as levitating objects (c4). Despite heavy overlaps, the object attributes (‘white Rolls Royce’, ‘yellow Ferrari’) remain correctly bound to respective objects without attribute mixing, highlighting effectiveness of our binding mechanism.
Figure 20: Taking control with SeeThrough3D: We demonstrate the individual controls that our approach offers over the scene composition, which includes 3D attributes such as (a) object orientation, (b) object size, (c) object location, (d) scene camera elevation, as well as (e) text prompt and object semantics, all while ensuring occlusion consistency. Notably, all images in this figure were generated using the same random seed , highlighting the effectiveness of control. Disentangled control: In (a), (b) and (c), we are able to control the 3D attributes of one object (red boat), without altering the other object, indicating disentangled control. Notice how occlusion consistency is preserved in challenging cases like (d1), where the camera elevation is very low. Despite heavy overlaps, the object attributes (‘green boat‘, ‘red boat‘) remain correctly bound to respective objects without attribute mixing, highlighting effectiveness of our binding mechanism.
Baselines
depth ord. ↑
obj. score ↑
angular err. ↓
text align. ↑
KID (×10−3)↓
180∘ flip ang. err. ↓
VODiff [ 37 ]
0.68
19.70
92.73
29.51
15.40
41.38
LooseControl [ 4 ]
0.82
20.02
89.88
28.43
14.32
37.48
Build-A-Scene [ 19 ]
0.89
21.0
91.62
28.05
20.12
32.23
LaRender [ 76 ]
1.02
21.83
89.63
30.20
13.46
41.19
CompassControl [ 48 ]
0.87
20.60
66.29
29.76
13.01
35.79
Ours
1.46
22.86
47.92
31.87
5.43
25.72
Appendix
Table 3: Quantitative comparison: In the main paper, we did not compare against methods that only enable partial 3D control: Compass Control [ 48 ] and ORIGEN [ 43 ] . These baselines do not allow for 3D layout control, and primarily focus on object orientation control. While Compass Control allows for 2D layout control, ORIGEN does not, and hence it is not compatible with our quantitative evaluation (see Evaluation metrics in the main paper). Results for Compass Control are presented in yellow. It implicitly encodes orientation using an adapter, hence performs better than the other baselines in angular error. Further, we evaluate a relaxed angular error, which does not penalize 180∘ flips in the generated object (violet column). This caters to layout depth based methods LooseControl [ 4 ] and Build-A-Scene [ 19 ] , which do not encode a front-facing direction for the objects, thus result in such 180∘ flips. Our OSCR representation explicitly encodes orientation in the image space using color-coding, thus enabling precise orientation control, outperforming all baselines.
Figure 21: Qualitative comparisons with additional baselines: We present comparisons with baselines Compass Control [ 48 ] and ORIGEN [ 43 ] . We observe that Compass Control is not able able to handle complex occlusions (A1-4), and mixes object attributes in case of heavy overlaps (A5-6). ORIGEN fails to generate some objects in the scene (B1-6), and its outputs contain visual artifacts arising from poor noise optimization (B2). Additionally, ORIGEN is limited to one-step generative models, and hence suffers from low image fidelity. In contrast, our method is able to model complex occlusions (E1-6) without attribute mixing, indicating its effectiveness.
Figure 22: Personalization: We show that SeeThrough3D can be finetuned for occlusion-aware 3D control of personalized objects. This is achieved by learning a separate ‘subject’ LoRA to fuse appearance attributes from personalized object image into the generation process, building upon prior work on conditioning diffusion transformers [ 61 , 62 , 79 ] . This approach achieves such personalized 3D control without need for any test-time tuning . As shown in (a), we can compose objects from multiple modalities , such as dog (text) and royal chair (image). Interestingly, our model can personalize object categories not seen during training, such as bottle and glasses in (c), indicating strong generalization.
Figure 23: Adapting the training dataset for personalization: (a) given a rendered image from the training dataset, we randomly choose an object and apply a texture to it in Blender [ 12 ] (see Fig. 24 for examples of generated textures). (b) We separately render the textured 3D asset, and use it as reference object condition. (c) Finally, the textured object is placed back into the original image, and used as ground truth target for training the model.
Figure 24: Examples of generated textures: We generate textures using FLUX [ 35 ] , for preparing data for the personalization task. Notice how the textures contain high frequency features, induced by text and sharp patterns.
Figure 25: Limitations: (a) Our model is built upon FLUX [ 35 ] which fails to generate some out of distribution cases, such as a parrot outside a cage. Consequently, our model also struggles to generate such cases.
Figure 26: We built an intuitive UI to enable the user to easily design layouts for using our model. The UI enables the user to add objects, edit their placement, orientation and dimensions, and provide a text description for each object. Additionally, it allows the user to set the camera parameters.
Figure 27: We present qualitative comparisons with the baselines from the main paper, which are categorized into 3D layout control: LooseControl [ 4 ] and Build-A-Scene [ 19 ] ; Occlusion control: VODiff [ 37 ] and LaRender [ 76 ] . Each example has been analyzed with reference to layout adherence, occlusion consistency and realism ( red text above each example).
Figure 28: We present qualitative comparisons with the baselines from the main paper, which are categorized into 3D layout control: LooseControl [ 4 ] and Build-A-Scene [ 19 ] ; Occlusion control: VODiff [ 37 ] and LaRender [ 76 ] . Each example has been analyzed with reference to layout adherence, occlusion consistency and realism ( red text above each example).
Figure 29: Qualitative results: We present additional results of our method. For each example, we have shown the OSCR layout alongside the generated image; the correspondence from boxes to individual objects has been omitted here for clarity.
Image-to-3D generation faces inherent semantic ambiguity under occlusion, where partial observation alone is often insufficient to determine object category. In this work, we formalize text-driven amodal 3D generation, where text prompts steer the completion of unseen regions while strictly preserving input observation. Crucially, we identify that these objectives demand distinct control granularities: rigid control for the observation versus relaxed structural control for the prompt. To this end, we propose RelaxFlow, a training-free dual-branch framework that decouples control granularity via a Multi-Prior Consensus Module and a Relaxation Mechanism. Theoretically, we prove that our relaxation is equivalent to applying a low-pass filter on the generative vector field, which suppresses high-frequency instance details to isolate geometric structure that accommodates the observation. To facilitate evaluation, we introduce two diagnostic benchmarks, ExtremeOcc-3D and AmbiSem-3D. Extensive experiments demonstrate that RelaxFlow successfully steers the generation of unseen regions to match the prompt intent without compromising visual fidelity.
Jiayin Zhu, Guoji Fu, Xiaolu Liu +3
National University of Singapore · Zhejiang University · University of Science and Technology of China
Recent 3D generative models produce high-quality geometry from a single image using large-scale priors and diffusion architectures. However, they assume complete visibility and single-view inputs, limiting applicability in multi-view, occluded, or editing scenarios. Although prior works address these challenges individually, they lack a unified framework for controllable 3D completion under diverse conditioning signals. We present Axolotl3D, a multi-modal and occlusion-aware 3D generation model that jointly conditions on images, visibility masks, camera parameters, and a partial point cloud. The point cloud serves as a geometric anchor promoting faithful shape completion, while camera parameters ensure consistent multi-view alignment in a shared 3D coordinate system. A unified training strategy synthesizes diverse conditioning regimes from large-scale 3D data, enabling robust cross-modal reasoning. Experiments on Toys4K and OmniObject3D demonstrate state-of-the-art performance under both clean and occluded settings, as well as strong results in real-world reconstruction and geometry-consistent editing.
Text-to-image models often struggle to synthesize correct occlusion relationships among multiple objects, especially in densely overlapping regions. Many training-free layout-guided methods enforce 2D spatial constraints but do not explicitly resolve depth-dependent attention competition, which can cause concept mixing and implausible occlusion. To address this problem, we propose DepthArb, a training-free framework that formulates occlusion generation as attention arbitration within a unified denoising trajectory. DepthArb employs two core occlusion-control mechanisms: Attention Arbitration Modulation suppresses background-object attention within foreground support according to relative depth, while Spatial Compactness Control limits attention dispersion to preserve object coherence. Because interference varies during generation, Occlusion Conflict Estimation constructs a shared spatial conflict field to adaptively weight both objectives. Through a unified spatial-text attention interface, DepthArb operates on U-Net cross-attention and the image-to-text component of MMDiT joint attention without model retraining. We further introduce OcclBench, a benchmark with continuous relative-depth specifications and occlusion-specific evaluation metrics. Experiments on OcclBench and public benchmarks show that DepthArb improves several layout and occlusion metrics over the evaluated baselines while maintaining competitive text-image alignment.
Hongjin Niu, Jiahao Wang, Xirui Hu +4
School of Computer Science and Technology, Xi’an Jiaotong University · China Telecom