OccluRank: Controllable Occlusion-Aware Layout-to-Image Generation by Adding Just an Ordinal Rank
Authors: Wenyang Hong, Yuan Wang, Yanbin Hao, Lanqing Xue, Ke Wang, Xiang Wang, Kuien Liu, Richang Hong
Organizations: Hefei University of Technology · School of Cyber Science and Technology, University of Science and Technology of China · LCFC · ByteDance Inc. · School of Artificial Intelligence and Data Science, University of Science and Technology of China · Institute of Software, Chinese Academy of Sciences
Layout-to-image generation enables explicit spatial control through bounding-box layouts, yet bounding boxes specify only instance locations and cannot represent their occlusion order. Existing methods may rely on additional geometric conditions, employ complex inference procedures, or aggregate independently constructed instance representations without explicitly modeling their occlusion-dependent interactions. We propose OccluRank, a simple and controllable occlusion-aware layout-to-image framework that augments each bounding box with only one ordinal rank. OccluRank encodes the user-specified occlusion order through lightweight rank-based conditioning and introduces an Order-aware Instance Interaction (OII) module to jointly update rank-conditioned instance representations before aggregation. This allows the specified order to guide information exchange among occluding instances without additional geometric inputs or specialized inference-time optimization. We further construct OccluLayout, a synthetic training dataset whose occlusion order and amodal annotations are derived directly from known scene geometry rather than estimated from partially occluded images using auxiliary prediction models. For comprehensive evaluation, we introduce OccluLayout-Bench, which uses multiple multimodal large language model evaluators to assess instance presence, spatial layout, attributes, and occlusion order, together with FID for overall image quality. Experiments show that OccluRank more reliably preserves target instances, follows specified layouts, and realizes desired occlusion relationships while maintaining comparable attribute consistency and overall image quality.
Figures & tables
Figure 1: Comparison with representative occlusion-control approaches. (a) IFAdapter is order-agnostic and cannot reliably produce the desired occlusion order, whereas OccluRank uses ordinal ranks and OII for explicit order control. (b) SeeThrough3D requires more complex geometric conditions, including an Occlusion-Aware Scene Representation (OSCR) and instance masks, whereas OccluRank assigns only one ordinal rank to each instance.
Figure 2: Overview of OccluRank. OccluRank constructs spatially aligned instance features, incorporates ordinal rank embeddings, and performs location-wise cross-instance interaction before aggregating into an Instance Semantic Map (ISM). The masked ISM is residually injected into selected cross-attention layers of SDXL.
Figure 3: Overview of the OccluLayout construction.
Evaluator (MLLM)
Method
Venue
Presence ↑
Box mIoU ↑
Color ↑
Texture ↑
Strict Pair ↑
Strict Image ↑
FID ↓
Qwen3-VL-235B- A22B
CreatiLayout
ICCV’2025
87.92
53.83
81.06
71.70
67.08
50.78
62.754
LaRender
ICCV’2025
55.14
16.88
46.25
38.41
16.63
7.90
96.556
OcclusionFormer †
ICML’2026
76.49
50.36
70.45
57.89
55.28
39.77
72.707
OcclusionFormer
ICML’2026
78.88
49.29
74.28
66.11
51.82
35.49
57.124
VODiff
CVPR’2025
56.29
1.93
25.45
15.69
25.10
12.84
85.396
IFAdapter
ICCV’2025
90.17
60.80
83.46
74.84
66.06
51.55
62.923
Table 1: Quantitative comparison on OccluLayout-Bench using three MLLM evaluators. VODiff is evaluated at 512×512 and all other methods at 1024×1024 . † denotes the official checkpoint without further training on OccluLayout.
Figure 4: Qualitative comparison under overlapping layouts. † denotes the official checkpoint without OccluLayout training.
Figure 5: Qualitative comparison of the ablation variants under the same input layout and front-to-back order.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Source
#Img.
#Inst.
Res.
Vocab.
Inst. Text
BBox
Mask
Z-order
Amodal
COCO 2017 ( Lin et al. 2014 )
COCO
≈0.1M
≈0.88M
Low
80
Class
✓
✓
–
–
InstaOrder ( Lee and Park 2022 )
COCO
≈0.1M
≈0.50M
Low
80
Class
✓
✓
Human
–
COCOA ( Zhu et al. 2017 )
COCO/BSDS
≈5.5K
≈69.0K
Low
80
Class
✓
✓
Human
Human
EliGen-Data ( Zhang et al. 2025a )
EliGen
≈0.5M
≈1.26M
High
Open
Phrase
✓
×
–
–
LayoutSAM ( Zhang et al. 2025b )
SA-1B
≈2.0M
≈10.7M
High
Open
Phrase
✓
×
–
–
SACap-1M ( Li et al. 2025 )
SA-1B
≈1.0M
≈5.88M
High
Open
Phrase
✓
✓
–
–
Appendix
Table 2: Statistical comparison of layout and occlusion-related datasets. Resolution is classified as High when the image’s long edge exceeds 1000 pixels. BBox and Mask indicate whether the released dataset provides bounding-box and object-level mask annotations, respectively. Z-order and Amodal report how the corresponding annotations are obtained: “Human” denotes manual annotation, “Predicted” denotes model prediction, “Geometry” denotes direct derivation from the underlying 3D scene geometry, and “–” indicates that the annotation is unavailable. Vocabulary denotes the number of distinct instance labels; duplicate asset-name suffixes are merged when counting OccluLayout. † The released SeeThrough3D data include raw Blender renders and depth-to-image variants, and multiple images may share the same underlying 3D layout.
Statistic
Train
Test
# Images
33,496
1,000
# Instances
114,047
3,386
Avg. Instances / Image
3.40
3.39
Images with Box Overlap
78.8%
77.2%
Mean Max Pairwise Box IoU
0.2215
0.2190
Mean Relative BBox Area
0.0630
0.0631
Appendix
Table 3: Statistics of the OccluLayout training and test splits. Box overlap denotes images containing at least one pair of overlapping bounding boxes.
Method
Venue
Occ. ↑
Dep. ↓
CreatiLayout
ICCV’2025
0.7552
0.2244
LaRender
ICCV’2025
0.5908
0.4392
OcclusionFormer †
ICML’2026
0.8184
0.2126
OcclusionFormer
ICML’2026
0.7531
0.2423
VODiff
CVPR’2025
0.7065
0.2548
IFAdapter
ICCV’2025
0.7987
0.1993
Appendix
Table 4: Additional quantitative comparison on OccluLayout-Bench using occlusion-order F1 (Occ.) and depth-order WHDR (Dep.). VODiff is evaluated at 512×512 and all other methods at 1024×1024 . † denotes the official checkpoint without further training on OccluLayout.
Variant
Venue
Occ. ↑
Dep. ↓
IFAdapter
ICCV’2025
0.7987
0.1993
Scalar Reweight
–
0.8127
0.2073
w/o Local Loss
–
0.8400
0.1866
w/o Rank Emb.
–
0.8279
0.1859
Ours
–
0.8577
0.1844
Appendix
Table 5: Ablation study on OccluLayout-Bench using occlusion-order F1 (Occ.) and depth-order WHDR (Dep.).
Evaluator (MLLM)
Variant
Venue
Presence ↑
Box mIoU ↑
Color ↑
Texture ↑
Strict Pair ↑
Strict Image ↑
FID ↓
Qwen3-VL-235B- A22B
IFAdapter
ICCV’2025
90.17
60.80
83.46
74.84
66.06
51.55
62.923
Scalar Reweight
–
90.37
59.74
84.21
75.78
71.40
57.38
62.277
w/o Local Loss
–
90.96
63.10
84.09
75.39
72.81
56.61
62.322
w/o Rank Emb.
–
90.79
63.32
83.93
74.18
73.89
59.33
64.332
Ours
–
91.58
63.20
85.23
76.02
76.45
62.44
62.746
Qwen3-VL-32B
IFAdapter
ICCV’2025
91.44
60.87
82.59
73.19
66.46
51.42
62.923
Appendix
Table 6: Ablation study on OccluLayout-Bench using four MLLM evaluators.
Figure 6: Qualitative comparison under reversed instance order. The variant without rank embeddings fails to respond to the order change, while our full model successfully controls the front-to-back relationship.
Figure 7: Qualitative examples on categories unseen during training. Numeric prefixes denote the prescribed front-to-back order. OccluRank generalizes its occlusion control to unseen categories, preserving the requested instances while correctly realizing the specified occlusion relationships.
Recent layout-to-image models have achieved remarkable progress in spatial controllability. However, they still struggle with inter-object occlusion. When bounding boxes overlap, most existing methods lack explicit occlusion information, which makes the generation in intersection regions inherently ambiguous and hinders the determination of complex occlusion relationships. As a result, they often produce entangled textures or physically inconsistent layering in the overlapped areas. To address this issue, we first construct SA-Z, a large-scale dataset enriched with explicit occlusion ordering and pixel-level annotations. Building upon our proposed dataset, we introduce OcclusionFormer, a novel occlusion-aware Diffusion Transformer framework that explicitly models Z-order priority by decoupling instances and compositing them via volume rendering. Furthermore, to ensure fine-grained spatial precision, we introduce a queried alignment loss that explicitly supervises individual instances and enhances semantic consistency. The proposed method effectively reduces ambiguity in overlapping regions, enforces correct occlusion dependencies, and preserves structural integrity, leading to substantial accuracy gains across diverse scenes.
Ziye Li, Henghui Ding
Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, China.
Text-to-image diffusion models fail to generate correct object counts in dense scenes, where overlapping instances collapse into indistinguishable structures despite appearing visually plausible. We identify this as instance ownership collapse: tokens from overlapping objects interact freely through attention, while heavily occluded instances receive weak supervision due to their small visible areas. We address this through layout-aware attention biases that softly bias token interactions toward region-consistent grouping and suppress cross-instance leakage, paired with an amodal-balanced loss that amplifies gradients for occluded objects based on their occlusion level. To enable systematic evaluation, we introduce OverlapDepth-45K, a benchmark of densely overlapping scenes with amodal supervision. Our approach substantially improves count accuracy and prevents instance merging while preserving image quality. Project page: https://bachngoh.github.io/AIBL
Bach-Hoang Ngo, Si-Tri Ngo, Hieu Le +1
University of Science, Ho Chi Minh City, Vietnam · 2Vietnam National University, Ho Chi Minh City, Vietnam · 3UNC Charlotte, North Carolina, United States
Text-to-image models often struggle to synthesize correct occlusion relationships among multiple objects, especially in densely overlapping regions. Many training-free layout-guided methods enforce 2D spatial constraints but do not explicitly resolve depth-dependent attention competition, which can cause concept mixing and implausible occlusion. To address this problem, we propose DepthArb, a training-free framework that formulates occlusion generation as attention arbitration within a unified denoising trajectory. DepthArb employs two core occlusion-control mechanisms: Attention Arbitration Modulation suppresses background-object attention within foreground support according to relative depth, while Spatial Compactness Control limits attention dispersion to preserve object coherence. Because interference varies during generation, Occlusion Conflict Estimation constructs a shared spatial conflict field to adaptively weight both objectives. Through a unified spatial-text attention interface, DepthArb operates on U-Net cross-attention and the image-to-text component of MMDiT joint attention without model retraining. We further introduce OcclBench, a benchmark with continuous relative-depth specifications and occlusion-specific evaluation metrics. Experiments on OcclBench and public benchmarks show that DepthArb improves several layout and occlusion metrics over the evaluated baselines while maintaining competitive text-image alignment.
Hongjin Niu, Jiahao Wang, Xirui Hu +4
School of Computer Science and Technology, Xi’an Jiaotong University · China Telecom