OccluRank: Controllable Occlusion-Aware Layout-to-Image Generation by Adding Just an Ordinal Rank

Wenyang Hong; Yuan Wang; Yanbin Hao; Lanqing Xue; Ke Wang; Xiang Wang; Kuien Liu; Richang Hong

cs.CVAug 21, 2026

OccluRank: Controllable Occlusion-Aware Layout-to-Image Generation by Adding Just an Ordinal Rank

Authors: Wenyang Hong, Yuan Wang, Yanbin Hao, Lanqing Xue, Ke Wang, Xiang Wang, Kuien Liu, Richang Hong

Organizations: Hefei University of Technology · School of Cyber Science and Technology, University of Science and Technology of China · LCFC · ByteDance Inc. ·

Abstract

Layout-to-image generation enables explicit spatial control through bounding-box layouts, yet bounding boxes specify only instance locations and cannot represent their occlusion order. Existing methods may rely on additional geometric conditions, employ complex inference procedures, or aggregate independently constructed instance representations without explicitly modeling their occlusion-dependent interactions. We propose OccluRank, a simple and controllable occlusion-aware layout-to-image framework that augments each bounding box with only one ordinal rank. OccluRank encodes the user-specified occlusion order through lightweight rank-based conditioning and introduces an Order-aware Instance Interaction (OII) module to jointly update rank-conditioned instance representations before aggregation. This allows the specified order to guide information exchange among occluding instances without additional geometric inputs or specialized inference-time optimization. We further construct OccluLayout, a synthetic training dataset whose occlusion order and amodal annotations are derived directly from known scene geometry rather than estimated from partially occluded images using auxiliary prediction models. For comprehensive evaluation, we introduce OccluLayout-Bench, which uses multiple multimodal large language model evaluators to assess instance presence, spatial layout, attributes, and occlusion order, together with FID for overall image quality. Experiments show that OccluRank more reliably preserves target instances, follows specified layouts, and realizes desired occlusion relationships while maintaining comparable attribute consistency and overall image quality.

Figures & tables

Figure 1: Comparison with representative occlusion-control approaches. (a) IFAdapter is order-agnostic and cannot reliably produce the desired occlusion order, whereas OccluRank uses ordinal ranks and OII for explicit order control. (b) SeeThrough3D requires more complex geometric conditions, including an Occlusion-Aware Scene Representation (OSCR) and instance masks, whereas OccluRank assigns only one ordinal rank to each instance.

Figure 2: Overview of OccluRank. OccluRank constructs spatially aligned instance features, incorporates ordinal rank embeddings, and performs location-wise cross-instance interaction before aggregating into an Instance Semantic Map (ISM). The masked ISM is residually injected into selected cross-attention layers of SDXL.

Figure 3: Overview of the OccluLayout construction.

Evaluator (MLLM)	Method	Venue	Presence $\uparrow$	Box mIoU $\uparrow$	Color $\uparrow$	Texture $\uparrow$	Strict Pair $\uparrow$	Strict Image $\uparrow$	FID $\downarrow$
Qwen3-VL-235B- A22B	CreatiLayout	ICCV’2025	87.92	53.83	81.06	71.70	67.08	50.78	62.754
	LaRender	ICCV’2025	55.14	16.88	46.25	38.41	16.63	7.90	96.556
	OcclusionFormer †	ICML’2026	76.49	50.36	70.45	57.89	55.28	39.77	72.707
	OcclusionFormer	ICML’2026	78.88	49.29	74.28	66.11	51.82	35.49	57.124
	VODiff	CVPR’2025	56.29	1.93	25.45	15.69	25.10	12.84	85.396
	IFAdapter	ICCV’2025	90.17	60.80	83.46	74.84	66.06	51.55	62.923

Table 1: Quantitative comparison on OccluLayout-Bench using three MLLM evaluators. VODiff is evaluated at

512{\times}512

and all other methods at

1024{\times}1024

. † denotes the official checkpoint without further training on OccluLayout.

Figure 4: Qualitative comparison under overlapping layouts. † denotes the official checkpoint without OccluLayout training.

Figure 5: Qualitative comparison of the ablation variants under the same input layout and front-to-back order.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Source	#Img.	#Inst.	Res.	Vocab.	Inst. Text	BBox	Mask	Z-order	Amodal
COCO 2017 ( Lin et al. 2014 )	COCO	$\approx 0.1\mathrm{M}$	$\approx 0.88\mathrm{M}$	Low	80	Class	$\checkmark$	$\checkmark$	–	–
InstaOrder ( Lee and Park 2022 )	COCO	$\approx 0.1\mathrm{M}$	$\approx 0.50\mathrm{M}$	Low	80	Class	$\checkmark$	$\checkmark$	Human	–
COCOA ( Zhu et al. 2017 )	COCO/BSDS	$\approx 5.5\mathrm{K}$	$\approx 69.0\mathrm{K}$	Low	80	Class	$\checkmark$	$\checkmark$	Human	Human
EliGen-Data ( Zhang et al. 2025a )	EliGen	$\approx 0.5\mathrm{M}$	$\approx 1.26\mathrm{M}$	High	Open	Phrase	$\checkmark$	$\times$	–	–
LayoutSAM ( Zhang et al. 2025b )	SA-1B	$\approx 2.0\mathrm{M}$	$\approx 10.7\mathrm{M}$	High	Open	Phrase	$\checkmark$	$\times$	–	–
SACap-1M ( Li et al. 2025 )	SA-1B	$\approx 1.0\mathrm{M}$	$\approx 5.88\mathrm{M}$	High	Open	Phrase	$\checkmark$	$\checkmark$	–	–

Appendix

Table 2: Statistical comparison of layout and occlusion-related datasets. Resolution is classified as High when the image’s long edge exceeds 1000 pixels. BBox and Mask indicate whether the released dataset provides bounding-box and object-level mask annotations, respectively. Z-order and Amodal report how the corresponding annotations are obtained: “Human” denotes manual annotation, “Predicted” denotes model prediction, “Geometry” denotes direct derivation from the underlying 3D scene geometry, and “–” indicates that the annotation is unavailable. Vocabulary denotes the number of distinct instance labels; duplicate asset-name suffixes are merged when counting OccluLayout. † The released SeeThrough3D data include raw Blender renders and depth-to-image variants, and multiple images may share the same underlying 3D layout.

Statistic	Train	Test
# Images	33,496	1,000
# Instances	114,047	3,386
Avg. Instances / Image	3.40	3.39
Images with Box Overlap	78.8%	77.2%
Mean Max Pairwise Box IoU	0.2215	0.2190
Mean Relative BBox Area	0.0630	0.0631

Appendix

Table 3: Statistics of the OccluLayout training and test splits. Box overlap denotes images containing at least one pair of overlapping bounding boxes.

Method	Venue	Occ. $\uparrow$	Dep. $\downarrow$
CreatiLayout	ICCV’2025	0.7552	0.2244
LaRender	ICCV’2025	0.5908	0.4392
OcclusionFormer †	ICML’2026	0.8184	0.2126
OcclusionFormer	ICML’2026	0.7531	0.2423
VODiff	CVPR’2025	0.7065	0.2548
IFAdapter	ICCV’2025	0.7987	0.1993

Appendix

Table 4: Additional quantitative comparison on OccluLayout-Bench using occlusion-order F1 (Occ.) and depth-order WHDR (Dep.). VODiff is evaluated at

512{\times}512

and all other methods at

1024{\times}1024

. † denotes the official checkpoint without further training on OccluLayout.

Variant	Venue	Occ. $\uparrow$	Dep. $\downarrow$
IFAdapter	ICCV’2025	0.7987	0.1993
Scalar Reweight	–	0.8127	0.2073
w/o Local Loss	–	0.8400	0.1866
w/o Rank Emb.	–	0.8279	0.1859
Ours	–	0.8577	0.1844

Appendix

Table 5: Ablation study on OccluLayout-Bench using occlusion-order F1 (Occ.) and depth-order WHDR (Dep.).

Evaluator (MLLM)	Variant	Venue	Presence $\uparrow$	Box mIoU $\uparrow$	Color $\uparrow$	Texture $\uparrow$	Strict Pair $\uparrow$	Strict Image $\uparrow$	FID $\downarrow$
Qwen3-VL-235B- A22B	IFAdapter	ICCV’2025	90.17	60.80	83.46	74.84	66.06	51.55	62.923
	Scalar Reweight	–	90.37	59.74	84.21	75.78	71.40	57.38	62.277
	w/o Local Loss	–	90.96	63.10	84.09	75.39	72.81	56.61	62.322
	w/o Rank Emb.	–	90.79	63.32	83.93	74.18	73.89	59.33	64.332
	Ours	–	91.58	63.20	85.23	76.02	76.45	62.44	62.746
Qwen3-VL-32B	IFAdapter	ICCV’2025	91.44	60.87	82.59	73.19	66.46	51.42	62.923

Appendix

Table 6: Ablation study on OccluLayout-Bench using four MLLM evaluators.

Figure 6: Qualitative comparison under reversed instance order. The variant without rank embeddings fails to respond to the order change, while our full model successfully controls the front-to-back relationship.

Figure 7: Qualitative examples on categories unseen during training. Numeric prefixes denote the prescribed front-to-back order. OccluRank generalizes its occlusion control to unseen categories, preserving the requested instances while correctly realizing the specified occlusion relationships.

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 463 papers

May 20, 2026

OcclusionFormer: Arranging Z-Order for Layout-Grounded Image Generation

Recent layout-to-image models have achieved remarkable progress in spatial controllability. However, they still struggle with inter-object occlusion. When bounding boxes overlap, most existing methods lack explicit occlusion information, which makes the generation in intersection regions inherently ambiguous and hinders the determination of complex occlusion relationships. As a result, they often produce entangled textures or physically inconsistent layering in the overlapped areas. To address this issue, we first construct SA-Z, a large-scale dataset enriched with explicit occlusion ordering and pixel-level annotations. Building upon our proposed dataset, we introduce OcclusionFormer, a novel occlusion-aware Diffusion Transformer framework that explicitly models Z-order priority by decoupling instances and compositing them via volume rendering. Furthermore, to ensure fine-grained spatial precision, we introduce a queried alignment loss that explicitly supervises individual instances and enhances semantic consistency. The proposed method effectively reduces ambiguity in overlapping regions, enforces correct occlusion dependencies, and preserves structural integrity, leading to substantial accuracy gains across diverse scenes.

Ziye Li, Henghui Ding

Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, China.

Diffusion Transformer Occlusion Reasoning Image Generation Compositional Image Generation Diffusion-Based Image Generation

Ask PDF

Jul 3, 2026

Learning to Generate Multiple Objects from Dense and Occluded Layouts

Text-to-image diffusion models fail to generate correct object counts in dense scenes, where overlapping instances collapse into indistinguishable structures despite appearing visually plausible. We identify this as instance ownership collapse: tokens from overlapping objects interact freely through attention, while heavily occluded instances receive weak supervision due to their small visible areas. We address this through layout-aware attention biases that softly bias token interactions toward region-consistent grouping and suppress cross-instance leakage, paired with an amodal-balanced loss that amplifies gradients for occluded objects based on their occlusion level. To enable systematic evaluation, we introduce OverlapDepth-45K, a benchmark of densely overlapping scenes with amodal supervision. Our approach substantially improves count accuracy and prevents instance merging while preserving image quality. Project page: https://bachngoh.github.io/AIBL

Bach-Hoang Ngo, Si-Tri Ngo, Hieu Le +1

University of Science, Ho Chi Minh City, Vietnam · 2Vietnam National University, Ho Chi Minh City, Vietnam · 3UNC Charlotte, North Carolina, United States

Image Generation Compositional Image Generation Compositional T2I Generation Attention Mechanisms T2I Generation Evaluation T2I Diffusion Models

Ask PDF

Mar 25, 2026

DepthArb: Training-Free Depth-Arbitrated Generation for Occlusion-Robust Image Synthesis

Text-to-image models often struggle to synthesize correct occlusion relationships among multiple objects, especially in densely overlapping regions. Many training-free layout-guided methods enforce 2D spatial constraints but do not explicitly resolve depth-dependent attention competition, which can cause concept mixing and implausible occlusion. To address this problem, we propose DepthArb, a training-free framework that formulates occlusion generation as attention arbitration within a unified denoising trajectory. DepthArb employs two core occlusion-control mechanisms: Attention Arbitration Modulation suppresses background-object attention within foreground support according to relative depth, while Spatial Compactness Control limits attention dispersion to preserve object coherence. Because interference varies during generation, Occlusion Conflict Estimation constructs a shared spatial conflict field to adaptively weight both objectives. Through a unified spatial-text attention interface, DepthArb operates on U-Net cross-attention and the image-to-text component of MMDiT joint attention without model retraining. We further introduce OcclBench, a benchmark with continuous relative-depth specifications and occlusion-specific evaluation metrics. Experiments on OcclBench and public benchmarks show that DepthArb improves several layout and occlusion metrics over the evaluated baselines while maintaining competitive text-image alignment.

Hongjin Niu, Jiahao Wang, Xirui Hu +4

School of Computer Science and Technology, Xi’an Jiaotong University · China Telecom

T2I Diffusion Models Compositional T2I Generation Attention Control in Diffusion Models

Ask PDF

Abstract

Image Generation

Explore similar work

OcclusionFormer: Arranging Z-Order for Layout-Grounded Image Generation

Learning to Generate Multiple Objects from Dense and Occluded Layouts

DepthArb: Training-Free Depth-Arbitrated Generation for Occlusion-Robust Image Synthesis