OverLay++: Dense-Overlap Layout-to-Image Generation Dataset

Shivansh Aggarwal; Shresth Grover; Divyansh Srivastava; Haiyang Xu; Bingnan Li; Xiang Zhang; Ethan J. Armand; Chuan Li

cs.CVOct 6, 2026

OverLay++: Dense-Overlap Layout-to-Image Generation Dataset

Authors: Shivansh Aggarwal, Shresth Grover, Divyansh Srivastava, Haiyang Xu, Bingnan Li, Xiang Zhang, Ethan J. Armand, Chuan Li, +2 more

Organizations: UC San Diego · Lambda, Inc.

Abstract

Layout-to-Image generation has made substantial progress in spatial and object-level control. However, existing methods still struggle with complex scenes containing many overlapping and interacting objects. We argue that training data is a particular bottleneck: existing datasets lack examples with dense, complex object interactions. To address this gap, we introduce OverLay++, a large-scale Layout-to-Image dataset with structurally complex scenes. OverLay++ contains approximately 500K images with an average of 6.6 objects per image, exceeding existing datasets by 1.67 times in annotation density. Beyond annotation density, OverLay++ provides rich semantic detail with object captions over six times longer than in current datasets. Our dataset generation pipeline is simple and produces dense, overlapping object annotations with rich per-object captions. Across multiple benchmarks, state-of-the-art Layout-to-Image methods trained on the OverLay++ dataset show consistent improvement and faster convergence, demonstrating the importance of dense, overlap-aware, and caption-rich supervision for controllable image generation.

Figures & tables

Figure 1 : OverLay++: A dense and overlap-aware dataset for Layout-to-Image generation. Left: OverLay++ provides dense object annotations with rich local captions for fine-grained spatial and attribute control. Center: Our dataset contains 500K images with 6.6 objects per image, yielding denser annotations and richer captions. Right: Training on OverLay++ improves generation quality in dense and overlapping scenes, producing better object fidelity and spatial alignment.

Figure 2 : Dataset generation pipeline for the OverLay++ dataset. Four stages: (I) image filtering via aesthetics, resolution, and center cropping; (II) object detection for bounding boxes and labels; (III) scene filtering to remove low-quality or inconsistent samples, and (IV) hierarchical captioning producing scene-level and object-level captions (long and short).

Figure 3 : Qualitative comparison of L2I generation. Models trained on OverLay++ produce more coherent and detailed images than those trained on prior datasets. Common failure cases of prior datasets include missing objects, missing attributes, and poor handling of dense object layouts.

Panel of Figure 4 — Figure 4 : Convergence analysis. Comparison of mIoU and O-mIoU versus training steps for Simple, Regular, and Complex settings. The model trained on OverLay++ achieves higher scores across all difficulty levels and demonstrates stable convergence behavior.

Model	mIoU(%) $\uparrow$	Color(%) $\uparrow$	Texture(%) $\uparrow$	Shape(%) $\uparrow$
EliGen-SD3 ( Zhang et al., 2025a )	45.79	37.65	39.98	39.76
EliGen-SD3 (Ours)	46.92 (+2.47%)	40.61 (+7.86%)	42.91 (+7.33%)	42.82 (+7.70%)
SiamLayout-SD3 ( Zhang et al., 2025b )	15.54	11.69	12.34	12.17
SiamLayout-SD3 (Ours)	17.10 (+10.04%)	13.15 (+12.49%)	13.77 (+11.59%)	13.61 (+11.83%)

Table 4 : Results on DenseLayout ( Xiang et al., 2025 ) . EliGen-SD3 and SiamLayout-SD3 trained on OverLay++ outperform their corresponding baselines on DenseLayout benchmark. Improvements over baselines are highlighted in green , and bold indicates the best results among all compared methods.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Split	Model	Long-CLIP G $\uparrow$
Simple	EliGen-SD3	0.2564
	EliGen-SD3 (Ours)	0.2573
Regular	EliGen-SD3	0.2556
	EliGen-SD3 (Ours)	0.2559
Complex	EliGen-SD3	0.2563
	EliGen-SD3 (Ours)	0.2570

Appendix

Table 8 : Global text–image alignment with Long-CLIP on OverLayBench. Higher scores indicate stronger alignment. These scores use a different encoder from CLIP G in the main tables and are not directly comparable in magnitude.

Model	Steps	Spatial $\uparrow$	Color $\uparrow$	Texture $\uparrow$	Shape $\uparrow$	FID $\downarrow$	IS $\uparrow$
SD3 (pretrained)	–	78.78	65.94	68.43	67.07	20.68	21.74
EliGen-SD3	5K	93.14	82.13	86.60	86.15	25.88	19.20
EliGen-SD3 (Ours)	5K	95.61	90.86	93.12	92.94	19.54	19.94
EliGen-SD3	10K	93.04	82.96	86.73	86.35	25.17	19.21
EliGen-SD3 (Ours)	10K	95.45	90.03	93.14	92.81	19.61	19.54

Appendix

Table 10: LayoutSAM-Eval during EliGen-SD3 training. The pretrained SD3 backbone is shown for context. These 5K- and 10K-step runs use the main training settings and are separate from the 2K-step, overlap-matched ablation in Table 9 .

Explore similar work

Learning to Generate Multiple Objects from Dense and Occluded Layouts
Jul 3, 2026Bach-Hoang Ngo, Si-Tri Ngo, Hieu Le +1Diverse Occlusion-And-Revelation Scenarios Scene Reasoning
University of Science, Ho Chi Minh City, Vietnam · 2Vietnam National University, Ho Chi Minh City, Vietnam · 3UNC Charlotte, North Carolina, United States
OcclusionFormer: Arranging Z-Order for Layout-Grounded Image Generation
May 20, 2026Ziye Li, Henghui DingDiverse Occlusion-And-Revelation Scenarios Occlusion
Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, China.
Object-Centric Dataset Resources for Constrained-Data Image Generation and Augmentation
Jun 19, 2026Vasile Marian, Yong-Bin Kang, Alexander BudderyPedestrian Detection Large-Scale 3D Editing Dataset
The University of Queensland Brisbane, Australia · Swinburne University of Technology Melbourne, Australia

Jul 3, 2026, cs.CVJ/K move · Enter open · S save

Learning to Generate Multiple Objects from Dense and Occluded Layouts

Bach-Hoang Ngo, Si-Tri Ngo, Hieu Le, Trung-Nghia Le

University of Science, Ho Chi Minh City, Vietnam · 2Vietnam National University, Ho Chi Minh City, Vietnam · 3UNC Charlotte, North Carolina, United States

Diverse Occlusion-And-Revelation Scenarios Scene Reasoning Objects Dense Scene Understanding Dynamic Attention+1

Open paper AskPDF arXiv

Text-to-image diffusion models fail to generate correct object counts in dense scenes, where overlapping instances collapse into indistinguishable structures despite appearing visually plausible. We identify this as instance ownership collapse: tokens from overlapping objects interact freely through attention, while heavily occluded instances receive weak supervision due to their small visible areas. We address this through layout-aware attention biases that softly bias token interactions toward region-consistent grouping and suppress cross-instance leakage, paired with an amodal-balanced loss that amplifies gradients for occluded objects based on their occlusion level. To enable systematic evaluation, we introduce OverlapDepth-45K, a benchmark of densely overlapping scenes with amodal supervision. Our approach substantially improves count accuracy and prevents instance merging while preserving image quality. Project page: https://bachngoh.github.io/AIBL

Abstract

Explore similar work

Learning to Generate Multiple Objects from Dense and Occluded Layouts

OcclusionFormer: Arranging Z-Order for Layout-Grounded Image Generation

Object-Centric Dataset Resources for Constrained-Data Image Generation and Augmentation