Region-instructed layout control in text-to-image generation is highly practical, yet existing methods suffer from limitations: (i) training-based approaches inherit data bias and often degrade image quality, and (ii) current techniques struggle with occlusion order, limiting real-world usability. To address these issues, we propose LayerBind. By modeling regional generation as distinct layers and binding them during the generation, our method enables precise regional and occlusion controllability. Our motivation stems from the observation that spatial layout and occlusion are established at a very early denoising stage, suggesting that rearranging the early latent structure is sufficient to modify the final output. Building on this, we structure the scheme into two phases: instance initialization and subsequent semantic nursing. (1) First, leveraging the contextual sharing mechanism in multimodal joint attention, Layer-wise Instance Initialization creates per-instance branches that attend to their own regions while anchoring to the shared background. At a designated early step, these branches are fused according to the layer order to form a unified latent with a pre-established layout. (2) Then, Layer-wise Semantic Nursing reinforces regional details and maintains the occlusion order via a layer-wise attention enhancement. Specifically, a sequential layered attention path operates alongside the standard global path, with updates composited under a layer-transparency scheduler. LayerBind is training-free and plug-and-play, serving as a regional and occlusion controller across Diffusion Transformers. Beyond generation, it natively supports editable workflows, allowing for flexible modifications like changing instances or rearranging visible orders. Both qualitative and quantitative results demonstrate LayerBind's effectiveness, highlighting its strong potential for creative applications.
Figures & tables
Figure 2: (a, b) Observation: simply rearranging the latent structure at an early step directly manipulates the final spatial layout and occlusion order. (c) Our LayerBind scheme: initializing the instance layout first, then conducting semantic nursing for instance detail while maintaining layout and occlusions.
Figure 3: Overview of the LayerBind pipeline. (a) Layer-wise Instance Initialization splits early denoising into background and instance branches. Each instance generates independently while sharing background context (via Contextual Attention, CTA, Eq. 3 ), then they are fused to form the initialized early latent. (b) Layer-wise Semantic Nursing reinforces following generation. It conduct layer-wise sequential CTA updates for each region, modulated by a Layer Transparency Scheduler, to refine instance details and maintain occlusions. Note: For simplicity, only image token updates are visualized; the detailed strategy will be described in the following sections.
Figure 4: Attention response weights of foreground to background and text across different FLUX [ 3 ] layers. We select layer 0 [ 44 , 1 ] and layers with strong text response for hard instance binding. More analysis is presented in the Appendix A .
Figure 5: Visualization of occlusion control abilities. Compared to the previous methods, LayerBind achieves more precise layer-wise control, avoiding errors such as instance neglect and concept blending. More visualizations are available in the Appendix F
Table 1: Quantitative comparison for occlusion control, measuring: depth relationship (UniDet), T2I alignment (CLIP-G/L), Layout alignment (L Acc/VQA ), occlusion perception score (O VQA ), and image quality (HPS). Inference speed includes the percentage of overhead introduced by the controller relative to the base model. * evaluated at 512x512 resolution; all other methods are at 1024x1024.
Figure 6: Visualization results on T2I alignment evaluations. LayerBind can serve as a plug-and-play layout controller for improving T2I alignment ability without quality degradation.
Attribute Binding
Spatial
Numeracy
Complex
Color
Shape
Texture
FLUX [ 3 ]
77.53
60.16
69.64
39.09
59.81
37.01
CreatiLayout [ 53 ]
76.94
59.92
73.45
60.33
71.51
37.45
InstanceAssemble [ 47 ]
71.64
55.13
62.15
64.38
56.90
38.33
HybridLayout [ 45 ]
84.15
68.82
77.31
63.39
64.57
40.15
RAGD [ 8 ]
80.39
60.16
70.85
51.93
53.76
43.77
Table 2: The quantitative evaluation results of T2I alignment tasks.
Table 3: The quantitative ablation results of applying Hard Binding (Sec. 4.2 , HB) and Layer-wise Semantic Nursing (Sec. 4.3 , LSN) on BindBench dataset.
Figure 7: Visualization of effect of Hard Binding. It prevents instances from being ignored due to modality competition [ 31 ] .
Figure 8: Applications. Top) As also shown in Fig.1, LayerBind supports flexible occlusion control and instance modifications. Bottom) Treat an original generation as background context and branching edit instructions. LayerBind also achieves composited image edits.
Figure 9: Visualization of effects of different η1 with LSN strategy. To illustrate the details refinement of LSN, we add color attributes to each region (e.g., golden turtle, blue chicken, red bicycle).
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Attention response weights of foreground to background, self, and text tokens across different attention blocks (in FLUX and SD3.5).
Figure 11: The simplified prompt template used for our LLM-based Layout Parser. Specific rules are defined for each part of the parsing, and these rules are made comprehensible to the model through In-Context Examples.
Figure 12: Illustration of leveraging QWen-VL2.5 [ 2 ] as a VQA judge for evaluating 3D spatial relationships. It can accurately perceive spatial relationships and score the image content on whether it satisfies the question.
Model
1
2
3
4
5
6
FLUX
18%
31%
45%
60%
76%
89%
SD3.5
24%
39%
55%
73%
92%
107%
Appendix
Table 4: LayerBind’s additional inference time when inputting different numbers of regions. Each region occupies 25% of the image tokens (e.g., 1024 tokens). The inference cost of LayerBind increases linearly with the number of additional tokens.
Figure 13: The illustration of the effectiveness of the proposed LSN and naive regional prompting [ 4 ] strategies. Without layer-wise updates, errors such as concept blending and failure in occlusion control may occur.
Figure 14: The performance is stabilized across a robust η1 range, while fine-tuning η1 can further optimize specific cases.
Figure 15: The visualization of the estimated alpha mask and the effectiveness of different branch blending strategies.
Figure 16: The proposed LayrBind can be integrated with external visual adapters. a) With the FLUX-Redux plugin, LayerBind can directly generate the background by referencing an existing image. b) With IP-adapter [ 50 ] , LayerBind can inject visual references into regions. c) LayerBind supports instance position editing.
Figure 17: Visualization of generating transparent instances.
Figure 18: LayerBind successfully handles rational complex spatial relationships, while failing in counterfactually incorrect layout arrangements.
Figure 19: Illustration of typical failure cases and repair measures.
Figure 20: Visualization of occlusion control abilities on T2ICompBench-3D dataset.
Figure 21: Visualization of occlusion control abilities on BindBench dataset.
Figure 22: Visualization of T2I alignment tasks on T2ICompBench dataset.
Recent layout-to-image models have achieved remarkable progress in spatial controllability. However, they still struggle with inter-object occlusion. When bounding boxes overlap, most existing methods lack explicit occlusion information, which makes the generation in intersection regions inherently ambiguous and hinders the determination of complex occlusion relationships. As a result, they often produce entangled textures or physically inconsistent layering in the overlapped areas. To address this issue, we first construct SA-Z, a large-scale dataset enriched with explicit occlusion ordering and pixel-level annotations. Building upon our proposed dataset, we introduce OcclusionFormer, a novel occlusion-aware Diffusion Transformer framework that explicitly models Z-order priority by decoupling instances and compositing them via volume rendering. Furthermore, to ensure fine-grained spatial precision, we introduce a queried alignment loss that explicitly supervises individual instances and enhances semantic consistency. The proposed method effectively reduces ambiguity in overlapping regions, enforces correct occlusion dependencies, and preserves structural integrity, leading to substantial accuracy gains across diverse scenes.
Ziye Li, Henghui Ding
Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, China.
Recent open-weight text-to-image (T2I) diffusion models still struggle with multi-instance prompts, often omitting or merging instances and mixing semantics among similar objects. We trace these failures to early denoising steps, before instance boundaries are reliably stabilized. Existing training-free guidance is largely driven by cross-attention or other token-conditioned semantic signals. Such guidance can separate concepts at the token level, but largely assumes that distinct instance regions have already emerged. In early denoising steps, it cannot reliably carve out these regions, so count failures and semantic mixing persist. By contrast, self-attention exposes class-agnostic instance layouts during early denoising. To exploit this asymmetry, we propose ISAC (Instance-to-Semantic Attention Control), a training-free, model-agnostic objective that first stabilizes self-attention layouts and then binds cross-attention semantics within them, without fine-tuning or external vision models. Across T2I-CompBench, HRS-Bench, and our newly curated IntraCompBench, ISAC consistently outperforms prior training-free methods. Furthermore, ISAC enhances layout-to-image controllers by refining coarse, overlapping bounding boxes into dense instance masks. Code and IntraCompBench are available at https://shjo-april.github.io/ISAC.
Sanghyun Jo, Wooyeol Lee, Ziseok Lee +3
OGQ, Seoul, Korea · Seoul National University, Seoul, Korea
While Diffusion models produce high-quality images, real-world visual design requires intentionally preserving clean regions for headlines, logos, or UI elements. Existing layout-aware text-to-image methods struggle to strictly preserve reserved regions, often introducing unwanted content that complicates downstream editing. Existing approaches either manipulate attention to displace content or suppress conditioning via noise-level interventions. However, these methods sacrifice visual coherence, producing flat regions or noticeable style inconsistencies between reserved and non-reserved areas. To address this limitation, we propose a Prompt-Aware Shared-Latent Training-Free Annealed Diffusion Sampling (PASTA), a training-free framework that enforces reserved-region constraints by selectively applying semantic guidance to object-allowed regions during sampling. PASTA operates on a shared latent trajectory, where full-prompt and context-prompt guidance are fused through mask-guided sampling. An annealed schedule suppresses semantic leakage during early denoising stages while gradually restoring visual consistency in later stages. Extensive experiments demonstrate that PASTA improves reserved-region fidelity while preserving overall image realism and perceptual consistency.
Ren Ajiki, Ryugo Morita, Stanislav Frolov +5
Faculty of Science and Engineering, Hosei University, Tokyo, Japan · RPTU Kaiserslautern-Landau & DFKI GmbH, Kaiserslautern, Germany