EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding

Kai Zou; Hongbo Liu; Dian Zheng; Jianxiong Gao; Zhiwei Zhao; Bin Liu

cs.CVMar 18, 2026

EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding

Authors: Kai Zou, Hongbo Liu, Dian Zheng, Jianxiong Gao, Zhiwei Zhao, Bin Liu

Organizations: School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Tongji University · Sun Yat-sen University · Fudan University · Hefei University of Technology

Abstract

In this work, we present EchoGen, a unified framework for layout-to-image generation and image grounding, capable of generating images with accurate layouts and high fidelity to text descriptions (e.g., spatial relationships), while grounding the image robustly at the same time. We believe that image grounding possesses strong text and layout understanding abilities, which can compensate for the corresponding limitations in layout-to-image generation. At the same time, images generated from layouts exhibit high diversity in content, thereby enhancing the robustness of image grounding. Jointly training both tasks within a unified model can promote performance improvements for each. However, we identify that this joint training paradigm encounters several optimization challenges and results in restricted performance. To address these issues, we propose progressive training strategies. First, the Parallel Multi-Task Pre-training (PMTP) stage equips the model with basic abilities for both tasks, leveraging shared tokens to accelerate training. Next, the Dual Joint Optimization (DJO) stage exploits task duality to sequentially integrate the two tasks, enabling unified optimization. Finally, the Cycle RL stage eliminates reliance on visual supervision by using consistency constraints as rewards, significantly enhancing the model's unified capabilities via the GRPO strategy. Extensive experiments demonstrate state-of-the-art results on both layout-to-image generation and image grounding benchmarks, and reveal clear synergistic gains from optimizing the two tasks together.

Figures & tables

Figure 1: Our unified approach faithfully parses complex prompts—e.g., color attributes and relative spatial relations—and, combined with layout conditions, achieves more accurate semantic alignment. In contrast, specialized methods (e.g., Gligen and MIGC ) often fail to handle such constraints. For instance, in the first row of MIGC results, the model generates a vertical spatial relation (top–middle–bottom), whereas the prompt specifies a depth-based relation (front–middle–back).

Figure 2: Overview of EchoGen . Left—Parallel Multi‑task Pre‑training: a unified autoregressive transformer is trained on layout‑to‑image and image grounding tasks in parallel, yielding fast acquisition of base capabilities. Middle—Dual Joint Optimization: the image tokens generated by the generation forward pass are directly reused as the input to grounding, forming a single joint objective, which strengthens layout

\to

image

\to

layout cycle consistency. Right—Cycled RL: we execute the layout

\to

image

\to

layout loop and treat the discrepancy between input and recovered layouts as a continuous reward, enabling self-supervised Reinforcement Learning without explicit supervision of intermediate visual outputs.

Figure 3: Attention Mask of parallel multi-task pre-training.

Gen. only
Method	MS COCO					LayoutSAM-Eval
Method	$\mathrm{AP}$ $\uparrow$	$\mathrm{AP}_{50}$ $\uparrow$	$\mathrm{AP}_{75}$ $\uparrow$	CLIP $\uparrow$	FID $\downarrow$	Spatial $\uparrow$	Color $\uparrow$	Texture $\uparrow$	Shape $\uparrow$
GLIGEN	$30.99$	$41.44$	$36.89$	$24.81$	$27.93$	$77.53$	$49.41$	$55.29$	$52.72$
MIGC	$46.16$	$58.29$	$50.21$	$25.04$	$25.35$	$85.66$	$66.97$	$71.24$	$69.06$
InstanceDiffusion	$49.97$	$61.33$	$52.65$	$25.15$	$25.00$	$87.99$	$69.16$	$72.78$	$71.08$
IFAdapter	$43.31$	$56.29$	$52.76$	$26.39$	$23.45$	$84.32$	$68.32$	$71.37$	$68.37$

Table 1: Quantitative results on MS COCO and LayoutSAM-Eval benchmark. Method marked as Gen. only perform layout-to-image generation only, while Und. and Gen. denotes unified layout-to-image generation and grounding models.

Stage	Samples	Steps
Parallel Multi‑task Pre‑training	4M	125K
Dual Joint Optimization	2M	60K
Cycled RL	50K	50K

Table 2: Training recipe.

Figure 4: Quantitative comparison on layout-to-image generation. The first column lists the global text prompts; the second column visualizes the layout conditions, rendered as (referring-expression, bounding-box) pairs. We highlight a subset of negatives and positives with red and green boxes; note that negatives may correspond to incorrect spatial locations or attributes (e.g., color) .

Method	Ref-L4
Method	$\mathrm{Acc}_{0.5}$	$\mathrm{Acc}_{0.75}$	$\mathrm{Acc}_{0.9}$	$\mathrm{mAcc}$
Und. only
KOSMOS-2	48.53	38.34	17.54	34.72
Qwen-VL-Chat	73.80	58.05	37.16	55.94
CogVLM-g.	81.70	70.77	48.35	66.09
Und. and Gen.

Table 3: Quantitative results on Ref-L4. Method marked as Und. only perform image grounding only, while Und. and Gen. denotes unified layout-to-image generation and grounding models.

	Stage 1	Stage 2	Stage 3	AP $\uparrow$	FID $\downarrow$
#1	✓			47.26	21.98
#2	✓	✓		52.38	22.23
#3	✓		✓	48.01	20.32
#4	✓	✓	✓	54.61	20.12

Table 4: Ablation over training stages.

Figure 5: Ablation on stage transition points. Left: gains when switching from Stage 1

\!\rightarrow

Stage 2; Right: gains when switching from Stage 2

\!\rightarrow

Stage 3. The x - axis records the

\mathrm{AP}

at entry to the current stage (determined by the preceding stage’s training budget), while the y - axis reports the

\mathrm{AP}

improvement achieved by that stage.

Dataset	AP	$\mathbf{AP}_{\mathbf{50}}$	$\mathbf{AP}_{\mathbf{75}}$
From real dataset	54.61	68.85	65.01
Random	53.96	68.02	64.32
Generated by GPT–4o	54.51	68.94	64.59

Table 5: Ablation on Stage 3 (RL) training set.

Explore similar work

Mar 16, 2026

CyCLeGen: Cycle-Consistent Layout Prediction and Image Generation in Vision Foundation Models

We present CyCLeGen, a unified vision-language foundation model capable of both image understanding and image generation within a single autoregressive framework. Unlike existing vision models that depend on separate modules for perception and synthesis, CyCLeGen adopts a fully integrated architecture that enforces cycle-consistent learning through image->layout->image and layout->image->layout generation loops. This unified formulation introduces two key advantages: introspection, enabling the model to reason about its own generations, and data efficiency, allowing self-improvement via synthetic supervision under a reinforcement learning objective guided by cycle consistency. Extensive experiments show that CyCLeGen achieves significant gains across diverse image understanding and generation benchmarks, highlighting the potential of unified vision-language foundation models.

Xiaojun Shan, Haoyu Shen, Yucheng Mao +5

UC San Diego

Image Generation Autoregressive Image Generation

Ask PDF

Jul 17, 2026

Think, Plan, Paint: Layout-Aware Reasoning for Controllable Image Generation in Unified Models

Unified Multimodal Large Language Models (MLLMs) offer a promising paradigm for unifying visual understanding and generation, yet they still struggle to follow complex spatial instructions and logical constraints in controllable image generation. To address this gap, we present ATLAS, a unified framework that equips MLLMs with a human-like "Think, Plan, and Paint" paradigm. We adopt layout as the shared representation that connects the three stages, enabling the model to reason about spatial requirements, plan explicit object arrangements, and render the final image. We further improve plan-to-image fidelity with reinforcement-learning-based layout alignment. We instantiate ATLAS at 7B and 80B scales, achieving state-of-the-art performance among MLLMs on image generation benchmarks and an average 65.31% improvement over existing layout-based unified MLLMs. On spatially related tasks, ATLAS obtains an average 23.06% gain over the base models. Through the same layout interface, ATLAS also supports instruction-guided editing and multimodal grounding. We further introduce ATLAS-Reasoning, a benchmark for evaluating generation under complex spatial instructions.

Junhao Liu, Jian-Wei Zhang, Tao Huang +3

Hunyuan, Tencent · School of Computer Science, Peking University

Unified Multimodal Models Image Generation Image Generation Evaluation Controllable Image Generation

Ask PDF

Sep 8, 2026

Dreaming in Flow: Generative Grounding Feedback for Self-Evolving Unified Multimodal Models

Unified multimodal models integrate visual understanding and generation within a single network, yet the two capabilities are commonly optimized as separate tasks. We introduce Generative Grounding Feedback(GGF), a self-evolving post-training framework that uses only text prompts and the model's own visual experience. Given a prompt, the model first generates a visual ``dream.'' Flow-level feedback compares text-, image-, and repair-conditioned predictions at the same noisy latent state, transferring image-grounded generation directions to the prompt condition. Dream replay grounding replays this dream through captioning and re-imagination, training claim-level evidence to remain consistent across the replay while separating unrelated visual experiences. Jointly optimized, these two directions let generation provide visual grounding for understanding and understanding refine subsequent generation without paired image--text supervision. Experiments across unified models with different understanding--generation integration designs show consistent improvements in text-to-image generation together with modest gains in visual understanding.

Ke Hao, Yuanzhi Liang, Tingxi Chen +5

Shanghai Jiao Tong University, Shanghai, China · Institute of Artificial Intelligence, China Telecom (TeleAI), Shanghai, China · University of Science and Technology of China, Hefei, China

Multimodal Grounding Unified Multimodal Models Multimodal Generation Multimodal Learning

Ask PDF