CyCLeGen: Cycle-Consistent Layout Prediction and Image Generation in Vision Foundation Models

Xiaojun Shan; Haoyu Shen; Yucheng Mao; Xiang Zhang; Abhay Anand; Bingnan Li; Haiyang Xu; Zhuowen Tu

cs.CVMar 16, 2026

CyCLeGen: Cycle-Consistent Layout Prediction and Image Generation in Vision Foundation Models

Authors: Xiaojun Shan, Haoyu Shen, Yucheng Mao, Xiang Zhang, Abhay Anand, Bingnan Li, Haiyang Xu, Zhuowen Tu

Organizations: UC San Diego

Abstract

We present CyCLeGen, a unified vision-language foundation model capable of both image understanding and image generation within a single autoregressive framework. Unlike existing vision models that depend on separate modules for perception and synthesis, CyCLeGen adopts a fully integrated architecture that enforces cycle-consistent learning through image->layout->image and layout->image->layout generation loops. This unified formulation introduces two key advantages: introspection, enabling the model to reason about its own generations, and data efficiency, allowing self-improvement via synthetic supervision under a reinforcement learning objective guided by cycle consistency. Extensive experiments show that CyCLeGen achieves significant gains across diverse image understanding and generation benchmarks, highlighting the potential of unified vision-language foundation models.

Figures & tables

Figure 1: Overview of the CyCLeGen framework. CyCLeGen employs a unified autoregressive transformer that jointly performs layout understanding (image → layout) and layout-conditioned image generation (layout → image) within a shared token space. Given image tokens, the model predicts layout and text tokens through the understanding branch; conversely, conditioned on layout and text tokens, the generation branch produces image tokens. Training enforces bidirectional cycle consistency across the two directions, forming image → layout → image and layout → image → layout loops. Cycle rewards/losses encourage the generated outputs in one direction to remain structurally and semantically recoverable in the reverse direction, aligning visual understanding with controllable image generation.

Figure 2: Overview of CycleGRPO. CycleGRPO trains a unified autoregressive transformer for both layout understanding and layout-conditioned generation through two complementary directions. The Und $\rightarrow$ Gen direction ( blue arrows ) starts from an input image, where the model autoregressively predicts layouts via trajectory rollouts and decodes them into bounding boxes. The Gen $\rightarrow$ Und direction ( purple arrows ) starts from a layout prompt, where the model generates images and enforces that the generated images can recover the original layout through understanding. Multiple trajectories are sampled in both directions and optimized with GRPO, providing cycle-consistent reinforcement signals that align visual understanding with image generation.

OverlayBench-Simple
Type	Method	# Params	mIoU(%) $\uparrow$	O-mIoU(%) $\uparrow$	SR ${}_{\text{E}}$ (%) $\uparrow$	SR ${}_{\text{R}}$ (%) $\uparrow$	CLIP ${}_{\text{Global}}$ $\uparrow$	CLIP ${}_{\text{Local}}$ $\uparrow$
Diffusion	GLIGEN [ 18 ]	1.07B	60.54	36.22	49.99	78.72	34.17	24.75
	InstanceDiff [ 29 ]	1.23B	71.21	49.99	77.71	87.49	34.25	27.69
	MIGC [ 45 ]	0.86B	58.64	32.15	63.41	81.60	33.07	26.49
	HiCo [ 7 ]	1.22B	69.47	47.23	67.75	86.08	35.25	27.04
	CreatiLayout [ 40 ]	3.31B	58.78	32.52	72.34	84.45	37.29	27.49

Table 1: Comparison of diffusion-based and autoregressive layout-to-image generation models on the OverlayBench [ 17 ] benchmark across Simple, Regular, and Complex splits. Improvements are computed relative to PlanGen.

Figure 3: Qualitative comparison of layout-to-image generation. Given the same input layouts (Ground Truth), our CyCLeGen model produces images that more faithfully preserve object geometry, spatial configuration, and fine-grained structure compared to PlanGen and diffusion-based baselines (InstanceDiff [ 29 ] , HiCo [ 7 ] , CreatiLayout [ 40 ] ). Across diverse scenes—including people, architectural facades, cable cars, and airplanes—CyCLeGen demonstrates stronger adherence to layout constraints and generates visually coherent, detail-rich images.

Type	Method	Region-wise Quality				Global-wise Quality
Type	Method	Spatial $\uparrow$	Color $\uparrow$	Texture $\uparrow$	Shape $\uparrow$	Pick $\uparrow$	FID $\downarrow$	IS $\uparrow$
	Real Images	98.95	98.45	98.90	98.80	-	-	-
Diffusion	GLIGEN [ 18 ]	77.53	49.41	55.29	52.72	20.78	21.92	20.57
	InstanceDiff [ 29 ]	87.99	69.16	72.78	71.08	21.01	19.67	20.02
	MIGC [ 45 ]	85.66	66.97	71.24	69.06	20.71	21.19	19.65
	Hico [ 7 ]	90.22	69.89	73.95	72.75	22.44	19.30	22.50

Table 2: Layout-to-image quantitative comparison on the LayoutSAM-Eval [ 40 ] benchmark. We report region-wise quality (Spatial, Color, Texture, Shape) and global perceptual metrics (PickScore, FID, IS).

Setting	AP ↑	AP50 ↑	AP75 ↑	AR ↑
GroundingDINO [ 20 ]	42.53	47.20	44.60	83.61
Janus-Pro [ 5 ]	9.05	13.68	7.20	14.67
PlanGen-baseline [ 11 ]	27.66	33.94	30.56	51.58
CyCLeGen (ours)	28.33 ( $+0.67$ )	36.99 ( $+3.05$ )	30.81 ( $+0.25$ )	71.05 ( $+19.47$ )

Table 3: Quantitative results on layout understanding evaluated on the LayoutSamEval benchmark [ 40 ] . We compare our model with the PlanGen baseline, along with strong vision-language and detection baselines, including Grounding-DINO [ 20 ] and Janus-Pro [ 5 ] . Metrics include AP, AP50, AP75, and AR. Improvements are computed relative to PlanGen.

Figure 4: Qualitative comparison of layout understanding results. We compare CyCLeGen (top) with PlanGen (middle) and GroundingDINO (bottom) across diverse scenes. CyCLeGen produces more complete and precise detections, with fewer missed objects, cleaner bounding boxes, and more semantically coherent structured descriptions. In contrast, PlanGen often yields fragmented boxes, duplicated predictions, and under-detection in complex scenes. These results demonstrate that our cycle-consistent training substantially boosts structural grounding quality.

Method	BLEU-1 ↑	BLEU-2 ↑	BLEU-3 ↑	BLEU-4 ↑	METEOR ↑	ROUGE-L ↑	CIDEr ↑
Janus-Pro [ 5 ]	35.26	19.73	11.19	6.66	21.78	26.04	12.80
PlanGen-baseline [ 11 ]	43.06	30.20	22.68	17.92	21.92	33.81	37.48
CyCLeGen (ours) SFT	51.72 ( $+8.66$ )	37.97 ( $+7.77$ )	29.27 ( $+6.60$ )	23.58 ( $+5.66$ )	25.43 ( $+3.51$ )	40.10 ( $+6.29$ )	57.90 ( $+20.42$ )
RL w/o cycle	46.63 (+3.57)	31.83 (+1.63)	23.20 (+0.52)	17.99 (+0.07)	21.85 (-0.07)	34.92 (+1.11)	38.86 (+1.38)
CyCLeGen (ours) RL	52.99 ( $+9.93$ )	39.91 ( $+9.71$ )	31.41 ( $+8.73$ )	25.68 ( $+7.76$ )	26.84 ( $+4.92$ )	42.35 ( $+8.54$ )	66.46 ( $+28.98$ )

Table 4: Quantitative comparison of the image captioning performance on the LayoutSamEval benchmark [ 40 ] . We report BLEU-1/2/3/4, METEOR, ROUGE-L, and CIDEr scores for the baseline PlanGen [ 11 ] and Janus Pro [ 5 ] , our SFT-variant model, and reinforcement learning. Note that CyCLeGen has not been trained on this captioning task, yet, it demonstrates a substantial improvement over the baselines, clearly demonstrating the strong advantage of adding the cycle-consistency to the basic visual foundation model. Improvements are computed relative to PlanGen.

Model	AP $\uparrow$	AP50 $\uparrow$	AP75 $\uparrow$	AR $\uparrow$	Spatial $\uparrow$	Color $\uparrow$	Texture $\uparrow$	Shape $\uparrow$
PlanGen	27.66	33.94	30.56	51.58	92.21	82.69	86.53	85.36
PlanGen- Und	28.00	37.78	29.40	48.17	88.16 (-4.05)	76.92 (-5.77)	80.71 (-5.82)	79.71 (-5.65)
PlanGen- Gen	8.41 (-19.25)	16.33 (-17.61)	7.27 (-23.29)	17.62 (-33.96)	91.41	81.83	85.84	84.81
PlanGen- Joint	24.28 (-3.38)	34.26 (+0.32)	25.38 (-5.18)	45.54 (-6.04)	91.00 (-1.21)	81.75 (-0.94)	85.64 (-0.89)	84.28 (-1.08)
PlanGen- w/o cycle	27.83 (+0.17)	36.77 (+2.83)	29.63 (-0.93)	51.61 (+0.03)	91.72 (-0.49)	82.45 (-0.24)	86.78 (+0.25)	86.19 (+0.83)
CyCLeGen (ours)	28.33 ( $+0.67$ )	36.99 ( $+3.05$ )	30.81 ( $+0.25$ )	71.05 ( $+19.47$ )	92.71 (+0.50)	84.22 (+1.53)	87.97 (+1.44)	87.02 (+1.66)

Table 5: Comparison with PlanGen and our variants on LayoutSAM-Eval [ 40 ] . Understanding metrics are on the left, and generation metrics on the right.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Type	Method	HPSv2
Diffusion	InstanceDiff [ 29 ]	21.25
	HiCo [ 7 ]	27.51
	CreatiLayout [ 40 ]	25.95
AR	PlanGen [ 11 ]	24.50
AR	CyCLeGen (ours)	25.96

Appendix

Table 6: HPSv2 comparison between diffusion-based methods and autoregressive models.

Method	Pick $\uparrow$	FID $\downarrow$	IS $\uparrow$
GT Layout	21.53	15.62	20.42
PlanGen	21.40	16.18	20.18
CyCLeGen	21.32	19.27	21.05

Appendix

Table 7: Layout planning quantitative comparison of CyCLeGen with Plangen [ 11 ] and GT layout on LayoutSAM-Eval [ 40 ] .

Setting	AP $\uparrow$	AP50 $\uparrow$	AP75 $\uparrow$	AR $\uparrow$	Spatial $\uparrow$	Color $\uparrow$	Texture $\uparrow$	Shape $\uparrow$
PlanGen	27.66	33.94	30.56	51.58	92.21	82.69	86.53	85.36
w / o SFT	27.89	36.77	30.62	62.91	92.69	83.79	87.29	86.46
CyCLeGen	28.33	36.99	30.81	71.05	92.71	84.22	87.97	87.02
$\Delta$	+0.44	+0.22	+0.19	+8.14	+0.02	+0.43	+0.68	+0.56

Appendix

Table 8: Comparison between PlanGen, training without SFT, and CyCLeGen. Understanding metrics are on the left, and generation metrics on the right. Improvements are computed relative to the case without SFT.

Method	LocalCLIP $\uparrow$	IS $\uparrow$	SR (%) $\uparrow$
PlanGen	25.28	9.08	14.50
CycleGen	25.31	9.18	17.00

Appendix

Table 9: Comparison of image editing performance between PlanGen and CyCLeGen. Higher is better for all metrics.

Method	LocalCLIP $\downarrow$	IS $\uparrow$	SR (%) $\uparrow$
PlanGen	23.08	9.37	87.50
CycleGen	22.92	9.09	86.00

Appendix

Table 10: Object removal results on 200 samples from the COCO test set. Higher is better for all metrics, except for LocalCLIP.

Explore similar work

EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding
Mar 18, 2026Kai Zou, Hongbo Liu, Dian Zheng +3Image Generation Vision-Language Grounding
School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Tongji University +3
Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards
Jun 25, 2026Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar +5Unified Multimodal Models Multimodal Generation
1Mohamed bin Zayed University of Artificial Intelligence · 4Linköping University · 3Australian National University +1
ABACUS: Adapting Unified Foundation Model for Bridging Image Count Understanding and Generation
Jun 22, 2026Anindya Mondal, Sauradip Nag, Anjan DuttaVLM Adaptation Crowd Counting
University of Surrey, United Kingdom & Simon Fraser University, Canada · University of Surrey, United Kingdom

Mar 18, 2026J/K move · Enter open · S save

EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding

Kai Zou, Hongbo Liu, Dian Zheng, Jianxiong Gao+2

School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Tongji University+3

Image Generation Vision-Language Grounding Compositional Image Generation

Open paper AskPDF arXiv

In this work, we present EchoGen, a unified framework for layout-to-image generation and image grounding, capable of generating images with accurate layouts and high fidelity to text descriptions (e.g., spatial relationships), while grounding the image robustly at the same time. We believe that image grounding possesses strong text and layout understanding abilities, which can compensate for the corresponding limitations in layout-to-image generation. At the same time, images generated from layouts exhibit high diversity in content, thereby enhancing the robustness of image grounding. Jointly training both tasks within a unified model can promote performance improvements for each. However, we identify that this joint training paradigm encounters several optimization challenges and results in restricted performance. To address these issues, we propose progressive training strategies. First, the Parallel Multi-Task Pre-training (PMTP) stage equips the model with basic abilities for both tasks, leveraging shared tokens to accelerate training. Next, the Dual Joint Optimization (DJO) stage exploits task duality to sequentially integrate the two tasks, enabling unified optimization. Finally, the Cycle RL stage eliminates reliance on visual supervision by using consistency constraints as rewards, significantly enhancing the model's unified capabilities via the GRPO strategy. Extensive experiments demonstrate state-of-the-art results on both layout-to-image generation and image grounding benchmarks, and reveal clear synergistic gains from optimizing the two tasks together.

Abstract

Explore similar work

EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding

Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards

ABACUS: Adapting Unified Foundation Model for Bridging Image Count Understanding and Generation