Look Back to Move Forward: Learning to Revisit for Agentic Compositional Image Generation

Zixu Wang; Xiaoran Zhao; Yahui Liu; Feng-ao Wang; Junhao Xiao; You Shu; Bing Liu; Yiting Liu

cs.CVOct 12, 2026

Look Back to Move Forward: Learning to Revisit for Agentic Compositional Image Generation

Authors: Zixu Wang, Xiaoran Zhao, Yahui Liu, Feng-ao Wang, Junhao Xiao, You Shu, Bing Liu, Yiting Liu,

Organizations: NUDT · PDL Lab · Xiaohongshu · HNU ·

Abstract

Text-to-image models have made remarkable progress in visual fidelity and instruction following, yet complex compositional generation remains challenging. Existing agentic approaches address this limitation through iterative generation and editing, adapting their actions based on interaction history. However, they typically fix the continuation state (i.e., the image from which refinement proceeds) to the latest image. Since iterative editing is not necessarily monotonic, the agent may continue refining from the latest image even when it has already degraded. In this work, we introduce GenRevisit, an agentic framework that extends action selection to joint continuation-state and action selection. The agent can not only use interaction history to adjust editing instructions but also select an image candidate to edit, jointly deciding where to continue from and how to refine. We train GenRevisit with SFT followed by agentic reinforcement learning. To address the coarse credit assignment of final outcome reward in GRPO, which assigns the same trajectory-level advantage to all actions, we further propose Normalized Action Credit Assignment (NACA) to redistribute positive advantage according to each action's contribution to reducing the remaining objective. On GenEval2, GenRevisit achieves GM scores of 84.3 and 74.4 with Qwen-Image-2512 and Z-Image-Turbo, surpassing the strongest baselines using the same generators by 12.7 and 10.9 points respectively. GenRevisit achieves state-of-the-art performance on GenEval, GenEval++, and GenEval2 among baselines.

Figures & tables

Figure 1: An illustrative comparison between latest-state refinement and our GenRevisit.

Figure 2: Overview of GenRevisit. Panel (a) presents the overall workflow. At each decision step, the multimodal planner selects the next action based on the decision context and, when editing, jointly selects a source image as the continuation state and a corresponding edit specification. The resulting image is verified and recorded in the interaction history to inform subsequent decisions, while submission returns the historical GM-best image. Panel (b) illustrates an editing example. The planner selects $I_{1}$ , which is neither the latest candidate $I_{2}$ nor the GM-best candidate $I_{0}$ , and corrects the backpack count while preserving the other constraints. The resulting candidate $I_{3}$ passes all constraint-level checks and becomes the new GM-best image.

Panel of Table 1 — Table 1: Main results on GenEval2. GM averages prompt-level geometric means of atom scores; Skill Avg. averages the five skill-wise AM scores. Bold and underlined scores mark the best and second-best results among (i) all direct-generation models and Qwen-Image-2512 workflows, or (ii) Z-Image-Turbo workflows alone. $\dagger$ marks the best external agentic baseline within each generator block; $\uparrow$ / $\downarrow$ shows absolute score-point changes relative to it.

Method	Agent	Generator	Overall $\uparrow$
Direct generation
FLUX.1-dev [ 41 , 13 ]	–	FLUX.1-dev	67.0
Qwen-Image [ 34 , 13 ]	–	Qwen-Image	87.0
Qwen-Image-2512 † [ 9 ]	–	Qwen-Image-2512	74.0
Z-Image-Turbo † [ 9 ]	–	Z-Image-Turbo	82.0
Agentic generation and refinement

Table 2: Cross-benchmark evaluation on GenEval and GenEval++.

Figure 3: Qualitative comparison of GenRevisit and baseline models.

Variant	Overall			Skill-wise AM
	GM	$\bm{\Delta}$ GM	Skill Avg.	Object	Attribute	Position	Count	Verb
Qwen-Image-2512	29.0	–	62.0	97.9	65.2	58.1	66.9	22.1
Untuned Qwen3-VL-8B-Instruct †	49.7	$+20.7$	74.6	97.9	88.9	64.4	72.4	49.6
SFT only	81.5	$+31.8$	85.3	98.9	96.1	91.7	88.8	51.1
SFT + GRPO	80.3	$-1.2$	85.1	98.7	96.0	91.1	88.4	51.4
SFT + NACA	84.3	$+4.0$	86.7	99.4	97.2	92.5	90.3	53.9

Table 3: Ablation study on GenEval2.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Role	Model or component	Use
Planner	Qwen3-VL-8B-Instruct	Language parameters are updated; vision tower and multimodal projector are frozen.
Teacher	GPT-5.5	Collects multi-round demonstrations with executed actions and verifier feedback.
Generator	Qwen-Image-2512	Produces a new candidate without an editing input.
Editor	Qwen-Image-Edit-2511	Edits the candidate named in the action.
Verifier	Qwen3-VL-8B-Instruct	Frozen VQA evaluator; provides atomic feedback and Soft-TIFA scores.
Guidance	Fixed skill library	Returns requested guidance; does not choose the next action or image.

Appendix

Table A.1: Model roles and trainable components. Backend routing is controlled by the environment, not predicted in an action.

Component	Information
Task	Original prompt, fixed atomic requirements and questions, maximum image attempts.
Candidate history	Every completed candidate, its bound image, parent identifier, GM, constraint results, and the action that produced it.
Control	Current rankings, visible and editable candidate identifiers, legal submissions, and remaining image budget.
Retrieved guidance	Actual returned skill content and its associated target constraints.
Query control	Available skills, completed and pending queries, and remaining query allowance.
Action error, when present	The last unresolved format or validation error, used to request a new complete action.

Appendix

Table A.2: Decision-time information in the Joint input. All entries are environment-provided inputs.

Interface	Visible candidates	Editable candidates	Submission
Joint	All completed images	All completed images	GM-best
Best-only	All completed images	Current GM-best	GM-best
Latest-only	All completed images	Current latest	GM-best

Appendix

Table A.3: Candidate permissions under the GM-control interface. Rankings and permissions are recomputed after each completed image. This table defines the interfaces, not additional trained variants.

Setting	Value
Planner initialization	Qwen3-VL-8B-Instruct
Prompt membership	800 train / 100 validation / 100 test
Teacher prompt pool	1,000 prompts across the three splits
Learning rate / global batch size / epochs	$10^{-5}$ / 32 / 2
Selected action targets	3,473 train / 437 validation / 413 test
Targets per exported record	One canonical assistant action

Appendix

Table A.4: SFT settings and supervision interface, aligned with the main-paper configuration.

Setting	Value
Training protocol
Initialization	Joint selective SFT checkpoint
Rounds / prompt groups per round	5 / 64
Rollouts per group	4
Collected trajectories	256 per round; 1,280 total
Optimizer steps	8 per round; 40 total

Appendix

Table A.5: RL settings for the five-round training chain. Hardware is specified only for the final round where it was confirmed.

Action	Selection rule
Initial generation	Retain the required first generation.
Editing	Retain when $g_{\mathrm{child}}>g_{\mathrm{edit}}+\varepsilon_{\mathrm{SFT}}$ .
Subsequent generation	Retain when $g_{\mathrm{child}}>g_{\mathrm{archive}}+\varepsilon_{\mathrm{SFT}}$ .
Submission	Retain the actually executed submission when it selects the current GM-best and uses a valid reason.
Guidance retrieval	Require a real returned response. The first subsequent executed image action must itself be selected, and its target constraints must intersect the query’s target constraints.

Appendix

Table B.1: Selection rules for positive SFT targets. All comparisons use the record before the candidate action and its actual execution outcome.

Split	Generate	Edit	Retrieve	Submit	Total
Train	961	1,076	584	852	3,473
Validation	108	140	103	86	437
Test	110	114	99	90	413
Total	1,179	1,330	786	1,028	4,323

Appendix

Table B.2: Selected targets in the current Joint SFT export. Counts are action responses, not distinct prompts or raw trajectories.

Case	Handling
Normal submission	Reward is the utility of the submitted image.
Policy-invalid trajectory	Reward is $-1$ and participates in group statistics. Retained sampled responses are trained if their group is not filtered.
Image backend, transport, or verifier failure	Retry or recover; do not substitute a score or treat the failure as a policy violation.
Image budget exhausted, followed by valid submission	Score the submitted image normally; budget exhaustion alone adds no penalty.
No valid submission	Use the recorded termination class to distinguish policy invalidity from incomplete execution or infrastructure failure; do not invent a submission.
Zero-variance group	Remove the group from optimizer data, including KL.

Appendix

Table C.1: Termination and failure handling during RL data preparation.

Response or input	Advantage	KL term
Eligible image action, $A_{i}>0$ , $q_{i,t}>\delta$	$A_{i}w_{i,t}$	Retained
Eligible image action, $A_{i}>0$ , $q_{i,t}\leq\delta$	$0$	Retained
Eligible image action, positive-trajectory fallback	$A_{i}$	Retained
Image action in a nonpositive-advantage trajectory	$A_{i}$	Retained
Retrieval, submission, or retained invalid response	$A_{i}$	Retained
Context, tool results, and verifier feedback	Not a target	Excluded

Appendix

Table C.2: Credit and KL for responses in retained optimizer data. The first two rows assume effective positive progress is available.

GenEval
Method	Single obj. $\uparrow$	Two obj. $\uparrow$	Counting $\uparrow$	Colors $\uparrow$	Position $\uparrow$	Color attr. $\uparrow$	Overall $\uparrow$
Direct generation
FLUX.1-dev [ 41 ]	0.9900	0.8100	0.7500	0.8000	0.2100	0.4800	0.6700
Qwen-Image [ 34 ]	0.9900	0.9200	0.8900	0.8800	0.7600	0.7700	0.8700
Agentic generation and refinement
T2I-Copilot † [ 2 , 14 ]	0.9900	0.9100	0.6800	0.8600	0.5500	0.4600	0.7400

Appendix

Table E.1: Cross-benchmark generalization. Bold scores indicate the best result in each column.

Method	Image backend	Category-wise accuracy							Overall
		Color	Count	Color/ Count	Color/ Pos	Pos/ Count	Pos/ Size	Multi- Count
Direct generation
FLUX.1-dev [ 13 ]		FLUX.1-dev	0.4000	0.6000	0.2500	0.2500	0.0750	0.4000		0.3000	0.3250
BAGEL [ 38 ]	BAGEL	0.3250	0.6000	0.2500	0.3250	0.2500	0.4750	0.3750	0.3710
Qwen-Image [ 13 ]	Qwen-Image	0.8750	0.7250	0.7250	0.6000	0.4750	0.7250	0.5500	0.6680
Echo-4o [ 38 ]	BAGEL	0.8000	0.5750	0.5500	0.7750	0.6250	0.8000	0.6250	0.6790

Appendix

Table E.2: Comparison on GenEval++. The seven categories follow the benchmark taxonomy. Higher is better; bold marks the highest displayed result in each column.

Panel of Figure 18 — Figure F.1: Joint and Best-only from a shared prefix. Source labels identify the actual editing inputs. Joint’s final parent is a_002 , not J:a_003 ; Best-only submits shared a_001 , not B:a_004 . Suffix identifiers are branch-local. The final Joint instruction appears in Appendix G.2 .

Panel of Figure 19 — Figure F.2: An archived historical-access versus latest-only pair. Both branches use two additional calls. Full-history has parent relations a001 $\to$ H:a003 and a001 $\to$ H:a004 ; Latest-only follows a002 $\to$ L:a003 $\to$ L:a004 . The detail panels reproduce the same-region crops from the archived figure and refer to the last generated images, not necessarily to the submissions. Dotted lines denote detail views, not edits.

Panel of Figure 20 — Figure F.3: A successful continuation from below the current best. The only executed edge shown is a001 $\to$ a004 . a000 is a comparison candidate, not an intermediate image in that edit. Discrete labels report the saved verifier outcomes. This is one observed trajectory rather than a paired continuation from both possible starting images.

Image action	Pass; GM	$L_{i,t}$	$q_{i,t}$	$w_{i,t}$	GRPO	NACA
Generate	5/6; 62.03	218	.824812	1.898143	1.730187	3.284142
$\mathrm{a000}\!\to\!\mathrm{a001}$	5/6; 82.54	342	.046820	.107747	1.730187	.186422
$\mathrm{a001}\!\to\!\mathrm{a002}$	5/6; 54.16	376	0	0	1.730187	0
$\mathrm{a001}\!\to\!\mathrm{a003}$	6/6; 95.64	380	.989549	2.277251	1.730187	3.940071

Appendix

Table F.1: Recorded image-action credit. GM is on a 0–100 scale. The printed coefficients are rounded; token-weighted conservation applies to the stored full-precision values.

Panel of Figure 22 — Figure F.4: NACA changes action credit within a single positive-advantage rollout. Solid bars show NACA action advantages; the dashed line is the common GRPO broadcast value. Each bar represents an image action, not a token count. The zero bar removes positive policy-gradient credit, not the response’s KL term. Retrieval and submission are not shown and keep their trajectory advantage.

Figure F.5: Selected outputs on designed prompts. Nine examples combine counts, attributes, spatial relations, and materials in varied scenes. The prompt key summarizes each request. Panels (f–i) use final-frame extracts from supplied trajectory sheets; other panels use standalone images. Tile sizes organize the display and do not encode scores.

Figure F.6: Selected GenEval2 outputs from the SFT policy. Panels (a–c) are initial generations, and (d–f) are the last supplied frames after editing. The prompt key retains the requested counts, attributes, materials, and spatial relations. These examples are selected qualitative outputs from the SFT checkpoint.

Figure F.7: Selected GenEval++ outputs. Nine images span the benchmark’s seven compositional categories. These are selected qualitative examples from GenRevisit. Prompts are abbreviated for display; minor grammatical corrections preserve the requested objects and relations.

Figure F.8: Executed revisits on a designed prompt. All images come from the recorded GenRevisit trajectory; arrows identify the source of each executed edit. Intermediate errors remain visible alongside the submitted result.

Aspect	Teacher collection	Joint student / SFT
Input packaging	Task, planner context, candidate observations, GM continuation control	Dedicated structured decision-time input
Candidate score key	primary_score	gm
Control key	gm_continuation_control	control
Editing instruction	Four semantic parts requested in the collection guidance	One instruction string; no extra four-part schema
Candidate access	All completed images available for joint editing	All completed images visible and editable
Output	One structured action	One structured action

Appendix

Table G.1: Teacher and student input contracts. Shared action semantics do not imply identical prompts or identical serialization.

Action	Required arguments
generate_image	target_constraint_ids , preserve_constraint_ids , instruction
edit_image	source_attempt_id , target_constraint_ids , preserve_constraint_ids , instruction
query_skill	skill_ids , target_constraint_ids
submit_attempt	selected_attempt_id , reason_code

Appendix

Table G.2: Task-relevant arguments of the four actions. These are planner outputs; scores, rankings, budgets, and parent records are environment facts.

Skill	Focus
Counting and instance layout	Count and instance layout
Spatial relations	Spatial relations
Attribute binding	Attribute-to-entity binding
Local edit preservation	Local edits and preservation
Action and pose relations	Action, pose, and relation errors
Object identity and presence	Object identity and presence

Appendix

Table G.3: Documented skill catalog. Each entry supplies guidance rather than a new action type.

Constraint	Fixed verification question	Status	Probability
c_001	How many flamingos are in the image?	Pass	0.999979
c_002	Are the flamingos green?	Pass	0.999994
c_003	Are the flamingos yellow?	Pass	0.808171
c_004	Are there any flamingos in the image?	Pass	0.999774

Appendix

Table G.4: Recorded verifier feedback on the submitted Joint flamingo image. Probabilities are rounded only for display.

Compositional Image Generation

Compositional image generation studies how to combine objects, attributes, spatial relations, and visual references into coherent, controllable images.

See all 82 papers

Generation Navigator: A State-Aware Agentic Framework for Image Generation
May 18, 2026Jinming Liu, Ruoyu Feng, Yuqi Wang +2T2I Generation Agentic Image Generation
Shanghai Jiao Tong University · Independent · Eastern Institute of Technology, Ningbo, China
EPIC: Efficient Predicate-Guided Inference-Time Control for Compositional Text-to-Image Generation
May 12, 2026Sunung Mun, Sunghyun Cho, Jungseul OkCompositional T2I Generation Controllable Image Generation
Graduate School of Artificial Intelligence, POSTECH · Department of Computer Science & Engineering, POSTECH
coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generation
Mar 13, 2026Chunhan Li, Qifeng Wu, Jia-Hui Pan +7Layout-Guided T2I Generation T2I Generation
Lingnan University. · China University of Petroleum · CMU +4

May 18, 2026J/K move · Enter open · S save

Generation Navigator: A State-Aware Agentic Framework for Image Generation

Jinming Liu, Ruoyu Feng, Yuqi Wang, Wenjun Zeng+1

Shanghai Jiao Tong University · Independent · Eastern Institute of Technology, Ningbo, China

T2I Generation Agentic Image Generation RL for Image Generation Controllable Image Generation

Open paper AskPDF arXiv

Despite rapid advances in text-to-image generation, faithfully realizing user intent remains challenging, often requiring manual multi-turn trial and error. To automate this process, existing systems rely on either simple prompt rewriting or closed-loop agents driven by hand-crafted rules, rather than learning to adapt actions to the evolving generation process. In this paper, we reformulate image generation as a state-conditioned action-making problem and propose Generation Navigator, a multi-turn T2I agent that learns to dynamically steer the generation trajectory and output the next action. However, training this agent via reinforcement learning introduces a critical credit assignment challenge: naively rewarding a trajectory based solely on a single state assigns equal credit to all actions in the rollout, ignores the quality dynamics across turns, and fails to distinguish actions that improve the trajectory from those that degrade it or waste turns without progress. We resolve this with PRE-GRPO (Peak-Retention-Efficiency Group Relative Policy Optimization), a trajectory-level reinforcement learning objective that explicitly rewards discovering a high-quality image (Peak), avoiding subsequent quality degradation across turns (Retention), and minimizing unnecessary turns (Efficiency). Experiments show substantial improvements across benchmarks, reaching a WISE score of 0.90 and 79.06% reasoning accuracy on T2I-ReasonBench.

Abstract

Compositional Image Generation

Explore similar work

Generation Navigator: A State-Aware Agentic Framework for Image Generation

EPIC: Efficient Predicate-Guided Inference-Time Control for Compositional Text-to-Image Generation

coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generation