WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing

Hui Zhang; Zongkai Liu; Liqiang Niu; Juntao Liu; Han Li; Zhen Cao; Wenchao Chen; Chengduo Zhao

cs.CVSep 4, 2026

WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing

Authors: Hui Zhang, Zongkai Liu, Liqiang Niu, Juntao Liu, Han Li, Zhen Cao, Wenchao Chen, Chengduo Zhao, +1 more

Organizations: Weixin AI, Tencent

Abstract

Image generation and editing models have advanced rapidly, yet remain unreliable when prompts require external world knowledge. Bounded and long-tail parametric knowledge prevents direct or reason-then-generate approaches from recovering the required facts and visual appearances. Existing agentic generation and editing methods mitigate this limitation with retrieval tools, yet remain constrained by insufficient visual verification, overloaded policy models, and weak integration of retrieved textual and visual evidence. To address these limitations, we present WeAgent-MMGenEdit, a full-stack recipe including a multimodal harness, a scalable data construction pipeline, a comprehensive benchmark, and post-training methods for the agent policy and image backend. We first introduce WeAgent-Harness, a multimodal runtime with persistent evidence management and dedicated verification and integration tools that organize retrieved multimodal evidence into a dense carrier. Upon this, we develop a scalable pipeline for prompt synthesis and agentic trajectory collection, yielding 23K supervised trajectories and 14.7K RL tasks with three-layer verifiable checklists. We further introduce WeBench-MMGenEdit, a bilingual benchmark covering both knowledge-intensive image generation and multi-image editing. Finally, a two-sided post-training recipe based on SFT and RL improves the agent policy and image backend. Together, WeAgent-MMGenEdit enables a 30B-total/3B-active policy to outperform similarly sized policy models and approach the performance of a 1T-parameter agent.

Figures & tables

Figure 1: WeAgent-MMGenEdit improves image generation and editing, outperforming similarly sized policy models.

Figure 2: WeAgent-MMGenEdit on knowledge-intensive generation (top) and multi-reference editing (bottom).

Figure 3: Three paradigms for multimodal knowledge-intensive image generation and editing. (A) Closed-book generation relies solely on parametric knowledge and suffers from severe factual hallucinations. (B) Existing agentic systems retrieve external evidence but fail to effectively verify and integrate it because of insufficient visual inspection, an overloaded policy model, or weak entity–attribute–layout binding. (C) WeAgent-MMGenEdit explicitly verifies retrieved visual evidence and organizes multimodal evidence into a dense rendered carrier that provides both content and layout guidance for final generation, yielding the most accurate and faithful generation results.

Figure 4: WeAgent-Harness: A Multimodal Runtime for Retrieval, Verification, and Integration. Search tools acquire external textual and visual evidence, which is stored under stable identifiers. Dedicated visual verification and code-based integration modules inspect candidate images and organize verified multimodal evidence into a dense carrier for final image generation and editing. The final response interleaves the generated image with concise textual information; all interactions and artifacts are recorded for training and evaluation.

Role	Tool	Function
Retrieve	Web_Search	Queries the open web and returns titles, links, snippets, and dates.
	Web_Extract	Fetches a webpage and extracts information relevant to the current need.
	Search_for_Image	Retrieves candidate images using a textual query.
	Search_by_Image	Searches the web using an image for identification and provenance.
Verify	Vision_Analyze	Answers focused questions about registered images.
Integrate	Execute_Code	Integrates verified multimodal evidence and renders it into a draft image.

Table 1: Tool interfaces in WeAgent-Harness , organized by the retrieve–verify–integrate–deliver workflow.

Figure 5: Construction of WeDataset-MMGenEdit. Top: bilingual multimodal knowledge is organized into a concept bank and knowledge graph, from which multi-hop tasks and three layers of verifiable checklists are synthesized simultaneously. Bottom: tasks are executed in WeAgent-Harness, and the resulting agentic process, generation input, and final image are independently graded and routed to SFT, RL, or evaluation pools.

Figure 6: Composition of WeBench-MMGenEdit. The benchmark contains $300$ human-audited cases, exactly balanced between generation and editing and between English and Chinese. It emphasizes deep multimodal retrieval and knowledge-intensive generation and editing: $94\%$ of cases require at least five retrieval hops, and $69.3\%$ of editing cases contain multiple user-provided images.

Figure 7: Checklist-grounded agentic RL. Grouped trajectories are sampled in WeAgent-Harness and scored by isolated process and generation-input judges. Valid trajectories are normalized within each prompt group and optimized with GSPO using only policy-generated tokens. Rollout, reward evaluation, and actor optimization run asynchronously, with updated parameters periodically synchronized to rollout workers.

Figure 8: Two-stage image editing post-training. (a) Multi-reference Image Editing SFT learns stable conditioning over heterogeneous references using independent resolution buckets and target-only supervision. (b) Diffusion-NFT RL stage samples multiple candidates, evaluates them with a decomposed multimodal reward, and optimizes group-relative preferences while remaining anchored to the SFT model.

Model	Policy Model	Gen/Edit Model	Harness	Image Generation					Image Editing
Model	Policy Model	Gen/Edit Model	Harness	IA	TKA	VKA	VQ	W_Avg	IA	TKA	VKA	CP	VQ	W_Avg
Direct Gen/Edit
GPT-Image-2	N/A	GPT-Image-2	N/A	63.20	33.83	42.18	85.13	48.79	53.86	23.67	47.95	81.06	82.33	48.99
Gemini3.1-Flash	N/A	Gemini3.1-Flash	N/A	67.18	46.50	45.70	81.41	55.12	56.35	31.49	54.60	75.08	77.72	51.79
Gemini3.1-Flash-Lite	N/A	Gemini3.1-Flash-Lite	N/A	65.77	46.22	43.84	79.06	53.81	53.07	30.52	47.95	66.71	76.27	48.51
Seedream-5.0-pro	N/A	Seedream-5.0-pro	N/A	58.07	29.91	35.56	80.01	43.58	54.09	26.37	47.72	77.34	78.18	48.60

Table 2: Quantitative Results on Image Generation and Editing. Comparison of direct generation/editing models, reasoning-based pipelines, existing agentic methods, agentic generation/editing within WeAgent-Harness, and our WeAgent-MMGenEdit models. TKA and VKA , highlighted in light yellow, are the primary knowledge-oriented evaluation dimensions.

Models	Science & Knowledge				Pop Culture & News				Overall
Models	Visual cor.	Text acc.	Faithfulness	Aesthetics	Visual cor.	Text acc.	Faithfulness	Aesthetics	Overall
Direct Gen/Edit
GPT-Image-1	20.92	27.89	72.79	63.95	19.43	31.98	84.64	61.60	34.19
GPT-Image-1.5	29.25	40.14	81.29	77.21	29.43	46.22	89.64	71.17	44.97
Gemini 2.5 Flash Image	18.03	19.39	72.79	65.82	14.24	26.04	84.39	70.91	30.24
Gemini 3 Pro Image	39.46	49.32	86.22	70.92	30.51	53.37	91.07	68.75	50.38

Table 3: Quantitative Results on the KnowGen Benchmark. We compare direct image generation models and agentic generation methods on the Science

\&

Knowledge and Pop Culture

\&

News subsets.

Model Name	Knowledge-Driven					Reasoning-Driven					Overall
Model Name	SE	Weather	MC	IP	WK	SL	Poem	Life Reason	GU	Math	Overall
Direct Gen/Edit
GPT-Image-1	0.32	0.06	0.22	0.02	0.16	0.32	0.10	0.24	0.10	0.12	0.17
GPT-Image-1.5	0.36	0.18	0.22	0.04	0.30	0.34	0.08	0.34	0.10	0.02	0.21
Gemini 2.5 Flash Image	0.30	0.10	0.12	0.00	0.30	0.32	0.36	0.20	0.04	0.08	0.18
Gemini 3 Pro Image	0.50	0.36	0.40	0.16	0.56	0.62	0.68	0.30	0.16	0.46	0.41

Table 4: Quantitative Results on Mind-Bench. We compare direct image generation models and agentic generation methods across knowledge-driven and reasoning-driven tasks.

Figure 9: Qualitative comparison on knowledge-intensive generation and editing. WeAgent-MMGenEdit more effectively retrieves, verifies, and integrates multimodal evidence, yielding more accurate and consistent outputs across different image-generation backends than direct and existing agentic baselines.

Model Name	Harness	Image Generation							Image Editing
		Agentic Chain Score				Prompt Score			Agentic Chain Score				Prompt Score
		TKR	VKR	MEI	Avg	TKS	VKS	Avg	TKR	VKR	MEI	Avg	TKS	VKS	Avg
Open-Source Agentic Gen/Edit
Mind Brush (GPT-5.1)	Mind Brush	50.31	46.95	49.27	48.84	52.91	44.10	48.51	20.76	20.17	36.10	25.68	26.71	10.01	18.36
GenSearcher-8B	GenSearcher	45.41	44.17	41.54	43.71	43.89	38.73	41.31	18.66	19.13	27.54	21.78	22.55	8.42	15.49
GenEvolve-8B	GenEvolve	42.14	53.66	42.71	46.17	46.03	46.47	46.25	22.38	21.59	35.92	26.63	29.92	17.37	23.65

Table 5: Comparison of Agentic Chain and Prompt Scores. We evaluate existing agentic systems and different policies within WeAgent-Harness on generation and editing.

Ablation						Performance
Search Tools	Verify Tools	Integrate Tools	Agent SFT	Agent RL	Process Reward	Gen. W-Avg	Edit. W-Avg
Harness Ablation
$\checkmark$	$\times$	$\times$	–	–	–	47.22	42.14
$\checkmark$	$\checkmark$	$\times$	–	–	–	49.03	43.30
$\checkmark$	$\times$	$\checkmark$	–	–	–	48.70	42.91
$\checkmark$	$\checkmark$	$\checkmark$	–	–	–	51.86	45.80

Table 6: Ablation study of WeAgent-MMGenEdit. Contributions of WeAgent-Harness components and different stages of agent post-training are evaluated on image generation and editing using weighted-average performance.

	Agent SFT	Agent RL
Initialization	Qwen3-VL-30B-A3B	WeAgent-SFT
Training data	$\sim 23$ K expert trajectories	$\sim 14.7$ K RL task pool
Batching	Global batch 32	64 prompts $\times$ 8 trajectories
Maximum context	49,152 tokens	32,768-token episode context
Optimizer	Adam	Adam
Learning rate	$1\times 10^{-5}$	$5\times 10^{-7}$

Table 7: Main configurations for agent-side post-training.

	WeEdit-M-SFT	WeEdit-M-RL
Initialization	Qwen-Image-Edit-2509	WeEdit-M-SFT
Training data	$\sim 50$ K samples	Hard subset from $\sim 15$ K RL candidates
Trainable parameters	LoRA, rank 256	LoRA, rank 256
Reference images	1–5	1–5
Learning rate	$5\times 10^{-5}$	$5\times 10^{-5}$
Candidates / condition	–	$K=12$

Table 8: Main configurations for image-side post-training.

Figure 10: Agentic trajectory for Case 1 (Part 1 of 4).

Figure 11: Agentic trajectory for Case 1 (Part 2 of 4).

Figure 12: Agentic trajectory for Case 1 (Part 3 of 4).

Figure 13: Agentic trajectory for Case 1 (Part 4 of 4).

Figure 14: Agentic trajectory for Case 2 (Part 1 of 4).

Figure 15: Agentic trajectory for Case 2 (Part 2 of 4).

Figure 16: Agentic trajectory for Case 2 (Part 3 of 4).

Figure 17: Agentic trajectory for Case 2 (Part 4 of 4).

Figure 18: Agentic trajectory for Case 3 (Part 1 of 6).

Figure 19: Agentic trajectory for Case 3 (Part 2 of 6).

Figure 20: Agentic trajectory for Case 3 (Part 3 of 6).

Figure 21: Agentic trajectory for Case 3 (Part 4 of 6).

Figure 22: Agentic trajectory for Case 3 (Part 5 of 6).

Figure 23: Agentic trajectory for Case 3 (Part 6 of 6).

Figure 24: Agentic trajectory for Case 4 (Part 1 of 3).

Figure 25: Agentic trajectory for Case 4 (Part 2 of 3).

Figure 26: Agentic trajectory for Case 4 (Part 3 of 3).

Explore similar work

Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis
Mar 31, 2026Shuang Chen, Quanxin Shou, Hangting Chen +16Image Generation Agentic Image Generation
University of California, Los Angeles · Tencent Hunyuan · The Hong Kong University of Science and Technology +1
RS-Gen: A Multi-Stage Agentic Framework for Reasoning and Search-Augmented Image Generation
Jun 22, 2026Feifei Bian, Zhimin Zheng, Wei Deng +2Image Generation Agentic Image Generation
MiLM Plus, Xiaomi Inc.
APE: Agentic Prompt Enhancer for Image Generation and Editing
May 29, 2026Zijian Huang, Jay Zhangjie Wu, Zian Wang +5Text-Guided Image Editing T2I Generation
1NVIDIA · University of Michigan

Mar 31, 2026J/K move · Enter open · S save

Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis

Shuang Chen, Quanxin Shou, Hangting Chen, Yucheng Zhou+15

University of California, Los Angeles · Tencent Hunyuan · The Hong Kong University of Science and Technology+1

Image Generation Agentic Image Generation Unified Multimodal Models Multimodal RAG

Open paper AskPDF arXiv

Unified multimodal models provide a natural and promising architecture for understanding diverse and complex real-world knowledge while generating high-quality images. However, they still rely primarily on frozen parametric knowledge, which makes them struggle with real-world image generation involving long-tail and knowledge-intensive concepts. Inspired by the broad success of agents on real-world tasks, we explore agentic modeling to address this limitation. Specifically, we present Unify-Agent, a unified multimodal agent for world-grounded image synthesis, which reframes image generation as an agentic pipeline consisting of prompt understanding, multimodal evidence searching, grounded recaptioning, and final synthesis. To train our model, we construct a tailored multimodal data pipeline and curate 143K high-quality agent trajectories for world-grounded image synthesis, enabling effective supervision over the full agentic generation process. We further introduce FactIP, a benchmark covering 12 categories of culturally significant and long-tail factual concepts that explicitly requires external knowledge grounding. Extensive experiments show that our proposed Unify-Agent substantially improves over its base unified model across diverse benchmarks and real world generation tasks, while approaching the world knowledge capabilities of the strongest closed-source models. As an early exploration of agent-based modeling for world-grounded image synthesis, our work highlights the value of tightly coupling reasoning, searching, and generation for reliable open-world agentic image synthesis.

Abstract

Explore similar work

Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis

RS-Gen: A Multi-Stage Agentic Framework for Reasoning and Search-Augmented Image Generation

APE: Agentic Prompt Enhancer for Image Generation and Editing