Organizations: University of California, Los Angeles · Tencent Hunyuan · The Hong Kong University of Science and Technology · The Chinese University of Hong Kong
Unified multimodal models provide a natural and promising architecture for understanding diverse and complex real-world knowledge while generating high-quality images. However, they still rely primarily on frozen parametric knowledge, which makes them struggle with real-world image generation involving long-tail and knowledge-intensive concepts. Inspired by the broad success of agents on real-world tasks, we explore agentic modeling to address this limitation. Specifically, we present Unify-Agent, a unified multimodal agent for world-grounded image synthesis, which reframes image generation as an agentic pipeline consisting of prompt understanding, multimodal evidence searching, grounded recaptioning, and final synthesis. To train our model, we construct a tailored multimodal data pipeline and curate 143K high-quality agent trajectories for world-grounded image synthesis, enabling effective supervision over the full agentic generation process. We further introduce FactIP, a benchmark covering 12 categories of culturally significant and long-tail factual concepts that explicitly requires external knowledge grounding. Extensive experiments show that our proposed Unify-Agent substantially improves over its base unified model across diverse benchmarks and real world generation tasks, while approaching the world knowledge capabilities of the strongest closed-source models. As an early exploration of agent-based modeling for world-grounded image synthesis, our work highlights the value of tightly coupling reasoning, searching, and generation for reliable open-world agentic image synthesis.
Figures & tables
Figure 1: High-quality samples from our Unify-Agent , highlighting its excellence in unified multi-image generation and agentic search enhanced world knowledge integration. It delivers strong cross-image consistency, broad stylistic versatility, and more faithful, knowledge-grounded visual generation across diverse concepts and scenarios.
Figure 2: Absolute performance improvement over prompt-only Bagel on rare-IP factual generation under different knowledge injection strategies. We compare text injection, visual injection, and joint text+visual injection across three FactIP categories: Scene, Character, and Object. The purple hatched bars denote the overall average gain across all evaluated samples.
Figure 3: Overview of our data pipeline. Starting from long-tail IP collection, we construct user instructions and Ground Truth images, build multimodal research trajectories with textual and visual evidence, and finally perform evidence-grounded recaption annotation to obtain high-quality training samples. The resulting dataset supports both SFT trajectory learning and the FactIP benchmark, which evaluates generation quality in terms of clarity, content, aesthetics, and relevance.
Figure 4: Overview of the agentic pipeline of our method. Given an input prompt, our framework first performs prompt understanding and cognitive gap detection to identify missing but visually critical attributes. It then acquires complementary multimodal evidence through textual evidence searching and visual evidence searching. Based on the collected evidence, the model grounds the generation process with two types of constraints: identity-preserving constraints that capture character-specific visual traits, and scene-compositional constraints that specify pose, environment, garment, and overall mood. These grounded constraints are then integrated into an evidence-grounded recaptioning module, which produces a detailed caption for the downstream image generator to synthesize the final image.
Model
Clarity
Content
Aesthetics
Relevance
Overall
Character
Object
Scene
Character
Object
Scene
Character
Object
Scene
Character
Object
Scene
Commercial Models
GPT-Image-1
94.6
94.1
96.2
83.5
71.1
78.4
89.2
88.1
94.1
61.9
60.2
76.4
69.3
GPT-Image-1.5
94.4
94.2
95.6
82.7
73.2
77.9
89.5
88.1
93.8
62.6
61.5
77.2
69.9
Seedream-4
93.9
83.7
88.0
81.0
92.3
78.1
88.2
81.7
91.5
80.5
90.5
86.0
83.0
Seedream-4.5
94.1
83.1
87.8
80.1
92.2
76.9
88.4
79.7
90.8
81.0
91.3
83.7
82.0
Table 1: FactIP benchmark results with category-level weighted averages. Character includes celebrity, animation, game, comic, mythology, and mascot prompts; Object includes animals/plants, food, cultural relic/art, and toy prompts; Scene includes landmark and celebration prompts. xx denotes the best score and xx the second-best within each group.
Model
Cultural
Time
Space
Biology
Physics
Chemistry
Overall
Commercial Models
GPT-Image-1
0.81
0.71
0.89
0.83
0.79
0.74
0.80
Nano Banana-Pro
0.89
0.80
0.89
0.88
0.86
0.85
0.87
Nano Banana
0.89
0.87
0.95
0.89
0.89
0.79
0.89
Generation Only
SD-v1-5
0.34
0.35
0.32
0.28
0.29
0.21
0.32
Table 2: Performance comparison on the WiSE benchmark across different knowledge dimensions (WiScore). xx denotes the best score and xx the second-best within each group.
Model
Aircraft
Vehicle
Cuisine
Flower
Insect
Landmark
Plant
Sport
Overall
Text
Entity
Text
Entity
Text
Entity
Text
Entity
Text
Entity
Text
Entity
Text
Entity
Text
Entity
Text
Entity
All
Generation Only
SD
3.06
2.34
3.62
3.12
2.97
3.09
4.22
2.28
2.84
1.22
3.78
1.97
3.69
1.69
3.47
3.44
3.46
2.39
2.92
Flux
3.31
2.41
4.31
2.97
4.28
2.47
4.12
1.75
3.50
1.12
3.94
1.72
4.44
1.22
3.75
2.69
3.96
2.04
3.00
Custom-Diff
3.22
2.94
3.50
3.41
3.53
1.78
3.72
3.28
2.66
1.88
3.44
3.34
3.47
2.91
2.75
2.03
3.29
2.70
3.00
Imagen
3.16
2.81
3.78
3.16
3.75
3.25
4.12
2.72
2.88
1.12
3.53
2.41
3.94
1.59
3.75
3.19
3.61
2.53
3.07
Table 3: KiTTEN benchmark results (scores range from 0 to 5). Each category shows Text alignment and Entity alignment scores. xx denotes the best score and xx the second-best within each group.
Model
SKCM
SKCI
MKCC
Concept
Concept
Instantiation
All
Concept
Instantiation
Composition
All
Generation Only
SD v1.5
40.5
52.9
53.9
53.4
37.6
13.4
15.1
22.0
Pixart
26.4
46.2
55.3
50.8
35.8
19.8
24.3
26.6
SD XL
45.8
59.9
65.8
62.8
51.7
28.0
35.4
38.4
Playground
45.6
66.1
62.5
64.3
53.8
35.4
44.8
44.7
Table 4: Comparison on T2I-FactBench (SKCM, SKCI, and MKCC). xx denotes the best score and xx the second-best within each group.
Variant
Clarity
Content
Aesthetics
Relevance
Overall
Baseline (Vanilla Bagel)
91.3
64.2
84.0
44.9
50.9
Pipeline Ablations
w/o Text-Search
90.7 (-0.6)
70.9 (+6.7)
84.3 (+0.3)
64.6 (+19.7)
65.4 (+14.5)
w/o Image-Search
92.1 (+0.8)
73.1 (+8.9)
85.0 (+1.0)
50.8 (+5.9)
56.2 (+5.3)
w/o Recaption
83.0 (-8.3)
69.0 (+4.8)
74.5 (-9.5)
60.2 (+15.3)
62.9 (+12.0)
Constraint Ablations
Table 5: Ablation study of Unify-Agent on the FactIP benchmark. We report top-level weighted averages over Clarity , Content , Aesthetics , and Relevance . The middle rows show expected performance drops when removing key pipeline components, grounded constraint types, or recaption-stage architectural modules. Numbers in parentheses denote changes relative to the vanilla Bagel baseline; red indicates improvement and green indicates degradation. The best result in each column is marked in bold , and the second-best is underlined .
Figure 5: Qualitative comparison of multi-image generation results on knowledge-intensive prompts involving historical figures, fictional characters, products, and stylized toys. Our method consistently produces images that better preserve subject identity, fine-grained attributes, and prompt-specific details, while achieving stronger real-world knowledge grounding than competing baselines, including Flux-1, Bagel-7b, Hunyuan, and Stable Diffusion.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Value
Model & Architecture Setup
Base LLM
Bagel-14B
Vision Encoder (ViT)
SigLIP-SO400M-14 (NaViT)
Visual Generation VAE
FLUX VAE (Frozen)
ViT Patch Size
14
VAE Latent Patch Size
2
Appendix
Table 6: Hyperparameters for the Supervised Fine-Tuning (SFT) stage.
Figure 6: Hybrid attention masking strategy for agentic image generation. The matrix shows causal attention for reasoning traces and textual dialog (blue) and full attention for reference visual-textual evidence (pink/orange/green). Notably, the generation phase (grey) employs a restricted mask that filters out historical reasoning noise, forcing the latent flow-matching tokens to attend only to relevant reference images and recaptions for high-fidelity synthesis.
Figure 7: (a) Hierarchical category distribution of FactIP Bench, consisting of three major groups (Character, Scene, and Object) and 12 fine-grained subcategories. (b) Category-wise comparison of different methods on FactIP Bench, where the radar chart presents the overall scores across all subcategories.
Category
Subcategory
Description
Num
CHARACTER
Animation
Animated characters, creatures, equipment, and iconic locations from anime and animated media.
438
Comic
Characters and visual elements originating from comic books and manga series.
363
Celebrity
Prominent figures across diverse domains, including scientists, political leaders, business executives, athletes, and entertainment personalities.
300
Game
Video game characters, weapons, equipment, and other in-game visual elements.
272
Mascot
Official mascots representing Olympic Games, regional events, and corporate brands.
77
Mythology
Universally recognized mythological narratives and legendary figures, e.g., Kuafu Chasing the Sun.
50
Appendix
Table 7: Detailed category statistics of our benchmark. The full benchmark contains three major categories and 12 fine-grained subcategories, totaling 2,462 prompts.
Model
Character
Object
Scene
Overall
Celebrity
Animation
Game
Comic
Mythology
Mascot
Animal
Food
Art
Toy
Landmark
Festival
Commercial Models
GPT-Image-1
77.0
63.1
66.4
60.3
95.7
64.7
89.0
88.6
40.9
52.2
76.1
94.2
69.3
GPT-Image-1.5
77.7
64.0
64.1
61.9
93.0
69.4
88.1
92.1
42.8
48.9
76.0
95.4
69.9
Seedream-4.5
86.0
75.6
82.3
80.8
88.9
88.1
77.0
92.4
64.8
83.0
83.5
88.9
82.0
Seedream-4
86.8
76.1
85.3
81.0
88.9
85.2
80.1
91.4
70.2
83.8
85.8
87.5
83.0
Appendix
Table 8: Per-subtask overall scores on the FactIP benchmark. Character includes celebrity, animation, game, comic, mythology, and mascot prompts; Object includes animals/plants, food, cultural relic/art, and toy prompts; Scene includes landmark and celebration prompts. xx denotes the best score and xx the second-best within each group.
Figure 8: Image generated for the prompt: "The copper is burning, highlighting the color"
Figure 9: Image generated for the "DUDOO" art toy.
Figure 10: Image generated for William Butler Yeats writing a manuscript.
Figure 11: Image generated for Grigory Perelman scribbling mathematical equations.
Figure 12: Image generated for Bruce Beutler reviewing data on a monitor.
Figure 13: MLLM evaluation for Popovich drawing a play.
Figure 14: MLLM evaluation for Max Weber delivering a passionate speech.
Figure 15: MLLM evaluation for the mascot character Habatan in the Baroque style.
Figure 16: MLLM evaluation for Scottie Pippen during an NBA playoff game.
Text-to-image (T2I) models can produce visually compelling images, yet they remain limited on open-world tasks that require complex semantic understanding, multi-step reasoning, and the integration of external world knowledge. Existing efforts introduce agent capabilities into image generation, but they either prescribe a fixed workflow or place only a subset of the open-world image generation process under agent control. Consequently, reasoning, tool invocation, and image generation are not coordinated by a single policy. We propose ToolArtist, a fully agentic image generation model obtained by post-training a Unified Multimodal Model (UMM). ToolArtist dynamically orchestrates reasoning, external tool use, and native image generation within one unified policy. During Supervised Fine-Tuning (SFT), we equip a teacher agent with search tools alongside an image-generation tool. We then convert the collected trajectories into a UMM compatible format, where the image-generation tool is concealed while the resulting generated images are retained. During Reinforcement Learning (RL), we develop an agentic RL infrastructure for UMMs and introduce Reason-Act-Draw GRPO (RAD-GRPO), which uses complementary intent and quality rewards to jointly optimize the model. Experiments show that placing the entire open-world image-generation process under an agent policy consistently outperforms approaches with fixed pipelines or only partially agent-controlled components. We release the training data and the complete post-training infrastructure.
Image generation and editing models have advanced rapidly, yet remain unreliable when prompts require external world knowledge. Bounded and long-tail parametric knowledge prevents direct or reason-then-generate approaches from recovering the required facts and visual appearances. Existing agentic generation and editing methods mitigate this limitation with retrieval tools, yet remain constrained by insufficient visual verification, overloaded policy models, and weak integration of retrieved textual and visual evidence. To address these limitations, we present WeAgent-MMGenEdit, a full-stack recipe including a multimodal harness, a scalable data construction pipeline, a comprehensive benchmark, and post-training methods for the agent policy and image backend. We first introduce WeAgent-Harness, a multimodal runtime with persistent evidence management and dedicated verification and integration tools that organize retrieved multimodal evidence into a dense carrier. Upon this, we develop a scalable pipeline for prompt synthesis and agentic trajectory collection, yielding 23K supervised trajectories and 14.7K RL tasks with three-layer verifiable checklists. We further introduce WeBench-MMGenEdit, a bilingual benchmark covering both knowledge-intensive image generation and multi-image editing. Finally, a two-sided post-training recipe based on SFT and RL improves the agent policy and image backend. Together, WeAgent-MMGenEdit enables a 30B-total/3B-active policy to outperform similarly sized policy models and approach the performance of a 1T-parameter agent.
Recent image generation models have shown strong capabilities in generating high-fidelity and photorealistic images. However, they are fundamentally constrained by frozen internal knowledge, thus often failing on real-world scenarios that are knowledge-intensive or require up-to-date information. In this paper, we present Gen-Searcher, as the first attempt to train a search-augmented image generation agent, which performs multi-hop reasoning and search to collect the textual knowledge and reference images needed for grounded generation. To achieve this, we construct a tailored data pipeline and curate two high-quality datasets, Gen-Searcher-SFT-10k and Gen-Searcher-RL-6k, containing diverse search-intensive prompts and corresponding ground-truth synthesis images. We further introduce KnowGen, a comprehensive benchmark that explicitly requires search-grounded external knowledge for image generation and evaluates models from multiple dimensions. Based on these resources, we train Gen-Searcher with SFT followed by agentic reinforcement learning with dual reward feedback, which combines text-based and image-based rewards to provide more stable and informative learning signals for GRPO training. Experiments show that Gen-Searcher brings substantial gains, improving Qwen-Image by around 16 points on KnowGen and 15 points on WISE. We hope this work can serve as an open foundation for search agents in image generation, and we fully open-source our data, models, and code.