cs.CVMar 31, 2026
SaveUnify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis
Organizations: University of California, Los Angeles · Tencent Hunyuan · The Hong Kong University of Science and Technology · The Chinese University of Hong Kong
Abstract
Unified multimodal models provide a natural and promising architecture for understanding diverse and complex real-world knowledge while generating high-quality images. However, they still rely primarily on frozen parametric knowledge, which makes them struggle with real-world image generation involving long-tail and knowledge-intensive concepts. Inspired by the broad success of agents on real-world tasks, we explore agentic modeling to address this limitation. Specifically, we present Unify-Agent, a unified multimodal agent for world-grounded image synthesis, which reframes image generation as an agentic pipeline consisting of prompt understanding, multimodal evidence searching, grounded recaptioning, and final synthesis. To train our model, we construct a tailored multimodal data pipeline and curate 143K high-quality agent trajectories for world-grounded image synthesis, enabling effective supervision over the full agentic generation process. We further introduce FactIP, a benchmark covering 12 categories of culturally significant and long-tail factual concepts that explicitly requires external knowledge grounding. Extensive experiments show that our proposed Unify-Agent substantially improves over its base unified model across diverse benchmarks and real world generation tasks, while approaching the world knowledge capabilities of the strongest closed-source models. As an early exploration of agent-based modeling for world-grounded image synthesis, our work highlights the value of tightly coupling reasoning, searching, and generation for reliable open-world agentic image synthesis.
Figures & tables
Figure 1: High-quality samples from our Unify-Agent , highlighting its excellence in unified multi-image generation and agentic search enhanced world knowledge integration. It delivers strong cross-image consistency, broad stylistic versatility, and more faithful, knowledge-grounded visual generation across diverse concepts and scenarios.
Figure 2: Absolute performance improvement over prompt-only Bagel on rare-IP factual generation under different knowledge injection strategies. We compare text injection, visual injection, and joint text+visual injection across three FactIP categories: Scene, Character, and Object. The purple hatched bars denote the overall average gain across all evaluated samples.
Figure 3: Overview of our data pipeline. Starting from long-tail IP collection, we construct user instructions and Ground Truth images, build multimodal research trajectories with textual and visual evidence, and finally perform evidence-grounded recaption annotation to obtain high-quality training samples. The resulting dataset supports both SFT trajectory learning and the FactIP benchmark, which evaluates generation quality in terms of clarity, content, aesthetics, and relevance.
Figure 4: Overview of the agentic pipeline of our method. Given an input prompt, our framework first performs prompt understanding and cognitive gap detection to identify missing but visually critical attributes. It then acquires complementary multimodal evidence through textual evidence searching and visual evidence searching. Based on the collected evidence, the model grounds the generation process with two types of constraints: identity-preserving constraints that capture character-specific visual traits, and scene-compositional constraints that specify pose, environment, garment, and overall mood. These grounded constraints are then integrated into an evidence-grounded recaptioning module, which produces a detailed caption for the downstream image generator to synthesize the final image.
| Model | Clarity | Content | Aesthetics | Relevance | Overall | ||||||||
| Character | Object | Scene | Character | Object | Scene | Character | Object | Scene | Character | Object | Scene | ||
| Commercial Models | |||||||||||||
| GPT-Image-1 | 94.6 | 94.1 | 96.2 | 83.5 | 71.1 | 78.4 | 89.2 | 88.1 | 94.1 | 61.9 | 60.2 | 76.4 | 69.3 |
| GPT-Image-1.5 | 94.4 | 94.2 | 95.6 | 82.7 | 73.2 | 77.9 | 89.5 | 88.1 | 93.8 | 62.6 | 61.5 | 77.2 | 69.9 |
| Seedream-4 | 93.9 | 83.7 | 88.0 | 81.0 | 92.3 | 78.1 | 88.2 | 81.7 | 91.5 | 80.5 | 90.5 | 86.0 | 83.0 |
| Seedream-4.5 | 94.1 | 83.1 | 87.8 | 80.1 | 92.2 | 76.9 | 88.4 | 79.7 | 90.8 | 81.0 | 91.3 | 83.7 | 82.0 |
Table 1: FactIP benchmark results with category-level weighted averages. Character includes celebrity, animation, game, comic, mythology, and mascot prompts; Object includes animals/plants, food, cultural relic/art, and toy prompts; Scene includes landmark and celebration prompts. xx denotes the best score and xx the second-best within each group.
| Model | Cultural | Time | Space | Biology | Physics | Chemistry | Overall |
| Commercial Models | |||||||
| GPT-Image-1 | 0.81 | 0.71 | 0.89 | 0.83 | 0.79 | 0.74 | 0.80 |
| Nano Banana-Pro | 0.89 | 0.80 | 0.89 | 0.88 | 0.86 | 0.85 | 0.87 |
| Nano Banana | 0.89 | 0.87 | 0.95 | 0.89 | 0.89 | 0.79 | 0.89 |
| Generation Only | |||||||
| SD-v1-5 | 0.34 | 0.35 | 0.32 | 0.28 | 0.29 | 0.21 | 0.32 |
Table 2: Performance comparison on the WiSE benchmark across different knowledge dimensions (WiScore). xx denotes the best score and xx the second-best within each group.
| Model | Aircraft | Vehicle | Cuisine | Flower | Insect | Landmark | Plant | Sport | Overall | ||||||||||
| Text | Entity | Text | Entity | Text | Entity | Text | Entity | Text | Entity | Text | Entity | Text | Entity | Text | Entity | Text | Entity | All | |
| Generation Only | |||||||||||||||||||
| SD | 3.06 | 2.34 | 3.62 | 3.12 | 2.97 | 3.09 | 4.22 | 2.28 | 2.84 | 1.22 | 3.78 | 1.97 | 3.69 | 1.69 | 3.47 | 3.44 | 3.46 | 2.39 | 2.92 |
| Flux | 3.31 | 2.41 | 4.31 | 2.97 | 4.28 | 2.47 | 4.12 | 1.75 | 3.50 | 1.12 | 3.94 | 1.72 | 4.44 | 1.22 | 3.75 | 2.69 | 3.96 | 2.04 | 3.00 |
| Custom-Diff | 3.22 | 2.94 | 3.50 | 3.41 | 3.53 | 1.78 | 3.72 | 3.28 | 2.66 | 1.88 | 3.44 | 3.34 | 3.47 | 2.91 | 2.75 | 2.03 | 3.29 | 2.70 | 3.00 |
| Imagen | 3.16 | 2.81 | 3.78 | 3.16 | 3.75 | 3.25 | 4.12 | 2.72 | 2.88 | 1.12 | 3.53 | 2.41 | 3.94 | 1.59 | 3.75 | 3.19 | 3.61 | 2.53 | 3.07 |
Table 3: KiTTEN benchmark results (scores range from 0 to 5). Each category shows Text alignment and Entity alignment scores. xx denotes the best score and xx the second-best within each group.
| Model | SKCM | SKCI | MKCC | |||||
| Concept | Concept | Instantiation | All | Concept | Instantiation | Composition | All | |
| Generation Only | ||||||||
| SD v1.5 | 40.5 | 52.9 | 53.9 | 53.4 | 37.6 | 13.4 | 15.1 | 22.0 |
| Pixart | 26.4 | 46.2 | 55.3 | 50.8 | 35.8 | 19.8 | 24.3 | 26.6 |
| SD XL | 45.8 | 59.9 | 65.8 | 62.8 | 51.7 | 28.0 | 35.4 | 38.4 |
| Playground | 45.6 | 66.1 | 62.5 | 64.3 | 53.8 | 35.4 | 44.8 | 44.7 |
Table 4: Comparison on T2I-FactBench (SKCM, SKCI, and MKCC). xx denotes the best score and xx the second-best within each group.
| Variant | Clarity | Content | Aesthetics | Relevance | Overall |
|---|---|---|---|---|---|
| Baseline (Vanilla Bagel) | 91.3 | 64.2 | 84.0 | 44.9 | 50.9 |
| Pipeline Ablations | |||||
| w/o Text-Search | 90.7 (-0.6) | 70.9 (+6.7) | 84.3 (+0.3) | 64.6 (+19.7) | 65.4 (+14.5) |
| w/o Image-Search | 92.1 (+0.8) | 73.1 (+8.9) | 85.0 (+1.0) | 50.8 (+5.9) | 56.2 (+5.3) |
| w/o Recaption | 83.0 (-8.3) | 69.0 (+4.8) | 74.5 (-9.5) | 60.2 (+15.3) | 62.9 (+12.0) |
| Constraint Ablations | |||||
Table 5: Ablation study of Unify-Agent on the FactIP benchmark. We report top-level weighted averages over Clarity , Content , Aesthetics , and Relevance . The middle rows show expected performance drops when removing key pipeline components, grounded constraint types, or recaption-stage architectural modules. Numbers in parentheses denote changes relative to the vanilla Bagel baseline; red indicates improvement and green indicates degradation. The best result in each column is marked in bold , and the second-best is underlined .
Figure 5: Qualitative comparison of multi-image generation results on knowledge-intensive prompts involving historical figures, fictional characters, products, and stylized toys. Our method consistently produces images that better preserve subject identity, fine-grained attributes, and prompt-specific details, while achieving stronger real-world knowledge grounding than competing baselines, including Flux-1, Bagel-7b, Hunyuan, and Stable Diffusion.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Hyperparameter | Value |
|---|---|
| Model & Architecture Setup | |
| Base LLM | Bagel-14B |
| Vision Encoder (ViT) | SigLIP-SO400M-14 (NaViT) |
| Visual Generation VAE | FLUX VAE (Frozen) |
| ViT Patch Size | 14 |
| VAE Latent Patch Size | 2 |
Table 6: Hyperparameters for the Supervised Fine-Tuning (SFT) stage.
Figure 6: Hybrid attention masking strategy for agentic image generation. The matrix shows causal attention for reasoning traces and textual dialog (blue) and full attention for reference visual-textual evidence (pink/orange/green). Notably, the generation phase (grey) employs a restricted mask that filters out historical reasoning noise, forcing the latent flow-matching tokens to attend only to relevant reference images and recaptions for high-fidelity synthesis.
Figure 7: (a) Hierarchical category distribution of FactIP Bench, consisting of three major groups (Character, Scene, and Object) and 12 fine-grained subcategories. (b) Category-wise comparison of different methods on FactIP Bench, where the radar chart presents the overall scores across all subcategories.
| Category | Subcategory | Description | Num |
|---|---|---|---|
| CHARACTER | Animation | Animated characters, creatures, equipment, and iconic locations from anime and animated media. | 438 |
| Comic | Characters and visual elements originating from comic books and manga series. | 363 | |
| Celebrity | Prominent figures across diverse domains, including scientists, political leaders, business executives, athletes, and entertainment personalities. | 300 | |
| Game | Video game characters, weapons, equipment, and other in-game visual elements. | 272 | |
| Mascot | Official mascots representing Olympic Games, regional events, and corporate brands. | 77 | |
| Mythology | Universally recognized mythological narratives and legendary figures, e.g., Kuafu Chasing the Sun. | 50 |
Table 7: Detailed category statistics of our benchmark. The full benchmark contains three major categories and 12 fine-grained subcategories, totaling 2,462 prompts.
| Model | Character | Object | Scene | Overall | |||||||||
| Celebrity | Animation | Game | Comic | Mythology | Mascot | Animal | Food | Art | Toy | Landmark | Festival | ||
| Commercial Models | |||||||||||||
| GPT-Image-1 | 77.0 | 63.1 | 66.4 | 60.3 | 95.7 | 64.7 | 89.0 | 88.6 | 40.9 | 52.2 | 76.1 | 94.2 | 69.3 |
| GPT-Image-1.5 | 77.7 | 64.0 | 64.1 | 61.9 | 93.0 | 69.4 | 88.1 | 92.1 | 42.8 | 48.9 | 76.0 | 95.4 | 69.9 |
| Seedream-4.5 | 86.0 | 75.6 | 82.3 | 80.8 | 88.9 | 88.1 | 77.0 | 92.4 | 64.8 | 83.0 | 83.5 | 88.9 | 82.0 |
| Seedream-4 | 86.8 | 76.1 | 85.3 | 81.0 | 88.9 | 85.2 | 80.1 | 91.4 | 70.2 | 83.8 | 85.8 | 87.5 | 83.0 |
Table 8: Per-subtask overall scores on the FactIP benchmark. Character includes celebrity, animation, game, comic, mythology, and mascot prompts; Object includes animals/plants, food, cultural relic/art, and toy prompts; Scene includes landmark and celebration prompts. xx denotes the best score and xx the second-best within each group.
Figure 8: Image generated for the prompt: "The copper is burning, highlighting the color"
Figure 9: Image generated for the "DUDOO" art toy.
Figure 10: Image generated for William Butler Yeats writing a manuscript.
Figure 11: Image generated for Grigory Perelman scribbling mathematical equations.
Figure 12: Image generated for Bruce Beutler reviewing data on a monitor.
Figure 13: MLLM evaluation for Popovich drawing a play.
Figure 14: MLLM evaluation for Max Weber delivering a passionate speech.
Figure 15: MLLM evaluation for the mascot character Habatan in the Baroque style.
Figure 16: MLLM evaluation for Scottie Pippen during an NBA playoff game.