cs.CVMar 30, 2026
SaveGEMS: Agent-Native Multimodal Generation with Memory and Skills
Organizations: Shanghai AI Laboratory · Nanjing University · Shanghai Jiao Tong University · CUHK
Abstract
Recent multimodal generation models have achieved remarkable progress on general-purpose generation tasks, yet continue to struggle with complex instructions and specialized downstream tasks. Inspired by the success of advanced agent frameworks such as Claude Code, we propose GEMS (Agent-Native Multimodal GEneration with Memory and Skills), a framework that pushes beyond the inherent limitations of foundational models on both general and downstream tasks. GEMS is built upon three core components. Agent Loop introduces a structured multi-agent framework that iteratively improves generation quality through closed-loop optimization. Agent Memory provides a persistent, trajectory-level memory that hierarchically stores both factual states and compressed experiential summaries, enabling a global view of the optimization process while reducing redundancy. Agent Skill offers an extensible collection of domain-specific expertise with on-demand loading, allowing the system to effectively handle diverse downstream applications. Across five mainstream tasks and four downstream tasks, evaluated on multiple generative backends, GEMS consistently achieves significant performance gains. Most notably, it enables the lightweight 6B model Z-Image-Turbo to surpass the state-of-the-art Nano Banana 2 on GenEval2, demonstrating the effectiveness of agent harness in extending model capabilities beyond their original limits.
Figures & tables
Figure 2: The system architecture of GEMS. The framework consists of three primary pillars: Agent Loop, Agent Memory, and Agent Skill. The user prompt is augmented with domain-specific expertise from Agent Skill, and then iteratively refined within the Agent Loop, with Agent Memory managing the historical context to guide the generation process.
Figure 3: Architecture of the Agent Skill system, highlighting its scalable and on-demand nature.
| Model | Method | GenEval | GenEval2 | DPG-Bench | OneIG-EN | OneIG-ZH | WISE | Avg. |
|---|---|---|---|---|---|---|---|---|
| Closed-Source Models | ||||||||
| Nano Banana | – | 0.75 | 44.6 | 85.23 | 0.550 | 0.405 | – | – |
| GPT-Image 1 | – | 0.84 | – | 85.15 | 0.533 | 0.474 | 0.80 | – |
| Seedream 4 [ 53 ] | – | 0.84 | – | 88.25 | 0.576 | 0.553 | 0.78 | – |
| Open-Source Models | ||||||||
| Bagel [ 10 ] | – | 0.82 | 23.1 | 85.10 | 0.361 | 0.370 | 0.70 | 59.35 |
Table 1: Evaluation on Mainstream Tasks. For inference-time scaling methods, all results are evaluated by us, with the best and second-best performances highlighted in bold and underlined , and relative improvements or decreases compared to the baseline indicated by and , respectively. Other results are sourced from public data. The “Avg." column represents the mean of normalized scores, with OneIG-EN and OneIG-ZH pre-averaged as a single metric before final aggregation.
| Model | Method | LongText-EN | LongText-ZH | SpatialGenEval | CREA | ArtiMuse | Avg. |
|---|---|---|---|---|---|---|---|
| Closed-Source Models | |||||||
| Nano Banana | – | – | – | 61.7 | – | – | – |
| GPT-Image 1 | – | 0.956 | 0.619 | 60.5 | – | – | – |
| Seedream 4 [ 53 ] | – | – | – | 62.7 | – | – | – |
| Open-Source Models | |||||||
| Bagel [ 10 ] | – | 0.373 | 0.310 | 57.0 | – | – | – |
Table 2: Evaluation on Downstream Tasks. For inference-time scaling methods, all results are evaluated by us, with the best and second-best performances highlighted in bold and underlined , and relative improvements or decreases compared to the baseline indicated by and , respectively. Other results are sourced from public data. The “Avg.” column represents the mean of normalized scores, with LongText-EN and LongText-ZH pre-averaged as a single metric before final aggregation.
Figure 4: Ablation study on GenEval2 with Z-Image-Turbo. (Left) Performance gains contributed by individual components, including Agent Loop, Agent Memory, and Agent Skill. (Right) Detailed analysis of the performance improvements of Agent Memory.
Figure 5: Average passed criteria over iterations.
Figure 8: Iteration distribution across GEMS ablation variants. By incorporating Memory and Skill, the system shifts the distribution toward earlier rounds and reducing the average number of iterations.
Figure 9: Skill distribution across tasks and relative performance improvement. Each stacked bar illustrates the percentage composition of skills triggered within the respective benchmarks. The red numerical labels (e.g., +3.6%) above the bars denote the relative performance gain achieved compared to the version without Agent Skill.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Prompt for Planner (Skill Selection).
Figure 11: Prompt for Planner (Skill Application).
Figure 12: Prompt for Decomposer.
Figure 13: Prompt for Verifier.
Figure 14: Prompt for Refiner.
Figure 15: Prompt for Compressor.
| Base Model | Method | Single Obj | Two Obj | Counting | Colors | Position | Attr Binding | Overall |
|---|---|---|---|---|---|---|---|---|
| Closed-Source Models | ||||||||
| Nano Banana | – | – | – | – | – | – | – | 0.75 |
| GPT-Image 1 | – | 0.99 | 0.92 | 0.85 | 0.92 | 0.75 | 0.61 | 0.84 |
| Seedream 4 [ 53 ] | – | – | – | – | – | – | – | 0.84 |
| Open-Source Models | ||||||||
| Bagel [ 10 ] | – | – | – | – | – | – | – | 0.82 |
Table 4: Detailed Performance Breakdown on GenEval. We report the fine-grained scores across six dimensions: Single Object, Two Object, Counting, Colors, Position, and Attribute Binding, with the overall performance shown in the last column.
| Base Model | Method | Object | Attribute | Count | Position | Verb |
|---|---|---|---|---|---|---|
| Closed-Source Models | ||||||
| Nano Banana | – | 99.0 | 91.4 | 70.1 | 70.2 | 86.7 |
| Nano Banana 2 | – | 97.0 | 95.1 | 76.4 | 83.7 | 56.1 |
| Open-Source Models | ||||||
| Bagel [ 10 ] | – | 92.9 | 75.9 | 55.6 | 50.6 | 57.8 |
| Qwen-Image [ 64 ] | – | 99.1 | 85.6 | 70.3 | 60.2 | 71.1 |
Table 5: Detailed Performance Breakdown on GenEval2. We report the fine-grained scores across five dimensions: Object, Attribute, Count, Position, and Verb.
| Base Model | Method | Global | Entity | Attribute | Relation | Other | Overall |
|---|---|---|---|---|---|---|---|
| Closed-Source Models | |||||||
| Nano Banana | – | – | – | – | – | – | 85.23 |
| GPT-Image 1 | – | 88.89 | 88.94 | 89.84 | 92.63 | 90.96 | 85.15 |
| Seedream 4 [ 53 ] | – | – | – | – | – | – | 88.25 |
| Open-Source Models | |||||||
| Bagel [ 10 ] | – | – | – | – | – | – | 85.10 |
Table 6: Detailed Performance Breakdown on DPG-Bench. We report the fine-grained scores across five dimensions: Global, Entity, Attribute, Relation, and Other, with the overall performance shown in the last column.
| Base Model | Method | Alignment | Text | Reasoning | Style | Diversity | Overall |
|---|---|---|---|---|---|---|---|
| Closed-Source Models | |||||||
| Nano Banana | – | 0.878 | 0.894 | 0.346 | 0.450 | 0.182 | 0.550 |
| GPT-Image 1 | – | 0.851 | 0.857 | 0.345 | 0.462 | 0.151 | 0.533 |
| Seedream 4 [ 53 ] | – | 0.894 | 0.981 | 0.352 | 0.458 | 0.197 | 0.576 |
| Open-Source Models | |||||||
| Bagel [ 10 ] | – | 0.769 | 0.244 | 0.173 | 0.367 | 0.251 | 0.361 |
Table 7: Detailed Performance Breakdown on OneIG-EN. We report the fine-grained scores across five dimensions: Alignment, Text, Reasoning, Style, and Diversity, with the overall performance shown in the last column.
| Base Model | Method | Alignment | Text | Reasoning | Style | Diversity | Overall |
|---|---|---|---|---|---|---|---|
| Closed-Source Models | |||||||
| Nano Banana | – | 0.825 | 0.276 | 0.298 | 0.427 | 0.198 | 0.405 |
| GPT-Image 1 | – | 0.812 | 0.650 | 0.300 | 0.449 | 0.159 | 0.474 |
| Seedream 4 [ 53 ] | – | 0.847 | 0.982 | 0.286 | 0.443 | 0.206 | 0.553 |
| Open-Source Models | |||||||
| Bagel [ 10 ] | – | 0.672 | 0.365 | 0.186 | 0.357 | 0.268 | 0.370 |
Table 8: Detailed Performance Breakdown on OneIG-ZH. We report the fine-grained scores across five dimensions: Alignment, Text, Reasoning, Style, and Diversity, with the overall performance shown in the last column.
| Base Model | Method | Cultural | Time | Space | Biology | Physics | Chemistry | Overall |
|---|---|---|---|---|---|---|---|---|
| Closed-Source Models | ||||||||
| GPT-Image 1 | – | 0.81 | 0.71 | 0.89 | 0.83 | 0.79 | 0.74 | 0.80 |
| Seedream 4 [ 53 ] | – | 0.78 | 0.73 | 0.85 | 0.79 | 0.84 | 0.67 | 0.78 |
| Open-Source Models | ||||||||
| Bagel [ 10 ] | – | 0.76 | 0.69 | 0.75 | 0.65 | 0.75 | 0.58 | 0.70 |
| Qwen-Image [ 64 ] | – | 0.62 | 0.63 | 0.77 | 0.57 | 0.75 | 0.40 | 0.62 |
Table 9: Detailed Performance Breakdown on WISE. We report the fine-grained scores across six dimensions: Cultural, Time, Space, Biology, Physics, and Chemistry, with the overall performance shown in the last column.
| Base Model | Method | Object | Attribute | Position | Orientation | Layout | Comparison | Proximity | Occlusion | Motion | Causal | Overall |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Closed-Source Models | ||||||||||||
| Nano Banana | – | 58.5 | 75.3 | 55.5 | 58.9 | 70.9 | 31.8 | 68.7 | 33.5 | 81.4 | 82.2 | 61.7 |
| GPT-Image 1 | – | 56.3 | 74.1 | 53.3 | 58.9 | 70.4 | 31.4 | 66.8 | 30.2 | 80.9 | 82.2 | 60.5 |
| Seedream 4 [ 53 ] | – | 59.9 | 80.2 | 57.2 | 58.9 | 70.1 | 32.1 | 68.3 | 33.8 | 83.0 | 83.8 | 62.7 |
| Open-Source Models | ||||||||||||
| Bagel [ 10 ] | – | 55.3 | 73.7 | 51.2 | 54.0 | 62.9 | 28.6 | 64.1 | 29.0 | 74.4 | 76.7 | 57.0 |
Table 10: Detailed Performance Breakdown on SpatialGenEval. We report the fine-grained scores across ten dimensions: Object, Attribute, Position, Orientation, Layout, Comparison, Proximity, Occlusion, Motion, and Causal, with the overall performance shown in the last column.
| Base Model | Method | Originality | Expressiveness | Aesthetic | Technical | Unexpected | Interpretability | Overall |
|---|---|---|---|---|---|---|---|---|
| Inference-Time Scaling | ||||||||
| Z-Image-Turbo | Original | 1.62 | 1.57 | 2.92 | 3.19 | 1.37 | 1.17 | 11.84 |
| Rewrite | 2.39 | 2.63 | 3.72 | 3.80 | 2.22 | 1.99 | 16.75 | |
| Promptist [ 17 ] | 1.85 | 2.02 | 3.36 | 3.76 | 1.57 | 1.45 | 14.00 | |
| Search [ 41 ] | 1.87 | 1.88 | 3.07 | 3.33 | 1.56 | 1.28 | 12.98 | |
| Maestro [ 59 ] | 2.32 | 2.46 | 3.66 | 3.76 | 1.92 | 1.68 | 15.81 | |
Table 11: Detailed Performance Breakdown on CREA. We report the fine-grained scores across six dimensions: Originality, Expressiveness, Aesthetic, Technical, Unexpected, and Interpretability, with the overall performance shown in the last column.
Figure 16: Baseline vs GEMS for prompt: a view of the Earth from the moon.
Figure 17: Baseline vs GEMS for prompt: A high-speed photography shot of clear water being splashed upwards from a black surface. In mid-air, the splashing water droplets perfectly form the shape of a detailed, symmetrical butterfly. The wings are made entirely of liquid water ripples. The background is completely pitch black. The water butterfly is physically connected to the splash below.
More in this topic
Multimodal Generation
Multimodal generation creates coordinated text, images, audio, video, 3D, and other outputs from mixed inputs while preserving meaning and consistency.
See all 243 papers