cs.CVNov 3, 2026
SaveYANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction
Organizations: The Hong Kong University of Science and Technology · Independent Researcher · The University of Melbourne · The Chinese University of Hong Kong, Shenzhen ·
Abstract
Unified Multimodal Models (UMMs) integrate visual understanding, generation, and editing within a single unified framework. However, existing unified multimodal datasets and benchmarks remain dominated by single-turn tasks. Even in recent multi-turn datasets, many turns can be solved independently without tracking information across turns. Consequently, they provide limited supervision and evaluation for multimodal stateful interaction, which requires models to understand, maintain, update, and faithfully realize evolving dialogue states across turns. To address this gap, we introduce YANAMI, the first large-scale multimodal dialogue dataset and benchmark for stateful interaction, comprising YANAMI-400K and YANAMI-Bench. YANAMI-400K contains 400K multi-turn multimodal dialogues, with more than 2M images and 2.6M dialogue turns. The dialogues span four representative scenarios: Multi-Image Fusion, Image Editing, World Modeling, and Visual Reasoning, and require models to continuously track, preserve, and revise evolving multimodal dialogue states across turns. YANAMI-Bench comprises 400 multi-turn dialogues for systematically evaluating multimodal stateful interaction. It jointly evaluates Dialogue State Understanding and Visual Realization, characterizing the gap between models' ability to understand evolving dialogue states and their ability to faithfully realize them in visual outputs. Experiments show that fine-tuning BAGEL on YANAMI-400K improves image editing, generation, and multimodal stateful interaction by 10.3%, 2.0%, and 13.23%, respectively, while preserving its general visual understanding capabilities. Evaluations on YANAMI-Bench further reveal that existing UMMs still struggle to maintain, update, and visually realize evolving multimodal dialogue states.
Figures & tables
Figure 1: YANAMI: the first large-scale multimodal dialogue dataset and benchmark for stateful interaction.
Figure 2: Representative examples from YANAMI-400K. The four columns present evolving multimodal dialogues in four interaction scenarios: (a) multi-image fusion, (b) image editing, (c) world modeling, and (d) visual reasoning. Across turns, subsequent responses retrieve, maintain, and update the accumulated dialogue state. Colored text highlights six key capabilities.
Figure 3: Overview of the YANAMI data pipeline. The pipeline comprises four stages: (a) Data Preparation, (b) Data Construction, (c) Automatic Verification, and (d) Human Verification.
| Index | Model | Size | Modality | Format | Fusion | Editing | Modeling | Reasoning | Overall |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-Image-2 ( OpenAI, 2026b ) | – | 86.78 | 90.54 | 81.40 | 73.46 | 83.05 | ||
| 2 | Nano Banana 2 ( Google, 2026a ) | – | 84.73 | 87.32 | 74.74 | 76.93 | 80.93 | ||
| 3 | Step1X-v1p2 ( Liu et al., 2025 ) | 12B | 56.08 | 63.36 | 51.66 | 48.00 | 54.77 | ||
| 4 | Qwen-Edit ( Wu et al., 2025a ) | 20B | 49.66 | 54.70 | 28.14 | 37.72 | 42.55 | ||
| 5 | FLUX.2 [dev] ( Labs, 2025 ) | 32B | 72.44 | 74.54 | 65.54 | 56.68 | 67.30 | ||
| 6 | UniPic 3.0 ( Wei et al., 2026a ) | 20B | 63.44 | 70.66 | 56.96 | 48.82 | 59.97 |
Table 2: Results on YANAMI-Bench across four interaction scenarios. denotes models supporting image generation/editing only; denotes models supporting both understanding and generation. denotes concatenated image inputs; denotes placing all available images at the beginning; and denotes interleaved image–text inputs. Red , blue , and yellow cells highlight the best results among closed-source models, open-source image-editing models, and UMMs. Underlined results denote the second-best performance within each model group.
| Model | Fus. | Edi. | Mod. | Rea. | Avg. |
|---|---|---|---|---|---|
| Show-o2 | 74.00 | 64.25 | 79.50 | 54.00 | 67.94 |
| BAGEL | 85.75 | 76.50 | 85.25 | 78.25 | 81.44 |
| BAGEL + Ours | 96.25 | 98.00 | 95.50 | 93.00 | 95.69 |
Table 3: Selected average DSU results. An upward arrow indicates an improvement over BAGEL. Full results are provided in Appendix D.2 .
Figure 6: Analysis of stateful multi-turn multimodal interaction. (a) Measuring the understanding-realization gap, lower is better; (b) comparing performance at the first and last image-generation turns under GT-History, to assess the effect of increasingly accumulated context and state dependencies; (c) comparing GT-History with Self-History on dialogues to measure error accumulation when self-generated images are used as subsequent visual context.
| Model | Understanding | GenEval | ||||||||
| POPE | MME-P | MMBench | Single Obj. | Two Obj. | Count. | Color | Position | Attri. | Overall | |
| Emu3 ( Wang et al., 2024 ) | 85.2 | 1243.8 | 58.5 | 0.99 | 0.81 | 0.42 | 0.80 | 0.49 | 0.45 | 0.66 |
| Show-o ( Xie et al., 2025 ) | 80.0 | 1097.2 | – | 0.98 | 0.80 | 0.66 | 0.84 | 0.31 | 0.50 | 0.68 |
| Janus-Pro-7B ( Chen et al., 2025d ) | 87.4 | 1567.1 | 79.2 | 0.99 | 0.89 | 0.59 | 0.90 | 0.79 | 0.66 | 0.80 |
| MetaQuery-XL ( Pan et al., 2025 ) | – | 1685.2 | 83.5 | – | – | – | – | – | – | 0.80 |
| Ovis-U1 ( Wang et al., 2025a ) | – | 1508.0 | 77.8 | 0.98 | 0.98 | 0.90 | 0.92 | 0.79 | 0.75 | 0.89 |
Table 4: Comparison of different models on visual understanding tasks and GenEval. An upward arrow indicates an improvement over BAGEL.
| Model | BG | Color | Mat. | Motion | Port. | Style | Add | Remove | Replace | Text | Tone | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AnyEdit ( Yu et al., 2025 ) | 4.31 | 4.25 | 2.64 | 0.67 | 1.90 | 1.95 | 3.72 | 3.75 | 3.23 | 0.77 | 4.21 | 2.85 |
| InstructPix2Pix ( Brooks et al., 2023 ) | 3.94 | 5.40 | 3.52 | 1.27 | 2.62 | 4.39 | 3.07 | 1.50 | 3.48 | 1.13 | 5.10 | 3.22 |
| OmniGen ( Xiao et al., 2025 ) | 5.23 | 5.93 | 5.44 | 3.12 | 3.17 | 4.88 | 6.33 | 6.35 | 5.34 | 4.31 | 4.96 | 5.01 |
| Step1X-Edit ( Liu et al., 2025 ) | 7.03 | 6.26 | 6.46 | 3.66 | 5.23 | 7.24 | 7.17 | 6.42 | 7.39 | 7.40 | 6.62 | 6.44 |
| Step1X-Edit-v1p2 ( Liu et al., 2025 ) | – | – | – | – | – | – | – | – | – | – | – | 7.24 |
| OmniGen2 ( Wu et al., 2025c ) | 6.99 | 6.66 | 4.88 | 2.55 | 3.66 | 6.08 | 7.09 | 6.60 | 6.65 | 4.49 | 6.03 | 5.57 |
Table 5: Comparison of fine-tuned BAGEL and other models on GEdit-EN-full benchmark. An upward arrow indicates an improvement over BAGEL.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Scale | Domain | Open Data | Multi-turn Format | Hybrid Evaluation | Explicit Dialogue-State Management | Irreducible to Single-Turn |
|---|---|---|---|---|---|---|---|
| InstructPix2Pix ( Brooks et al., 2023 ) | 450K editing pairs | E | Yes | No | No | No | No |
| AnyEdit ( Yu et al., 2025 ) | 2.5M editing pairs | E | Yes | No | No | No | No |
| CoMM ( Chen et al., 2025c ) | 227K documents | U/G | Yes | No | No | No | No |
| Zebra-CoT ( Li et al., 2025 ) | 182,384 samples | U/G | Yes | No | No | No | No |
| Pico-Banana-400K ( Qian et al., 2026 ) | 400K editing pairs; 72K multi-turn | E | Yes | Partial | No | No | No |
| WEAVE-100K ( Chow et al., 2026b ) | 100,750 dialogues | U/G/E | Yes | Yes | Yes | No | No |
Table 6: Comparison with representative multimodal datasets. U, G, and E denote visual understanding, image generation, and image editing, respectively. Multi-turn Format indicates that data are organized as multi-turn user–assistant interactions; Hybrid Evaluation indicates a unified protocol combining complementary evaluation signals rather than merely reporting multiple independent metrics. Explicit Dialogue-State Management is marked Yes only when a dataset explicitly supervises the establishment, maintenance, invalidation, or update of the currently valid multimodal state; generic history conditioning, context tracking, or a claim of “stateful dialogue” alone does not qualify. Irreducible to Single-Turn indicates that target turns cannot be reformulated as conventional single-turn tasks simply by directly supplying the relevant historical images. Partial denotes implicit or subset-level coverage. Scale follows the unit reported by each work and is therefore not directly comparable across all rows. ‡ ChatUMM reports approximately 40B consumed training tokens but does not report the number of dialogues, images, or samples. § No official ChatUMM training-data release was publicly available as of 2026-09-24; its categorization is therefore based on the construction procedure and examples reported in the paper. Although ChatUMM describes its interactions as stateful, its published pipeline primarily converts standard single-turn tasks into dialogues through historical dependencies and distractor turns, without explicit dialogue-state supervision. Once the relevant historical input is supplied directly, the reported target turns can generally be reformulated as conventional single-turn generation, editing, or composition tasks.
Figure 7: Human-verification interface for YANAMI-Bench. Annotators can browse the 100 benchmark dialogues in each interaction scenario, inspect the complete multi-turn context and visual outputs, and review or revise the corresponding atomic evaluation criteria. The interface also records whether a sample is approved or requires image refinement, supporting the exhaustive verification and correction process used to construct the final benchmark.
| Corpus and Sequence Statistics | Dataset Distributions | |||
| Statistic | Value | Images per Dialogue | Dialogues | Share |
| Total Dialogues | 402,639 | 80,771 | 20.06% | |
| Total Images | 2,047,236 | 106,841 | 26.54% | |
| Total Dialogue Turns | 2,643,291 | 76,358 | 18.96% | |
| Total User Messages | 2,643,291 | 321,868 | 79.94% | |
| Total Assistant Messages | 2,643,291 | 215,027 | 53.40% | |
Table 7: Detailed statistics of YANAMI-400K. Each dialogue turn contains one user message and one assistant response; Total Messages therefore counts both sides of the interaction. Scenario shares are rounded percentages corresponding to the training-set distribution shown in Fig. 4 .
| Scenario | Dialogues | Images | Dialogue Turns | Understanding QAs | Atomic Checks | Avg. Checks |
|---|---|---|---|---|---|---|
| Multi-Image Fusion | 100 | 400 | 777 | 400 | 1,890 | 18.90 |
| Image Editing | 100 | 499 | 754 | 400 | 3,408 | 34.08 |
| World Modeling | 100 | 467 | 598 | 400 | 1,141 | 11.41 |
| Visual Reasoning | 100 | 517 | 1,044 | 400 | 2,028 | 20.28 |
| Total | 400 | 1,883 | 3,173 | 1,600 | 8,467 | 21.17 |
Table 8: Detailed statistics of YANAMI-Bench by interaction scenario. Images include all reference and target images contained in the benchmark dialogues. Understanding QAs comprise Reference Resolution, Context Retrieval, Constraint Tracking, and Correction Updating questions, with one question of each type per dialogue. Atomic Checks counts the independently verifiable visual criteria used in the benchmark evaluation and excludes capability-specific checks; Avg. Checks reports the average number of evaluated atomic checks per dialogue.
| Model | Generation mechanism | Visual-input handling | Principal inference settings |
|---|---|---|---|
| BAGEL ( Deng et al., 2025 ) and +YANAMI-400K | Unified MoT for understanding and generation | Native interleaved image–text sequence; no image concatenation | 50 steps; text/image CFG ; parallel CFG; timestep shift |
| Emu3.5 ( Cui et al., 2025 ) | Unified autoregressive generation with discrete visual tokens | Native interleaved image–text sequence; no image concatenation | Image/text guidance ; image top- 10,240; top- |
| OmniGen2 ( Wu et al., 2025c ) | Unified diffusion pipeline | Separate visual inputs; references are concatenated into a grid only when more than three are present | 50 steps; text guidance ; resolution alignment disabled |
| Ovis-U1 ( Wang et al., 2025a ) | Unified MLLM with a diffusion generation head | Ordered image placeholders; multiple references are concatenated into a grid | 50 steps; text/image CFG |
| InternVL-U ( Tian et al., 2026 ) | Unified VLM with diffusion-based generation | Multiple references are concatenated into a grid | 50 steps; deterministic per-sample seed |
| Janus-Pro ( Chen et al., 2025d ) | Autoregressive generation with discrete visual tokens | Interleaved inputs for understanding; generation uses the flattened dialogue text without reference-image conditioning | 576 visual tokens per image; CFG ; greedy decoding |
Table 9: Baseline inference configurations used on YANAMI-Bench. Only settings that materially affect input organization or output generation are reported; all models otherwise follow the shared evaluation protocol described in the text.
| Model and System Configuration | Optimization and Sequence Configuration | ||
|---|---|---|---|
| Setting | Value | Setting | Value |
| Base model | BAGEL-7B-MoT | Learning rate | |
| Decoder layer | Qwen2MoTDecoderLayer | Scheduler | Constant |
| Compute resources | NVIDIA A800 80GB; NVIDIA A100 80GB | Warm-up steps | 100 |
| Launch configuration | 1 node, 6 processes | CE / MSE loss weights | 0.25 / 1.0 |
| Sharding strategy | FSDP FULL_SHARD | Gradient accumulation steps | 1 |
Table 10: Training configuration. Settings used to fine-tune BAGEL-7B-MoT on YANAMI-400K, including the model and system setup, optimization hyperparameters, and sequence-length settings.
| Model | Fusion | Editing | Modeling | Reasoning | Avg. |
|---|---|---|---|---|---|
| OmniGen2 ( Wu et al., 2025c ) | 46.75 | 50.00 | 65.00 | 42.75 | 51.13 |
| Janus-Pro ( Chen et al., 2025d ) | 68.00 | 41.75 | 69.25 | 51.50 | 57.63 |
| Show-o2 ( Xie et al., 2026a ) | 74.00 | 64.25 | 79.50 | 54.00 | 67.94 |
| Ovis-U1 ( Wang et al., 2025a ) | 83.75 | 25.25 | 65.75 | 55.25 | 57.50 |
| Emu3.5 ( Cui et al., 2025 ) | 80.00 | 47.50 | 85.50 | 71.00 | 71.00 |
| InternVL-U ( Tian et al., 2026 ) | 82.75 | 73.00 | 88.75 | 63.00 | 76.88 |
Table 11: Dialogue-state understanding results across the four interaction scenarios. Fusion, Editing, Modeling, and Reasoning denote Multi-Image Fusion, Image Editing, World Modeling, and Visual Reasoning, respectively. Each scenario score is the arithmetic mean of RR, CR, CT, and CU, and Avg. is the macro-average across the four scenarios. All results are reported on a 0–100 scale, and higher values are better.
| Index | Model | Size | Modality | Format | RR | CR | CT | CU | DSU Avg. |
|---|---|---|---|---|---|---|---|---|---|
| 1 | OmniGen2 ( Wu et al., 2025c ) | 7B | 15.0 | 66.0 | 28.0 | 62.0 | 42.75 | ||
| 2 | Janus-Pro ( Chen et al., 2025d ) | 7B | 30.0 | 71.0 | 41.0 | 64.0 | 51.50 | ||
| 3 | Show-o2 ( Xie et al., 2026a ) | 7B | 18.0 | 81.0 | 43.0 | 74.0 | 54.00 | ||
| 4 | Ovis-U1 ( Wang et al., 2025a ) | 3B | 34.0 | 70.0 | 42.0 | 75.0 | 55.25 | ||
| 5 | Emu3.5 ( Cui et al., 2025 ) | 34B | 34.0 | 87.0 | 71.0 | 92.0 | 71.00 | ||
| 6 | InternVL-U ( Tian et al., 2026 ) | 4B | 25.0 | 90.0 | 53.0 | 84.0 | 63.00 |
Table 12: Dialogue-state understanding results for Visual Reasoning. RR, CR, CT, and CU denote Reference Resolution, Context Retrieval, Constraint Tracking, and Correction Updating, respectively. DSU Avg. is the arithmetic mean of the four capability scores. All results are reported on a 0–100 scale, and higher values are better. An upward arrow indicates an improvement over BAGEL.
| Index | Model | Size | Modality | Format | DSU Avg. | SC | VQ | Overall Avg. |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-Image-2 ( OpenAI, 2026b ) | – | – | 75.2 | 70.8 | 73.46 | ||
| 2 | Nano Banana 2 ( Google, 2026a ) | – | – | 80.7 | 71.3 | 76.93 | ||
| 3 | Step1X-v1p2 ( Liu et al., 2025 ) | 12B | – | 54.0 | 39.0 | 48.00 | ||
| 4 | Qwen-Edit ( Wu et al., 2025a ) | 20B | – | 39.4 | 35.2 | 37.72 | ||
| 5 | FLUX.2 [dev] ( Labs, 2025 ) | 32B | – | 49.0 | 68.2 | 56.68 | ||
| 6 | UniPic 3.0 ( Wei et al., 2026a ) | 20B | – | 56.1 | 37.9 | 48.82 |
Table 13: Understanding and visual-realization results for Visual Reasoning. SC and VQ denote State Consistency and Visual Quality, respectively. All component and aggregate results are reported on a 0–100 scale; VQ scores are converted from their original 0–10 scale by multiplying by 10. Overall Avg. is computed using the weighting defined in Appendix B.3 . Image-generation-only models do not provide DSU scores; a dash denotes an unsupported or unavailable component. Modality and input-format icons follow the conventions in Table 2 . An upward arrow indicates an improvement over BAGEL.
| Index | Model | Size | Modality | Format | RR | CR | CT | CU | DSU Avg. |
|---|---|---|---|---|---|---|---|---|---|
| 1 | OmniGen2 ( Wu et al., 2025c ) | 7B | 73.0 | 44.0 | 44.0 | 39.0 | 50.00 | ||
| 2 | Janus-Pro ( Chen et al., 2025d ) | 7B | 18.0 | 55.0 | 62.0 | 32.0 | 41.75 | ||
| 3 | Show-o2 ( Xie et al., 2026a ) | 7B | 70.0 | 60.0 | 75.0 | 52.0 | 64.25 | ||
| 4 | Ovis-U1 ( Wang et al., 2025a ) | 3B | 32.0 | 25.0 | 35.0 | 9.0 | 25.25 | ||
| 5 | Emu3.5 ( Cui et al., 2025 ) | 34B | 2.0 | 66.0 | 81.0 | 41.0 | 47.50 | ||
| 6 | InternVL-U ( Tian et al., 2026 ) | 4B | 67.0 | 83.0 | 87.0 | 55.0 | 73.00 |
Table 14: Dialogue-state understanding results for Image Editing. RR, CR, CT, and CU denote Reference Resolution, Context Retrieval, Constraint Tracking, and Correction Updating, respectively. DSU Avg. is the arithmetic mean of the four capability scores. All results are reported on a 0–100 scale, and higher values are better. An upward arrow indicates an improvement over BAGEL.
| Index | Model | Size | Modality | Format | DSU Avg. | SC | VQ | Overall Avg. |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-Image-2 ( OpenAI, 2026b ) | – | – | 93.6 | 85.9 | 90.54 | ||
| 2 | Nano Banana 2 ( Google, 2026a ) | – | – | 91.1 | 81.6 | 87.32 | ||
| 3 | Step1X-v1p2 ( Liu et al., 2025 ) | 12B | – | 64.0 | 62.4 | 63.36 | ||
| 4 | Qwen-Edit ( Wu et al., 2025a ) | 20B | – | 53.7 | 56.2 | 54.70 | ||
| 5 | FLUX.2 [dev] ( Labs, 2025 ) | 32B | – | 73.3 | 76.4 | 74.54 | ||
| 6 | UniPic 3.0 ( Wei et al., 2026a ) | 20B | – | 68.3 | 74.2 | 70.66 |
Table 15: Understanding and visual-realization results for Image Editing. SC and VQ denote State Consistency and Visual Quality, respectively. All component and aggregate results are reported on a 0–100 scale; VQ scores are converted from their original 0–10 scale by multiplying by 10. Overall Avg. is computed using the weighting defined in Appendix B.3 . Image-generation-only models do not provide DSU scores; a dash denotes an unsupported or unavailable component. Modality and input-format icons follow the conventions in Table 2 . An upward arrow indicates an improvement over BAGEL.
| Index | Model | Size | Modality | Format | RR | CR | CT | CU | DSU Avg. |
|---|---|---|---|---|---|---|---|---|---|
| 1 | OmniGen2 ( Wu et al., 2025c ) | 7B | 78.0 | 65.0 | 48.0 | 69.0 | 65.00 | ||
| 2 | Janus-Pro ( Chen et al., 2025d ) | 7B | 68.0 | 47.0 | 93.0 | 69.0 | 69.25 | ||
| 3 | Show-o2 ( Xie et al., 2026a ) | 7B | 70.0 | 75.0 | 100.0 | 73.0 | 79.50 | ||
| 4 | Ovis-U1 ( Wang et al., 2025a ) | 3B | 58.0 | 56.0 | 81.0 | 68.0 | 65.75 | ||
| 5 | Emu3.5 ( Cui et al., 2025 ) | 34B | 73.0 | 76.0 | 99.0 | 94.0 | 85.50 | ||
| 6 | InternVL-U ( Tian et al., 2026 ) | 4B | 81.0 | 87.0 | 97.0 | 90.0 | 88.75 |
Table 16: Dialogue-state understanding results for World Modeling. RR, CR, CT, and CU denote Reference Resolution, Context Retrieval, Constraint Tracking, and Correction Updating, respectively. DSU Avg. is the arithmetic mean of the four capability scores. All results are reported on a 0–100 scale, and higher values are better. An upward arrow indicates an improvement over BAGEL.
| Index | Model | Size | Modality | Format | DSU Avg. | SC | VQ | Overall Avg. |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-Image-2 ( OpenAI, 2026b ) | – | – | 79.2 | 84.7 | 81.40 | ||
| 2 | Nano Banana 2 ( Google, 2026a ) | – | – | 71.6 | 79.4 | 74.74 | ||
| 3 | Step1X-v1p2 ( Liu et al., 2025 ) | 12B | – | 56.7 | 44.1 | 51.66 | ||
| 4 | Qwen-Edit ( Wu et al., 2025a ) | 20B | – | 27.1 | 29.7 | 28.14 | ||
| 5 | FLUX.2 [dev] ( Labs, 2025 ) | 32B | – | 60.9 | 72.5 | 65.54 | ||
| 6 | UniPic 3.0 ( Wei et al., 2026a ) | 20B | – | 62.2 | 49.1 | 56.96 |
Table 17: Understanding and visual-realization results for World Modeling. SC and VQ denote State Consistency and Visual Quality, respectively. All component and aggregate results are reported on a 0–100 scale; VQ scores are converted from their original 0–10 scale by multiplying by 10. Overall Avg. is computed using the weighting defined in Appendix B.3 . Image-generation-only models do not provide DSU scores; a dash denotes an unsupported or unavailable component. Modality and input-format icons follow the conventions in Table 2 . An upward arrow indicates an improvement over BAGEL.
| Index | Model | Size | Modality | Format | RR | CR | CT | CU | DSU Avg. |
|---|---|---|---|---|---|---|---|---|---|
| 1 | OmniGen2 ( Wu et al., 2025c ) | 7B | 31.0 | 63.0 | 60.0 | 33.0 | 46.75 | ||
| 2 | Janus-Pro ( Chen et al., 2025d ) | 7B | 45.0 | 79.0 | 72.0 | 76.0 | 68.00 | ||
| 3 | Show-o2 ( Xie et al., 2026a ) | 7B | 48.0 | 71.0 | 90.0 | 87.0 | 74.00 | ||
| 4 | Ovis-U1 ( Wang et al., 2025a ) | 3B | 84.0 | 78.0 | 88.0 | 85.0 | 83.75 | ||
| 5 | Emu3.5 ( Cui et al., 2025 ) | 34B | 41.0 | 91.0 | 95.0 | 93.0 | 80.00 | ||
| 6 | InternVL-U ( Tian et al., 2026 ) | 4B | 66.0 | 84.0 | 89.0 | 92.0 | 82.75 |
Table 18: Dialogue-state understanding results for Multi-Image Fusion. RR, CR, CT, and CU denote Reference Resolution, Context Retrieval, Constraint Tracking, and Correction Updating, respectively. DSU Avg. is the arithmetic mean of the four capability scores. All results are reported on a 0–100 scale, and higher values are better. An upward arrow indicates an improvement over BAGEL.
| Index | Model | Size | Modality | Format | DSU Avg. | SC | VQ | Overall Avg. |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-Image-2 ( OpenAI, 2026b ) | – | – | 90.3 | 81.4 | 86.78 | ||
| 2 | Nano Banana 2 ( Google, 2026a ) | – | – | 89.7 | 77.3 | 84.73 | ||
| 3 | Step1X-v1p2 ( Liu et al., 2025 ) | 12B | – | 61.2 | 48.4 | 56.08 | ||
| 4 | Qwen-Edit ( Wu et al., 2025a ) | 20B | – | 48.5 | 51.4 | 49.66 | ||
| 5 | FLUX.2 [dev] ( Labs, 2025 ) | 32B | – | 74.8 | 68.9 | 72.44 | ||
| 6 | UniPic 3.0 ( Wei et al., 2026a ) | 20B | – | 63.6 | 63.2 | 63.44 |
Table 19: Understanding and visual-realization results for Multi-Image Fusion. SC and VQ denote State Consistency and Visual Quality, respectively. All component and aggregate results are reported on a 0–100 scale; VQ scores are converted from their original 0–10 scale by multiplying by 10. Overall Avg. is computed using the weighting defined in Appendix B.3 . Image-generation-only models do not provide DSU scores; a dash denotes an unsupported or unavailable component. Modality and input-format icons follow the conventions in Table 2 . An upward arrow indicates an improvement over BAGEL.
| Model | First | Last | |
|---|---|---|---|
| BAGEL original | 0.543 | 0.518 | |
| BAGEL + YANAMI-400K | 0.703 | 0.634 | |
| Emu3.5 | 0.657 | 0.650 | |
| OmniGen2 | 0.568 | 0.539 | |
| Ovis-U1 | 0.609 | 0.481 | |
| InternVL-U | 0.573 | 0.508 |
Table 20: Complete first- versus last-generation-turn results under GT-History. Only dialogues containing at least two generation turns are included. Higher scores are better, and denotes Last minus First.
| Model | CK | CU | Overall |
|---|---|---|---|
| BAGEL original | 0.5667 | 0.5922 | 0.5795 |
| BAGEL + YANAMI-400K | 0.4158 | 0.3270 | 0.3714 |
| Emu3.5 | 0.3765 | 0.3889 | 0.3827 |
| OmniGen2 | 0.5188 | 0.5126 | 0.5157 |
| Ovis-U1 | 0.4835 | 0.4609 | 0.4722 |
| InternVL-U | 0.5478 | 0.5707 | 0.5592 |
Table 21: Understanding–realization gaps computed on shared subsets that all evaluated models understand correctly. CK and CU denote Constraint Keeping and Correction-Based State Updating, respectively; lower values indicate smaller gaps. Overall is the macro-average of CK and CU.
Figure 8: Complete qualitative example for Image Editing. Across seven turns, the dialogue progressively replaces and modifies visual elements while preserving the active appearance constraints, incorporating a user correction, and retrieving earlier requirements for the final composition. The left and right panels show the first and second halves of the same dialogue, respectively.
Figure 9: Complete qualitative example for Multi-Image Fusion. The dialogue combines information from multiple visual references while tracking the intended setting, object placement, and persistent restrictions, culminating in a jointly composed visual output. The left and right panels show the first and second halves of the same dialogue, respectively.
Figure 10: Complete qualitative example for World Modeling. The interaction requires the model to maintain the scene layout, update an initially incorrect depth relation, and render a new viewpoint while preserving the corrected spatial state and other persistent scene properties. The left and right panels show the first and second halves of the same dialogue, respectively.
Figure 11: Complete qualitative example for Visual Reasoning. The model follows a multi-step action plan, revises the plan after a user correction, generates intermediate visual states, and subsequently retrieves those states to describe the completed sequence. The left and right panels show the first and second halves of the same dialogue, respectively.