Humans routinely consume multimodal content, where textual and visual information is naturally interleaved. Although Unified Multimodal Models (UMMs) can jointly generate text and images, multimodal interleaved generation remains largely underexplored. Progress in this area is hindered by two key limitations: (1) the lack of well-defined interleaved generation tasks and corresponding training data, and (2) the absence of fine-grained evaluation protocols that can reliably diagnose model performance across different capability dimensions. To this end, we present IntReal, comprising two complementary components: IntReal-100K, a real-world dataset of multimodal interleaved samples spanning four major categories, and the IntReal Evaluation Suite, which evaluates model performance across seven complementary dimensions using specialized expert models, enabling coarse-to-fine diagnosis. Extensive experiments validate our evaluation suite through low inter-metric redundancy (max pairwise absolute Spearman's ρ≤0.42 ), high agreement with human judgments (83.3% accuracy), and strong consistency across different expert evaluators (Spearman's ρ>0.9 ), supporting the complementarity and reliability of our metrics. By benchmarking four families of UMMs, we show that IntReal Evaluation Suite reveals distinct strengths and weaknesses across models. Post-training experiments demonstrate the utility of IntReal-100K for improving interleaved generation and further show that training unimodal generation can also benefit interleaved generation. These findings suggest that interleaved generation can benefit from modality-specific optimization, rather than requiring joint text-image training at every stage.
Figures & tables
Figure 1: Overview of IntReal. IntReal-100K contains 100,000 multimodal interleaved samples with 538,452 images, curated from 7 real-world sources and organized into 4 categories. The IntReal Evaluation Suite performs coarse-to-fine diagnosis via 7 fine-grained dimensions.
Figure 2: Overview of the IntReal Evaluation Suite. We select 600 high-quality samples from IntReal-100K to construct this suite. Each sample is evaluated at two granularities: holistic evaluation generates the complete interleaved sequence at once, while fine-grained evaluation generates the next image conditioned on the preceding interleaved context to enable controllable diagnosis.
Model
Holistic Evaluation
Fine-grained Evaluation
Mean
Text Richness
Text Logicality
Image Aesthetic
Image Diversity
Quantity Compliance
Context Fidelity
Subject Consistency
Emu3.5
14.02
75.17
81.20
75.51
39.70
65.60
61.46
58.95
U1-A3B-MoT
24.67
69.17
80.70
88.70
72.50
86.81
63.45
69.43
Bagel +SFT
17.50
63.50
76.82
91.35
62.83
74.19
60.61
63.83
Bagel +SFT+RL
16.67
62.50
81.94
90.31
62.50
76.04
65.43
65.06
U1-8B-MoT
21.83
67.33
79.60
88.36
95.33
87.29
63.38
71.88
Table 3: IntReal evaluation result per dimension . We evaluate four UMM families across seven dimensions. All scores are scaled to the range of [0,100] , with higher values indicating better performance. The Δ vs. base rows report relative changes after post-training with respect to the corresponding base model, where green and red indicate improvements and degradations, respectively. Gray rows denote the closed-source API system, and Δ vs. OSS Best reports its relative difference with respect to the best open-source result in each dimension.
Figure 4Figure 5
Figure 7: Case study on SenseNova-U1-8B-MoT. SFT produces more detailed text but degrades visual details (distorted brush), while RL preserves the text improvements and restores image quality.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Source
# Samples
# Images
Images / Sample
# Text Tokens
Text Tokens / Sample
Planning
Wikivoyage
21,055
56,527
2.68
21.5M
1,021.13
Wikitravel
5,055
12,254
2.42
5.2M
1,020.53
Knowledge
Wikipedia
21,138
100,198
4.74
32.2M
1,522.34
Tutorial
Xiachufang
34,771
213,818
6.15
52.3M
1,504.77
Story
Curious George
7,298
63,088
8.64
14.2M
1,941.43
Rabbids Invasion
3,949
34,284
8.68
7.8M
1,965.36
Appendix
Table 4: Statistics of IntReal-100K across different data sources. We report the number of samples, number of images, average number of images per sample, total number of text tokens, and average number of text tokens per sample. Text tokens are computed using the Qwen2.5 tokenizer.
Figure 8: Calibration case for Text Richness. A low-scoring sample from Bagel+SFT , where the response fails to provide sufficiently specific insect characteristics and offers only limited detail when distinguishing the target insect from similar species, leading to a low Text Richness score.
Figure 9: Calibration case for Text Logicality. A low-scoring sample from U1-8B-MoT , where spatial and factual inconsistencies in the description lead to a low Text Logicality score.
Figure 10: Calibration case for Image Aesthetic. A low-scoring sample from U1-8B-MoT-SFT , where image quality degrades over steps, with later images exhibiting excessively strong saturation and contrast and appearing less photorealistic, leading to a low Image Aesthetic score.
Figure 11: Calibration case for Image Diversity. A low-scoring sample from Emu3.5 , where Images 1&2, 3&4, and 5&6 are excessively similar to each other, resulting in limited visual variation and a low Image Diversity score.
Figure 12: Calibration case for Quantity Compliance. A low-scoring sample from Bagel+SFT+RL , where the prompt requests eight images but the response generates only six, leading to a low Quantity Compliance score.
Figure 13: Calibration case for Context Fidelity. A low-scoring sample from U1-8B-MoT+SFT+RL , where the generated images fail to faithfully follow the described cooking process: the bread is insufficiently toasted in the first image, while the second image omits the preparation and spreading of the caramelized onions, leading to a low Context Fidelity score.
Figure 14: Calibration case for Subject Consistency. A low-scoring sample from U1-A3B-MoT , where the main subjects are not consistently preserved across images. For example, the green dinosaur differs across Images 1, 4, and 5, while the teddy bear changes color between Images 3 and 6. Since the reference sequence maintains a stable subject appearance throughout, such internal inconsistency in the generated images necessarily manifests as low similarity to the reference at the corresponding positions, leading to a low Subject Consistency score.
Figure 15
Hyperparameter
Bagel
SenseNova-U1
Initialization
BAGEL-7B-MoT
SenseNova-U1-8B-MoT-Interleaved
Training steps
6,700
7,000
Optimizer
AdamW
AdamW
Learning rate
2×10−5
2×10−5
LR schedule
Cosine
Cosine
Warmup steps
670
700
Appendix
Table 5: SFT configurations for Bagel and SenseNova-U1. We report the main optimization, batching, loss, and distributed training hyperparameters used in our post-training experiments.
Hyperparameter
Bagel
SenseNova-U1
Training resolution
1024
1024
Rollout denoising steps
10
20
Stochastic scheduler
Flow-CPS
Flow-SDE
Noise level
0.7
0.4
Candidate stochastic step indices
{1,2,3}
{1,2,3}
Selected stochastic steps / rollout
1
1
Appendix
Table 6: RL configurations for Bagel and SenseNova-U1. We report the main rollout, stochastic sampling, reward, and evaluation configurations used in our reinforcement learning experiments.
Model
Holistic Evaluation
Fine-grained Evaluation
Mean
Text Richness
Text Logicality
Image Aesthetic
Image Diversity
Quantity Compliance
Context Fidelity
Subject Consistency
Emu3.5
13.34
69.93
81.20
75.51
39.70
71.06
61.46
58.89
U1-A3B-MoT
29.50
69.00
80.70
88.70
72.50
89.11
63.45
70.42
Bagel +SFT
19.67
54.33
76.82
91.35
62.83
78.71
60.61
63.48
Bagel +SFT+RL
20.67
50.33
81.94
90.31
62.50
80.76
65.43
64.56
U1-8B-MoT
25.67
58.83
79.60
88.36
95.33
89.88
63.38
71.58
Appendix
Table 7: IntReal evaluation results with an alternative evaluator. We report results following the protocol in table 3 , using Qwen3.6-35B-A3B in place of Gemma4 as the evaluator.
Figure 17: IntReal evaluation results per category for individual models. We report the score of each metric for each category separately for each model.
Figure 18: Qualitative example from the Planning category generated by Nanobanana2. The response interleaves text and images to introduce the geography, history, and landmarks of Mayfair and Marylebone.
Figure 19: Qualitative example from the Knowledge category generated by Emu3.5. The response interleaves text and images to describe the appearance, reproductive behavior, and development of the copper demoiselle damselfly.
Figure 20: Qualitative example from the Tutorial category generated by U1-8B-MoT+SFT+RL. The response interleaves text and images to explain how to roast cherry tomatoes with garlic and rosemary and serve them with bread.
Figure 21: Qualitative example from the Story category generated by Bagel+SFT+RL. The response interleaves text and images to narrate a group of dinosaurs exploring and playing together in a lush jungle.
Multimodal generation creates coordinated text, images, audio, video, 3D, and other outputs from mixed inputs while preserving meaning and consistency.