Unified multimodal models (UMMs) provide an interface for interleaved text-image generation, where generated visual states can directly inform subsequent reasoning and synthesis. However, existing systems are still primarily demonstrated on short trajectories or simplified visual content. We develop a large UMM natively trained on interleaved multimodal data, with strong capabilities in image synthesis, text rendering, complex composition, and multimodal planning, and study how to extend these native capabilities to substantially longer autoregressive trajectories. The main challenge is that generated images continuously accumulate in context: irrelevant or imperfect visual states interfere with future synthesis, while previously established characters and scenes must still be recovered after long intervals. We introduce a visual-memory framework that maintains persistent visual references, selectively retrieves scene-relevant historical states while preserving the complete semantic history, and uses few-step self-rollout to adapt the model to imperfect generated histories. We further introduce LongMIG-Bench for evaluating long-horizon text-to-multi-image and image-to-multi-image generation. Our model achieves high-fidelity long-horizon interleaved generation with strong character and scene consistency, demonstrating how visual memory can scale native interleaved generation to substantially longer trajectories.
Figures & tables
Figure 1: High-fidelity long-horizon interleaved generation across diverse visual domains. Our model supports photorealistic, stylized, text-rich, and design-oriented generation while maintaining visual consistency over extended trajectories.
Figure 2: Overview of our long-horizon interleaved generation framework. Built on our unified multimodal backbone, the model constructs persistent visual references, retrieves scene-relevant history during autoregressive generation, and uses few-step self-rollout for memory adaptation.
Method
Character Identity ↑
Scene Memory
Visual Fidelity ↑
Visual Memory ↑
Aesthetic Quality ↑
Text Rendering ↑
Scene ↑
Gap-S ↑
Gap-D ↑
Agent-Planned
StoryDiffusion
1.61
1.61
0.0615
0.1196
2.92
1.55
2.31
1.17
Qwen-Image
3.16
3.05
0.0694
0.2127
3.83
3.14
3.73
2.22
UniSpace
2.41
2.33
0.0346
0.1321
2.48
2.03
2.57
1.53
HiDream
3.35
2.58
0.0576
0.1620
3.19
2.76
2.71
2.78
Table 1: Long-horizon generation on LongMIG-Bench. Best and second-best results are shown in bold and underline , respectively.
Figure 3: High-fidelity long-form generation. Left: image-conditioned generation with identity preservation across diverse views. Right: long-form interleaved generation with consistent characters and visual states over an extended trajectory.
Figure 4: Native interleaved generation over extended trajectories. Representative frames from long autoregressive generations across diverse visual domains, including 50+ image trajectories.
Variant
Setting
Identity ↑
Scene ↑
Gap-D ↑
LR Scene ↑
LR Gap-D ↑
Visual Fidelity ↑
Scene-aware retrieval
Predicted Routing
Predicted Scene
3.36
3.26
0.164
3.10
0.119
3.15
GT Scene
Annotated Scene
3.21
3.32
0.173
3.15
0.121
3.03
Noisy Scene
30% Corrupted
3.30
2.81
0.145
2.76
0.099
3.14
Recent-4
Scene-Agnostic
3.15
2.85
0.147
2.79
0.092
2.97
Recent-2
Scene-Agnostic
3.05
2.58
0.114
2.47
0.071
2.89
Table 2: Controlled analysis of visual-memory components. Ablations of scene-aware retrieval, persistent visual references, and self-rollout adaptation on the 24-image Long+Ext stress test.
Figure 5: Effect of self-rollout adaptation. Base and GT-FT exhibit progressive reddish color drift and loss of fine-grained facial details, whereas Self-Rollout better preserves color appearance and local visual structure throughout the trajectory.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Long-form visual process generation for Portuguese egg tarts. All methods are prompted to generate a sequence documenting the preparation process. The comparison highlights differences in step progression, visual diversity, repetition, and consistency across the complete trajectory.
Figure 7: Long-form visual process generation for braised pork. Each method generates a multi-image sequence describing the cooking process. The examples illustrate how models differ in representing successive visual states rather than repeatedly depicting similar intermediate or final results.
Figure 8: Long-form generation across diverse visual domains. Representative trajectories cover (a) cinematic storytelling, (b) cooking photography, (c) minimalist typography, (d) architectural visualization, (e) playful food illustration, (f) graphic identity design, (g) image-conditioned product design, and (h) image-conditioned pet photography. Input reference images for image-conditioned examples are shown separately.
Figure 9: Complete native interleaved planning-and-generation trajectory. Given a request for an eight-page nutrition presentation, the model first forms an overall plan and generates a reusable visual reference, then alternates between textual planning and image generation until completing the requested sequence. Generated images are inserted back into the multimodal context and remain available to subsequent generation steps.
Data type
Fraction
Multi-shot storyboards
45.3%
Illustrated books with visual references
18.5%
Illustrated books without visual references
7.9%
Social-media comics
14.5%
Long comics with background continuity
9.1%
Brand and design sequences
4.7%
Appendix
Table 3: Composition of the long-horizon post-training corpus.
T2MI
I2MI
Stories
50
50
Specified shots
462
460
Avg. shots / story
9.2
Min. revisited shots / story
≥4
Avg. revisited shots
6.16
6.04
Distinct styles
50
50
Appendix
Table 4: Composition of LongMIG-Bench.
Figure 10: Examples from LongMIG-Bench. Each story contains structured character and location definitions together with an ordered sequence of shots. Colors indicate recurring scenes, and arrows highlight long-range revisits after intervening scene transitions. The I2MI track additionally provides a user reference image.
Sec./Image ↓
Last Prefill (s) ↓
Horizon
Full History
Ours
Full History
Ours
8
27.8
26.4
6.8
4.9
16
49.3
37.6
20.7
9.8
24
73.8
46.5
41.4
14.0
32
102.9
54.9
69.0
19.4
Appendix
Table 5: Efficiency across increasing generation horizons. Ours and Full History use identical model weights and generation settings. Sec./Image is the mean wall-clock time per image over the first h images; Last Prefill is the AR prefill time of the h -th image.
Figure 11: Scaling of image-synthesis context and generation latency. Left: selective visual-memory retrieval substantially slows context growth compared with Full History. Right: the corresponding per-image latency gap increases with generation horizon. Solid curves denote measured trajectories, while dotted extensions indicate linear extrapolation.