Overview

Multimodal generation studies models that create outputs such as images, video, speech, motion, or scientific signals from text, media, or combinations of them. Common approaches include diffusion and flow-matching decoders, autoregressive or discrete-token models, shared latent spaces, and systems that connect vision-language models to modality-specific generators; planning and reinforcement learning can add control for complex or interleaved outputs. A major direction is building models that both understand and generate, with benchmarks measuring prompt adherence, fidelity, temporal and cross-modal coherence, and task-specific validity. An open challenge is enabling shared models to turn understanding into faithful, controllable generation without modality imbalance or loss of specialized quality, especially over long sequences and varied real-world inputs.

Latest papers

CardsList

255 papers21 papers in the last four weeks, down 22% on the four weeks before. 0.1% of all new papers.

  1. Leveraging Complementary Event Frame and Voxel Representations for RGB-Guided Multimodal Pre-training

    Nov 9, 2026Wentao Wu, Xiao Wang, Yi Wang +3Multimodal PretrainingMultimodal SSL

  2. Split and Bridge: Multimodal Generation via Diffusion Bridging

    Nov 7, 2026Ahmad Arrabi, Xiaohan Zhang, Xingyu Li +1Multimodal GenerationMultimodal Diffusion Models

  3. UniMore: Fine-Grained Supervision for Unified Understanding and Generation

    Nov 6, 2026Haoyue Bai, Yinyi Luo, Yicheng He +3Unified Multimodal ModelsMultimodal Learning

  4. WeLM-Image: A Unified Image Generation Model towards Visual Intelligence

    Nov 5, 2026Canmiao Fu, Chen Li, Guohui Zhang +13Image GenerationMultimodal Generation

  5. Prism: Bidirectional Self-Teaching for Unified Multimodal Models

    Nov 4, 2026Peng Zhang, Qian Chen, Xianyin Zhang +6Unified Multimodal ModelsMultimodal Generation

  6. Generative AI for urban planning: synthesizing land-use maps with satellite imagery via DualControlNet

    Nov 4, 2026Qingqi Song, Yuzhou Chen, Lingqian Hu +1Urban PlanningImage Generation

  7. Movens-Zero: Scaling Humanoid Motion Foundation Models with Large-Scale Human Videos

    Oct 30, 2026Wendong Bu, Tinghong Ye, Qizhou Wang +21Human Motion GenerationMultimodal Generation

  8. S2TDR: Structured Multimodal Reconstruction for Sticker-Aware Dialogue Understanding

    Oct 30, 2026Yuanchen Shi, Fang KongMultimodal UnderstandingMultimodal Generation

  9. IntReal: A Real-World Dataset and Benchmark for Multimodal Interleaved Generation

    Oct 28, 2026Shaohan Yu, Jie Qin, Wende Tan +4Multimodal GenerationMultimodal Dataset

  10. Making MLLM Latents Matter: Factor-Aware Predictive Learning for Cinematic Video Generation

    Oct 22, 2026Songlin Yang, Zhe Wang, Xiaotong Zhao +8Multimodal Video GenerationVideo Generation

  11. JUICE: Unified Multimodal Modeling in Pretrained Representation Spaces

    Oct 20, 2026Jiahan Li, Huangyuan Su, Shengbang Tong +9Unified Multimodal ModelsMultimodal Generation

  12. Drive2Gauss: FeedForward 4D Gaussian Reconstruction from Multimodal Generative Driving Videos

    Oct 19, 2026Tong Zhao, Zhiyuan Han, Lin Chen +10Multimodal Video GenerationDynamic Scene Reconstruction

  13. MIRA: A Multimodal Illustrated Deep Research Agent for Controllable Reports

    Oct 19, 2026Hongrui Wu, Kaiwen Tuo, Congcong Wang +7Deep Research AgentsMultimodal Agents

  14. Modaliff: A Unified Token Diffusion Model for Any-to-Any Generation

    Oct 18, 2026Duc Kieu, Kien Do, Minh Khoa Le +3Multimodal GenerationUnified Multimodal Models

  15. Beyond Reward Scoring: Online Multi-Agent Preference Learning for Multimodal Response Generation

    Oct 16, 2026Ziyun Zou, Jiwoo Hong, Yifan Lu +5Preference AlignmentPreference-Based RL

  16. Full-Duplex Interaction with Speech, Expressions and Gestures

    Oct 15, 2026Siyin Wang, Wenyi Yu, Terumi Chiba +5Facial Expression SynthesisMultimodal Generation

  17. MSP: Modality Self-Play for Training Multimodal LLMs Without Paired Cross-Modal Data

    Oct 14, 2026Robert Moseley, Md Kaykobad Reza, Ameya Patil +2Multimodal LearningMultimodal Foundation Models

  18. ReasonUni: Reasoning-Augmented Reconstruction Advances Unified Multimodal Models

    Oct 14, 2026Feize Wu, Zhiyuan Yan, Yuhuan Hong +10Unified Multimodal ModelsMultimodal Generation

  19. WorldCast: Exploring Intelligent Visual Generation in World State Prediction

    Oct 14, 2026Wenhao Yan, Haodong Li, Xiaotong Li +9World ModelsMultimodal Generation

  20. Scaling Unified Multimodal Models to Long-Horizon Interleaved Generation

    Oct 12, 2026Ziqiao Peng, Yutao Cui, Wang Lin +12Multimodal GenerationUnified Multimodal Models

  21. OmniRSI: Taming Latent Diffusion for Unified Multimodal Remote Sensing Image Generation

    Oct 11, 2026Hebaixu Wang, Jing Zhang, Haonan Guo +3Multimodal GenerationUnified Multimodal Models

  22. Understanding Unified VLMs: From Attention Probing to Guidance Tuning for Multimodal Generation

    Oct 10, 2026Zhengyang Luo, Xiaoyu Qiu, Zhiheng Tan +3Unified Multimodal ModelsMultimodal Generation

  23. Multimodal Co-Generation With Continuous Bitstream Diffusion

    Oct 9, 2026Georgios Batzolis, Georgios Kontos, Bruno Trentini +4Multimodal GenerationMultimodal Diffusion Models

  24. UniData: Universal Multimodal Instruction Generation Pipeline

    Oct 8, 2026Jiaqi Tang, Yi-Feng Wu, Yuting Zhang +12Synthetic Data Generation for Language ModelsMultimodal Generation

  25. Self-correction Optimization for Interleaved Multimodal Generation

    Oct 7, 2026Xin You, Zhiwei Ning, Zukai Chen +7Multimodal GenerationTemporal Consistency in Video Generation