Overview

Multimodal generation studies models that create outputs such as images, video, speech, motion, or scientific signals from text, media, or combinations of them. Common approaches include diffusion and flow-matching decoders, autoregressive or discrete-token models, shared latent spaces, and systems that connect vision-language models to modality-specific generators; planning and reinforcement learning can add control for complex or interleaved outputs. A major direction is building models that both understand and generate, with benchmarks measuring prompt adherence, fidelity, temporal and cross-modal coherence, and task-specific validity. An open challenge is enabling shared models to turn understanding into faithful, controllable generation without modality imbalance or loss of specialized quality, especially over long sequences and varied real-world inputs.

Latest papers

CardsList

255 papers18 papers in the last four weeks, down 31% on the four weeks before. 0.1% of all new papers.

  1. Self-correction Optimization for Interleaved Multimodal Generation

    Oct 7, 2026Xin You, Zhiwei Ning, Zukai Chen +7Multimodal GenerationTemporal Consistency in Video Generation

  2. The Poisoned Conversation: Privacy-Leaking Watermarks in Unified Multimodal Models

    Oct 4, 2026Tobias Braun, Jonas Henry Grebe, Emil Sivic +4Unified Multimodal ModelsMultimodal Generation

  3. Structural-Functional Brain Connectivity Generation via Multimodal Hypergraph-based Flow Matching

    Oct 2, 2026Chyong Yi Poh, Hwa Hui Tew, Junn Yong Loo +4Multimodal GenerationFunctional Connectivity

  4. Gestalt: Large Multimodal Interplay Model

    Sep 30, 2026Zequn Yang, Yu Miao, Haotian Ni +8Unified Multimodal ModelsCross-Modal Alignment

  5. ScaffoldM3C: A Multimodal Sequential Monte Carlo Framework for Generative Stable Construction Planning

    Sep 30, 2026Gadiel Sznaier Camps, Chengyang He, Guillaume Sartoretti +2Robotic ConstructionMultimodal Generation

  6. Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces

    Sep 30, 2026Hongyuan Tao, Xinggang Wang, Lianghui Zhu +7Unified Multimodal ModelsMultimodal Pretraining

  7. Beyond Layers: Position-Resolved Gradient Conflict and Position-Aware Modulation for Unified Multimodal Models

    Sep 29, 2026Shuyang Jiang, Fucheng Deng, Yuchuan Luo +1Unified Multimodal ModelsGradient Interference

  8. What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation

    Sep 29, 2026Sieun Hyeon, Yejoon Lee, Mintaek Lim +3Multimodal GenerationMultimodal Model Evaluation

  9. Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Models

    Sep 28, 2026Wentao Zhou, Weijie Gan, Jiayun WangAdversarial TrainingUnified Multimodal Models

  10. Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy

    Sep 28, 2026Abhinav Sharma, Sai Karthik Navuluru, Wang Wei +21Audio-Video GenerationMultimodal Generation

  11. Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks

    Sep 25, 2026Nilotpal Kakati, Daniel Murnane, Baran Hashemi +5Unified Multimodal ModelsHigh-Energy Physics

  12. Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?

    Sep 20, 2026Jiyao Yang, Yang Liu, Zhenyue Qin +2Fake News DetectionMultimodal Large Language Models

  13. Bio-MF: Low-Latency and High-Fidelity EEG-to-fNIRS Cross-Modal Generation for Hybrid Motor-Imagery Brain--Computer Interfaces

    Sep 17, 2026Boyuan Zhao, Sifan Zhang, Luping ChenBrain-Computer InterfacesMultimodal Generation

  14. BirdsongChat: A Hybrid Multi-Agent Framework for Multimodal Embodied Behavior Simulation

    Sep 16, 2026Callie C. Liao, Duoduo Liao, Ellie L. ZhangMultimodal AgentsEmbodied AI

  15. MOCC-R1: Reinforcing Reasoning-Response Consistency for Multimodal Counselor Response Generation

    Sep 15, 2026Wenjie Zheng, Qiming Xie, Jianfei Yu +1Multimodal GenerationReasoning Consistency in Language Models

  16. Long-Tail Adaptive Flow Matching with Explicit Conditional Consistency Guidance for Precise Multimodal Face Synthesis

    Sep 14, 2026Yushe Cao, Xuechao Zou, Xing Xi +3Multimodal GenerationDiffusion Model Guidance

  17. UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation

    Sep 11, 2026Danning Zhang, Yijing Lin, Shuhan Zhuang +4Subject-Driven Image GenerationMultimodal Generation

  18. Multi-Modal Controlled Coherent Motion Generation

    Sep 10, 2026Yifei Liu, Qiong Cao, Hongwei Yi +2Human Motion Generation3D Avatar Generation

  19. Dreaming in Flow: Bidirectional Self-Refinement for Unified Multimodal Models

    Sep 8, 2026Ke Hao, Yuanzhi Liang, Tingxi Chen +5Multimodal GroundingUnified Multimodal Models

  20. SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign

    Sep 3, 2026Jiarui Lu, Yuyang Wang, Yizhe Zhang +4Unified Multimodal ModelsGenerative Modeling

  21. PrivateHub: Contrastive Diffusion Model for Private Sensor-Intensive Environment Data Generation

    Sep 2, 2026Jiechao Gao, Yuandong Pan, Jie Wang +2Privacy-Preserving MLSynthetic Data Generation