Overview

Multimodal generation studies models that create outputs such as images, video, speech, motion, or scientific signals from text, media, or combinations of them. Common approaches include diffusion and flow-matching decoders, autoregressive or discrete-token models, shared latent spaces, and systems that connect vision-language models to modality-specific generators; planning and reinforcement learning can add control for complex or interleaved outputs. A major direction is building models that both understand and generate, with benchmarks measuring prompt adherence, fidelity, temporal and cross-modal coherence, and task-specific validity. An open challenge is enabling shared models to turn understanding into faithful, controllable generation without modality imbalance or loss of specialized quality, especially over long sequences and varied real-world inputs.

Latest papers

CardsList

255 papers21 papers in the last four weeks, down 22% on the four weeks before. 0.1% of all new papers.

  1. The Poisoned Conversation: Privacy-Leaking Watermarks in Unified Multimodal Models

    Oct 4, 2026Tobias Braun, Jonas Henry Grebe, Emil Sivic +4Unified Multimodal ModelsMultimodal Generation

  2. Structural-Functional Brain Connectivity Generation via Multimodal Hypergraph-based Flow Matching

    Oct 2, 2026Chyong Yi Poh, Hwa Hui Tew, Junn Yong Loo +4Multimodal GenerationFunctional Connectivity

  3. Gestalt: Large Multimodal Interplay Model

    Sep 30, 2026Zequn Yang, Yu Miao, Haotian Ni +8Unified Multimodal ModelsCross-Modal Alignment

  4. ScaffoldM3C: A Multimodal Sequential Monte Carlo Framework for Generative Stable Construction Planning

    Sep 30, 2026Gadiel Sznaier Camps, Chengyang He, Guillaume Sartoretti +2Robotic ConstructionMultimodal Generation

  5. Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces

    Sep 30, 2026Hongyuan Tao, Xinggang Wang, Lianghui Zhu +7Unified Multimodal ModelsMultimodal Pretraining

  6. Beyond Layers: Position-Resolved Gradient Conflict and Position-Aware Modulation for Unified Multimodal Models

    Sep 29, 2026Shuyang Jiang, Fucheng Deng, Yuchuan Luo +1Unified Multimodal ModelsGradient Interference

  7. What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation

    Sep 29, 2026Sieun Hyeon, Yejoon Lee, Mintaek Lim +3Multimodal GenerationMultimodal Model Evaluation

  8. Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Models

    Sep 28, 2026Wentao Zhou, Weijie Gan, Jiayun WangAdversarial TrainingUnified Multimodal Models

  9. Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy

    Sep 28, 2026Abhinav Sharma, Sai Karthik Navuluru, Wang Wei +21Audio-Video GenerationMultimodal Generation

  10. Prompting Particle Physics: Tokenized Multi-modal Foundation Models for Combinatorially Many Tasks

    Sep 25, 2026Nilotpal Kakati, Daniel Murnane, Baran Hashemi +5Unified Multimodal ModelsHigh-Energy Physics

  11. Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?

    Sep 20, 2026Jiyao Yang, Yang Liu, Zhenyue Qin +2Fake News DetectionMultimodal Large Language Models

  12. Bio-MF: Low-Latency and High-Fidelity EEG-to-fNIRS Cross-Modal Generation for Hybrid Motor-Imagery Brain--Computer Interfaces

    Sep 17, 2026Boyuan Zhao, Sifan Zhang, Luping ChenBrain-Computer InterfacesMultimodal Generation

  13. BirdsongChat: A Hybrid Multi-Agent Framework for Multimodal Embodied Behavior Simulation

    Sep 16, 2026Callie C. Liao, Duoduo Liao, Ellie L. ZhangMultimodal AgentsEmbodied AI

  14. MOCC-R1: Reinforcing Reasoning-Response Consistency for Multimodal Counselor Response Generation

    Sep 15, 2026Wenjie Zheng, Qiming Xie, Jianfei Yu +1Multimodal GenerationReasoning Consistency in Language Models

  15. Long-Tail Adaptive Flow Matching with Explicit Conditional Consistency Guidance for Precise Multimodal Face Synthesis

    Sep 14, 2026Yushe Cao, Xuechao Zou, Xing Xi +3Multimodal GenerationDiffusion Model Guidance

  16. UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation

    Sep 11, 2026Danning Zhang, Yijing Lin, Shuhan Zhuang +4Subject-Driven Image GenerationMultimodal Generation

  17. Multi-Modal Controlled Coherent Motion Generation

    Sep 10, 2026Yifei Liu, Qiong Cao, Hongwei Yi +2Human Motion Generation3D Avatar Generation

  18. Dreaming in Flow: Bidirectional Self-Refinement for Unified Multimodal Models

    Sep 8, 2026Ke Hao, Yuanzhi Liang, Tingxi Chen +5Multimodal GroundingUnified Multimodal Models

  19. SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign

    Sep 3, 2026Jiarui Lu, Yuyang Wang, Yizhe Zhang +4Unified Multimodal ModelsGenerative Modeling

  20. PrivateHub: Contrastive Diffusion Model for Private Sensor-Intensive Environment Data Generation

    Sep 2, 2026Jiechao Gao, Yuandong Pan, Jie Wang +2Privacy-Preserving MLSynthetic Data Generation

  21. Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System

    Sep 1, 2026Penghao Wu, Haiwen Diao, Weichen Fan +3Unified Multimodal ModelsMultimodal Generation