Overview

Unified multimodal models study how one system can process and generate text, images, video, audio, and sometimes spatial or action data. Common approaches use shared tokenizers and autoregressive or diffusion-based backbones, often alongside modality-specific encoders, decoders, or experts; training may combine supervised instruction tuning, alignment, and reinforcement learning. Evaluation spans tasks such as visual question answering, text-to-image generation, video understanding, editing, and cross-modal consistency. A central challenge is that shared architecture does not guarantee useful transfer: understanding and generation can interfere or remain semantically misaligned, while efficiency, safety, and reliable evaluation across interacting modalities remain open concerns.

Latest papers

CardsList

314 papers26 papers in the last four weeks, down 33% on the four weeks before. 0.2% of all new papers.

  1. Unified Multimodal Model for Brain MRI Imputation and Understanding

    Jun 15, 2026Zhiyun Song, Che Liu, Tian Xia +2Missing-Modality LearningUnified Multimodal Models

  2. UniDDT: Unifying Multimodal Understanding and Generation with Decoupled Diffusion Transformer

    Jun 15, 2026Shuai Wang, Liang Li, Yang Chen +3Diffusion TransformerUnified Multimodal Models

  3. HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers

    Jun 11, 2026Guozhen Zhang, Xuerui Qiu, Yutao Cui +11Unified Multimodal ModelsVision Transformer

  4. ARM: An AutoRegressive Large Multimodal Model with Unified Discrete Representations

    Jun 9, 2026Junke Wang, Xiao Wang, Jiacheng Pan +16Text-Guided Image EditingUnified Multimodal Models

  5. IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generation

    Jun 8, 2026Lingyi Meng, Zecong Tang, Haoran Li +12Unified Multimodal ModelsMultimodal Generation

  6. Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing

    Jun 4, 2026Yuxiao Ye, Haoran He, Fangyuan Kong +4Reinforcement LearningImage Editing Evaluation

  7. UniCanvas: A Diffusion-base Unified Model for Text-in-Image Joint Generation

    Jun 2, 2026Zeyuan Yang, Hao-Wei Chen, Xueyang Yu +5Unified Multimodal ModelsMultimodal Generation

  8. Foley-Omni: A Unified Multimodal Generation Model from Task-Level Audio Synthesis to Complete Video Soundtrack Generation

    Jun 2, 2026Ye Tao, Lupeng Liu, Xuenan Xu +6Audio-Video GenerationUnified Multimodal Models

  9. Attend to Anything: Foundation Model for Unified Human Attention Modeling

    Jun 2, 2026Wenzhuo Zhao, Ronghao Xian, Keren Fu +1Visual AttentionUnified Multimodal Models

  10. On the Limits of Token Reduction for Efficient Unified Vision Language Training

    May 31, 2026Siyi Chen, Weiming Zhuang, Jingtao Li +1Vision-Language ModelsUnified Multimodal Models

  11. ProductWebGen: Benchmarking Multimodal Product Webpage Generation

    May 31, 2026Zhihong Liu, Siqi Kou, Zheng Li +5Web Application GenerationUnified Multimodal Models

  12. Do Text Edits Generalize to Visual Generation? Benchmarking Cross-Modal Knowledge Editing in UMMs

    May 30, 2026Xin Gao, Cheng Yang, Chufan Shi +1Knowledge EditingUnified Multimodal Models

  13. Representation Forcing for Bottleneck-Free Unified Multimodal Models

    May 29, 2026Yuqing Wang, Zhijie Lin, Ceyuan Yang +10Unified Multimodal ModelsConditional Image Generation

  14. UNISON: A Unified Sound Generation and Editing Framework via Deep LLM Fusion

    May 29, 2026Zhaoqing Li, Haoning Xu, Jingran Su +9Text-to-Audio GenerationTTS Synthesis

  15. Archon: A Unified Multimodal Model for Holistic Digital Human Generation

    May 28, 2026Chong Bao, Shichen Liu, Lijun Yu +9Unified Multimodal ModelsMultimodal Pretraining

  16. Mind-Omni: A Unified Multi-Task Framework for Brain-Vision-Language Modeling via Discrete Diffusion

    May 28, 2026Yizhuo Lu, Changde Du, Qingyu Shi +5Brain-Computer InterfacesUnified Multimodal Models

  17. AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling

    May 28, 2026Yiheng Li, Zhuo Li, Ruibing Hou +4Cross-Modal LearningMasked Modeling

  18. Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs

    May 27, 2026Xiang Fang, Wanlong Fang, Changshuo Wang +4Missing-Modality LearningVLM Robustness

  19. How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

    May 26, 2026Qian Yang, Ankur Sikarwar, Huy Le +4Visual Spatial ReasoningUnified Multimodal Models

  20. Particle-Lund Multimodality in Jet Taggers

    May 26, 2026Loukas Gouskos, Benedikt MaierUnified Multimodal ModelsJet Tagging

  21. Toward Native Multimodal Modeling: A Roadmap

    May 25, 2026Siyu An, Junru Lu, Junnan Dong +18Unified Multimodal ModelsMultimodal Generation

  22. DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement

    May 25, 2026Renjie Lu, Xulong Zhang, Xiaoyang Qu +2Multimodal Disentangled Representation LearningUnified Multimodal Models

  23. Guess the Unified Model: How Much Can We Recover from Generated Images?

    May 24, 2026Jasin Cekinmez, Ryo Mitsuhashi, Addison J. Wu +1Data ProvenanceUnified Multimodal Models

  24. Unified 3D Scene Understanding Through Physical World Modeling

    May 23, 2026Wanhee Lee, Klemen Kotar, Rahul Mysore Venkatesh +4Unified Multimodal ModelsDepth Estimation

  25. Bernini: Latent Semantic Planning for Video Diffusion

    May 21, 2026Bernini Team, Chenchen Liu, Junyi Chen +9Video Diffusion ModelsUnified Multimodal Models

  26. Uni-Edit: Intelligent Editing Is A General Task For Unified Model Tuning

    May 20, 2026Dian Zheng, Manyuan Zhang, Hongyu Li +4Text-Guided Image EditingUnified Multimodal Models