Overview

Unified multimodal models study how one system can process and generate text, images, video, audio, and sometimes spatial or action data. Common approaches use shared tokenizers and autoregressive or diffusion-based backbones, often alongside modality-specific encoders, decoders, or experts; training may combine supervised instruction tuning, alignment, and reinforcement learning. Evaluation spans tasks such as visual question answering, text-to-image generation, video understanding, editing, and cross-modal consistency. A central challenge is that shared architecture does not guarantee useful transfer: understanding and generation can interfere or remain semantically misaligned, while efficiency, safety, and reliable evaluation across interacting modalities remain open concerns.

Latest papers

CardsList

314 papers26 papers in the last four weeks, down 33% on the four weeks before. 0.2% of all new papers.

  1. Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion

    Mar 6, 2026Lijiang Li, Zuwei Long, Yunhang Shen +6Masked Diffusion ModelsUnified Multimodal Models

  2. AV-Unified: A Unified Framework for Audio-visual Scene Understanding

    Mar 6, 2026Guangyao Li, Xin Wang, Wenwu ZhuAudio-Visual UnderstandingUnified Multimodal Models

  3. Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models

    Mar 6, 2026Jiadong Pan, Liang Li, Yuxin Peng +6Unified Multimodal ModelsT2I Generation

  4. UniPAR: A Unified Framework for Pedestrian Attribute Recognition

    Mar 5, 2026Minghe Xu, Rouying Wu, Jiarui Xu +5Pedestrian Attribute RecognitionUnified Multimodal Models

  5. A Simple Baseline for Unifying Understanding, Generation, and Editing via Vanilla Next-token Prediction

    Mar 5, 2026Jie Zhu, Hanghang Ma, Jia Wang +6Unified Multimodal ModelsMultimodal Understanding

  6. Crab+^{+}: A Scalable and Unified Audio-Visual Scene Understanding Model with Explicit Cooperation

    Mar 4, 2026Dongnuan Cai, Henghui Du, Chang Zhou +5Audio-Visual UnderstandingUnified Multimodal Models

  7. Modeling Cross-vision Synergy for Unified Large Vision Model

    Mar 3, 2026Shengqiong Wu, Lanhu Wu, Mingyang Bao +5Vision Foundation ModelsUnified Multimodal Models

  8. Beyond Language Modeling: An Exploration of Multimodal Pretraining

    Mar 3, 2026Shengbang Tong, David Fan, John Nguyen +18Multimodal PretrainingUnified Multimodal Models

  9. UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?

    Mar 3, 2026Zimo Wen, Boxiu Li, Wanbo Zhang +11Unified Multimodal ModelsMultimodal Model Evaluation

  10. ACE-Brain-0: Spatial Intelligence as a Shared Scaffold for Universal Embodiments

    Mar 3, 2026Ziyang Gong, Zehang Luo, Anke Tang +21Embodied Spatial ReasoningEmbodied AI

  11. Towards an Incremental Unified Multimodal Anomaly Detection: Augmenting Multimodal Denoising From an Information Bottleneck Perspective

    Mar 3, 2026Kaifang Long, Lianbo Ma, Jiaqi Liu +2Multimodal Anomaly DetectionUnified Multimodal Models

  12. LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Model

    Mar 1, 2026Zebin You, Xiaolu Zhang, Jun Zhou +2Diffusion ModelsMultimodal Diffusion Models

  13. Omni-C: Compressing Heterogeneous Modalities into a Single Dense Encoder

    Feb 27, 2026Kin Wai Lau, Yasar Abbas Ur Rehman, Lai-Man Po +1Multimodal LearningUnified Multimodal Models

  14. Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?

    Feb 27, 2026Hongbo Jiang, Jie Li, Yunhang Shen +4Unified Multimodal ModelsVLM Reasoning

  15. OmniZip: Learning a Unified and Lightweight Lossless Compressor for Multi-Modal Data

    Feb 25, 2026Yan Zhao, Zhengxue Cheng, Junxuan Zhang +4Lossless Data CompressionUnified Multimodal Models

  16. Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking

    Feb 24, 2026Shengqiong Wu, Bobo Li, Xinkai Wang +6VLM ReasoningUnified Multimodal Models

  17. Mobile-O: Unified Multimodal Understanding and Generation on Mobile Device

    Feb 23, 2026Abdelrahman Shaker, Ahmed Heakl, Jaseel Muhammad +8Unified Multimodal ModelsMultimodal Generation

  18. MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models

    Feb 23, 2026Mingrui Wu, Hang Liu, Jiayi Ji +2Image GenerationUnified Multimodal Models

  19. Multimodal Crystal Flow: Any-to-Any Modality Generation for Unified Crystal Modeling

    Feb 23, 2026Kiyoung Seong, Sungsoo Ahn, Sehui Han +1Crystal Structure GenerationUnified Multimodal Models

  20. Continual-NExT: A Unified Comprehension And Generation Continual Learning Framework

    Feb 20, 2026Jingyang Qiao, Zhizhong Zhang, Xin Tan +3Cross-Modal Representation TransferMultimodal Large Language Models

  21. TimeOmni-VL: Unified Models for Time Series Understanding and Generation

    Feb 19, 2026Tong Guan, Sheng Pan, Johan Barthelemy +5Time Series GenerationTime-Series Foundation Models

  22. Understanding vs. Generation: Navigating Optimization Dilemma in Multimodal Models

    Feb 17, 2026Sen Ye, Mengde Xu, Shuyang Gu +3Multimodal GenerationMultimodal Optimization

  23. LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Models

    Feb 15, 2026Shufan Li, Yuchen Zhu, Jiuxiang Gu +6Unified Multimodal ModelsMultimodal Reasoning

  24. Kelix Technical Report

    Feb 10, 2026Boyang Ding, Chenglong Chu, Dunju Zang +28Unified Multimodal ModelsMultimodal Large Language Models

  25. UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models

    Feb 9, 2026Cheng Yang, Chufan Shi, Bo Shui +7Unified Multimodal ModelsImage Generation

  26. Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation

    Feb 2, 2026Xinshun Wang, Peiming Li, Ziyi Wang +5Cross-Modal Representation LearningUnified Multimodal Models

  27. Quantifying the Gap between Understanding and Generation within Unified Multimodal Models

    Feb 2, 2026Chenlong Wang, Yuhang Chen, Zhihan Hu +4Unified Multimodal ModelsCross-Modal Alignment