Overview

Unified multimodal models study how one system can process and generate text, images, video, audio, and sometimes spatial or action data. Common approaches use shared tokenizers and autoregressive or diffusion-based backbones, often alongside modality-specific encoders, decoders, or experts; training may combine supervised instruction tuning, alignment, and reinforcement learning. Evaluation spans tasks such as visual question answering, text-to-image generation, video understanding, editing, and cross-modal consistency. A central challenge is that shared architecture does not guarantee useful transfer: understanding and generation can interfere or remain semantically misaligned, while efficiency, safety, and reliable evaluation across interacting modalities remain open concerns.

Latest papers

CardsList

314 papers26 papers in the last four weeks, down 33% on the four weeks before. 0.2% of all new papers.

  1. MedUAG: Unified Understanding and Generation for Medical Multimodal Models

    Aug 19, 2026Zijie Meng, Yuncheng Zhang, Hualiang Wang +8Unified Multimodal ModelsMultimodal Generation

  2. Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models

    Aug 18, 2026Zongyang Qiu, Yihan Wu, Kaixuan Fan +2Unified Multimodal ModelsMultimodal Generation

  3. MUPA2^{2}E: Multimodal Unified Perception with Asymmetric Attention for Emotion Assessment

    Aug 17, 2026Stefanos Gkikas, Eric Nichols, Christian Arzate Cruz +1Dataset BiasMultimodal Emotion Recognition

  4. Fusion Anything: A Generalized Multimodal Foundation Model

    Aug 16, 2026Huizi Cui, Zongbo Han, Chenggong Ding +6Unified Multimodal ModelsMultimodal Pretraining

  5. SPARGen: Unifying Spatial Perception and Reasoning through Native Multimodal Generation

    Aug 14, 2026Jinsheng Quan, Jianhua Li, Siyi Xie +7Multimodal Spatial Reasoning3D Spatial Reasoning

  6. A Model-Internal Protocol for Assessing Multimodal Models as Integrated Systems

    Aug 12, 2026Hao Zhang, Jiaxin Qi, Zhijiang Tang +1Unified Multimodal ModelsMultimodal Reasoning

  7. MedUP: Awakening Unified Understanding and Perception in Medical Vision-Language Models

    Aug 11, 2026Yuan Wang, Hualiang Wang, Yixin Chen +6Unified Multimodal ModelsVision-Language Grounding

  8. Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models

    Aug 10, 2026Zhi Zeng, Cheng Zhang, Zesheng Yang +9Unified Multimodal ModelsAudio-Visual QA

  9. AnyTrack: Unifying Visual Object Tracking with Any Modalities

    Aug 7, 2026Hao Li, Yunzhi Zhuge, Wenning Hao +4Multimodal Visual Object TrackingMissing-Modality Learning

  10. ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation

    Aug 5, 2026Jiahao Zhao, Xiaomin Yu, Zhongxiang Sun +5Unified Multimodal ModelsAgentic Image Generation

  11. Allocation Before Ranking: Decoupled Token Compression for OmniLLMs

    Aug 3, 2026Zhenghui Guo, Yilin Yang, Yuanbin Man +5Audio Token CompressionUnified Multimodal Models

  12. Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning

    Aug 2, 2026Yuqi Li, Yuedong Tan, Huiran Duan +7Multimodal Visual Object TrackingEfficient Multimodal Inference

  13. Do Unified Multimodal Models Think in One Space? A Lens Through Cross-Branch Steering

    Jul 29, 2026Yu Wang, Sharon LiUnified Multimodal ModelsCross-Modal Alignment

  14. MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities

    Jul 28, 2026Mingqiao Ye, Zhaochong An, Zhitong Gao +11Unified Multimodal ModelsDecoder-Only Language Models

  15. UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling

    Jul 27, 2026Zhipeng Bao, Zhen Zhu, Nupur Kumari +4Unified Multimodal ModelsVisual Autoregressive Models

  16. Twins: Learn to Predict Unified Representations with Focal Loss

    Jul 24, 2026Kaixiong Gong, Xin Cai, Bin Lin +9Focal LossDiffusion Transformer

  17. OPOD: On-Policy Omni Distillation

    Jul 23, 2026Tong Zhao, Yuyang Hu, Yutao Zhu +5Cross-Modal Knowledge DistillationUnified Multimodal Models

  18. Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editing

    Jul 18, 2026Weiwei Tan, Junxian Li, Rui Wang +3Unified Multimodal Models

  19. Think, Plan, Paint: Layout-Aware Reasoning for Controllable Image Generation in Unified Models

    Jul 17, 2026Junhao Liu, Jian-Wei Zhang, Tao Huang +3Unified Multimodal ModelsImage Generation

  20. Monkey King Bang: A Unified Scientific Multimodal Foundation Model

    Jul 17, 2026Hesen Chen, Xinyu Su, Xiaomeng Yang +11AI for ScienceUnified Multimodal Models

  21. RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination

    Jul 15, 2026Haotian Liang, Mingkang Chen, Yufei Huang +27Unified Multimodal ModelsRobot Foundation Models