Overview

Unified multimodal models study how one system can process and generate text, images, video, audio, and sometimes spatial or action data. Common approaches use shared tokenizers and autoregressive or diffusion-based backbones, often alongside modality-specific encoders, decoders, or experts; training may combine supervised instruction tuning, alignment, and reinforcement learning. Evaluation spans tasks such as visual question answering, text-to-image generation, video understanding, editing, and cross-modal consistency. A central challenge is that shared architecture does not guarantee useful transfer: understanding and generation can interfere or remain semantically misaligned, while efficiency, safety, and reliable evaluation across interacting modalities remain open concerns.

Latest papers

CardsList

314 papers26 papers in the last four weeks, down 33% on the four weeks before. 0.2% of all new papers.

  1. Next-Dense-Stride Prediction for Multimodal Autoregressive Visual Modeling

    Jul 10, 2026Chicago Y. Park, Jialin Mao, Xiaojian Xu +3Unified Multimodal ModelsAutoregressive Image Generation

  2. ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts

    Jul 10, 2026Jiawen Li, Tian Guan, Huijuan Shi +5Computational PathologyUnified Multimodal Models

  3. Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing

    Jul 10, 2026Chenxu Peng, Chongtian zhou, Dicheng Liu +5Event-Based VisionVisual Representation Learning

  4. DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models

    Jul 9, 2026Pengjie Wang, Linger Deng, Zujia Zhang +6Efficient Multimodal InferenceUnified Multimodal Models

  5. ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation

    Jul 7, 2026Tianjiao Yu, Xinzhuo Li, Yifan Shen +43D Foundation ModelsUnified Multimodal Models

  6. Vision as Unified Multimodal Generation

    Jul 7, 2026Xiaoyang Han, Jianhua Li, Kewang Deng +14Unified Multimodal ModelsMultimodal Generation

  7. Transferability Between Understanding and Generation in Unified Multimodal Models

    Jul 5, 2026Jiwon Kang, Heeji Yoon, Jaewoo Jung +5Unified Multimodal ModelsMultimodal Generation

  8. Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Process

    Jul 4, 2026Zican Hu, Xuyang Hu, Yiming Liu +10Unified Multimodal ModelsMultimodal Reasoning

  9. Gemma 4 Technical Report

    Jul 2, 2026Gemma Team, Sherif El Abd, Vaibhav Aggarwal +298Efficient Multimodal InferenceUnified Multimodal Models

  10. Rosetta: Composable Native Multimodal Pretraining

    Jul 1, 2026Xiangyue Liu, Zijian Zhang, Miles Yang +3Unified Multimodal ModelsMultimodal Pretraining

  11. UniTac: A Unified Multimodal Model for Cross-Sensor Tactile Understanding and Generation

    Jun 30, 2026Jiahang Tu, Fengyu Yang, Chenyang Ma +8Tactile SensingUnified Multimodal Models

  12. BrainJanus: A Unified Model for Understanding and Generation across Brain, Vision, and Language

    Jun 29, 2026Haitao Wu, Qirui Zhang, Zhouheng Yao +8Cross-Modal Representation LearningUnified Multimodal Models

  13. Illuminating Unified Multimodal Model for Free-form Interleaved Text-Image Generation

    Jun 29, 2026Chonghuinan Wang, Zhikai Chen, Chunwei Wang +9Unified Multimodal ModelsMultimodal Model Evaluation

  14. COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Models

    Jun 27, 2026Ziqi Zhou, Weize Quan, Mining Tan +6Multimodal GroundingUnified Multimodal Models

  15. Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards

    Jun 25, 2026Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar +5Unified Multimodal ModelsMultimodal Generation

  16. Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation

    Jun 25, 2026Jinyu Liu, Xincheng Shuai, Henghui Ding +1Unified Multimodal ModelsMultimodal Foundation Models

  17. iFLYTEK-Embodied-Omni Technical Report

    Jun 24, 2026Yuan Zhang, Jingfei Ni, Guanchen Lu +12Unified Multimodal ModelsRobot Motion Generation

  18. UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translation

    Jun 23, 2026Jiahao Lyu, Pei Fu, Zhenhang Li +6Image Text TranslationUnified Multimodal Models

  19. SPAR: Semantic-Pixel Self-Alignment and Adaptive Routing for Unified Multimodal Models

    Jun 22, 2026Hongxiang Li, Hongxu Chen, Chenyang Zhu +5Diffusion Model AlignmentUnified Multimodal Models

  20. BioMatrix: Towards a Comprehensive Biological Foundation Model Spanning the Modality Matrix of Sequences, Structures, and Language

    Jun 20, 2026Qizhi Pei, Zhimeng Zhou, Yi Duan +9Multimodal UnderstandingMultimodal Pretraining

  21. Odoriko: A Shape-Aware Multimodal Diffusion Framework for Human Motion

    Jun 19, 2026Dongseok Shim, Julian Tanke, Kengo Uchida +5Unified Multimodal ModelsText-to-Motion Generation

  22. Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning

    Jun 17, 2026Pengyu Li, Zhitao Gao, Lingling Zhang +5Cross-Modal Knowledge DistillationEfficient Multimodal Inference

  23. Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification

    Jun 16, 2026Wujian Peng, Lingchen Meng, Yuxuan Cai +7Unified Multimodal ModelsAutoregressive Image Generation

  24. Pareto LoRA: Mitigating Modality Imbalance in Unified Multimodal Models via Pareto-Optimal Gradient Integration

    Jun 15, 2026Xiwen Wei, Mark Nutter, Madhusudhanan Srinivasan +1Unified Multimodal ModelsModality Imbalance