Overview

Unified multimodal models study how one system can process and generate text, images, video, audio, and sometimes spatial or action data. Common approaches use shared tokenizers and autoregressive or diffusion-based backbones, often alongside modality-specific encoders, decoders, or experts; training may combine supervised instruction tuning, alignment, and reinforcement learning. Evaluation spans tasks such as visual question answering, text-to-image generation, video understanding, editing, and cross-modal consistency. A central challenge is that shared architecture does not guarantee useful transfer: understanding and generation can interfere or remain semantically misaligned, while efficiency, safety, and reliable evaluation across interacting modalities remain open concerns.

Latest papers

CardsList

314 papers26 papers in the last four weeks, down 33% on the four weeks before. 0.2% of all new papers.

  1. Gaspard: Expressive unified speech and gesture synthesis with continuous autoregressive flow-matching and decoupled classifier-free guidance

    Nov 18, 2026Téo Guichoux, Théodor Lemerle, Catherine Pelachaud +1Multimodal GenerationUnified Multimodal Models

  2. The Elephant in the Question: Unified Models Picture the Concepts Behind Their Answers

    Nov 18, 2026Tobias Braun, Marcus Rohrbach, Lorenzo CavallaroMultimodal Model InterpretabilityUnified Multimodal Models

  3. Pixel is All You Need

    Nov 17, 2026Jie Chu, Haoting Shi, Fucheng Zhang +4Generative ModelingUnified Multimodal Models

  4. Report of the 8th LSVOS Challenge: Mobile Unified Multimodal Understanding

    Nov 12, 2026Chang Liu, Henghui Ding, Tong Yang +10Unified Multimodal ModelsMultimodal Model Evaluation

  5. Zero-Shot Benchmarking of Unified Multimodal Models for Shallow-Water UAV Image Distortion Assessment and Restoration

    Nov 6, 2026Dario Scilla, Omar A. Lopez, Mattia Rosso +2Image RestorationUnderwater Image Enhancement

  6. UniMore: Fine-Grained Supervision for Unified Understanding and Generation

    Nov 6, 2026Haoyue Bai, Yinyi Luo, Yicheng He +3Unified Multimodal ModelsMultimodal Learning

  7. Prism: Bidirectional Self-Teaching for Unified Multimodal Models

    Nov 4, 2026Peng Zhang, Qian Chen, Xianyin Zhang +6Unified Multimodal ModelsMultimodal Generation

  8. From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models

    Nov 3, 2026Gayatri Deshmukh, Onkar Susladkar, Adheesh Juvekar +2Unified Multimodal ModelsMultimodal Diffusion Models

  9. YANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction

    Nov 3, 2026Shengyu Wang, Tengyue Nong, Junming Liu +10Multimodal DatasetMulti-Turn Dialogue Evaluation

  10. OCR in the Foundation Model Era: From Specialist Models to Unified Models

    Nov 3, 2026Zhiheng Li, Zongyang Ma, Jiaxian Chen +7Optical Character RecognitionUnified Multimodal Models

  11. Is Forgetting Unified? Exploring Concept Unlearning in Unified Multimodal Models

    Nov 3, 2026Zhongkai Xue, Junkai Chen, Mingcong Lei +7Concept ErasureUnified Multimodal Models

  12. Latent Reasoning in Continuous Space for Unified Multimodal Models

    Oct 30, 2026Byungwoo Jeon, Yoonwoo Jeong, Hyunseok Lee +2Continuous Latent ReasoningUnified Multimodal Models

  13. DiFuse: Fusing Native Diffusion Representations for Spatial Reasoning in Unified Multimodal Models

    Oct 29, 2026Ruixin Fan, Weining Wang, Jiahui Sun +2Multimodal Spatial ReasoningUnified Multimodal Models

  14. Calibrating Attention-Based Token Pruning for Efficient Multi-Image Synthesis in UMMs

    Oct 29, 2026Zimeng Wu, Qijian Tian, Penghui Du +3Efficient Multimodal InferenceMultimodal Token Compression

  15. Continual Adaptation Across Modalities: Learning and Forgetting in Unified Multimodal Models

    Oct 28, 2026Xinrui Wang, Zhihao Chen, Jia Tang +4Parameter IsolationUnified Multimodal Models

  16. IntReal: A Real-World Dataset and Benchmark for Multimodal Interleaved Generation

    Oct 28, 2026Shaohan Yu, Jie Qin, Wende Tan +4Multimodal GenerationMultimodal Dataset

  17. Cross-modal Steering Vectors Exist Across Both Image and Text Generation

    Oct 26, 2026Donghao Zeng, Nishant Subramani, Tamar Rott Shaham +1Cross-Modal Representation TransferUnified Multimodal Models

  18. RynnUnify: Towards Unified Embodied Brain

    Oct 23, 2026Minghao Zhu, Bohan Hou, Kun Xiang +20Robot Foundation ModelsEmbodied World Models

  19. JUICE: Unified Multimodal Modeling in Pretrained Representation Spaces

    Oct 20, 2026Jiahan Li, Huangyuan Su, Shengbang Tong +9Unified Multimodal ModelsMultimodal Generation

  20. Beyond Omni: A Unified MLLM for Embodied Sensory Understanding

    Oct 19, 2026Shiyun Xiong, Dongming Wu, Peiwen Sun +5Unified Multimodal ModelsMultimodal Large Language Models

  21. UniJev: Separating Accuracy from Calibration in Trimodal System One Decision Models

    Oct 19, 2026Jianing Wang, Youwen Jian, Derong Wang +5Model CalibrationVLM Calibration