Overview

Unified multimodal models study how one system can process and generate text, images, video, audio, and sometimes spatial or action data. Common approaches use shared tokenizers and autoregressive or diffusion-based backbones, often alongside modality-specific encoders, decoders, or experts; training may combine supervised instruction tuning, alignment, and reinforcement learning. Evaluation spans tasks such as visual question answering, text-to-image generation, video understanding, editing, and cross-modal consistency. A central challenge is that shared architecture does not guarantee useful transfer: understanding and generation can interfere or remain semantically misaligned, while efficiency, safety, and reliable evaluation across interacting modalities remain open concerns.

Latest papers

CardsList

314 papers26 papers in the last four weeks, down 33% on the four weeks before. 0.2% of all new papers.

  1. Uni-RS: A Spatially Faithful Unified Understanding and Generation Model for Remote Sensing

    Jan 25, 2026Weiyu Zhang, Yuan Hu, Yong Li +1Unified Multimodal ModelsMultimodal Generation

  2. MedVL-SAM2: A unified 3D medical vision-language model for multimodal reasoning and prompt-driven segmentation

    Jan 14, 2026Yang Xing, Jiong Wu, Savas Ozdemir +4Interactive Medical Image SegmentationUnified Multimodal Models

  3. Unified Text-Image Generation with Weakness-Targeted Post-Training

    Jan 7, 2026Jiahui Chen, Philippe Hansen-Estruch, Xiaochuang Han +7Unified Multimodal ModelsT2I Generation

  4. SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models

    Oct 14, 2025Weiyang Jin, Yuwei Niu, Jiaqi Liao +4Unified Multimodal ModelsT2I Generation

  5. UniVideo: Unified Understanding, Generation, and Editing for Videos

    Oct 9, 2025Cong Wei, Quande Liu, Zixuan Ye +5Video GenerationUnified Multimodal Models

  6. HunyuanImage 3.0 Technical Report

    Sep 28, 2025Tencent Hunyuan Foundation Model TeamUnified Multimodal ModelsAutoregressive Image Generation

  7. Reconstruction Alignment Improves Unified Multimodal Models

    Sep 8, 2025Ji Xie, Trevor Darrell, Luke Zettlemoyer +1Unified Multimodal ModelsCross-Modal Alignment

  8. Mitigating Task Objective Conflicts for Continual Learning in Unified Multimodal Models

    Jun 4, 2025Jiaxing Zhang, Hao TangTask InterferenceUnified Multimodal Models

  9. Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision

    Apr 7, 2025Yuandong Pu, Le Zhuo, Kaiwen Zhu +7Diffusion Models for Image RestorationImage Enhancement

  10. Q-SiT: Teaching LMMs for Image Quality Scoring and Interpreting

    Mar 12, 2025Zicheng Zhang, Haoning Wu, Ziheng Jia +2Unified Multimodal ModelsMulti-Task Learning