Overview

Unified multimodal models study how one system can process and generate text, images, video, audio, and sometimes spatial or action data. Common approaches use shared tokenizers and autoregressive or diffusion-based backbones, often alongside modality-specific encoders, decoders, or experts; training may combine supervised instruction tuning, alignment, and reinforcement learning. Evaluation spans tasks such as visual question answering, text-to-image generation, video understanding, editing, and cross-modal consistency. A central challenge is that shared architecture does not guarantee useful transfer: understanding and generation can interfere or remain semantically misaligned, while efficiency, safety, and reliable evaluation across interacting modalities remain open concerns.

Latest papers

CardsList

314 papers26 papers in the last four weeks, down 33% on the four weeks before. 0.2% of all new papers.

  1. LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Model

    Apr 2, 2026Jiachun Jin, Zetong Zhou, Xiao Yang +4Unified Multimodal ModelsMultimodal Reasoning

  2. Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Models

    Apr 1, 2026Zixiang Peng, Yongxiu Xu, Qin-Yi Zhang +5Unified Multimodal ModelsLLM Safety Benchmarks

  3. Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis

    Mar 31, 2026Shuang Chen, Quanxin Shou, Hangting Chen +16Image GenerationAgentic Image Generation

  4. LVRPO: Language-Visual Alignment with GRPO for Multimodal Understanding and Generation

    Mar 29, 2026Shentong Mo, Sukmin YunVision-Language AlignmentUnified Multimodal Models

  5. Customized Visual Storytelling with Unified Multimodal LLMs

    Mar 29, 2026Wei-Hua Li, Cheng Sun, Chu-Song ChenMultimodal GenerationControllable Video Generation

  6. OmniColor: A Unified Framework for Multi-modal Lineart Colorization

    Mar 29, 2026Xulu Zhang, Haoqian Du, Xiaoyong Wei +1Multimodal GenerationUnified Multimodal Models

  7. UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomy

    Mar 25, 2026Yicheng Xu, Jiangning Zhang, Zhucun Xue +5Multimodal ICLUnified Multimodal Models

  8. OmniACBench: A Benchmark for Evaluating Context-Grounded Acoustic Control in Omni-Modal Models

    Mar 25, 2026Seunghee Kim, Bumkyu Park, Kyudan Jung +5Multimodal GroundingUnified Multimodal Models

  9. UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generation

    Mar 24, 2026Jie Liu, Zilyu Ye, Linxiao Yuan +8Image GenerationRL for Generative Models

  10. UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation

    Mar 23, 2026Ziyi Wang, Xinshun Wang, Shuang Chen +2Cross-Modal LearningUnified Multimodal Models

  11. Enhancing Alignment for Unified Multimodal Models via Semantically-Grounded Supervision

    Mar 20, 2026Jiyeong Kim, Yerim So, Hyesong Choi +2Unified Multimodal ModelsMultimodal Generation

  12. One Model, Two Minds: Task-Conditioned Reasoning for Unified Image Quality and Aesthetic Assessment

    Mar 20, 2026Wen Yin, Cencen Liu, Dingrui Liu +3Image Quality AssessmentUnified Multimodal Models

  13. UniSAFE: A Comprehensive Benchmark for Safety Evaluation of Unified Multimodal Models

    Mar 18, 2026Segyu Lee, Boryeong Cho, Hojung Jung +8Unified Multimodal ModelsMultimodal Model Evaluation

  14. Ruyi2.5 Technical Report

    Mar 18, 2026Huan Song, Shuyu Tian, Qingfei Zhao +5Edge-Cloud Collaborative InferenceMultimodal Foundation Models

  15. Rethinking UMM Visual Generation: Masked Modeling for Efficient Image-Only Pre-training

    Mar 17, 2026Peng Sun, Jun Xie, Tao LinUnified Multimodal ModelsMultimodal Pretraining

  16. Flash-Unified: A Training-Free and Task-Aware Acceleration Framework for Native Unified Models

    Mar 16, 2026Junlong Ke, Zichen Wen, Boxue Yang +6Efficient Multimodal InferenceUnified Multimodal Models

  17. HYDRA: Representation Harmonized Tokenization for Multimodal Generation and Understanding

    Mar 16, 2026Xuerui Qiu, Yutao Cui, Guozhen Zhang +9Unified Multimodal ModelsMultimodal Generation

  18. RS-WorldModel: a Unified Model for Remote Sensing Understanding and Future Sense Forecasting

    Mar 16, 2026Linrui Xu, Zhongan Wang, Fei Shen +4World Model LearningRemote Sensing VLMs

  19. Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation

    Mar 13, 2026Yichen Zhang, Da Peng, Zonghao Guo +19Unified Multimodal ModelsMultimodal Generation

  20. EvoTok: A Unified Image Tokenizer via Residual Latent Evolution for Visual Understanding and Generation

    Mar 12, 2026Yan Li, Ning Liao, Xiangyu Zhao +5Image TokenizationUnified Multimodal Models

  21. UniCompress: Token Compression for Unified Vision-Language Understanding and Generation

    Mar 11, 2026Ziyao Wang, Chen Chen, Jingtao Li +4Multimodal Token CompressionVisual Token Compression

  22. UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations

    Mar 11, 2026Yaqi Zhao, Wang Lin, Zijian Zhang +5Unified Multimodal ModelsMultimodal Generation

  23. InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing

    Mar 10, 2026Changyao Tian, Danni Yang, Guanzhou Chen +26Unified Multimodal ModelsMultimodal Generation

  24. Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization

    Mar 10, 2026Ming Nie, Chunwei Wang, Jianhua Han +2Multimodal GenerationUnified Multimodal Models

  25. Dynin-Omni: Omnimodal Unified Large Diffusion Language Model

    Mar 9, 2026Jaeik Kim, Woojin Kim, Jihwan Hong +8Unified Multimodal ModelsMultimodal Foundation Models

  26. How Long Can Unified Multimodal Models Generate Images Reliably? Taming Long-Horizon Interleaved Image Generation via Context Curation

    Mar 8, 2026Haoyu Chen, Qing Liu, Yuqian Zhou +7Multimodal GenerationUnified Multimodal Models