Overview

Unified multimodal models study how one system can process and generate text, images, video, audio, and sometimes spatial or action data. Common approaches use shared tokenizers and autoregressive or diffusion-based backbones, often alongside modality-specific encoders, decoders, or experts; training may combine supervised instruction tuning, alignment, and reinforcement learning. Evaluation spans tasks such as visual question answering, text-to-image generation, video understanding, editing, and cross-modal consistency. A central challenge is that shared architecture does not guarantee useful transfer: understanding and generation can interfere or remain semantically misaligned, while efficiency, safety, and reliable evaluation across interacting modalities remain open concerns.

Latest papers

CardsList

314 papers26 papers in the last four weeks, down 33% on the four weeks before. 0.2% of all new papers.

  1. Uni-HOI:A Unified framework for Learning the Joint distribution of Text and Human-Object Interaction

    Apr 30, 2026Mengfei Zhang, Jinlu Zhang, Zhigang TuHOI GenerationUnified Multimodal Models

  2. Beyond Accuracy: Benchmarking Cross-Task Consistency in Unified Multimodal Models

    Apr 27, 2026Weixing Wang, Liudvikas Zekas, Anton Hackl +5Unified Multimodal ModelsCross-Modal Alignment

  3. Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation

    Apr 27, 2026Zhiheng Liu, Weiming Ren, Xiaoke Huang +12Unified Multimodal ModelsMultimodal Pretraining

  4. Meta-CoT: Enhancing Granularity and Generalization in Image Editing

    Apr 27, 2026Shiyi Zhang, Yiji Cheng, Tiankai Hang +8Text-Guided Image EditingUnified Multimodal Models

  5. CheXmix: Unified Generative Pretraining for Vision Language Models in Medical Imaging

    Apr 24, 2026Ashwin Kumar, Robbie Holland, Corey Barrett +8Radiology Report GenerationChest X-Ray Classification

  6. Text-Guided Multimodal Unified Industrial Anomaly Detection

    Apr 24, 2026Zewen Li, Shuo Ye, Zitong Yu +2Unified Multimodal ModelsIndustrial Anomaly Detection

  7. Context Unrolling in Omni Models

    Apr 23, 2026Ceyuan Yang, Zhijie Lin, Yang Zhao +16Cross-Modal LearningUnified Multimodal Models

  8. Exploring Spatial Intelligence from a Generative Perspective

    Apr 22, 2026Muzhi Zhu, Shunyao Jiang, Huanyi Zheng +9Spatial ReasoningUnified Multimodal Models

  9. OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models

    Apr 20, 2026Yiwei Zhang, Xuesong Chen, Jin Gao +5VLMs for Autonomous DrivingEnd-to-End Autonomous Driving

  10. SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning

    Apr 19, 2026Yian Li, Yang Jiao, Bin Zhu +4Multimodal CoT ReasoningUnified Multimodal Models

  11. Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language Models

    Apr 18, 2026Xinru Yan, Boxi Cao, Yaojie Lu +4Unified Multimodal ModelsNeural Network Interpretability

  12. Bias-constrained multimodal intelligence for equitable and reliable clinical AI

    Apr 18, 2026Cheng Li, Weijian Huang, Jiarun Liu +6Unified Multimodal ModelsAlgorithmic Fairness

  13. CodeMMR: Bridging Natural Language, Code, and Image for Unified Retrieval

    Apr 17, 2026Jiahui Geng, Qing Li, Fengyu Cai +1Multimodal IRUnified Multimodal Models

  14. UniRect-CoT: Enhancing Generation in Unified Multimodal Models via Reflective Rectification with Inherent Understanding

    Apr 15, 2026Yibo Jiang, Tao Wu, Rui Jiang +4Diffusion Model GuidanceUnified Multimodal Models

  15. TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training

    Apr 12, 2026Yinyi Luo, Wenwen Wang, Hayes Bai +6Unified Multimodal ModelsMultimodal Model Evaluation

  16. Uni-ViGU: Towards Unified Video Generation and Understanding via A Diffusion-Based Video Generator

    Apr 9, 2026Luozheng Qin, Jia Gong, Qian Qiao +6Unified Multimodal ModelsVideo Generation

  17. Bridging Time and Space: Decoupled Spatio-Temporal Alignment for Video Grounding

    Apr 9, 2026Xuezhen Tu, Jingyu Wu, Fangyu Kang +4Vision-Language GroundingTemporal Video Grounding

  18. Can Vision-Language Models Think from the Sky? Unifying UAV Reasoning and Generation

    Apr 7, 2026Jintao Sun, Gangyi Ding, Donglin Di +2VLM ReasoningUnified Multimodal Models

  19. CLEAR: Unlocking Generative Potential for Degraded Image Understanding in Unified Multimodal Models

    Apr 6, 2026Xiangzhao Hao, Zefeng Zhang, Zhenyu Zhang +6Unified Multimodal ModelsMultimodal Robustness

  20. Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning

    Apr 6, 2026Lei Zhang, Junjiao Tian, Zhipeng Fan +9Image GenerationT2I Generation