Overview

Unified multimodal models study how one system can process and generate text, images, video, audio, and sometimes spatial or action data. Common approaches use shared tokenizers and autoregressive or diffusion-based backbones, often alongside modality-specific encoders, decoders, or experts; training may combine supervised instruction tuning, alignment, and reinforcement learning. Evaluation spans tasks such as visual question answering, text-to-image generation, video understanding, editing, and cross-modal consistency. A central challenge is that shared architecture does not guarantee useful transfer: understanding and generation can interfere or remain semantically misaligned, while efficiency, safety, and reliable evaluation across interacting modalities remain open concerns.

Latest papers

CardsList

314 papers26 papers in the last four weeks, down 33% on the four weeks before. 0.2% of all new papers.

  1. Modaliff: A Unified Token Diffusion Model for Any-to-Any Generation

    Oct 18, 2026Duc Kieu, Kien Do, Minh Khoa Le +3Multimodal GenerationUnified Multimodal Models

  2. FlexUMM: Flexible Representation Learning for Unified Multimodal Models

    Oct 15, 2026Yinyi Luo, Haoyue Bai, Yi Ge +3Task InterferenceUnified Multimodal Models

  3. PW-Pruning: Phase-Aware Expert Selection for Speech-Centric Omni-Modal Models

    Oct 15, 2026Junjie Cao, Fengping Tian, Wenjie Zhou +5Mixture-of-Experts PruningMixture-of-Experts Models

  4. ReasonUni: Reasoning-Augmented Reconstruction Advances Unified Multimodal Models

    Oct 14, 2026Feize Wu, Zhiyuan Yan, Yuhuan Hong +10Unified Multimodal ModelsMultimodal Generation

  5. What Generation Knows: Unlocking Fine-Grained Understanding in Unified Multimodal Models

    Oct 14, 2026Tengju Ru, Weitong Lian, Zecong Tang +6Multimodal UnderstandingUnified Multimodal Models

  6. Scaling Unified Multimodal Models to Long-Horizon Interleaved Generation

    Oct 12, 2026Ziqiao Peng, Yutao Cui, Wang Lin +12Multimodal GenerationUnified Multimodal Models

  7. OmniRSI: Taming Latent Diffusion for Unified Multimodal Remote Sensing Image Generation

    Oct 11, 2026Hebaixu Wang, Jing Zhang, Haonan Guo +3Multimodal GenerationUnified Multimodal Models

  8. Understanding Unified VLMs: From Attention Probing to Guidance Tuning for Multimodal Generation

    Oct 10, 2026Zhengyang Luo, Xiaoyu Qiu, Zhiheng Tan +3Unified Multimodal ModelsMultimodal Generation

  9. TartanMatch: Towards Universal Dense Correspondence Across Modalities

    Oct 10, 2026Hyeokjoon Kwon, Jiting Cai, Ruogu Li +7Cross-Modal Feature MatchingUnified Multimodal Models

  10. CrossEdit: Cross-Modal Training Enables Rich Audio-Visual Editing

    Oct 7, 2026William Chen, Prem Seetharaman, Ke Chen +10Instruction-Based Video EditingUnified Multimodal Models

  11. Visual Abstention in Unified Multimodal Models

    Oct 6, 2026Chufan Shi, Cheng Yang, Tiannuo Yang +4Unified Multimodal ModelsImage Editing

  12. The Poisoned Conversation: Privacy-Leaking Watermarks in Unified Multimodal Models

    Oct 4, 2026Tobias Braun, Jonas Henry Grebe, Emil Sivic +4Unified Multimodal ModelsMultimodal Generation

  13. Recursive Self-Improvement in Unified Multimodal Models

    Oct 2, 2026Huijuan Wang, Chufan Shi, Cheng Yang +3Unified Multimodal ModelsChart Understanding

  14. Octrees as an Explicit 3D Language

    Oct 1, 2026Ran Dan, Si-Tong Wei, Pengfei Xiong +33D Shape GenerationUnified Multimodal Models

  15. Gestalt: Large Multimodal Interplay Model

    Sep 30, 2026Zequn Yang, Yu Miao, Haotian Ni +8Unified Multimodal ModelsCross-Modal Alignment

  16. Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces

    Sep 30, 2026Hongyuan Tao, Xinggang Wang, Lianghui Zhu +7Unified Multimodal ModelsMultimodal Pretraining

  17. UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement

    Sep 30, 2026Fang Wu, Da Xing, Yanjie Huang +16Unified Multimodal ModelsOn-Policy Self-Distillation

  18. Beyond Layers: Position-Resolved Gradient Conflict and Position-Aware Modulation for Unified Multimodal Models

    Sep 29, 2026Shuyang Jiang, Fucheng Deng, Yuchuan Luo +1Unified Multimodal ModelsGradient Interference

  19. Does Gradient Conflict Predict the Understanding--Generation Trade-off? A Controlled Audit of Conflict-Metric Validity in Unified Multimodal Models

    Sep 29, 2026Shuyang Jiang, Fucheng Deng, Yuchuan Luo +1Unified Multimodal ModelsGradient Interference

  20. UniAfford: Token-Routed Multitask Learning for Generalizable 2D-3D Affordance Perception

    Sep 29, 2026Yuhao Liu, Yiming Zhong, Hanqing Wang +9Cross-Modal Representation LearningUnified Multimodal Models

  21. Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Models

    Sep 28, 2026Wentao Zhou, Weijie Gan, Jiayun WangAdversarial TrainingUnified Multimodal Models

  22. Learning Native Reflection in Unified Models with Interleaved Reinforcement Learning

    Sep 28, 2026Yijia Fan, Ziqi Huang, Zhongang Cai +5Unified Multimodal ModelsImage Generation