Multimodal Foundation Models

Ask about this topic

Momentum

13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 303

All topics
CardsList

  1. GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents

    Apr 29, 2026GLM-V Team, :, Wenyi Hong +95Multimodal Large Language ModelsMultimodal Reasoning

  2. Beyond Accuracy: Benchmarking Cross-Task Consistency in Unified Multimodal Models

    Apr 27, 2026Weixing Wang, Liudvikas Zekas, Anton Hackl +5Unified Multimodal ModelsCross-Modal Alignment

    Hasso Plattner Institute / University of Potsdam, Potsdam, Germany

  3. Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence

    Apr 27, 2026NVIDIA, :, Amala Sanjay Deshmukh +216Efficient Multimodal InferenceMultimodal Large Language Models

  4. Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation

    Apr 27, 2026Zhiheng Liu, Weiming Ren, Xiaoke Huang +12Unified Multimodal ModelsMultimodal Pretraining

    Meta AI · University of Waterloo · The University of Hong Kong

  5. X2SAM: Any Segmentation in Images and Videos

    Apr 27, 2026Hao Wang, Limeng Qiao, Chi Zhang +4Image SegmentationVideo Object Segmentation

    Sun Yat-Sen University · 2 Peng Cheng Laboratory · 3 Meituan Inc

  6. ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward

    Apr 23, 2026Jingpei Wu, Xiao Han, Weixiang Shen +3Process Reward ModelsMultimodal QA

    LMU Munich · Harvard University · University of Cambridge +2

  7. Context Unrolling in Omni Models

    Apr 23, 2026Ceyuan Yang, Zhijie Lin, Yang Zhao +16Cross-Modal LearningUnified Multimodal Models

  8. Latent Denoising Improves Visual Alignment in Large Multimodal Models

    Apr 23, 2026Dhruv Parikh, Jacob Fein-Ashley, Rajgopal Kannan +1VLM RobustnessMultimodal Large Language Models

    University of Southern California Los Angeles, CA, USA · DEVCOM ARL Army Research Office USA

  9. Focus Session: Hardware and Software Techniques for Accelerating Multimodal Foundation Models

    Apr 23, 2026Muhammad Shafique, Abdul Basit, Muhammad Abdullah Hanif +3Model CompressionEfficient Multimodal Inference

    eBRAIN Lab, New York University (NYU) Abu Dhabi, Abu Dhabi, United Arab Emirates

  10. ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Response

    Apr 23, 2026Stephan Xie, Ben Cohen, Mononito Goswami +6Time Series ReasoningTime Series QA

    Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA, USA · Datadog AI Research, New York, NY, USA · Amazon Web Services, Seattle, WA, USA

  11. LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model

    Apr 22, 2026Inclusion AI, Tiwei Bie, Haoxing Chen +15Unified Multimodal ModelsMultimodal Generation

    AGI Research Center, Inclusion AI

  12. Exploring Spatial Intelligence from a Generative Perspective

    Apr 22, 2026Muzhi Zhu, Shunyao Jiang, Huanyi Zheng +9Spatial Reasoning BenchmarksUnified Multimodal Models

    Zhejiang University, State Key Laboratory of CAD & CG · Ant Group · Zhejiang University of Technology +1

  13. SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models

    Apr 21, 2026Josue Torres-Fonseca, Naihao Deng, Yinpei Dai +5LLM Safety BenchmarksLarge Language Model-Based Robot Planning

    University of Michigan · Boise State University

  14. PLaMo 2.1-VL Technical Report

    Apr 21, 2026Tommi Kerola, Yuya Masuda, Takashi Masuko +5Visual Question AnsweringVision-Language Models

    Preferred Networks, Inc. (PFN)

  15. DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoning

    Apr 20, 2026Abrar Majeedi, Zhiyuan Ruan, Ziyi Zhao +3Cross-Modal LearningVisual Question Answering

    University of Wisconsin-Madison · Amazon · Northeastern University

  16. Multiplication in Multimodal LLMs: Computation with Text, Image, and Audio Inputs

    Apr 20, 2026Samuel G. Balter, Ethan Jerzak, Connor T. JerzakMultimodal Large Language ModelsMultimodal Reasoning

    University of Texas at Austin · National University of Singapore (NUS)

  17. Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs

    Apr 20, 2026Tingchao Fu, Wenkai Wang, Fanxiao Li +6Knowledge EditingMultimodal Large Language Models

    School of Information Science and Engineering, Yunnan University · School of Software, Yunnan University · 3National University of Singapore +1

  18. Cross-Modal Attention Analysis and Optimization in Vision-Language Models: A Study on Visual Reliability

    Apr 19, 2026Lijie ZhouCross-Modal LearningVLM Robustness

    School of Computer Science University of Nottingham Ningbo China Ningbo, China

  19. Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language Models

    Apr 18, 2026Xinru Yan, Boxi Cao, Yaojie Lu +4Unified Multimodal ModelsNeural Network Interpretability

    University of Chinese Academy of Sciences, Beijing, China · Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China

  20. π0.7π_{0.7}: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities

    Apr 16, 2026Physical Intelligence, Bo Ai, Ali Amin +85Robot Policy LearningRobot Skill Learning

  21. RaTA-Tool: Retrieval-based Tool Selection with Multimodal Large Language Models

    Apr 16, 2026Gabriele Mattioli, Evelyn Turri, Sara Sarto +4Multimodal Foundation ModelsTool Selection

    University of Modena and Reggio Emilia, Italy · University of Pisa, Italy

  22. Foundation Models in Robotics: A Comprehensive Review of Methods, Models, Datasets, Challenges and Future Research Directions

    Apr 16, 2026Aggelos Psiris, Vasileios Argyriou, Evangelos K. Markakis +4Large Language Model-Based Robot PlanningRobot Foundation Models

    Department of Informatics and Telematics, Harokopio University of Athens, Athens, Greece · Department of Networks and Digital Media, Kingston University, London, United Kingdom · Department of Electrical and Computer Engineering, Hellenic Mediterranean University, Heraklion, Greece +4

  23. CoLA: Cross-Modal Low-rank Adaptation for Multimodal Downstream Tasks

    Apr 1, 2026Wish Suharitdamrong, Tony Alex, Muhammad Awais +1Cross-Modal LearningVLM Adaptation

    Surrey Institute for People-Centred AI, University of Surrey, Guildford, GU2 7XH, UK · Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK

  24. UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomy

    Mar 25, 2026Yicheng Xu, Jiangning Zhang, Zhucun Xue +5Multimodal ICLUnified Multimodal Models

    1Zhejiang University, China. · 2Shanghai Jiaotong University, China. · 3National University of Singapore, Singapore. +1

  25. UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation

    Mar 23, 2026Ziyi Wang, Xinshun Wang, Shuang Chen +2Cross-Modal LearningUnified Multimodal Models

    State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School, China · Donghua University, China · School of Automation Science and Engineering, South China University of Technology

  26. ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models

    Mar 19, 2026Thomas De Min, Subhankar Roy, Stéphane Lathuilière +2Multimodal Large Language ModelsAI Agent Benchmarks

    University of Trento · University of Bergamo · Inria, Univ. Grenoble Alpes, CNRS, LJK +1

  27. Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs

    Mar 14, 2026Nimrod Shabtay, Moshe Kimhi, Artem Spector +5Tool Use in VLMsEfficient VLM Inference

    IBM Research · Tel-Aviv University · Technion +1

  28. Less Data, Faster Convergence: Goal-Driven Data Optimization for Multimodal Instruction Tuning

    Mar 12, 2026Rujie Wu, Haozhe Zhao, Hai Ci +1Video UnderstandingInstruction-Tuning Data Selection

    School of Computer Science, Peking University, Beijing, China · Institute for Artificial Intelligence, Peking University, Beijing, China · State Key Laboratory of General Artificial Intelligence, Peking University, Beijing, China +2

  29. Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion

    Mar 6, 2026Lijiang Li, Zuwei Long, Yunhang Shen +6Masked Diffusion ModelsUnified Multimodal Models

    Nanjing University · Tencent Youtu Lab · CASIA

  30. Designing UNICORN: a Unified Benchmark for Imaging in Computational Pathology, Radiology, and Natural Language

    Mar 3, 2026Michelle Stegeman, Lena Philipp, Fennie van der Graaf +20Computational PathologyMedical Imaging Foundation Models

    and on behalf of the UNICORN consortium · Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands · Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands +2