Multimodal Foundation Models

Ask about this topic

Momentum

13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 303

All topics
CardsList

  1. Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding

    Oct 7, 2026Parastoo Azizeddin, Omid Sharafi, Maryam M. ShanechiEEG DecodingMultimodal Foundation Models

    Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California · Thomas Lord Department of Computer Science, University of Southern California · Alfred E. Mann Department of Biomedical Engineering, University of Southern California +1

  2. SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models

    Oct 7, 2026Jeonghyeok Do, Munchurl KimRemote Sensing Image UnderstandingMultimodal Pretraining

    Korea Advanced Institute of Science and Technology (KAIST)

  3. RoMod: Temporal Routing Modulation via Mixture-of-Experts for Video Anomaly Detection

    Oct 4, 2026Chao Huang, Pengfei Wei, Benfeng Wang +5Video Anomaly DetectionMultimodal Foundation Models

    Sun Yat-sen University · University of Macau

  4. ShieldCLIP: Selective Safety Alignment for Harmful Content Mitigation in Multimodal Foundation Models

    Sep 30, 2026Tobia Poppi, Silvia Cappelletti, Samuele Poppi +4Cross-Modal AlignmentMultimodal Foundation Models

    University of Modena and Reggio Emilia, Modena, Italy. · University of Pisa, Pisa, Italy. · MBZUAI, Abu Dhabi, United Arab Emirates. +1

  5. Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology

    Sep 29, 2026Junbo Shen, Jinying Gao, Bo LeiCross-Modal AlignmentMultimodal Foundation Models

    Department of Computer Science and Engineering, The Chinese University of Hong Kong · State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Beijing Academy of Artificial Intelligence, Beijing, China +1

  6. AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models

    Sep 28, 2026Konstantinos D. Polyzos, Eleni Oikonomou, Tara JavidiMultimodal Large Language ModelsKernel Methods

    University of California San Diego

  7. QiYao-M: Multimodal Time Series Foundation Model with Role-Aware Modeling of Endogenous and Exogenous Modalities

    Sep 28, 2026Hanyin Cheng, Linfeng Wang, Zhengbo Qu +7Multivariate Time Series ForecastingMultimodal Foundation Models

    East China Normal University · Huawei Technologies Co., Ltd.

  8. InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision

    Sep 28, 2026Guanghao Zhu, Zeyu Liu, Zhitian Hou +9Multimodal PretrainingMultimodal QA

    The Hong Kong Polytechnic University · PolyU-Daya Bay Technology and Innovation Research Institute

  9. MAD-Guard: Controlled Study of Autoregressive Generation versus Direct Decision Interfaces for Closed Multimodal Forensic Tasks

    Sep 27, 2026Hao ChenEfficient Multimodal InferenceImage Forgery Detection

    School of Cyber Security, Guangdong Polytechnic Normal University, Guangzhou, China

  10. AstraLOD3: Zero-shot multimodal agentic reconstruction of LOD3 building models

    Sep 23, 2026Bryan G. Pantoja-Rosero3D ReconstructionMulti-View 3D Reconstruction

    Digital Intelligence for Structures and Construction (DISC) Laboratory, School of Civil and Environmental Engineering, Nanyang Technological University, Singapore

  11. OmniFysics-Nano-V2 Technical Report: Understanding the Physical World Across Modalities

    Sep 22, 2026Yizhou Liu, Jinghang Han, Kaixiang Qiu +8Multimodal GroundingPhysical Reasoning

    Physical Superintelligence Lab, Fysics AI · College of Intelligent Robotics and Advanced Manufacturing, Fudan University

  12. A Language-Guided Multimodal Foundation Model for Zero-Shot and Multi-Task Brain Signal Analysis

    Sep 14, 2026Mingzhi Chen, Yiyu Gui, Guibo Luo +1Cross-Modal AlignmentMultimodal Foundation Models

    New Cornerstone Science Laboratory, Guangdong Provincial Key Laboratory of In-Memory Computing Chips, School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University, Shenzhen, China · Center for Brain Inspired Intelligence, Chinese Institute for Brain Research (CIBR), Beijing, China · New Cornerstone Science Laboratory, Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, China

  13. The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models

    Sep 10, 2026Pengfei Zhang, Biao Tian, Xiangang Li +1Neural DecodingMultimodal Foundation Models

    AI Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · Alibaba Token Hub, Alibaba Group, Beijing, China.

  14. Beyond Similarity: Foundation Models as an Efficient Backbone for Training-Free Composed Video Retrieval

    Sep 9, 2026Dmitry Demidov, Muhammad Zaigham Zaheer, Omkar Thawakar +2Multimodal Foundation ModelsComposed Video Retrieval

    Mohamed bin Zayed University of Artificial Intelligence, UAE

  15. NOAH: Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting

    Sep 8, 2026Tobias Susetzky, Raphael Rehms, Dmitrii Seletkov +5Irregular Time-Series ModelingClinical Prediction

    Chair for AI in Healthcare and Medicine, TUM University Hospital and Technical University of Munich (TUM), Munich, Germany · Institute for Diagnostic and Interventional Radiology, TUM University Hospital, Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany +3

  16. A Composable Evaluation System for Reproducible Omni-Modal Foundation Model Evaluation

    Sep 1, 2026Hodong Lee, Sanghee Park, Dohoon Ryu +4ML ReproducibilityMultimodal Foundation Models

    NAVER Cloud AI · Korea University · KAIST AI +1

  17. Fusion Anything: A Generalized Multimodal Foundation Model

    Aug 16, 2026Huizi Cui, Zongbo Han, Chenggong Ding +6Unified Multimodal ModelsMultimodal Pretraining

    Tianjin University · Beijing University of Posts and Telecommunications

  18. CardioState-JEPA: Delay-Aware Cross-Modal Learning of a Shared Cardiac Representation

    Aug 13, 2026Hamza Shafiq, Hung Manh Pham, Bin Zhu +3Cross-Modal Representation LearningJoint-Embedding Predictive Architecture

    Eindhoven University of Technology, Netherlands · Singapore Management University, Singapore

  19. Continuous-Latent Predictive Modeling with Semantic Alignment for EEG-Language Foundation Models

    Aug 12, 2026Myeong-Ju Cho, Hye-Bin Shin, Seo-Hyun Lee +1EEG DecodingPredictive Representation Learning

    Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea · Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea

  20. Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation

    Aug 11, 2026Dazhao Du, Shiyan Du, Jian Liu +8Temporal Video GroundingMultimodal Foundation Models

    The Hong Kong University of Science and Technology · Tencent

  21. SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception

    Aug 11, 2026Yiyang Su, Jie Zhu, Feng Liu +2Biometric IdentificationFace Recognition

    Michigan State University · Drexel University · University of North Carolina, Chapel Hill

  22. VOICE: A Vision-Omics Foundation Model Integrating Direct and Retrieval-Based Prediction of In-situ Single-Cell Gene Expression

    Aug 8, 2026Xin Luo, Yicheng Tao, Haoxuan Zeng +6Computational PathologySpatial Transcriptomics

    University of Michigan Ann Arbor, Michigan, USA · Weill Cornell Medicine New York, New York, USA

  23. Old Tricks, New Models: How Simple Image Transformations Break Modern AI-based Content Moderation

    Jul 30, 2026Marco Alecci, Francesco Marchiori, Iyiola Emmanuel Olatunji +2Multimodal RobustnessAdversarial Attacks

    Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg · Department of Mathematics, University of Padua, Padua, Italy

  24. IndustryForge-27B: A Domain-Enhanced Multimodal Foundation Model for Industrial CAD

    Jul 30, 2026Nianchen Deng, Jiaxin Ai, Tao Hu +10Parametric CAD ModelingText-to-CAD Generation

    Shanghai Artificial Intelligence Laboratory · Wuhan University · Shanghai Innovation Institute +4

  25. Color Fundus Photography Analysis: Co-evolution of Data, Preprocessing, and Modeling toward Multimodal AI

    Jul 27, 2026Yu Li, Wengan He, Wenhui Xu +7Medical Imaging Foundation ModelsMedical Image Analysis

    aZhuhai College of Science and Technology, Zhuhai, Guangdong, China · bZhuhai UNO technology Co., Ltd., Zhuhai, Guangdong, China

  26. LunarFM: A Shared Multimodal Representation of the Moon's Surface

    Jul 24, 2026Marc Girona-Mata, Jakob Gawlikowski, Sumit Goski +6Geospatial Representation LearningMultimodal Pretraining

    German Aerospace Center (DLR) · European Space Resources Innovation Center (ESRIC) · Universidad de Antioquia

  27. Scaling Native Multimodal Pre-Training From Scratch

    Jul 24, 2026Haoyuan Wu, Aoqi Wu, Hai Wang +3Cross-Modal LearningMultimodal Pretraining

    1The Chinese University of Hong Kong · 2LLM Department, Tencent

  28. OPOD: On-Policy Omni Distillation

    Jul 23, 2026Tong Zhao, Yuyang Hu, Yutao Zhu +5Cross-Modal Knowledge DistillationUnified Multimodal Models

    1Renmin University of China · 2Tencent

  29. Look Less, Think Faster: Joint Token-Compute Adaptation for Multimodal LLMs

    Jul 22, 2026Pengcheng Wang, Zhiquan Wang, Jayoung Lee +5Efficient Multimodal InferenceEfficient VLM Inference

    Purdue University, West Lafayette, IN, USA · University of Wisconsin–Madison, Madison, WI, USA · NVIDIA, Santa Clara, CA, USA

  30. Test-Time Training for Modality Order Consistency in Vision-Language Models

    Jul 22, 2026Aditi Gupta, Yossi GandelsmanVLM RobustnessVLM Adaptation

    University of Chicago · Reve

  31. OmniReasoner: Thinking with Long Audio-Video via Native Tool Use

    Jul 21, 2026Yu Chen, Caorui Li, Ziyu Xiong +8Temporal Video GroundingAudio-Visual Understanding

    University of Chinese Academy of Sciences · Institute of Automation, CAS · Southeast University +3

  32. Fusion Embedding: A Unified Embedding Space for Text, Image, Video, and Audio

    Jul 21, 2026Abdul Basit Tonmoy, Kazi Fardinul Hoque, Md. Shahrier Islam Arham +1VLM AdaptationMultimodal Embedding

    1Eximius Labs · 2Wabash College · 3Skop Intelligence Co.

  33. Now We Know? A Systematic Comparison of TerraMind and THOR

    Jul 20, 2026Frederick Schindlegger, Kenzo Bounegta, Eva Gmelich Meijling +5Geospatial Foundation ModelsRemote Sensing

    University of Münster, Münster, Germany · 2ESA Φ-lab, ESRIN, Frascati, Italy · 3Rubicon, Paris, France +2

  34. RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model

    Jul 20, 2026Kehan Li, Bohan Hou, Minghao Zhu +28Robot Foundation ModelsCross-Embodiment Robot Learning

    1DAMO Academy, Alibaba Group

  35. TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

    Jul 19, 2026Yuhan Zhu, Changlian Ma, Xiangyu Zeng +12Temporal Video GroundingReinforcement Learning

    Nanjing University · Shanghai AI Laboratory · Shanghai Jiao Tong University +3

  36. Geometry-Enhanced Portion Estimation for Multimodal LLMs

    Jul 17, 2026Lin Liao, Peng LiMultimodal Large Language ModelsMultimodal Foundation Models

  37. One Modality to Forget Them All: Enhancing Cross-Modal Unlearning in Vision-Language Models

    Jul 17, 2026Sudharshan Balaji, Yili Ren, Guangjing Wang +2VLM UnlearningVLM Robustness

    University of South Florida · USA · University of Massachusetts Lowell

  38. Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos

    Jul 17, 2026Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar +19Audio-Visual UnderstandingMultimodal Large Language Models

    1NVIDIA, USA · University of Maryland, USA

  39. Monkey King Bang: A Unified Scientific Multimodal Foundation Model

    Jul 17, 2026Hesen Chen, Xinyu Su, Xiaomeng Yang +11AI for ScienceUnified Multimodal Models

    SAIS Team, Shanghai Academy of AI for Science

  40. Ask Twice, Look Twice: Prompt Echoing Resolves the Question-First Paradox in Vision-Language Models

    Jul 17, 2026Rakshanda Hassan Abhinandan, John Galeotti, Deva Ramanan +1Visual Question AnsweringVision-Language Models

    Carnegie Mellon University, USA

  41. AlphaWiSE: Adaptive Weight Interpolation for Continual Multimodal Representation Learning

    Jul 16, 2026Sarthak Jain, Qiran Hu, Zhen Zhu +1Cross-Modal Representation LearningCross-Modal Alignment

    University of Illinois Urbana-Champaign · Google DeepMind

  42. Video = World + Event Stream

    Jul 16, 2026Lianghua Huang, Zhi-Fan Wu, Yupeng Shi +24Audio-Visual UnderstandingStreaming Video Understanding

    Wan Team, Alibaba Group

  43. Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment

    Jul 16, 2026Harikrishnan P M, Goutham Vignesh, Ganesh Parab +4Multimodal GroundingEfficient Multimodal Inference

    Phi Labs, Quantiphi, India · Phi Labs, Quantiphi, Canada.

  44. A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi

    Jul 16, 2026A. C. Opus, J. Q. LuEfficient Transformer InferenceGPU Kernel Optimization

    Department of Physics, University of Puerto Rico, Mayaguez, PR 00680, USA

  45. RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination

    Jul 15, 2026Haotian Liang, Mingkang Chen, Yufei Huang +27Unified Multimodal ModelsRobot Foundation Models

    Tencent Robotics X Team · Futian Laboratory · Tencent Hy Team

  46. FM2^2: Unified Federated Foundation Models for Heterogeneous Multimodal Medical Imaging

    Jul 15, 2026Shengchao Chen, Ting ShuMultimodal Federated LearningCross-Modal Alignment

    School of Artificial Intelligence, Shenzhen University Shenzhen, China · Australian AI Institute, University of Technology Sydney Sydney, Australia

  47. Boogu-Image-0.1: Boosting Open-Source Unified Multimodal Understanding and Generation

    Jul 14, 2026Guoxuan Chen, Chufeng Xiao, Haoran Yang +30Text-Guided Image EditingUnified Multimodal Models

    Boogu Team

  48. Hy-Embodied-VLM-1.0: Efficient Physical-World Agents

    Jul 14, 2026Ziyi Wang, Xumin Yu, Yongming Rao +19Efficient VLA ModelsMultimodal Foundation Models

    Tencent Robotics X Hy Vision Team Futian Laboratory

  49. Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model

    Jul 13, 2026Xinghang Li, Jun Guo, Qiwei Li +21Robot Foundation ModelsCross-Embodiment Robot Learning

    Xiaomi Robotics

  50. MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

    Jul 13, 2026Yuliang Liu, Zhang Li, Ziyang Zhang +11Document ParsingMultimodal Foundation Models

    1Huazhong University of Science and Technology · 2Kingsoft Office

  51. LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models

    Jul 13, 2026Gangsu Kim, Won-Ki JeongComputational PathologyPathology Foundation Models

    Department of Computer Science and Engineering, College of Informatics, Korea University, Republic of Korea

  52. Model Guides You How to Draw: Adaptive Visual Gating for Unified Multimodal Reasoning

    Jul 10, 2026Wenxi Gao, Guanxi Lu, Didi Zhu +5Efficient Multimodal InferenceUnified Multimodal Models

    Imperial College London · Tsinghua University

  53. A Foundation Model for Multimodal Event Sequences in Financial Applications

    Jul 10, 2026Nikita Rusakov, Vladislav Meshkov, Konstantin Zorin +4Financial ServicesMultimodal Pretraining

    Sber Moscow, Russian Federation · Sber AI Lab Moscow, Russian Federation

  54. ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts

    Jul 10, 2026Jiawen Li, Tian Guan, Huijuan Shi +5Computational PathologyUnified Multimodal Models

    Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Department of Pathology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China · Department of Engineering Science, University of Oxford, Oxford, UK +1

  55. Test-Time Scaling for Small VLMs on Multilingual Visual MCQ

    Jul 10, 2026Spiros Baxevanakis, Peng-Jian YangVisual Question AnsweringTest-Time Scaling for VLMs

    University of Amsterdam, Science Park 904, 1098 XH Amsterdam, The Netherlands

  56. Mixture of Probes: Learning from Privileged Modalities in Multimodal LLMs Through Probing

    Jul 9, 2026Dominick Reilly, Qiyu Wu, Hiromi Wakaki +2Missing-Modality LearningCross-Modal Representation Learning

    Sony Group Corporation · University of North Carolina at Charlotte · Sony AI

  57. FabriVLA: A Lightweight Vision-Language-Action Model for Precise Multi-Task Manipulation

    Jul 9, 2026Shiyuan Yang, Borong Zhang, Jizheng Zhang +5Efficient VLA ModelsRobotic Manipulation

    University of Macau · 2Mese Technology Limited Co., Ltd. · 3FabriX team at Youibot Robotics Co., Ltd.

  58. Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning

    Jul 9, 2026Yiyang Fang, Pei Fu, Jinjie Li +7Efficient Multimodal InferenceRL for Language Model Reasoning

    Wuhan University · The Hong Kong University of Science and Technology · Xiaomi Inc +2

  59. DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models

    Jul 9, 2026Pengjie Wang, Linger Deng, Zujia Zhang +6Efficient Multimodal InferenceUnified Multimodal Models

    Huazhong University of Science and Technology · MiLM Plus, Xiaomi Inc.

  60. Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks

    Jul 8, 2026Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu +1VLM UnlearningMLLM Unlearning

    University of Maryland, Baltimore County · University of Notre Dame · UNC Chapel Hill