Multimodal Foundation Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 303

All topics

Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding
Oct 7, 2026Parastoo Azizeddin, Omid Sharafi, Maryam M. ShanechiEEG Decoding Multimodal Foundation Models
Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California · Thomas Lord Department of Computer Science, University of Southern California · Alfred E. Mann Department of Biomedical Engineering, University of Southern California +1
SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models
Oct 7, 2026Jeonghyeok Do, Munchurl KimRemote Sensing Image Understanding Multimodal Pretraining
Korea Advanced Institute of Science and Technology (KAIST)
RoMod: Temporal Routing Modulation via Mixture-of-Experts for Video Anomaly Detection
Oct 4, 2026Chao Huang, Pengfei Wei, Benfeng Wang +5Video Anomaly Detection Multimodal Foundation Models
Sun Yat-sen University · University of Macau
ShieldCLIP: Selective Safety Alignment for Harmful Content Mitigation in Multimodal Foundation Models
Sep 30, 2026Tobia Poppi, Silvia Cappelletti, Samuele Poppi +4Cross-Modal Alignment Multimodal Foundation Models
University of Modena and Reggio Emilia, Modena, Italy. · University of Pisa, Pisa, Italy. · MBZUAI, Abu Dhabi, United Arab Emirates. +1
Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology
Sep 29, 2026Junbo Shen, Jinying Gao, Bo LeiCross-Modal Alignment Multimodal Foundation Models
Department of Computer Science and Engineering, The Chinese University of Hong Kong · State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Beijing Academy of Artificial Intelligence, Beijing, China +1
AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models
Sep 28, 2026Konstantinos D. Polyzos, Eleni Oikonomou, Tara JavidiMultimodal Large Language Models Kernel Methods
University of California San Diego
QiYao-M: Multimodal Time Series Foundation Model with Role-Aware Modeling of Endogenous and Exogenous Modalities
Sep 28, 2026Hanyin Cheng, Linfeng Wang, Zhengbo Qu +7Multivariate Time Series Forecasting Multimodal Foundation Models
East China Normal University · Huawei Technologies Co., Ltd.
InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision
Sep 28, 2026Guanghao Zhu, Zeyu Liu, Zhitian Hou +9Multimodal Pretraining Multimodal QA
The Hong Kong Polytechnic University · PolyU-Daya Bay Technology and Innovation Research Institute
MAD-Guard: Controlled Study of Autoregressive Generation versus Direct Decision Interfaces for Closed Multimodal Forensic Tasks
Sep 27, 2026Hao ChenEfficient Multimodal Inference Image Forgery Detection
School of Cyber Security, Guangdong Polytechnic Normal University, Guangzhou, China
AstraLOD3: Zero-shot multimodal agentic reconstruction of LOD3 building models
Sep 23, 2026Bryan G. Pantoja-Rosero3D Reconstruction Multi-View 3D Reconstruction
Digital Intelligence for Structures and Construction (DISC) Laboratory, School of Civil and Environmental Engineering, Nanyang Technological University, Singapore
OmniFysics-Nano-V2 Technical Report: Understanding the Physical World Across Modalities
Sep 22, 2026Yizhou Liu, Jinghang Han, Kaixiang Qiu +8Multimodal Grounding Physical Reasoning
Physical Superintelligence Lab, Fysics AI · College of Intelligent Robotics and Advanced Manufacturing, Fudan University
A Language-Guided Multimodal Foundation Model for Zero-Shot and Multi-Task Brain Signal Analysis
Sep 14, 2026Mingzhi Chen, Yiyu Gui, Guibo Luo +1Cross-Modal Alignment Multimodal Foundation Models
New Cornerstone Science Laboratory, Guangdong Provincial Key Laboratory of In-Memory Computing Chips, School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University, Shenzhen, China · Center for Brain Inspired Intelligence, Chinese Institute for Brain Research (CIBR), Beijing, China · New Cornerstone Science Laboratory, Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, China
The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models
Sep 10, 2026Pengfei Zhang, Biao Tian, Xiangang Li +1Neural Decoding Multimodal Foundation Models
AI Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · Alibaba Token Hub, Alibaba Group, Beijing, China.
Beyond Similarity: Foundation Models as an Efficient Backbone for Training-Free Composed Video Retrieval
Sep 9, 2026Dmitry Demidov, Muhammad Zaigham Zaheer, Omkar Thawakar +2Multimodal Foundation Models Composed Video Retrieval
Mohamed bin Zayed University of Artificial Intelligence, UAE
NOAH: Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting
Sep 8, 2026Tobias Susetzky, Raphael Rehms, Dmitrii Seletkov +5Irregular Time-Series Modeling Clinical Prediction
Chair for AI in Healthcare and Medicine, TUM University Hospital and Technical University of Munich (TUM), Munich, Germany · Institute for Diagnostic and Interventional Radiology, TUM University Hospital, Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany +3
A Composable Evaluation System for Reproducible Omni-Modal Foundation Model Evaluation
Sep 1, 2026Hodong Lee, Sanghee Park, Dohoon Ryu +4ML Reproducibility Multimodal Foundation Models
NAVER Cloud AI · Korea University · KAIST AI +1
Fusion Anything: A Generalized Multimodal Foundation Model
Aug 16, 2026Huizi Cui, Zongbo Han, Chenggong Ding +6Unified Multimodal Models Multimodal Pretraining
Tianjin University · Beijing University of Posts and Telecommunications
CardioState-JEPA: Delay-Aware Cross-Modal Learning of a Shared Cardiac Representation
Aug 13, 2026Hamza Shafiq, Hung Manh Pham, Bin Zhu +3Cross-Modal Representation Learning Joint-Embedding Predictive Architecture
Eindhoven University of Technology, Netherlands · Singapore Management University, Singapore
Continuous-Latent Predictive Modeling with Semantic Alignment for EEG-Language Foundation Models
Aug 12, 2026Myeong-Ju Cho, Hye-Bin Shin, Seo-Hyun Lee +1EEG Decoding Predictive Representation Learning
Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea · Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea
Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation
Aug 11, 2026Dazhao Du, Shiyan Du, Jian Liu +8Temporal Video Grounding Multimodal Foundation Models
The Hong Kong University of Science and Technology · Tencent
SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception
Aug 11, 2026Yiyang Su, Jie Zhu, Feng Liu +2Biometric Identification Face Recognition
Michigan State University · Drexel University · University of North Carolina, Chapel Hill
VOICE: A Vision-Omics Foundation Model Integrating Direct and Retrieval-Based Prediction of In-situ Single-Cell Gene Expression
Aug 8, 2026Xin Luo, Yicheng Tao, Haoxuan Zeng +6Computational Pathology Spatial Transcriptomics
University of Michigan Ann Arbor, Michigan, USA · Weill Cornell Medicine New York, New York, USA
Old Tricks, New Models: How Simple Image Transformations Break Modern AI-based Content Moderation
Jul 30, 2026Marco Alecci, Francesco Marchiori, Iyiola Emmanuel Olatunji +2Multimodal Robustness Adversarial Attacks
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg · Department of Mathematics, University of Padua, Padua, Italy
IndustryForge-27B: A Domain-Enhanced Multimodal Foundation Model for Industrial CAD
Jul 30, 2026Nianchen Deng, Jiaxin Ai, Tao Hu +10Parametric CAD Modeling Text-to-CAD Generation
Shanghai Artificial Intelligence Laboratory · Wuhan University · Shanghai Innovation Institute +4
Color Fundus Photography Analysis: Co-evolution of Data, Preprocessing, and Modeling toward Multimodal AI
Jul 27, 2026Yu Li, Wengan He, Wenhui Xu +7Medical Imaging Foundation Models Medical Image Analysis
aZhuhai College of Science and Technology, Zhuhai, Guangdong, China · bZhuhai UNO technology Co., Ltd., Zhuhai, Guangdong, China
LunarFM: A Shared Multimodal Representation of the Moon's Surface
Jul 24, 2026Marc Girona-Mata, Jakob Gawlikowski, Sumit Goski +6Geospatial Representation Learning Multimodal Pretraining
German Aerospace Center (DLR) · European Space Resources Innovation Center (ESRIC) · Universidad de Antioquia
Scaling Native Multimodal Pre-Training From Scratch
Jul 24, 2026Haoyuan Wu, Aoqi Wu, Hai Wang +3Cross-Modal Learning Multimodal Pretraining
The Chinese University of Hong Kong · LLM Department, Tencent
OPOD: On-Policy Omni Distillation
Jul 23, 2026Tong Zhao, Yuyang Hu, Yutao Zhu +5Cross-Modal Knowledge Distillation Unified Multimodal Models
1Renmin University of China · 2Tencent
Look Less, Think Faster: Joint Token-Compute Adaptation for Multimodal LLMs
Jul 22, 2026Pengcheng Wang, Zhiquan Wang, Jayoung Lee +5Efficient Multimodal Inference Efficient VLM Inference
Purdue University, West Lafayette, IN, USA · University of Wisconsin–Madison, Madison, WI, USA · NVIDIA, Santa Clara, CA, USA
Test-Time Training for Modality Order Consistency in Vision-Language Models
Jul 22, 2026Aditi Gupta, Yossi GandelsmanVLM Robustness VLM Adaptation
University of Chicago · Reve

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding

Parastoo Azizeddin, Omid Sharafi, Maryam M. Shanechi

Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California · Thomas Lord Department of Computer Science, University of Southern California · Alfred E. Mann Department of Biomedical Engineering, University of Southern California+1

EEG Decoding Multimodal Foundation Models

Open paper AskPDF arXiv

Learning EEG representations that generalize across cognitive tasks, subjects, and recording conditions remains a key challenge in electroencephalography (EEG) decoding. Recent advances in foundation models have improved EEG decoding performance, yet a fundamental open question remains: how to effectively interface neural signals with these models to enable multi-task learning across datasets. To investigate this question, we introduce BraVista, a visual-language framework that encodes multichannel EEG signals as structured images and enables multi-task learning through instruction-conditioned vision-language models (VLMs). Our approach relies on continued post-training of a general-domain VLM, leveraging its visual and linguistic priors to adapt to neural signals without a separate large-scale EEG-specific pretraining stage. We evaluate BraVista on four datasets spanning sleep staging, emotion recognition, cognitive workload classification, and abnormal EEG detection, showing strong performance across these tasks. Further analyses show that the choice of EEG-to-image representation is critical to performance. Moreover, through controlled perturbations of the EEG signal, we observe a gradual performance degradation under increasing noise, suggesting that the model relies on EEG-relevant information rather than superficial visual patterns. Together, these findings establish structured visual representations as an effective and scalable interface between neural signals and general-domain foundation models for unified multi-task EEG decoding.

Momentum

Related topics

Latest papers 303