cs.LGNov 3, 2026

Recasting Visual Language Models as EEG Foundation Models for Clinical Neurodiagnostics

Authors: Xin Zhou, Elena Sergeeva, Nan Bi, Wenyu Song, Lijun Yin, Hao Deng

Organizations: Massachusetts General Hospital, Boston, MA, 02114, USA. · Binghamton University, State University of New York, Binghamton, NY, 13902, USA. · Harvard Medical School, Boston, MA, 02115, USA. · Brigham and Women’s Hospital, Boston, 02115, MA, USA.

Abstract

Historically automated electroencephalography (EEG) systems was done using task-specific supervised models trained on purposely annotated data. A more recent modeling paradigm relies on creating a unified representation, or a 'foundation model', for multiple modality-specific tasks through large-scale domain-specific pretraining on the unannotated EEG signal. The quality of this unified representation directly depends on the availability of the modality-specific datasets. At the same time, generalist Vision Language Models (VLMs) exhibit broad few-shot learning capacities as well as good fine-tuning adaptability to different visual and language tasks. Can VLMs learn how to read and interpret EEG visually, emulating to a degree the way human clinicians inspect the EEG patterns? In this work we make a case that they can: we show that generalist VLMs without large-scale EEG pretraining perform competitively across a collection of clinical neuro-diagnostic tasks, can adapt to different tasks in a few-shot learning setting better than specialist foundation models, and exhibit better performance with model scaling when modality-specific models fail to exhibit.

Figures & tables

Explore similar work

CardsList
  1. EEGProbe: Do Vision-Language Models Really Understand EEG? A Benchmark for Signal-Grounded EEG Signals Evaluation

    Nov 16, 2026Haozhe Zhu, Dan Yuan, Chenghao Gu +3VLM EvaluationMultimodal Model Evaluation

  2. Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding

    Oct 7, 2026Parastoo Azizeddin, Omid Sharafi, Maryam M. ShanechiEEG DecodingMultimodal Foundation Models