Organizations: Graduate School of Information Science and Electrical Engineering, Kyushu University, Fukuoka, 819-0395, Japan · Department of Civil Engineering, Sichuan University of Science and Engineering, Zigong, 643002, China · Faculty of Arts and Science, Kyushu University, Fukuoka, 819-0395, Japan
Vision-language models (VLMs) have shown strong multimodal capabilities across scientific and biomedical domains, yet their ability to understand electroencephalography (EEG) signals remains unclear. We introduce EEGProbe, a signal-grounded benchmark for evaluating whether general-purpose VLMs can extract and reason from minimally processed, channel-wise EEG without task-specific feature engineering or EEG-specific adaptation. EEGProbe contains 620 EEG-centered questions across 13 subtasks and three capability categories: EEG Signal Classification, EEG Cross-modal Alignment, and EEG Neurophysiological Reasoning. To reduce shortcut-based answering, EEGProbe preserves the temporal structure of the recordings where possible, minimizes benchmark-induced processing, and applies signal-necessity testing followed by expert verification. We evaluate 11 representative open- and closed-source VLMs under a unified zero-shot protocol. The strongest model, GPT-5.6 Sol, achieves an overall score of only 41.63%, with cross-modal EEG alignment emerging as a major bottleneck. We further examine model behavior from a neurophysiological perspective and analyze how temporal, spectral, spatial, and physiological EEG structure relates to model successes and failures. Inference-time neurophysiological guidance can reshape model predictions by helping models recognize the neurophysiological patterns present in the input and connect them with relevant neurological knowledge, narrowing the gap between general multimodal competence and EEG understanding. EEGProbe provides a testbed for current VLMs and a basis for developing future multimodal models that can interpret EEG evidence more reliably. The EEGProbe benchmark will be publicly available at https://huggingface.co/datasets/2122ZHU/EEGProbe.
Figures & tables
Figure 1: Overview of EEGProbe. The benchmark evaluates general-purpose vision-language models across three categories of EEG understanding: EEG Signal Classification, EEG Cross-modal Alignment, and EEG Neurophysiological Reasoning. EEGProbe contains 13 subtasks covering motor imagery, emotion, sleep, seizure, ERP, cross-modal matching with image, text, audio, and video stimuli, and reasoning about frequency, brain regions, temporal structure, and signal quality.
Figure 2: EEG data preparation and serialization pipeline in EEGProbe. EEG recordings undergo minimal source-aware preparation, followed by deterministic serialization of multichannel time series into structured numerical text. The serialized signals are then assembled into task-specific instances for the three benchmark dimensions.
Figure 3: Construction pipeline of EEGProbe. Candidate instances are collected from multiple EEG datasets and tasks (Phase 1), filtered through an EEG signal-necessity test (Phase 2), and then verified by domain experts for answer correctness and EEG grounding (Phase 3).
Figure 4: Representative task examples in EEGProbe. (a) EEG Signal Classification evaluates state and response recognition from EEG. (b) EEG Cross-modal Alignment assesses correspondence between EEG responses and external visual, textual, auditory, and video stimuli. (c) EEG Neurophysiological Reasoning evaluates spectral, spatial, temporal, and signal-quality interpretation.
Question Format
Response Requirement
Representative EEG Use
Multiple-choice
Models select the correct answer from four candidate options.
EEG Signal Classification and Cross-modal Alignment.
Closed-form
Models provide a constrained response, such as a category, numerical value, or short phrase.
Seizure detection, ERP recognition, temporal reasoning, and data quality assessment.
Open-ended response
Models provide a free-form interpretation grounded in the given EEG signal.
Frequency-band analysis and brain-region interpretation.
Table 1: Question formats used in EEGProbe.
Category
Abbr.
Subtask
# Instances
Question Format
EEG Signal Classification
MI
Motor Imagery Classification
50
Multiple-choice
ER
Emotion Recognition
50
Multiple-choice
SSC
Sleep Stage Classification
50
Multiple-choice
SD
Seizure Detection
50
Closed-form
ERP
Event-Related Potential Recognition
50
Closed-form
EEG Cross-Modal Alignment
EIM
EEG–Image Matching
50
Multiple-choice
Table 2: Task composition and question formats in EEGProbe.
Figure 5: Example of rubric-based evaluation for an open-ended EEGProbe question. Green text indicates content supported by the expert reference answer, while red text indicates incorrect or overstated claims.
EEG Signal Classification
EEG Cross-modal Alignment
Model
MI
ERP
ER
SSC
SD
EAM
EIM
ETM
EVM
GPT-5.6 Sol
34.00
56.00
28.00
34.00
64.00
22.00
24.00
18.00
24.00
Gemini-3.8-Flash †
24.00
62.00
26.00
34.00
58.00
24.00
36.00
26.00
22.00
DeepSeek-V4.1-Flash †
26.00
40.00
28.00
36.00
50.00
28.00
24.00
34.00
20.00
MiniMax-M3 †,§
24.00
48.00
24.00
28.00
50.00
20.00
30.00
20.00
30.00
GPT-5.6 Luna †
20.00
52.00
20.00
28.00
56.00
28.00
36.00
26.00
16.00
Table 3: Main results on EEGProbe across three evaluation categories and 13 subtasks. All values are percentages, and boldface indicates the best result in each column. Total is the primary metric. Subtasks are abbreviated as follows: MI, Motor Imagery Classification; ERP, Event-Related Potential Recognition; ER, Emotion Recognition; SSC, Sleep Stage Classification; SD, Seizure Detection; EAM, EEG–Audio Matching; EIM, EEG–Image Matching; ETM, EEG–Text Matching; EVM, EEG–Video Matching; BRI, Brain-Region Interpretation; DQA, Data Quality Assessment; TR, Temporal Reasoning; and FBA, Frequency-Band Analysis.
Figure 6: Benchmark composition and input characteristics of EEGProbe. (A) Distribution of model-visible EEG input shapes. (B) Organization of evaluation categories, subtasks, and response types. (C) EEG window-duration distributions across evaluation categories. (D) Distribution of task-response types.
Figure 7: Representative temporal patterns from EEGProbe source recordings. Signals are aligned to task-specific events or signal landmarks and standardized separately within each panel for visualization. The figure illustrates differences in temporal morphology and time scale; absolute amplitudes should not be compared across datasets.
Figure 8: Spectral characteristics of representative EEG regimes in EEGProbe. (A) Joint distribution of spectral centroid and aperiodic-adjusted peak prominence. (B) Subject-level distributions of spectral centroid. (C) Subject-level distributions of peak prominence above the fitted 1/f background. Each point represents a subject-level median; open circles indicate group medians, and the ellipses in (A) denote descriptive 80% covariance regions. These descriptors are used only for benchmark characterization and are not provided as model inputs.
Figure 9: Model capability profiles across EEGProbe. Models are ordered by descending Total score. (A) Scores across the 13 subtasks, grouped into EEG Signal Classification, EEG Cross-modal Alignment, and EEG Neurophysiological Reasoning. Cell values show raw scores (%), while color represents the within-subtask z -score across the 11 models. (B) Unweighted mean score within each evaluation category.
Figure 10: Subtask-level diagnostic map across the 11 evaluated models. Each subtask is positioned by its mean score and cross-model population standard deviation. Colors denote the three evaluation categories, while marker shapes distinguish exact-match from rubric-based evaluation. Higher cross-model dispersion indicates stronger empirical separation among models.
Figure 11: Neurophysiological patterns associated with motor-imagery classes. (A) Group-level scalp distributions of μ -band (8–13 Hz) and β -band (13–30 Hz) ERD/ERS for left-hand, right-hand, feet, and tongue imagery. Negative values indicate power suppression relative to the pre-cue baseline, whereas positive values indicate power increases. (B) Subject-level ERD/ERS across left (FC3–C3–CP3), midline (FCz–Cz–CPz), and right (FC4–C4–CP4) sensorimotor ROIs. Dots denote individual subjects, and diamonds indicate group means with 95% confidence intervals. The analysis uses 2,283 artifact-screened, benchmark-trial-disjoint trials from nine subjects in BCI Competition IV-2a.
Figure 12: Temporal, spatial, and representational organization of visual EEG evidence in benchmark-disjoint THINGS-EEG2 training recordings. (A) Stimulus-locked temporal profiles of image-specific and concept-level pattern-similarity contrasts. (B) Spatial–temporal distribution of visual evidence across posterior channel groups. (C) Representational similarity for repeated images, different images from the same concept, and images from different concepts. All ten participants exhibit the ordering same image > same concept but different image > different concept.
Method
Correct
Accuracy
Baseline
4/20
20.0%
Baseline + guidance
8/20
40.0%
Table 4: Held-out performance of GPT-5.6 Sol on EEG–image matching before and after neurophysiological guidance.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Criterion
Weight
Correctly identifies Signal A as having stronger posterior alpha activity.
0.30
Reports posterior regions or supporting channels consistent with the evidence, such as P3, P4, and P8.
0.25
Characterizes the effect as strong and correctly describes its posterior specificity.
0.25
Relates the pattern conservatively to alpha blocking or the Berger effect.
0.20
Total
1.00
Appendix
Table 5: Scoring rubric for the FBA example bandalpha_001 .
Criterion
Weight
Correctly identifies a late positive pattern with relative centroparietal prominence.
0.25
Reports positive polarity and a compatible peak latency near 371 ms ( ± 50 ms).
0.20
Uses sample-specific evidence such as P3, Pz, and CPz and the approximately 8.7 μ V centroparietal dominance.
0.25
Provides a conservative P3-like target-evaluation or context-updating interpretation.
0.20
Avoids definitive claims about component identity, source localization, or cognitive state.
0.10
Total
1.00
Appendix
Table 6: Scoring rubric for the BRI example rsni_v2_035 .
Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.
Electroencephalography (EEG) is widely used in clinical assessment, brain-computer interfaces, and cognitive-state analysis, yet EEG modeling remains fragmented across task paradigms. Recent multimodal large language models (MLLMs) provide a promising interface for flexible multimodal reasoning, but their ability to understand EEG inputs remains limited. In this work, we propose OneEEG, a unified EEG-MLLM framework for multi-task EEG understanding. By training on our curated OneEEG-Instruct, a multi-task EEG instruction-tuning dataset, OneEEG jointly models numerical EEG representations and visual EEG representations, and introduces a task-aware Mixture of Projectors (MoP) to align heterogeneous EEG features across different task families with the LLM embedding space. We further establish OneEEG-Bench, a benchmark for evaluating current MLLMs and our method on multi-task EEG understanding. Experiments show that existing MLLMs have limited ability to interpret EEG inputs, while OneEEG achieves stronger performance by integrating numerical and visual EEG representations with task-aware EEG-to-LLM projection. Extensive ablations and analyses further reveal the effects of visual EEG renderings, MoP variants, router behavior, and feature dispersion across EEG tasks.
Jiacheng Zhang, Xueyi Zhang, Xin Lu +2
School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China · Language, Intelligence, and Machine Center, Shenzhen Loop Area Institute, Shenzhen, China · School of Computing, National University of Singapore, Singapore +2
Historically automated electroencephalography (EEG) systems was done using task-specific supervised models trained on purposely annotated data. A more recent modeling paradigm relies on creating a unified representation, or a 'foundation model', for multiple modality-specific tasks through large-scale domain-specific pretraining on the unannotated EEG signal. The quality of this unified representation directly depends on the availability of the modality-specific datasets. At the same time, generalist Vision Language Models (VLMs) exhibit broad few-shot learning capacities as well as good fine-tuning adaptability to different visual and language tasks. Can VLMs learn how to read and interpret EEG visually, emulating to a degree the way human clinicians inspect the EEG patterns? In this work we make a case that they can: we show that generalist VLMs without large-scale EEG pretraining perform competitively across a collection of clinical neuro-diagnostic tasks, can adapt to different tasks in a few-shot learning setting better than specialist foundation models, and exhibit better performance with model scaling when modality-specific models fail to exhibit.
Xin Zhou, Elena Sergeeva, Nan Bi +3
Massachusetts General Hospital, Boston, MA, 02114, USA. · Binghamton University, State University of New York, Binghamton, NY, 13902, USA. · Harvard Medical School, Boston, MA, 02115, USA. +1
Learning EEG representations that generalize across cognitive tasks, subjects, and recording conditions remains a key challenge in electroencephalography (EEG) decoding. Recent advances in foundation models have improved EEG decoding performance, yet a fundamental open question remains: how to effectively interface neural signals with these models to enable multi-task learning across datasets. To investigate this question, we introduce BraVista, a visual-language framework that encodes multichannel EEG signals as structured images and enables multi-task learning through instruction-conditioned vision-language models (VLMs). Our approach relies on continued post-training of a general-domain VLM, leveraging its visual and linguistic priors to adapt to neural signals without a separate large-scale EEG-specific pretraining stage. We evaluate BraVista on four datasets spanning sleep staging, emotion recognition, cognitive workload classification, and abnormal EEG detection, showing strong performance across these tasks. Further analyses show that the choice of EEG-to-image representation is critical to performance. Moreover, through controlled perturbations of the EEG signal, we observe a gradual performance degradation under increasing noise, suggesting that the model relies on EEG-relevant information rather than superficial visual patterns. Together, these findings establish structured visual representations as an effective and scalable interface between neural signals and general-domain foundation models for unified multi-task EEG decoding.
Parastoo Azizeddin, Omid Sharafi, Maryam M. Shanechi
Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California · Thomas Lord Department of Computer Science, University of Southern California · Alfred E. Mann Department of Biomedical Engineering, University of Southern California +1