cs.LGSep 26, 2026

Benchmarking EEG Foundation Models at Scale: Lessons from 20,000 Evaluations

Authors: Zhige Chen, Shu Peng, Chengxuan Qin, Rui Liu, Rui Yang, Kay Chen Tan, Jibin Wu

Organizations: Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University · Department of Computer Science, University of Liverpool · Academy of Artificial Intelligence and Advanced Technology, Xi’an Jiaotong-Liverpool University

Abstract

Electroencephalography (EEG) foundation models (FMs) promise transferable neural representations, yet their advantages over strong supervised baselines and their prospects for further scaling remain unclear. To address these questions, we introduce EEG-Arena, an open-source benchmark covering 30 EEG FMs and 25 supervised baselines evaluated on 57 downstream tasks from 23 public datasets. Through more than 20,000 evaluations across five experimental protocols, we assess downstream performance, pretraining benefits, model size scaling, pretraining data scaling, and robustness to channel configuration. We find that (1) EEG FMs outperform strong task-specific supervised baselines on most evaluated tasks, particularly under non-bipolar settings; (2) compared with architecture-matched supervised training from scratch, pretraining improves both early optimization and final downstream performance, with larger and more consistent gains as more labeled downstream data become available; (3) existing EEG FMs do not exhibit a consistent positive relationship between parameter count and downstream performance; (4) under a fixed architecture, increasing the pretraining data scale yields sustained downstream gains; and (5) channel-flexible FMs achieve higher absolute performance than channel-constrained models across most evaluated channel configurations. Together, these findings demonstrate the downstream value of EEG FMs and identify pretraining data expansion as a promising direction for further progress. To support continued research, we release EEG-Arena as an open-source evaluation framework that provides shared infrastructure for reproducible benchmarking, model comparison, and community-driven development.

Figures & tables

Appendix figures & tables74 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. OmniEEG-Bench: A Standardized Evaluation Benchmark for EEG Foundation Models

    May 30, 2026Ziling Lu, Zongsheng Li, Xinke Shen +11Brain-Computer InterfacesBenchmark Design

  2. A Multi-dimensional Framework for Evaluating Generalization in EEG Foundation Models

    May 27, 2026Aditya Kommineni, Emily Zhou, Kleanthis Avramidis +2Neural Network GeneralizationEEG Decoding

  3. NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces

    May 14, 2026Konstantinos Kontras, Trui Osselaer, Stylianos G. Mouslech +12ElectroencephalographyBrain-Computer Interfaces