Towards Modality-Agnostic Seizure Classification: A Unified Deep Learning Framework for Heterogeneous EEG and Intracranial EEG Signals

Electroencephalography (EEG) models used for epilepsy are often limited to specific datasets and tasks. This limited approach can make it challenging to apply these models across different datasets or in various situations. However, recent studies in foundation models and self-supervised learning suggest that an adaptable EEG backbone could support a range of EEG related tasks. In this study, we have developed a multimodal EEG foundation model that combines a raw signal encoder based on the Mamba architecture, a Vision Transformer (ViT)-style encoder for time-frequency data, and a lightweight encoder for text, all within a shared embedding space. The pretraining process relies on several innovative techniques, such as masked modeling, cross-view contrastive alignment, and temporal consistency losses. These methods are designed to create rich, seizure-relevant representations without requiring labeled data. To assess the efficacy and generalization of our pretrained model, we fine-tuned it on the canonical CHB-MIT seizure detection benchmark and additional seizure detection datasets, and conducted extensive experiments comparing different model variants. On the standard CHB-MIT split, our best single model achieved an AUROC of 0.874, and an ensemble variant reached 0.878 AUROC, representing state-of-the-art performance on this benchmark. In addition to standard train-test splits, we evaluated performance under a leave-one-subject-out (LOSO) protocol, which is rarely reported in prior EEG seizure modeling work and highlights the difficulty of patient-independent seizure detection, with a mean LOSO balanced accuracy of 0.558 across 19 subjects. Across datasets and evaluation settings, our multimodal foundation model enabled robust seizure detection and straightforward adaptation to new seizure detection scenarios, while also supporting interpretable seizure localization.

Automated seizure detection from electroencephalography (EEG) is essential for continuous neurological monitoring, particularly for subclinical epileptic seizures that may exhibit only subtle electrographic changes. Existing time-series methods are often designed for fixed EEG channel configurations, thereby limiting their applicability to heterogeneous EEG recordings with irregular channel layouts. Although visual and language modeling offer promising alternatives, aligning heterogeneous EEG representations with clinical semantics remains challenging. We introduce ViTexSZ, a heterogeneous Vision-Text knowledge distillation framework for EEG seizure detection. ViTexSZ converts EEG recordings into structured waveform images and introduces a query-based multi-channel alignment module that maps source-dependent visual features into a unified token space. A heterogeneous teacher further integrates the aligned EEG representations with clinical prompts through a multimodal large language model, associating high-level clinical semantics with seizure-related evidence. Vision-text knowledge distillation then transfers the teacher representations to a lightweight student during detection. Experiments on four EEG seizure datasets demonstrate the generalizability of ViTexSZ across both subclinical and general seizure detection scenarios, achieving the highest accuracy on all datasets and relative improvements of up to 12.9% over the second-best baselines, showing its effectiveness.

Neonatal seizures carry substantial risks of acute brain injury and long-term neurodevelopmental impairment, motivating the need for timely identification of impending seizure activity. While continuous electroencephalogram (EEG) monitoring remains the clinical gold standard, reliable patient-independent seizure prediction in neonates remains largely under-explored. We propose an interpretable multimodal framework that combines EEG and electrocardiogram (ECG) signals for neonatal seizure prediction. Our method uses mel-frequency cepstral coefficient (MFCC) representations processed by a lightweight (approx. 70k-parameter) convolutional neural network with squeeze-and-excitation attention. The framework was evaluated on the Helsinki neonatal EEG-ECG dataset and the Siena scalp EEG-ECG dataset. Under strict leave-one-patient-out (LOPO) evaluation, patient-independent prediction proved challenging (mean accuracy 58.5%, F1-score 60.3%), highlighting substantial inter-subject variability in neonatal EEG. However, fine-tuning the pretrained model with only five minutes of patient-specific data per class improved performance to 95.87% accuracy and 95.85% F1-score, demonstrating the feasibility of rapid patient-adaptive transfer learning. SHapley Additive exPlanations (SHAP) identified informative EEG channels that showed partial spatial agreement with clinically reported seizure origins. These findings suggest that fully patient-independent neonatal seizure prediction remains challenging, but that patient-adaptive fine-tuning with a small amount of data may provide a practical pathway toward personalized neonatal seizure prediction systems.