Towards Modality-Agnostic Seizure Classification: A Unified Deep Learning Framework for Heterogeneous EEG and Intracranial EEG Signals

Ramya Sundaram; Muhammad Aslam Jarwar; M Uma; Sunder Ali Khowaja; Qurat-Ul-Ain Mastoi

cs.LGOct 12, 2026

Towards Modality-Agnostic Seizure Classification: A Unified Deep Learning Framework for Heterogeneous EEG and Intracranial EEG Signals

Authors: Ramya Sundaram, Muhammad Aslam Jarwar, M Uma, Sunder Ali Khowaja, Qurat-Ul-Ain Mastoi

Organizations: School of Computing and Digital Technologies, Sheffield Hallam University, Sheffield and University of Strathclyde, Glasgow, United Kingdom. · School of Computing and Digital Technologies, Sheffield Hallam University, Sheffield, United Kingdom · Department of Cmputational Intelligence, SRM Institute of Science and Technology,Kattankulathur-603203, India. · Faculty of Engineering and Computing, Dublin City University, Ireland. ·

Abstract

Seizure prediction and detection, essential for clinical treatment of epilepsy, remain the crucial task of this neurological disease. EEG and iEEG are widely used to analyze epilepsy due to its capability of capturing discriminative brain activities at high time resolution. However, existing works generally analyze either EEG or iEEG seizure activity separately, where EEG analysis has been well-researched, while more and more works consider iEEG for its higher specificity at the spatial level and signal quality. Currently, most of the existing seizure prediction systems are single-modal in nature, which limits its application to a variety of clinical conditions. Therefore, a common framework to analyze seizures with both EEG and iEEG signals needs to be developed. This paper presents a new framework of multimodal seizure classification integrating the heterogeneous EEG and iEEG recordings from different subjects and sources. A specific framework, called Unified Multimodal Signal Harmonization (UMSH), is developed for harmonizing signals with varied sampling rates and recording conditions. Moreover, to handle the extremely class-imbalanced problem in the data sets, we use a Conditional Generative Adversarial Network (cGAN) based data augmentation technique to generate synthetic samples to balance the class distribution. The proposed framework uses the EEGNet-Vision Transformer (EEGNet-ViT) as the feature extraction and classification network to learn the local discriminative patterns as well as global contextual information in the multimodal seizure data. Experimental results demonstrate the effectiveness of the proposed framework for robust seizure classification, emphasizing the potential of integrated EEG-iEEG multimodal learning in advancing next-generation epilepsy monitoring and clinical decision-support systems.

Figures & tables

Fig. 1: General block diagram of the proposed methodology.

Fig. 2: Proposed Multimodel dataset creation (Unified Multimodel Signal Harmonization.

Fig. 3: General block diagram of the EEGNet-ViT Architecture for multimodal classification.

Name	Original	After Down-sampling	Merging 4 Consecutive Records	Final Dataset
DatasetFnusa – Non seizure	$66581\times 15000$	$66581\times 768$	$66581/4=16645.25$	$16645\times 3072$
DatasetFnusa – Ictal	$18184\times 15000$	$18184\times 768$	$18184/4=4546.00$	$4546\times 3072$
DatasetMayo – Non seizure	$44259\times 15000$	$44259\times 768$	$44259/4=11064.75$	$11064\times 3072$
DatasetMayo – Ictal	$6099\times 15000$	$6099\times 768$	$6099/4=1524.75$	$1524\times 3072$

TABLE I: Dataset Details of the iEEG Dataset

Final Combined Dataset	Records	Total
Non-seizure (iEEG only)	27,709	27,709
Ictal (iEEG + CHB-MIT)	6,070 + 885	6,925
Preictal (CHB-MIT only)	8,613	8,613

TABLE II: Dataset Details of the EEG-iEEG Combined Dataset

Fig. 4: Performance analysis of proposed EEG–iEEG Seizure Classification Framework on Training, Testing, and Validation Datasets.

Fig. 5: Confusion matrix of proposed EEG–iEEG seizure classification model’s performance across training, testing, and validation datasets.

Fig. 6: Performance Comparison of the Proposed Framework with Existing State-of-the-Art Methods.

Layer	Output Shape	Parameters
InputLayer	(None, 3072, 1, 1)	0
Conv2D (F1, kernel_size = (64,1))	(None, 3072, 1, 16)	1024
BatchNormalization	(None, 3072, 1, 16)	64
Activation (ELU)	(None, 3072, 1, 16)	0
DepthwiseConv2D (kernel_size = (1,16), depth_multiplier = D)	(None, 3072, 1, 16)	16
BatchNormalization	(None, 3072, 1, 16)	64

TABLE III: EEGNet-Transformer Layer Configuration

Parameter	Value
$F_{1}$	16
$D$	2
$F_{2}$	32
$d_{rate}$	0.4
$embed_{dim}$	64
$num_{heads}$	4

TABLE IV: Parameter Details of the Proposed Model

Class	Source	Total	Grand Total
Non-seizure (iEEG)	Original – DatasetFnusa (First 8300) + DatasetMayo (First 8300)	16600	16600
Ictal (iEEG-Synthetic)	CGAN Generated	8300	83
Ictal (CHB-MIT-Synthetic)	CGAN Generated	8300	83
Preictal (CHB-MIT-Synthetic)	CGAN Generated	16600	16600
Total			49800

TABLE V: Training Dataset Used for Model Development

Class	Source	Total	Grand Total
Non-seizure (iEEG)	Remaining DatasetFnusa + DatasetMayo Samples	8344 + 2763	11107
Ictal (iEEG-Synthetic)	CGAN Generated (Unused in Training)	5500	11000
Ictal (CHB-MIT-Synthetic)	CGAN Generated (Unused in Training)	5500
Preictal (CHB-MIT-Synthetic)	CGAN Generated (Unused in Training)	11000	11000
Total			33107

TABLE VI: Validation Dataset Used for Model Evaluation

Modalities	Accuracy (%)
EEG Training, iEEG Testing	95.45
iEEG Training, EEG Testing	94.67
EEG + iEEG (Proposed)	98.63

TABLE VII: Ablation Study of Different EEG and iEEG Modality Configurations for Seizure Classification

Configuration	Accuracy (%)	Precision (%)	Recall (%)	F1 Score (%)
Without CGAN	94.76	94.77	94.76	94.76
With UMSH and CGAN	98.18	97.93	98.18	98.05

TABLE VIII: Effect of CGAN-based augmentation on seizure classification performance

EEG Seizure Detection

Electroencephalogram (EEG) seizure detection identifies epileptic events in brain recordings and supports timely diagnosis, monitoring, and intervention.

See all 28 papers

Multimodal Pretraining for Generalizable EEG Representation Learning
Jul 23, 2026Targol Bakhtiarvand, Jugal Kalita, Adham AtyabiCross-Subject EEG Decoding Electroencephalography
Department of Computer Science, University of Colorado Colorado Springs, Colorado Springs, CO, USA
ViTexSZ: Heterogeneous Vision-Text Knowledge Distillation for EEG Seizure Detection
Aug 19, 2026Chenxi Liu, Mingzhao Li, Yicong Liu +4EEG Seizure Detection Electroencephalography
Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong SAR · The First Affiliated Hospital, Sun Yat-sen University, China · The Hong Kong Polytechnic University, Hong Kong SAR +2
Interpretable Multimodal EEG-ECG Framework for Neonatal Seizure Prediction and Patient-Adaptive Transfer Learning
Nov 18, 2025Sithmini Ranasingha, Agasthi Haputhanthri, Hansa Marasinghe +5EEG Seizure Detection Cross-Subject EEG Decoding
Department of Electronic and Telecommunication Engineering, University of Moratuwa, Sri Lanka · Department of Electrical and Computer Engineering, University of British Columbia, Canada · Department of Biomedical Engineering, University of Melbourne, Australia +1

Jul 23, 2026J/K move · Enter open · S save

Multimodal Pretraining for Generalizable EEG Representation Learning

Targol Bakhtiarvand, Jugal Kalita, Adham Atyabi

Department of Computer Science, University of Colorado Colorado Springs, Colorado Springs, CO, USA

Cross-Subject EEG Decoding Electroencephalography EEG Seizure Detection Self-Supervised Pre-Training Multimodal Pretraining EEG Foundation Models

Open paper AskPDF arXiv

Electroencephalography (EEG) models used for epilepsy are often limited to specific datasets and tasks. This limited approach can make it challenging to apply these models across different datasets or in various situations. However, recent studies in foundation models and self-supervised learning suggest that an adaptable EEG backbone could support a range of EEG related tasks. In this study, we have developed a multimodal EEG foundation model that combines a raw signal encoder based on the Mamba architecture, a Vision Transformer (ViT)-style encoder for time-frequency data, and a lightweight encoder for text, all within a shared embedding space. The pretraining process relies on several innovative techniques, such as masked modeling, cross-view contrastive alignment, and temporal consistency losses. These methods are designed to create rich, seizure-relevant representations without requiring labeled data. To assess the efficacy and generalization of our pretrained model, we fine-tuned it on the canonical CHB-MIT seizure detection benchmark and additional seizure detection datasets, and conducted extensive experiments comparing different model variants. On the standard CHB-MIT split, our best single model achieved an AUROC of 0.874, and an ensemble variant reached 0.878 AUROC, representing state-of-the-art performance on this benchmark. In addition to standard train-test splits, we evaluated performance under a leave-one-subject-out (LOSO) protocol, which is rarely reported in prior EEG seizure modeling work and highlights the difficulty of patient-independent seizure detection, with a mean LOSO balanced accuracy of 0.558 across 19 subjects. Across datasets and evaluation settings, our multimodal foundation model enabled robust seizure detection and straightforward adaptation to new seizure detection scenarios, while also supporting interpretable seizure localization.

Abstract

EEG Seizure Detection

Explore similar work

Multimodal Pretraining for Generalizable EEG Representation Learning

ViTexSZ: Heterogeneous Vision-Text Knowledge Distillation for EEG Seizure Detection

Interpretable Multimodal EEG-ECG Framework for Neonatal Seizure Prediction and Patient-Adaptive Transfer Learning