RatSeizure: A Benchmark and Saliency-Context Transformer for Rat Seizure Localization

Ting Yu Tsai; An Yu; Lucy Lee; Felix X. -F. Ye; Damian S. Shin; Tzu-Jen Kao; Xin Li; Ming-Ching Chang

cs.CVMar 25, 2026

RatSeizure: A Benchmark and Saliency-Context Transformer for Rat Seizure Localization

Authors: Ting Yu Tsai, An Yu, Lucy Lee, Felix X. -F. Ye, Damian S. Shin, Tzu-Jen Kao, Xin Li, Ming-Ching Chang

Organizations: University at Albany, SUNY · University of Louisville · GE HealthCare

Abstract

Animal models, particularly rats, play a critical role in seizure research for studying epileptogenesis and treatment response. However, progress is limited by the lack of datasets with precise temporal annotations and standardized evaluation protocols. Existing animal behavior datasets often have limited accessibility, coarse labeling, and insufficient temporal localization of clinically meaningful events. To address these limitations, we introduce RatSeizure, the first publicly benchmark for fine-grained seizure behavior analysis. The dataset consists of recorded clips annotated with seizure-related action units and temporal boundaries, enabling both behavior classification and temporal localization. We further propose RaSeformer, a saliency-context Transformer for temporal action localization that highlights behavior-relevant context while suppressing redundant cues. Experiments on RatSeizure show that RaSeformer achieves strong performance and provides a competitive reference model for this challenging task. We also establish standardized dataset splits and evaluation protocols to support reproducible benchmarking.

Figures & tables

Figure 1: RatSeizure dataset composition and split distribution: (a) Concentric donut chart showing behavioral class frequencies before (inner ring) and after (outer ring) data augmentation, with example frames from four action categories annotated with temporal boundaries. (b) Label distribution across training and test splits after augmentation, showing the relative frequency (%) of each behavioral category and a consistent distribution with $\sim 65:35$ split ratio across categories.

Figure 2: The RaSeformer Pipeline: In phase 1 (a-c), raw rat videos are first processed through temporal segmentation, mirror artifact masking, and YOLO-based ROI cropping to produce standardized input tensors. These tensors are encoded by an I3D backbone into spatiotemporal feature sequences (phase 2). In the final phase, the sequences pass through the Salient Context Transformer Encoder, which uses per-head Top- $K$ pruning to focus on salient behavioral cues. A decoder then fuses the encoded features, with parallel heads for temporal boundary regression and behavior classification.

Explore similar work

Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding
May 21, 2026Lina Zhang, Tonmoy Monsoor, Peizheng Li +23Video Understanding Multimodal Large Language Models
University of California, Los Angeles · Mercedes-Benz AG · University of Pittsburgh +4
DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG Generic Representations
Jul 13, 2026Abdulkader Helwan, Lina Abou-Abbas, Hussein El Amouri +2Electroencephalography EEG Seizure Detection
Department of Electrical and Computer Engineering, Lebanese American University, Byblos, Lebanon · Institute of Applied Artificial Intelligence, T´ELUQ University, Montreal, Canada
Detecting seizure onset and offset times using human intelligence: A critical-transitions-based approach
Jul 29, 2026Andrew Flynn, Cian McCafferty, Klaus Lehnertz +4Electroencephalography EEG Seizure Detection
School of Mathematical Sciences, University College Cork, T12 XF62 Cork, Ireland. · INFANT Research Centre, University College Cork, T12 DC4A Cork, Ireland. · Department of Anatomy & Neuroscience, University College Cork, Cork, Ireland. +6

May 21, 2026J/K move · Enter open · S save

Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding

Lina Zhang, Tonmoy Monsoor, Peizheng Li, Jiarui Cui+22

University of California, Los Angeles · Mercedes-Benz AG · University of Pittsburgh+4

Video Understanding Multimodal Large Language Models Multimodal Model Evaluation

Open paper AskPDF arXiv

While Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in general video understanding, their capacity to interpret involuntary, and spatio-temporally evolving pathologic motor behaviors such as seizure semiology remains largely untested. To address this gap, we introduce Seizure-Semiology-Suite, a clinically grounded dataset and benchmark for fine-grained, structured seizure semiology understanding. The dataset includes 438 seizure videos annotated with over 35,000 dense labels covering 20 ILAE-defined semiological features. Building on this dataset, we propose a seven-task hierarchical benchmark that systematically evaluates MLLMs from low-level visual perception to temporal sequencing, narrative report generation, and seizure diagnosis. To enable clinically meaningful evaluation of generated reports, we further introduce the Report Quality Index for Seizure Semiology (Seizure-RQI). Extensive baselines across 11 open-weight MLLMs reveal systematic weaknesses in laterality reasoning, temporal localization, symptom sequencing, and clinically faithful reporting. We show that seizure-specific fine-tuning substantially improves performance across tasks, and that a two-stage neuro-symbolic framework achieves an F1 score of 0.96 on epileptic versus non-epileptic seizure classification. Seizure-Semiology-Suite establishes a rigorous benchmark for evaluating multimodal models in safety-critical medical video understanding and guides the development of clinically reliable, domain-adaptive multimodal intelligence.

Abstract

Explore similar work

Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understanding

DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG Generic Representations

Detecting seizure onset and offset times using human intelligence: A critical-transitions-based approach