Seeing Seizures: A Large-Scale Empirical Study of Video and EEG Deep Learning for Automated Rodent Seizure Detection and Severity Grading

Alloy Das; Shreyan Ganguly; Nyzil Massey; Hung Phi Doan; Zhanhong Jiang; Thimmasettappa Thippeswamy; Soumik Sarkar

cs.CVOct 15, 2026

Seeing Seizures: A Large-Scale Empirical Study of Video and EEG Deep Learning for Automated Rodent Seizure Detection and Severity Grading

Authors: Alloy Das, Shreyan Ganguly, Nyzil Massey, Hung Phi Doan, Zhanhong Jiang, Thimmasettappa Thippeswamy, Soumik Sarkar

Organizations: Iowa State University, Ames, IA, USA · Arkansas State University, Jonesboro, AR, USA

Abstract

Manual annotation of synchronized video and EEG recordings is a bottleneck in preclinical seizure research. We present an empirical study of seizure detection and behavioral severity grading using 24,131 paired clips from 601 sessions across 20 rats in a soman-induced epilepsy model. Ten video architectures and seven single-channel EEG models are evaluated independently on binary detection and three- and five-class severity tasks. In the session-exclusive reference comparison, R(2+1)D-18 and GRU achieve detection macro-F1 scores of 0.968 and 0.974, respectively; for five-class grading, the corresponding scores are 0.657 and 0.560. Subject-exclusive evaluation reduces five-class macro-F1 to 0.475 for video and 0.404 for EEG, highlighting the difficulty of grading unseen animals. Temporal-sampling experiments demonstrate the value of multiple frames, while detection performance varies relatively little across the evaluated video architectures. A separate video-only binary-detection evaluation on RodEpil achieves a mean seizure-class F1 of 0.979 after uniform re-encoding to reduce identified class-correlated encoding cues. The study characterizes existing models for behavior-defined severity labels under the evaluated protocol, motivating further investigation of expert-assisted annotation workflows. Our code and github page is available here.

Figures & tables

Figure 2: Recording setup: each rat is recorded in its home cage using a fixed overhead camera and implanted EEG sensors. Cage cropping reduces variation in the field of view but does not remove all differences in camera geometry. Identifying details are masked.

Figure 3: Binary-detection macro-F1 versus video model size under the evaluated training budget. Scores range from 0.945 to 0.974. X3D-M (4M parameters) achieves 0.969, compared with 0.971 for R(2+1)D-18 (33M). This comparison concerns detection, not equivalence across severity tasks (Tables 2 and 9 ).

Figure 4: A paired Stage 3 seizure from rat R7. Top: Four sample video frames. Bottom: The corresponding single-channel EEG trace ( $\sim$ 64 s), with the active seizure period highlighted. Watching these streams side-by-side provides a complete view of both the physical symptoms and the electrical brain activity. Identifying markers have been masked.

Task	Modality	Prec.	Rec.	F1	MCC
2-class (detection)	Video	0.968 $\pm$ 0.002	0.968 $\pm$ 0.002	0.968 $\pm$ 0.002	0.936 $\pm$ 0.004
2-class (detection)	EEG	0.975 $\pm$ 0.002	0.974 $\pm$ 0.003	0.974 $\pm$ 0.003	0.949 $\pm$ 0.005
3-class (severity)	Video	0.775 $\pm$ 0.016	0.788 $\pm$ 0.009	0.780 $\pm$ 0.007	0.853 $\pm$ 0.010
3-class (severity)	EEG	0.744 $\pm$ 0.014	0.764 $\pm$ 0.013	0.750 $\pm$ 0.010	0.823 $\pm$ 0.014
5-class (severity)	Video	0.689 $\pm$ 0.024	0.643 $\pm$ 0.006	0.657 $\pm$ 0.009	0.831 $\pm$ 0.006
5-class (severity)	EEG	0.556 $\pm$ 0.040	0.635 $\pm$ 0.035	0.560 $\pm$ 0.005	0.728 $\pm$ 0.032

Table 1: Reference R(2+1)D-18 video and single-channel GRU EEG results under session-exclusive evaluation (mean

\pm

std over five runs). P, R, and F1 are macro-averaged; MCC uses the full confusion matrix. Video has higher severity-classification macro-F1, whereas GRU has higher detection macro-F1. AUROC appears in App. E.5 , Table 15 . Best results are bold .

	Detection (2-class)				3-class grading				5-class grading
Backbone	P	R	F1	MCC	P	R	F1	MCC	P	R	F1	MCC
MViTv2-S [ 30 ]	$\underline{0.972}_{\pm 0.001}$	$\underline{0.972}_{\pm 0.001}$	$\underline{0.972}_{\pm 0.001}$	$\underline{0.944}_{\pm 0.003}$	$0.780_{\pm 0.014}$	$0.800_{\pm 0.027}$	$0.786_{\pm 0.005}$	$0.856_{\pm 0.017}$	$0.694_{\pm 0.032}$	$\mathbf{0.656}_{\pm 0.030}$	$0.660_{\pm 0.016}$	$0.832_{\pm 0.007}$
SlowFast-R50 [ 17 ]	$\mathbf{0.974}_{\pm 0.002}$	$\mathbf{0.974}_{\pm 0.002}$	$\mathbf{0.974}_{\pm 0.002}$	$\mathbf{0.948}_{\pm 0.004}$	$\underline{0.781}_{\pm 0.018}$	$\underline{0.804}_{\pm 0.013}$	$\underline{0.789}_{\pm 0.009}$	$\underline{0.859}_{\pm 0.012}$	$\underline{0.718}_{\pm 0.024}$	$\underline{0.655}_{\pm 0.014}$	$\mathbf{0.675}_{\pm 0.008}$	$\mathbf{0.851}_{\pm 0.003}$
R(2+1)D-18 [ 53 ]	$0.971_{\pm 0.001}$	$0.971_{\pm 0.001}$	$0.971_{\pm 0.001}$	$0.942_{\pm 0.002}$	$0.780_{\pm 0.007}$	$0.787_{\pm 0.007}$	$0.783_{\pm 0.002}$	$\mathbf{0.860}_{\pm 0.003}$	$\mathbf{0.734}_{\pm 0.027}$	$0.643_{\pm 0.011}$	$\underline{0.667}_{\pm 0.010}$	$\underline{0.841}_{\pm 0.007}$
X3D-M [ 18 ]	$0.969_{\pm 0.002}$	$0.969_{\pm 0.002}$	$0.969_{\pm 0.002}$	$0.937_{\pm 0.003}$	$\mathbf{0.782}_{\pm 0.008}$	$\mathbf{0.810}_{\pm 0.015}$	$\mathbf{0.794}_{\pm 0.005}$	$0.856_{\pm 0.008}$	$0.568_{\pm 0.004}$	$0.616_{\pm 0.013}$	$0.586_{\pm 0.003}$	$0.824_{\pm 0.007}$
MViTv1-B [ 16 ]	$0.969_{\pm 0.002}$	$0.969_{\pm 0.002}$	$0.969_{\pm 0.002}$	$0.938_{\pm 0.003}$	$0.771_{\pm 0.018}$	$0.794_{\pm 0.016}$	$0.779_{\pm 0.011}$	$0.849_{\pm 0.016}$	$0.687_{\pm 0.022}$	$0.643_{\pm 0.011}$	$0.653_{\pm 0.008}$	$0.836_{\pm 0.006}$

Table 2: We tested ten video models using the exact same data splits and a strict 12-epoch training limit. Scores show the mean

\pm

std over five test runs (SlowFast detection uses

n{=}4

). Basic detection scores barely changed, shifting by only 0.03 across all the models. You can find the AUROC scores and exact model setups in App. E.5 and App. B.1 . We bolded the best results and underlined the second-best.

	Detection		3-class		5-class
Frames	F1	MCC	F1	MCC	F1	MCC
1	0.569	0.149	0.417	0.152	0.314	0.147
4	0.940	0.881	0.739	0.760	0.612	0.741
8	0.959	0.918	0.763	0.838	0.627	0.789
16	0.970	0.941	0.780	0.857	0.662	0.830
32	0.972	0.943	0.793	0.858	0.667	0.835

Table 3: Temporal-sampling ablation for R(2+1)D (seed 42). Single-frame performance is poor, while performance peaks around 32 frames. Using 64 frames gives no detection gain and lowers 5-class F1. AUROC results are reported in App. E.5 , Table 17 .

Deep
	Detection (2-class)				3-class				5-class
EEG backbone	P	R	F1	MCC	P	R	F1	MCC	P	R	F1	MCC
GRU [ 19 ]	0.975 $\pm$ 0.003	0.974 $\pm$ 0.003	0.974 $\pm$ 0.003	0.949 $\pm$ 0.005	0.744 $\pm$ 0.014	0.764 $\pm$ 0.013	0.750 $\pm$ 0.010	0.823 $\pm$ 0.014	0.556 $\pm$ 0.040	0.635 $\pm$ 0.035	0.560 $\pm$ 0.005	0.728 $\pm$ 0.032
LSTM [ 21 ]	0.972 $\pm$ 0.008	0.971 $\pm$ 0.007	0.971 $\pm$ 0.008	0.942 $\pm$ 0.015	0.696 $\pm$ 0.041	0.741 $\pm$ 0.067	0.698 $\pm$ 0.053	0.743 $\pm$ 0.059	0.476 $\pm$ 0.042	0.555 $\pm$ 0.100	0.468 $\pm$ 0.052	0.598 $\pm$ 0.059
EEGNet [ 29 ]	0.926 $\pm$ 0.011	0.922 $\pm$ 0.013	0.923 $\pm$ 0.012	0.848 $\pm$ 0.023	0.646 $\pm$ 0.010	0.676 $\pm$ 0.018	0.620 $\pm$ 0.015	0.626 $\pm$ 0.031	0.448 $\pm$ 0.040	0.488 $\pm$ 0.025	0.409 $\pm$ 0.039	0.499 $\pm$ 0.037
EEG Conformer [ 50 ]	0.972 $\pm$ 0.001	0.972 $\pm$ 0.001	0.972 $\pm$ 0.001	0.943 $\pm$ 0.001	0.719 $\pm$ 0.009	0.784 $\pm$ 0.013	0.729 $\pm$ 0.010	0.780 $\pm$ 0.017	0.512 $\pm$ 0.004	0.624 $\pm$ 0.032	0.523 $\pm$ 0.005	0.683 $\pm$ 0.008

Table 4: Seven EEG baselines use the same 6 s windows, 3 s stride, 125 Hz sampling, and log-mean aggregation (App. B.5 ). TCN performs best for detection and 3-class grading, while GRU and TCN are similar at 5 classes. Two LSTM runs failed to converge, so its 3- and 5-class results use

n{=}4

. Best results are in bold ; second-best are underlined .

Panel of Figure 5 — Figure 5: Grad-CAM [ 48 ] for R(2+1)D-18. Four of eight frames are shown, with input frames above activation maps. Activations concentrate on the animal in the Stage 4 example and are more diffuse in the non-seizure example. These observations do not establish causal reliance on behavior or exclude shortcuts. Full sequences appear in App. D.2 , Fig. 14 . Identifying tags are blurred.

Figure 6: Per-fold binary-detection macro-F1 for unseen animals, as displayed in the plot: video

0.948\pm 0.016

and EEG

0.849\pm 0.055

. The plotted video results have a higher mean and lower fold-to-fold variability than the plotted EEG results.

		Session-excl.		Subject-exclusive (5-fold)
Task	Mod.	Macro-F1	MCC	Macro-F1	MCC
Detection	Video	0.968	0.936	0.948 $\pm$ 0.016	0.896 $\pm$ 0.031
Detection	EEG	0.974	0.949	0.972 $\pm$ 0.015	0.944 $\pm$ 0.030
3-class	Video	0.780	0.853	0.720 $\pm$ 0.028	0.761 $\pm$ 0.029
3-class	EEG	0.750	0.823	0.656 $\pm$ 0.022	0.793 $\pm$ 0.046
5-class	Video	0.657	0.831	0.475 $\pm$ 0.024	0.676 $\pm$ 0.041

Table 5: Testing new sessions versus new animals. Testing unseen animals keeps basic detection high but hurts exact grading. Video’s 5-class lead shrinks from 0.097 to 0.071. Scores show the mean

\pm

std over five tests.

Figure 7: Macro-F1 across task granularities for the results displayed in this plot. Video decreases from 0.970 for binary detection to 0.662 for five-class classification; EEG decreases from 0.966 to 0.534. Both modalities perform less well as the label set becomes more detailed. The plotted values differ from the run-averaged values in Table 1 .

	F0	F1	F2	F3	F4	Mean
Raw (leaked) F1	1.000	0.998	1.000	1.000	1.000	0.9996
Raw (leaked) MCC	1.000	0.998	1.000	1.000	1.000	0.9995
Re-encoded F1	0.965	0.978	0.997	0.977	0.979	0.979
Re-encoded MCC	0.957	0.976	0.996	0.967	0.964	0.972

Table 6: RodEpil [ 44 ] subject-wise five-fold evaluation. Uniform re-encoding removes the identified class-correlated encoding cues; R(2+1)D then reaches 0.979 mean F1 and 0.972 MCC.

Figure 8: Per-class F1 for the five-class results displayed in this plot. Video/EEG values are 0.97/0.78 for non-seizure, 0.67/0.45 for Stage 2, 0.90/0.38 for Stage 3, 0.36/0.10 for Stage 4, and 0.42/0.03 for Stage 5. Rare severe stages remain difficult for both modalities.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Representative progress	Remaining gap
EEG-only, animal	GRU/LSTM, CNN and temporal models [ 19 , 28 ]	Behavioral information
Video-only, human	Detection and semiology analysis [ 4 , 24 ]	Generalization, deployment
Video-only, animal	Severity [ 47 ] ; RodEpil [ 44 ]	Matched EEG comparison
Video $+$ EEG, human	Synchronized and fused systems [ 31 , 10 ]	Multi-site validation
Video $+$ EEG, animal	Recent multimodal studies [ 42 , 35 ]	Severity; paired benchmarks

Appendix

Table 7: The research landscape for automated seizure analysis broken down by data type and subject. The clearest open gap is multi-level severity grading using paired animal video-EEG data.

Study	Year	Method / vision component	Key result
Aghaei et al . [ 1 ]	2017	Combined video and EEG recordings for detection	Early video $+$ EEG fusion demonstration
Martini et al . [ 37 ]	2021	Deep anomaly detection, paired stereo-EEG $+$ video	Label-free seizure flagging
Cao et al . [ 10 ]	2024	Synchronized video-EEG, pediatric cohort	Detection in childhood epilepsy
Lin et al . (vEpiNet) [ 31 ]	2024	Patient detection $+$ frame-diff/keypoints $+$ EfficientNetV2, MLP fusion	AUROC 0.9902; false positives reduced by about one third

Appendix

Table 8: Representative human video-EEG systems. Animal fusion remains sparse and has concentrated almost entirely on simple yes-or-no seizure detection [ 42 , 35 ] .

Backbone	Paradigm	Par.
MViTv2-S [ 30 ]	multiscale	34M
SlowFast-R50 [ 17 ]	two-pathway	35M
R(2+1)D-18 [ 53 ]	conv	33M
X3D-M [ 18 ]	eff. conv	4M
MViTv1-B [ 16 ]	multiscale	36M
Video Swin-S [ 34 ]	hier. attn.	50M

Appendix

Table 9: Video backbones evaluated in Table 2 , outlining their core design paradigms and overall parameter counts. Network capacities range broadly from 4M to 121M parameters. The smallest model is marked in bold , while the second-smallest is underlined .

Panel of Figure 9 — Figure 9: A visual breakdown of our basic yes/no detection task. We show four sample frames from a normal, non-seizure period (top) alongside four frames from an active seizure (bottom). Because both examples feature the same rat during the same recording session, the animal’s appearance, the camera angle, and the overall background perfectly match. This careful pairing drastically reduces the chance that the model can cheat by memorizing static visual shortcuts in the scene. (All clips have been de-identified).

Panel of Figure 10 — Figure 10: Visual filmstrips for each seizure stage: Each row (S2-S5) shows five sequential frames taken from a typical seizure clip, with time moving from left to right. As the severity stage increases, the physical symptoms become much more obvious escalating from mild movements in Stage 2 to extreme rearing and violent convulsions in Stages 4 and 5. Crucially, these behavioral shifts are plainly visible on camera, whereas the recorded EEG signal only captures them indirectly through shifting brain waves. (All frames are pulled from de-identified clips and tightly cropped to hide any specific cage labels.)

Subject	Sess.	Clips	Non-sz.	Seiz.	S2	S3	S4	S5
R1	42	3,091	1,576	1,515	22	1,325	131	37
R2	44	2,830	1,431	1,399	113	1,094	181	11
R3	38	2,774	1,380	1,394	364	873	124	33
R4	42	2,701	1,351	1,350	22	1,062	226	40
R5	43	2,195	1,100	1,095	23	946	114	12
R6	43	1,963	955	1,008	218	782	8	0

Appendix

Table 10: A complete, animal-by-animal breakdown of our combined video and EEG dataset across all 20 rats. For every subject, we track the number of recording sessions, total clips, the split between non-seizure and seizure clips, and the exact breakdown of seizure severity (Stages 2 through 5). As the numbers show, the frequency of seizures is drastically uneven across the cohort. Just four heavily affected animals (R1, R2, R3, and R4) provide nearly half of the entire dataset, with individual totals swinging wildly from as few as 11 to as many as 3,091 clips. Across the board, the most violent events (Stage 5) remain extremely rare, yielding only 193 clips. It is worth noting that these counts represent the final, human-labeled dataset; a small handful of clips might still be dropped later when we format the data specifically for the video or EEG models. Finally, all true animal identities have been hidden, and the subjects are simply listed in order from highest to lowest clip count (R1 to R20).

Figure 11: A visual breakdown of the clip counts for each animal across the entire dataset, representing the stacked totals from Table 10 . The distribution of seizures is incredibly lopsided: the four most heavily affected rats (R1-R4) completely overshadow the rest of the group, with individual contributions ranging drastically from just 11 clips up to 3,091. More importantly, the most violent seizures (Stage 4 in dark shading, Stage 5 in the darkest) barely register as a thin sliver on any animal’s bar graph, yielding only 193 Stage 5 clips in the whole study. This extreme rarity is exactly why teaching models to perform detailed severity grading is such a difficult challenge.

Panel of Figure 12 — Figure 12: Visual examples of variation across different subjects: a single frame taken from six different animals (where ’R’ simply indicates a de-identified subject). The exact cage placement, camera angle, lighting conditions (which alternate between natural daylight and infrared), and even the animal’s coat vary drastically from rat to rat. This massive visual inconsistency is exactly why forcing a model to generalize to entirely unseen animals is so much harder than simply testing it on new recording sessions from familiar ones. It also highlights exactly why the video model’s consistent stability across different subjects (Fig. 6 ) is such a crucial finding.

Panel of Figure 13 — Figure 13: Examples of where the video model struggles the most. (a) A very mild Stage 2 seizure featuring only subtle movements, making it incredibly easy to confuse with normal, everyday behavior. (b) An animal that rarely experienced seizures (found at the bottom of Table 10 , App. C ), leaving the model with very few validation clips captured under poor, grainy infrared lighting. Ultimately, extremely mild symptoms and data-starved animals drive almost all of the network’s remaining errors. (All clips have been de-identified.)

Panel of Figure 14 — Figure 14: A full-width, extended version of Fig. 5 . This layout shows all eight sample frames from the exact same pair of clips. For each sequence, the raw video frames sit on top, with the model’s attention heatmaps (seizure-class activation) directly underneath. While this sharp contrast strongly supports the idea that the model is genuinely focusing on the animal to make its predictions, it does not act as definitive proof against hidden visual shortcuts. (All clips have been completely de-identified, and any specific cage identifying bands have been redacted.)

Panel of Figure 15 — Figure 15: Grad-CAM heatmaps for the R(2+1)D video model across different severity stages. The network’s attention consistently locks onto the animal across the illustrated Stages 2, 3, and 5, showing similar activation patterns in these selected examples. Still, we must treat these visualizations as descriptive rather than causal they show us where the model is looking, but they do not serve as absolute proof of how it makes its final decisions. (All clips have been de-identified, and any identifying cage bands have been completely redacted.)

Train subj.	Frac.	Clips	Macro-F1	MCC
2	0.10	62	0.790	0.583
5	0.25	1,905	0.893	0.787
10	0.50	10,969	0.957	0.914
12	0.60	11,832	0.946	0.892
15	0.75	16,358	0.960	0.921
20	1.00	19,208	0.970	0.941

Appendix

Table 11: R(2+1)D binary-detection data-size and pretraining experiments under session-exclusive evaluation. The validation set is fixed as the training-subject subset grows. Macro-F1 rises from 0.957 with ten animals to 0.970 with twenty; intermediate results are non-monotonic. Kinetics pretraining improves macro-F1 by 0.018. These experiments do not establish data saturation for other architectures or tasks.

Backbone	Seiz-F1	Macro-F1	AUROC	MCC
SlowFast-R50 † [ 17 ]	1.000	1.000	1.000	1.000 $\pm$ 0.000
R(2+1)D-18 [ 53 ]	0.978	0.986	0.999	0.972 $\pm$ 0.015
MViTv2-S [ 30 ]	0.916	0.946	0.989	0.891 $\pm$ 0.020
S3D [ 57 ]	0.912	0.944	0.991	0.889 $\pm$ 0.029
X3D-M [ 18 ]	0.895	0.933	0.983	0.866 $\pm$ 0.010
VideoMAE-B [ 52 ]	0.874	0.919	0.982	0.839 $\pm$ 0.025

Appendix

Table 12: Performance breakdown for all ten video models on the thoroughly sanitized RodEpil dataset [ 44 ] , evaluated under strict subject-exclusive five-fold cross-validation. F1 and AUROC scores represent the average across all five folds, while MCC includes the standard deviation (mean

\pm

std). Notice how drastically the model rankings shuffle compared to our native dataset a clear reminder that a network’s success is deeply tied to the specific visual quirks of its training data. † We have explicitly flagged the SlowFast model; its perfect score warrants further investigation for residual shortcuts, without establishing their presence. The highest non-flagged score is highlighted in bold .

Input size	Macro-F1	AUROC	MCC
$112\times 112$ (default)	0.970	0.993	0.941
$224\times 224$	0.967	0.993	0.934
$448\times 448$	0.952	0.988	0.904

Appendix

Table 13: Ablation experiment evaluating spatial resolution for the R(2+1)D detection model across three input image sizes (16 frames, session-exclusive split, seed 42, 10 epochs; batch sizes scaled to 16, 8, and 4, respectively). Because the native source footage is only

\sim

300 to 480 px, both of the higher resolution settings force the network to upsample the video. Consequently, performance drops steadily across all three metrics. In terms of AUROC, the gap between the top two settings only emerges past the third decimal place (

0.9934

vs.

0.9926

). Note that the 448 px run suffered an out-of-memory crash after completing 8 of its 10 epochs; we report metrics from its best saved checkpoint, meaning this row represents an optimistic upper bound for the largest resolution setting. The best score in each column is highlighted in bold .

Stage	n	P	R	F1	n	P	R	F1
	Video				EEG
non-seizure	2720	0.965	0.968	0.966	2737	0.914	0.955	0.934
Stage 2	220	0.681	0.650	0.665	256	0.491	0.863	0.626
Stage 3	2148	0.904	0.896	0.900	2064	0.867	0.717	0.785
Stage 4	167	0.330	0.401	0.362	229	0.268	0.162	0.202
Stage 5	34	0.579	0.324	0.415	33	0.074	0.364	0.123

Appendix

Table 14: Per-class diagnostic precision (P), recall (R), and F1 for five-class classification, with modality-specific support counts (n). These tabulated results expose differences between common and rare classes. They should not be conflated with the values displayed in Fig. 8 .

Task	Modality	Session-excl.	Subject-excl.
2-class (detection)	Video	0.993 $\pm$ 0.001	0.985 $\pm$ 0.006
2-class (detection)	EEG	0.996 $\pm$ 0.001	0.995 $\pm$ 0.005
3-class (severity)	Video	0.964 $\pm$ 0.003	0.942 $\pm$ 0.015
3-class (severity)	EEG	0.956 $\pm$ 0.002	0.916 $\pm$ 0.025
5-class (severity)	Video	0.939 $\pm$ 0.004	0.888 $\pm$ 0.026
5-class (severity)	EEG	0.941 $\pm$ 0.003	0.831 $\pm$ 0.076

Appendix

Table 15: AUROC scores comparing the standard session-exclusive evaluation against the strictly segregated subject-exclusive cross-validation. We use binary AUROC for basic detection and a macro one-vs-rest approach for multi-class severity grading. All values represent the mean

\pm

std across five independent runs.

Video
Backbone	Detection	3-class	5-class
MViTv2-S [ 30 ]	$0.993_{\pm 0.002}$	$\mathbf{0.971}_{\pm 0.002}$	$0.915_{\pm 0.011}$
SlowFast-R50 [ 17 ]	$\mathbf{0.995}_{\pm 0.001}$	$\underline{0.966}_{\pm 0.001}$	$\underline{0.939}_{\pm 0.011}$
R(2+1)D-18 [ 53 ]	$\underline{0.994}_{\pm 0.001}$	$0.960_{\pm 0.003}$	$0.928_{\pm 0.017}$
X3D-M [ 18 ]	$0.993_{\pm 0.001}$	$0.952_{\pm 0.005}$	$0.911_{\pm 0.003}$
MViTv1-B [ 16 ]	$0.991_{\pm 0.003}$	$0.965_{\pm 0.003}$	$0.935_{\pm 0.012}$

Appendix

Table 16: AUROC scores for all ten video and seven EEG backbones, following the exact same row order as Tables 2 and 4 for easy comparison. We use binary AUROC for the basic detection task and a macro one-vs-rest approach for multi-class severity grading.

Frames	Detection	3-class	5-class
1	0.615	0.599	0.634
4	0.983	0.950	0.926
8	0.989	0.957	0.919
16	0.993	0.962	0.946
32	0.994	0.971	0.951
64	0.994	0.971	0.950

Appendix

Table 17: AUROC scores for both the video frame-count sweep and the rigorously re-encoded RodEpil cross-validation folds. For the sake of clarity, we have broken out the specific RodEpil AUROC results into their own dedicated section immediately below the frame-count data.

overall: both correct $2161$ ( $76.4\%$ ), video only $401$ ( $14.2\%$ ),
True class	n	video only	EEG only
non-seizure	1488	73	44
Stage 2	192	7	44
Stage 3	1062	286	43
Stage 4	74	27	8
Stage 5	14	8	1

Appendix

Table 18: Direct head-to-head evaluation across the

2{,}830

held-out test clips shared between both modalities for 5-class severity grading. To ensure a strictly fair comparison, this analysis is restricted to the exact intersection of clips evaluated by both models. Per-class rows highlight “solo victories” instances where precisely one model correctly predicted the severity stage while the other failed. Notably, Stage 2 represents the sole category where the EEG model outperforms the video network in solo correct predictions.

			Video recall		EEG recall
True class	n	med.	low	high	low	high
non-seizure	466	1.53	0.983	0.966	0.906	1.000
Stage 2	58	1.57	0.655	0.586	0.966	0.862
Stage 3	350	1.78	0.891	0.943	0.537	0.777
Stage 4	21	2.17	0.182	0.700	0.182	0.000

Appendix

Table 19: Recall evaluated across the low- and high-motion halves of each severity class, partitioned strictly at the class-specific median motion energy (

900

clips sampled from the paired evaluation set). We define motion energy as the mean absolute frame difference across

24

evenly spaced

96\times 96

grayscale frames. Because this metric is computed over the entire duration of the clip including its

10

s pre- and post-event buffers it serves as a coarse but objective proxy for the overall physical intensity of the seizure.

Figure 16: Row-normalized confusion matrices comparing both modalities across all three evaluation tasks (drawn from a representative seed-42 diagnostic run; displaying true-class fractions with raw counts in parentheses). On the granular 5-class task, the video model’s mistakes cluster tightly along the diagonal, meaning its errors are almost entirely limited to adjacent severity levels. In stark contrast, the EEG baseline not only misses active seizures completely, but also frequently commits erratic, non-adjacent grading errors.

Figure 17: Error diagnostics for the 5-class severity grading task. (a) Numeric label-distance histogram. Because baseline non-seizure behavior falls completely outside the standard Racine scale, this panel remains purely descriptive and should not be treated as a strict ordinal metric. (b) Per-class recall annotated with absolute support counts, distinctly highlighting the Stage 2 performance reversal. (c) Direct head-to-head outcomes across the strictly shared intersection of

2{,}830

test clips evaluated by both modalities.

EEG Seizure Detection

Electroencephalogram (EEG) seizure detection identifies epileptic events in brain recordings and supports timely diagnosis, monitoring, and intervention.

See all 29 papers

Mar 25, 2026

RatSeizure: A Benchmark and Saliency-Context Transformer for Rat Seizure Localization

Animal models, particularly rats, play a critical role in seizure research for studying epileptogenesis and treatment response. However, progress is limited by the lack of datasets with precise temporal annotations and standardized evaluation protocols. Existing animal behavior datasets often have limited accessibility, coarse labeling, and insufficient temporal localization of clinically meaningful events. To address these limitations, we introduce RatSeizure, the first publicly benchmark for fine-grained seizure behavior analysis. The dataset consists of recorded clips annotated with seizure-related action units and temporal boundaries, enabling both behavior classification and temporal localization. We further propose RaSeformer, a saliency-context Transformer for temporal action localization that highlights behavior-relevant context while suppressing redundant cues. Experiments on RatSeizure show that RaSeformer achieves strong performance and provides a competitive reference model for this challenging task. We also establish standardized dataset splits and evaluation protocols to support reproducible benchmarking.

Ting Yu Tsai, An Yu, Lucy Lee +5

University at Albany, SUNY · University of Louisville · GE HealthCare

Animal Behavior Analysis Temporal Action Localization

Ask PDF

Apr 29, 2026

A Multimodal Pre-trained Network for Integrated EEG-Video Seizure Detection

Reliable seizure detection in mouse models is essential for preclinical epilepsy research, yet manual review of synchronized video-EEG recordings is labor-intensive and single-modality systems fail for complementary reasons: video-based methods are easily confounded by benign behaviors, whereas EEG-based methods are vulnerable to ictal motion artifacts. We present EEGVFusion, a multimodal framework that combines self-supervised EEG representation learning, spatio-temporal video encoding, optimal-transport alignment, and bidirectional cross-attention to integrate neural and behavioral evidence. We also curate an expert-annotated dataset of synchronized EEG and video recordings comprising 93 sessions from 15 mice for training and evaluation. In the random-session split, EEGVFusion achieved a Balanced Accuracy of 0.9957 with perfect event sensitivity and an Event FAR of 0.6250 FP/h, indicating strong seizure detection performance with a low false-alarm burden. In a single held-out-subject evaluation with Subject 110 reserved for testing, EEGVFusion achieved a Balanced Accuracy of 0.9718 and reduced Event FAR from 2.7250 FP/h for the EEG-only counterpart to 0.4833 FP/h while preserving perfect event sensitivity. Targeted ablations further showed that EEG pre-training and OT alignment help reduce false alarms while preserving event sensitivity.

Tong Lu, Ke Xu, Zimo Zhang +12

Beijing Institute for Brain Research, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, 102206, China · Chinese Institute for Brain Research, Beijing, Beijing, 102206, China · Department of Neurosurgery, SanBo Brain Hospital, Capital Medical University, Beijing, 100018, China +4

Electroencephalography EEG Seizure Detection Multimodal Pretraining Multimodal Classification Multimodal Fusion

Ask PDF

Jul 13, 2026

DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG seizure-related Representations

Deep learning for EEG-based seizure detection faces critical challenges: severe annotation scarcity and extreme class imbalance, where ictal events comprise less than 10% of clinical recordings. We present DiffEEG, a 9.6M-parameter self-supervised foundation model that addresses both limitations through denoising diffusion pre-training and reinforcement learning (RL)-based fine-tuning. Pre-trained on 1.3M unlabeled segments from the Temple University Hospital Seizure Corpus (TUHSZ), DiffEEG learns generic neural representations via a 1D U-Net with multi-head self-attention. For downstream adaptation, a reinforced decision layer employs policy gradient optimization to directly maximize F1-score, prioritizing sensitivity to rare seizure events over overall accuracy. Under strict patient-wise evaluation (279 patients, Leave-One-Fold-Out), DiffEEG achieves 61% accuracy and 59% F1 for 4-class seizure subtyping, and 81% accuracy with 85% weighted F1 for binary detection, maintaining clinically viable seizure recall (59%) despite extreme imbalance (6.7% prevalence). Segment-level evaluation establishes an upper bound of 97.6% accuracy, confirming strong architectural capacity. DiffEEG demonstrates that diffusion-based pre-training combined with metric-aware reinforcement learning enables clinically deployable seizure monitoring with minimal labeled data requirements.

Abdulkader Helwan, Lina Abou-Abbas, Hussein El Amouri +2

Department of Electrical and Computer Engineering, Lebanese American University, Byblos, Lebanon · Institute of Applied Artificial Intelligence, TÉLUQ University, Montreal, Canada

Electroencephalography EEG Seizure Detection Self-Supervised Time Series Representation Learning EEG Foundation Models Diffusion Models

Ask PDF

Abstract

EEG Seizure Detection

Explore similar work

RatSeizure: A Benchmark and Saliency-Context Transformer for Rat Seizure Localization

A Multimodal Pre-trained Network for Integrated EEG-Video Seizure Detection

DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG seizure-related Representations