Beyond Reconstruction: A Unified Framework for Trustworthy and Explainable EEG Artifact Removal Evaluation
Authors: Phat Lam, Hung Nguyen, Lam Pham, Huy Phan, Dat Tran, Ian McLoughlin, Roozbeh Razavi-Far, Hung Cao
Organizations: Analytics Everywhere Lab, University of New Brunswick, Canada. · Austrian Institute of Technology, Austria. · German Research Center for Artificial Intelligence (DFKI), Germany. · FPT University, Vietnam. · Singapore Institute of Technology, Singapore. · Trustworthy and Secure AI Lab, University of New Brunswick, Canada.
Deep-learning methods have achieved strong performance in EEG artifact removal, but evaluation remains dominated by reconstruction metrics such as error, correlation, and signal-to-noise ratio. Although these metrics assess similarity to clean EEG, they do not establish whether models effectively suppress artifacts, preserve neural activity, or provide reliable explanations. To address these limitations, we propose TriX-EEG, a three-level framework for trustworthy and explainable EEG artifact removal. Level 1, Signal Validity (SV), evaluates reconstruction quality and neural band-power preservation. Level 2, Removal Faithfulness (RF), assesses the removed component through artifact error and residual artifact/neural leakage. Level 3, Explanation Reliability (ER), examines model behavior through temporal alignment, causal relevance, and stability. We evaluate TriX-EEG using nine published neural denoisers across three public benchmarks derived from EEGDenoiseNet and the PhysioNet Motion Artifact Contaminated EEG Dataset, covering EOG, EMG, and motion artifacts. Results show that reconstruction metrics alone are insufficient for trustworthy denoiser assessment: RF complements it by revealing whether errors stem from residual artifact or suppressed neural activity, while ER captures distinct aspects of explanation behavior, supporting joint evaluation of three aspects within a unified framework rather than reliance on any single factor. External validation on an independent EEG/EOG dataset and the BCI Competition IV-2a motor-imagery dataset further demonstrates broadly consistent rankings under dataset shift. Higher-ranked denoisers generally preserve more task-relevant information or cause less downstream performance degradation. Overall, TriX-EEG provides a comprehensive basis for EEG denoiser assessment, encouraging models that are not only accurate but also faithful and explainable.
Figures & tables
Fig. 1: Overview of the TriX-EEG evaluation framework. (a) A denoiser fθ maps the contaminated EEG y=x+a to x ; the removed component a and the attribution S(y) of the removed energy g are evaluated against the references x and a . (b) Level 1 compares x with x . (c) Level 2 compares a with a . (d) Level 3 tests whether the explanation follows the artifact in time (TAAS) and frequency (BAC), is causally relevant (CAF) and is stable under a small perturbation δ (ASS). (e) The level scores are combined into TDS by a geometric mean. Arrows give the preferred direction of each metric; solid lines are references, dashed lines model estimates. Mini-plots are schematic and computed on simulated signals.
Fig. 2: An example of Soft Artifact Energy Qa(τ) and Model Attribution alignment MA(τ) on two concatenated windows, corresponding with SmoothGrad saliency and TAAS scores for the compared models on EEGDenoiseNet-EOG dataset.
Fig. 3: An example of Soft Artifact Energy Qa(τ) and Model Attribution alignment MA(τ) on two concatenated windows, corresponding with SmoothGrad saliency and TAAS scores for the compared models on EEGDenoiseNet-EMG dataset.
Method
EEGDenoiseNet-EOG
EEGDenoiseNet-EMG
EEG-Motion
RRMSE t↓
RRMSE s↓
CC ↑
Δ SNR ↑
RRMSE t↓
RRMSE s↓
CC ↑
Δ SNR ↑
RRMSE t↓
RRMSE s↓
CC ↑
Δ SNR ↑
FCNN [ 41 ]
0.5570
0.5865
0.8111
10.8011
0.6176
0.6687
0.7849
9.6891
0.8553
1.0593
0.4827
−0.9128
Simple-CNN [ 41 ]
0.4437
0.4432
0.8847
12.6351
0.7215
0.7263
0.7255
8.4879
0.7102
1.0616
0.7508
1.1927
1D-ResCNN [ 30 ]
0.4184
0.4046
0.8966
13.1761
0.7510
0.7093
0.7076
8.2550
0.7262
1.0736
0.7342
0.9455
RNN-LSTM [ 41 ]
0.6903
0.7231
0.7176
8.6099
0.6758
0.7215
0.7384
8.8269
0.7751
1.0000
0.6184
0.0798
EEGDNet [ 24 ]
0.4406
0.4125
0.8907
12.4712
0.6554
0.6196
0.7456
9.0505
0.6641
0.8578
0.7485
1.5212
TABLE I: Comparison of Performance of SV Metrics (Level 1) across models (Best Results are in Bold; Second-best Ones are Underlined).
Fig. 4: Visualization of CAF across five different fractions on EEGDenoiseNet-EOG dataset.
Fig. 5: SV across datasets. Left: BPS (higher is better). Right: relative band-power error EbBP per band and its mean EBP (lower is better); colours use a log scale shared by all datasets.
Parameter
Metric(s)
Configuration
Level 1: Signal Validity
Maximum SNR improvement
UΔSNR
ΔSNRmax=20 dB
Band-power error tolerance
BPS
EmaxBP=1
Level 2: Removal Faithfulness
Removed-energy deviation tolerance
URAER
EmaxRAER=1
ARE tolerance
UARE
EmaxARE=1
TABLE II: Configuration of the TriX-EEG evaluation framework.
Fig. 6: Level-1 metric profiles of denoising models across datasets, constructed by averaging per-window clipped utility scores ( Ut , Us : temporal and spectral RRMSE utilities; UCC : correlation utility). The shaded final axis presents the aggregate DS. The annotations highlight the best-performing value for each metric component.
Fig. 7: Level-2 RF profiles. The shaded axis gives the aggregate FS.
Fig. 8: Level-3 ER profiles. Points show the mean over five stratified 64-window subsets, and error annotations report the sample standard deviation across subset/operation replicates. The shaded final axis reports the aggregate RS.
Fig. 9: Level-wise metrics across datasets. Boxes give the best value on each axis; for RS and TDS, mean ± standard deviation over the five Level-3 subsets.
Rank
Model
Mean DS
Mean FS
Mean RS
Mean TDS
1
BandRouteNet
0.5878
0.7244
0.6185±0.0045
0.6357±0.0014
2
EEGDNet
0.5287
0.7200
0.6265±0.0054
0.6151±0.0015
3
DeepSeparator
0.5231
0.7064
0.6386±0.0043
0.6142±0.0013
4
LRR-UNet
0.5611
0.7209
0.5554±0.0065
0.6030±0.0021
5
SimpleCNN
0.5167
0.7113
0.5761±0.0072
0.5876±0.0021
6
1D-ResCNN
0.5285
0.7191
0.5552±0.0069
0.5845±0.0022
TABLE III: Mean TriX-EEG component scores across the three datasets. DS and FS are computed over the complete test sets, whereas RS and TDS are reported as the mean ± standard deviation, calculated using the root-mean-square of the per-dataset standard deviations. Models are ranked by their mean TDS.
Fig. 10: Sanity-check behavior of TriX-EEG . Panels (a)–(c) report the valid-minus-corrupted sensitivity gap for trained versus untrained model, real versus shuffled attributions, and the true artifact versus an artifact of wrong type, respectively. Positive gaps indicate the expected response. Panel (d) reports higher-is-better utilities when clean EEG is supplied with a zero-artifact reference.
Analysis set
DS–FS
DS–RS
FS–RS
EEGDenoiseNet-EOG
0.9833
-0.6500
-0.7167
EEGDenoiseNet-EMG
0.8333
-0.1667
-0.4833
EEG-Motion
0.7833
-0.7000
-0.6167
Pooled within-dataset
0.8667
-0.5056
-0.6056
TABLE IV: Rank correlations among the three TriX-EEG level scores. The dataset rows report Spearman correlations across nine models. The pooled row reports the correlation of normalized within-dataset ranks across all model–dataset evaluations.
Rank
Denoiser
RRMSEt
RRMSEs
CC
Qext
1
DeepSeparator
0.4009
0.4616
0.9130
0.6835±0.0658
2
LRR-UNet
0.4208
0.4407
0.8979
0.6789±0.0758
3
BandRouteNet
0.4265
0.4479
0.8970
0.6743±0.0722
4
GAN
0.4816
0.4749
0.8623
0.6365±0.0748
5
EEGDNet
0.4901
0.4833
0.8676
0.6314±0.0645
6
1D-ResCNN
0.5882
0.5605
0.8049
0.5521±0.0673
TABLE V: Reconstruction-only target-domain performance on EEG/EOG dataset. Lower RRMSE and higher CC/ Qext are better. The uncertainty of Qext is the sample standard deviation across subjects.
Rank
Denoiser
DS
FS
RS
TDS
1
DeepSeparator
0.5979
0.5398
0.5127±0.0075
0.5490±0.0027
2
BandRouteNet
0.5867
0.5251
0.4777±0.0097
0.5280±0.0036
3
LRR-UNet
0.5980
0.5334
0.4470±0.0118
0.5224±0.0046
4
EEGDNet
0.5294
0.4987
0.4936±0.0136
0.5070±0.0046
5
GAN
0.5484
0.4861
0.4724±0.0076
0.5012±0.0027
6
1D-ResCNN
0.4645
0.4714
0.4257±0.0090
0.4534±0.0032
TABLE VI: Complete target-domain TriX-EEG evaluation on EEG-EOG dataset. RS and TDS are mean ± sample standard deviation over five paired Level-3 subsets.
Target ordering
Source ranking
Spearman ρ
Exact p
Qext
DS
0.4500
0.2298
Qext
FS
0.4000
0.2912
Qext
TDS
0.7000
0.0433
TDSext
DS
0.5167
0.1618
TDSext
FS
0.4500
0.2298
TDSext
RS
0.0667
0.8801
TABLE VII: Rank agreement between the target-domain orderings and the source-domain TriX-EEG scores on EEGDenoiseNet-EOG. Exact p -values are two-sided permutation tests across nine denoisers.
C3
C4
Input/method
Acc. ↑
Δ Acc. ↑
F1 ↑
Δ F1 ↑
Acc. ↑
Δ Acc. ↑
F1 ↑
Δ F1 ↑
Clean EEG
0.3622
+0.0402
0.3470
+0.0906
0.3562
+0.0395
0.3279
+0.0807
Contaminated
0.3220
0.0000
0.2564
0.0000
0.3167
0.0000
0.2471
0.0000
BandRouteNet
0.3311
+0.0092
0.2820
+0.0256
0.3283
+0.0116
0.2705
+0.0234
LRR-UNet
0.3325
+0.0105
0.2810
+0.0246
0.3301
+0.0134
0.2699
+0.0228
DeepSeparator
0.3298
+0.0078
0.2734
+0.0170
0.3277
+0.0110
0.2644
+0.0172
TABLE VIII: Single-channel downstream performance on C3 and C4. Metrics are balanced over subjects and SNRs; deltas are relative to the contaminated input. Bold denotes the best denoiser recovery and underlining the runner-up in each delta column.
Input/method
Acc. ↑
Δ Acc. ↑
Macro-F1 ↑
Δ F1 ↑
Clean EEG
0.6440
+0.0653
0.6398
+0.0761
Contaminated
0.5787
0.0000
0.5637
0.0000
BandRouteNet
0.5702
−0.0085
0.5587
−0.0050
DeepSeparator
0.5720
−0.0067
0.5580
−0.0057
LRR-UNet
0.5558
−0.0229
0.5418
−0.0219
EEGDNet
0.5532
−0.0255
0.5349
−0.0288
TABLE IX: Multichannel downstream performance with 22-channel FBCSP–LDA classifier. Metrics are balanced over subjects and SNRs; deltas are relative to the contaminated input. Bold denotes the best denoiser preservation and underlining the runner-up in each delta column.
Fig. 11: Agreement between the artifact-matched denoiser ranking produced by TriX-EEG and the ranking by downstream macro-F1 recovery on BCI IV-2a motor imagery ( n=9 denoisers). Each row shows the Spearman correlation ρ between one TriX-EEG score (DS, FS, RS or TDS) and downstream recovery. Results are given over all SNR levels (Overall), severe contamination ( {−15,−10,−5} dB) and mild contamination ( {0,2,5} dB). Horizontal bars show approximate 95% confidence intervals from the Fisher z -transform. Bold values mark the strongest association within each regime.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 12: Qualitative comparison on EEGDenoiseNet-EOG at two evaluation levels: (a) Level 1 assesses preservation of the clean EEG signal, while (b) Level 2 assesses whether the removed component corresponds to the ground-truth artifact.
Fig. 13: Qualitative comparison on EEGDenoiseNet-EMG at two evaluation levels: (a) Level 1 assesses preservation of the clean EEG signal, while (b) Level 2 assesses whether the removed component corresponds to the ground-truth artifact.
Fig. 14: Qualitative comparison on EEG-Motion at two evaluation levels: (a) Level 1 assesses preservation of the clean EEG signal, while (b) Level 2 assesses whether the removed component corresponds to the ground-truth artifact.
Fig. 15: Temporal attribution maps on EEGDenoiseNet-EOG for nine denoisers and three explanation methods. The upper panel overlays the contaminated EEG and true artifact on the soft artifact-energy target Qa(τ) . The remaining panels show row-normalized saliency; labels report TAAS.
Fig. 16: Temporal attribution maps on EEGDenoiseNet-EMG for nine denoisers and three explanation methods.
Fig. 17: Temporal attribution maps on EEG-Motion for nine denoisers and three explanation methods.
Dataset
Attribution
TAAS
CAF
ASS
RS
EEGDenoiseNet-EOG
Real
0.5437±0.0128
0.4560±0.0120
0.9864±0.0004
0.5700±0.0030
EEGDenoiseNet-EOG
Shuffled
0.4387±0.0143
0.0300±0.0058
0.5593±0.0023
0.3309±0.0038
EEGDenoiseNet-EMG
Real
0.8188±0.0064
0.4478±0.0035
0.9905±0.0002
0.6912±0.0028
EEGDenoiseNet-EMG
Shuffled
0.7769±0.0094
0.0223±0.0034
0.5579±0.0017
0.4659±0.0033
EEG-Motion
Real
0.7497±0.0107
0.1640±0.0198
0.9894±0.0003
0.6183±0.0085
EEG-Motion
Shuffled
0.7341±0.0156
0.0241±0.0048
0.5864±0.0048
0.4792±0.0032
Appendix
TABLE X: Random-attribution sanity check. Only attribution-dependent Level-3 metrics are shown.
Dataset
Model
DS
FS
RS
TDS
EEGDenoiseNet-EOG
Trained
0.6664
0.8667
0.5700±0.0030
0.6905±0.0012
EEGDenoiseNet-EOG
Untrained
0.0274
0.4886
0.6354±0.0089
0.2041±0.0010
EEGDenoiseNet-EMG
Trained
0.5259
0.8088
0.6912±0.0028
0.6649±0.0009
EEGDenoiseNet-EMG
Untrained
0.0199
0.4798
0.7303±0.0026
0.1910±0.0002
EEG-Motion
Trained
0.3754
0.4811
0.6183±0.0085
0.4816±0.0022
EEG-Motion
Untrained
0.0094
0.2561
0.6385±0.0056
0.1154±0.0003
Appendix
TABLE XI: Level and overall scores for trained and untrained model
Dataset
Threshold
Low/default/high
Utility
Utility values
Score
Score values
EEGDenoiseNet-EOG
ΔSNRmax
10/20/30
UΔSNR
0.8973/0.6216/0.4169
DS
0.7215/0.6664/0.6255
EEGDenoiseNet-EOG
EmaxBP
0.5/1/2
BPS
0.3608/0.6793/0.8396
DS
0.6027/0.6664/0.6985
EEGDenoiseNet-EOG
EmaxRAER
0.5/1/2
URAER
0.9018/0.9509/0.9754
FS
0.8569/0.8668/0.8717
EEGDenoiseNet-EOG
EmaxARE
0.5/1/2
UARE
0.4761/0.7212/0.8606
FS
0.8177/0.8668/0.8946
EEGDenoiseNet-EOG
EmaxAGE
0.5/1/2
UAGE
0.8752/0.9376/0.9688
FS
0.8543/0.8668/0.8730
EEGDenoiseNet-EOG
EmaxCNL
0.5/1/2
UCNL
0.5539/0.7764/0.8882
FS
0.8223/0.8668/0.8891
Appendix
TABLE XII: Normalization-threshold sensitivity. Slash-separated entries correspond to the low, default, and high settings.
Dataset
Reference
RAS
URAER
UARE
UAGE
UCNL
FS
EEGDenoiseNet-EOG
True
0.9477
0.9509
0.7212
0.9376
0.7764
0.8668
EEGDenoiseNet-EOG
Wrong raw
0.0408
0.7683
0.0000
0.0436
0.6963
0.3098
EEGDenoiseNet-EOG
Wrong RMS-matched
0.0408
0.9509
0.0000
0.0401
0.6963
0.3456
EEGDenoiseNet-EMG
True
0.9185
0.9422
0.6349
0.8736
0.6748
0.8088
EEGDenoiseNet-EMG
Wrong raw
0.0391
0.7628
0.0000
0.0362
0.6500
0.2976
EEGDenoiseNet-EMG
Wrong RMS-matched
0.0391
0.9422
0.0000
0.0347
0.6500
0.3332
Appendix
TABLE XIII: Wrong-artifact pairing check.
Dataset
Method
TAAS
CAF
ASS
RS
EEGDenoiseNet-EOG
Gradient × Input
0.7480±0.0059
0.5328±0.0124
0.9869±0.0004
0.6408±0.0040
EEGDenoiseNet-EOG
Integrated Gradients
0.7593±0.0054
0.5436±0.0120
0.9917±0.0001
0.6475±0.0041
EEGDenoiseNet-EOG
SmoothGrad
0.5437±0.0128
0.4560±0.0120
0.9864±0.0004
0.5700±0.0027
EEGDenoiseNet-EMG
Gradient × Input
0.8571±0.0089
0.4723±0.0055
0.9906±0.0002
0.7066±0.0032
EEGDenoiseNet-EMG
Integrated Gradients
0.8594±0.0078
0.4787±0.0047
0.9925±0.0001
0.7092±0.0029
EEGDenoiseNet-EMG
SmoothGrad
0.8183±0.0064
0.4482±0.0040
0.9905±0.0000
0.6912±0.0028
Appendix
TABLE XIV: Sensitivity of Level-3 results to the attribution method.
Electroencephalography (EEG) is highly susceptible to artifact contamination, such as electrooculographic (EOG) and electromyographic (EMG) interference, which severely degrades signal quality and hinders reliable interpretation in applications including neurological diagnosis, brain-computer interfaces (BCIs), etc. Effective EEG denoising remains challenging because different artifact sources exhibit diverse and temporally varying distributions, together with distinct spectral characteristics across frequency bands. To address these issues, we propose BandRouteNet, an adaptive frequency-aware neural network for EEG denoising that jointly exploits band-specific processing and full-band contextual modeling. The proposed model performs band-wise denoising to explicitly capture frequency-dependent artifact patterns. Within this framework, we introduce a routing mechanism that adaptively determines where and to what extent denoising should be applied across temporal locations within each frequency band. In parallel, a full-band conditioner directly processes the original noisy EEG to extract global temporal context, producing both conditional parameters for modulating the band-wise pathway and a coarse-grained signal-level refinement to supplement the final reconstruction. Extensive experiments on the EEGDenoiseNet benchmark dataset demonstrate that BandRouteNet outperforms other methods under EOG, EMG, and mixed-artifact conditions in terms of Relative Root Mean Square Error (RRMSE) and Signal-to-Noise Ratio Improvement (SNRimp) under unified experimental settings, while remaining highly parameter-efficient with only 0.2M trainable parameters. These results highlight its strong potential for high-performance EEG artifact removal in resource-constrained applications.
Phat Lam
Ho Chi Minh City University of Technology, Vietnam
Deep learning EEG denoising architectures have scaled from tens of thousands to tens of millions of parameters, yet no prior study has isolated model capacity as the experimental variable or tested whether reconstruction metrics predict downstream neural-signal utility. We address both gaps by fixing architecture, loss, data split, and training recipe while sweeping only channel width from 1.05K to 40.26K parameters in a minimal depthwise-separable convolutional U-Net. Models were evaluated on the EEGDenoiseNet benchmark, cross-dataset BCI transfer tests, controlled baseline retraining, and downstream motor-imagery classification with five decoder families across all nine BCI Competition IV-2a subjects. Reconstruction performance saturated by 3-6.5K parameters, with post-elbow gains of at most 0.015 correlation coefficient per log10-parameter unit. An 8.46M-parameter baseline retrained under the same pipeline matched the 40.26K compact variant on EOG--a 200x parameter gap yielding no advantage--while a Patch-Transformer control reproduced the same diminishing-return shape. Downstream evaluation exposed a classifier-dependent metric-utility gap: reconstruction-optimized denoising significantly degraded CSP+LDA classification across all nine subjects and three artifact types (best denoised accuracy 0.547 vs. 0.612 noisy baseline; Bonferroni p=0.0488), persisting on naturally recorded trials (Delta=-0.047; BH-FDR q=0.0049). End-to-end neural decoders showed variable or neutral effects. Standard EEG denoising benchmarks are saturated far below current model capacity, and reconstruction metrics do not predict BCI utility. Ultra-compact models at 33-46 KB and 1.27-2.61M FLOPs/segment are practical for edge deployment. These findings argue for capacity-controlled evaluation, harder task-aware benchmarks, and mandatory downstream validation.
Jasmeet Singh Bindra, Siddharth Panwar
Indian Knowledge Systems and Mental Health Applications (IKSMHA) Center, Indian Institute of Technology Mandi, Himachal Pradesh, India · School of Computing and Electrical Engineering, Indian Institute of Technology Mandi, Himachal Pradesh, India
Foundation models for EEG analysis are still in their infancy, limited by two key challenges: (1) variability across datasets caused by differences in recording devices and electrode configurations, and (2) the low signal-to-noise ratio (SNR) of EEG, where neural activity is often buried under artifacts and non-brain sources. To address these challenges, we present EEG-X, a device-agnostic and noise-robust foundation model for EEG representation learning. EEG-X introduces a location-based channel embedding that enables robust transfer across heterogeneous EEG devices and electrode layouts. To improve robustness against noise, EEG-X employs a novel noise-aware masking-reconstruction strategy that reconstructs artifact-cleaned signals rather than raw noisy EEG and introduces a novel Dictionary-based Convolutional Transformation (DiCT) that improves reconstruction-based pretraining by comparing signals in a structured feature space instead of directly in the raw signal space. Experiments across datasets collected from diverse EEG devices show that EEG-X consistently outperforms state-of-the-art methods across multiple downstream tasks and demonstrates strong cross-domain generalization when pretraining and downstream datasets differ in electrode layouts and recording configurations paving the way towards foundation models. The models and code are available at https://github.com/Emotiv/EEG-X.
Navid Mohammadi Foumani, Soheila Ghane, Nam Nguyen +3
Emotiv Research, Melbourne, Australia · Emotiv Research, Sydney, Australia · Monash University, Melbourne, Australia