This study investigates robust speech-related decoding from non-invasive MEG signals using the LibriBrain phoneme-classification benchmark from the 2025 PNPL competition. We compare residual convolutional neural networks (CNNs), an STFT-based CNN, and a CNN--Transformer hybrid, while also examining the effects of group averaging, label balancing, repeated grouping, normalization strategies, and data augmentation. Across our in-house implementations, preprocessing and data-configuration choices matter more than additional architectural complexity, among which instance normalization emerges as the most influential modification for generalization. The strongest of our own models, a CNN with group averaging, label balancing, repeated grouping, and instance normalization, achieves 60.95% F1-macro on the test split, compared with 39.53% for the plain CNN baseline. However, most of our models, without instance normalization, show substantial validation-to-test degradation, indicating that distribution shift induced by different normalization statistics is a major obstacle to generalization in our experiments. By contrast, MEGConformer maintains 64.09% F1-macro on both validation and test, and saliency-map analysis is qualitatively consistent with this contrast: weaker models exhibit more concentrated or repetitive phoneme-sensitive patterns across splits, whereas MEGConformer appears more distributed. Overall, the results suggest that improving the reliability of non-invasive phoneme decoding will likely require better handling of normalization-related distribution shift while also addressing the challenge of single-trial decoding.
Figures & tables
Figure 1: Raw phoneme-class distribution for the train and validation splits.
Split
Trials
Phonemes
Standardization
Train
76
1,336,606
Train statistics
Validation
14
283,690
Train statistics
Test
14
283,690
Test statistics
Holdout
1
2,382
Holdout statistics
Total
91
1,622,678
–
Table 1: Summary of LibriBrain phoneme classification dataset splits.
Figure 2: Overall model architecture. (a) CNN backbone. (b) STFT-CNN. (c) CNN-Transformer hybrid. For all convolutional blocks, the stride S is set to 1 , and padding P is chosen to preserve the temporal resolution unless otherwise specified.
Configuration [-0.15ex] (Ungrouped)
Train [-0.15ex] (Ungrouped)
Val. [-0.15ex] (Ungrouped)
Test [-0.15ex] (Ungrouped)
Test -0.15ex
Holdout [-0.15ex] (Ungrouped)
CNN, Label-Balancing, Repeat ×10
90.81
71.95
47.47
2.73
35.40
CNN, +LayerNorm
96.62
44.49
43.17
–
24.40
CNN, No Group Averaging
34.55
45.08
39.53
N/A
13.20
CNN + Augmentation (Section 3.1.4 )
48.55
49.31
34.03
–
18.80
CNN-Transformer
85.83
68.02
30.70
0.14
3.90
CNN, Label-Balancing, Repeat ×10 , +InstanceNorm
99.43
62.75
60.95
4.00
–
Table 2: F1-macro scores (%) for LibriBrain phoneme classification.
Figure 3: Row-wise normalized layer saliency maps for MEGConformer across validation and test splits. The saliency is visually stable across splits and distributed across encoder blocks, matching the strong validation–test transfer reported in Table 2 .
Figure 4: Row-wise normalized layer saliency maps for the three models for which both standard and InstanceNorm variants are available. In each 3-column block, the upper row shows validation maps and the lower row shows test maps. Each row within a saliency map corresponds to a trainable sublayer, and each column corresponds to a phoneme class. The color intensity indicates the relative saliency of each phoneme for that layer, normalized within the row.
Figure 5: Layer-by-phoneme validation-test saliency similarity matrices for MEGConformer. Both metrics show near-uniformly high correspondence across layers and phoneme classes.
Figure 6: Layer-by-phoneme validation-test saliency similarity matrices for the three models with both standard and InstanceNorm variants. In each 3-column block, the upper row shows Pearson correlation and the lower row shows Spearman correlation. Brighter cells indicate stronger agreement between validation and test saliency patterns for the same layer-phoneme pair.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Train
Validation
Test
Holdout
Backbone
GroupAvg.
LabelBal.
Repeat
Note
90.81
71.95
47.47
35.40
(a)
✓
✓
10
96.62
44.49
43.17
24.40
(a)
✓
✓
1
+LayerNorm
80.06
71.77
44.28
21.60
(a)
✓
✓
1
48.55
49.31
34.03
18.80
(a)
✓
✗
1
+Augmentation
34.55
45.08
39.53
13.20
(a)
✗
✗
1
Baseline
68.16
64.27
47.74
3.60
(a)
✓
✓
1
+Augmentation
Appendix
Table 4: Extended comparison of model and data configurations. Performance is reported in F1-macro (%).
We propose MEBM-Phoneme, a multi-scale enhanced neural decoder for phoneme classification from non-invasive magnetoencephalography (MEG) signals. Built upon the BrainMagic backbone, MEBM-Phoneme integrates a short-term multi-scale convolutional module to augment the native mid-term encoder, with fused representations via depthwise separable convolution for efficient cross-scale integration. A convolutional attention layer dynamically weights temporal dependencies to refine feature aggregation. To address class imbalance and session-specific distributional shifts, we introduce a stacking-based local validation set alongside weighted cross-entropy loss and random temporal augmentation. Comprehensive evaluations on LibriBrain Competition 2025 Track2 demonstrate robust generalization, achieving competitive phoneme decoding accuracy on the validation and official test leaderboard. These results underscore the value of hierarchical temporal modeling and training stabilization for advancing MEG-based speech perception analysis.
Liang Jinghua, Zhang Zifeng, Li Songyi +1
Speech and Hearing Research Center, School of Intelligence Science and Technology, Peking University, Beijing, China 100871 · Center for BioMed-X Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China 100871
We propose MEBM-Speech, a multi-scale enhanced neural decoder for speech activity detection from non-invasive magnetoencephalography (MEG) signals. Built upon the BrainMagic backbone, MEBM-Speech integrates three complementary temporal modeling mechanisms: a multi-scale convolutional module for short-term pattern extraction, a bidirectional LSTM (BiLSTM) for long-range context modeling, and a depthwise separable convolutional layer for efficient cross-scale feature fusion. A lightweight temporal jittering strategy and average pooling further improve onset robustness and boundary stability. The model performs continuous probabilistic decoding of MEG signals, enabling fine-grained detection of speech versus silence states - an ability crucial for both cognitive neuroscience and clinical applications. Comprehensive evaluations on the LibriBrain Competition 2025 Track1 benchmark demonstrate strong performance, achieving an average F1 macro of 89.3% on the validation set and comparable results on the official test leaderboard. These findings highlight the effectiveness of multi-scale temporal representation learning for robust MEG-based speech decoding.
Li Songyi, Zheng Linze, Liang Jinghua +1
Speech and Hearing Research Center, School of Intelligence Science and Technology, Peking University, Beijing, China 100871 · Center for BioMed-X Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China 100871
Data-efficient neural decoding is a central challenge for speech brain-computer interfaces. We present the first demonstration of transfer learning and cross-task decoding for MEG-based speech models spanning perception and production. We pre-train a Conformer-based model on 50 hours of single-subject listening data and fine-tune on just 5 minutes per subject across 18 participants. Transfer learning yields consistent improvements, with in-task accuracy gains of 1-4% and larger cross-task gains of up to 5-6%. Not only does pre-training improve performance within each task, but it also enables reliable cross-task decoding between perception and production. Critically, models trained on speech production decode passive listening above chance, confirming that learned representations reflect shared neural processes rather than task-specific motor activity.
Xabier de Zuazo, Vincenzo Verbeni, Eva Navas +3
HiTZ Center, University of the Basque Country – UPV/EHU, Spain · Basque Center on Cognition, Brain and Language – BCBL, Spain · Laboratory of Functional Anatomy, Faculty of Human Motor Sciences, Université libre de Bruxelles (ULB), Belgium +3