Logit Averaging During Training Sets a New State of the Art for Non-Invasive Speech BCI

Miran Özdogan; Gereon Elvers; Michael Bronstein; Oiwi Parker Jones

cs.LGOct 19, 2026

Logit Averaging During Training Sets a New State of the Art for Non-Invasive Speech BCI

Authors: Miran Özdogan, Gereon Elvers, Michael Bronstein, Oiwi Parker Jones

Organizations: University of Oxford

Abstract

Non-invasive brain-computer interfaces are in practice limited by signal-to-noise ratio, and recent state-of-the-art methods address this limitation by aggregating over repeated trials of the same stimulus. Yet multi-trial aggregation has been treated inconsistently across the literature: some work averages brain signals at the model's input, other work averages predictions at the model's output, and neither treats aggregation as an architectural choice in its own right. We provide the first systematic comparison of aggregation strategies for non-invasive neural decoding, evaluating six paradigms (including logit averaging during training and three attention-based variants we introduce) across three datasets, four tasks, and two group sizes. We find that aggregation choice matters substantially: training-time logit averaging, the simplest method we propose, outperforms all others in 6 of 7 settings. On the phoneme classification setting of the 2025 PNPL competition, it sets a new state of the art, significantly outperforming the competition winner by 17.4 absolute percentage points in balanced accuracy. We further draw a theoretical connection between logit averaging and joint ensemble training, which has been shown to underperform independent ensembling in the deep learning literature; our results call into question recent explanations of why joint ensemble training fails. Together, these findings establish multi-trial aggregation as a key design choice for non-invasive neural decoding and identify training-time logit averaging as a simple, effective default.

Figures & tables

Figure 1: Multi-sample aggregation paradigms. A group of $g$ trials elicited by the same stimulus passes through a shared CNN backbone. The paradigms compared in this paper differ in where along the pipeline aggregation across trials occurs: at the input (prior work), at intermediate features via learned attention or weighting (ours), at the logits during training (ours), or after softmax at evaluation time (prior work).

		Prior work		Ours
Dataset	Task	Input avg.	Post-hoc avg.	Logit avg.
LibriBrain	Phoneme ( $g=10$ )	$.176\pm.008$	$.076\pm.003$	$\mathbf{.241\pm.011}^{\ast\ast}$
	Phoneme ( $g=100$ )	$.565\pm.028$	$.113\pm.014$	$\mathbf{.641\pm.051}^{\ast\ast}$
	Voicing ( $g=10$ )	$.780\pm.007$	$.508\pm.005$	$\mathbf{.794\pm.011}$
	Word top-10 ( $g=10$ )	$.544\pm.034$	$.400\pm.016$	$\mathbf{.564\pm.019}$
MegNIST	Digit ( $g=10$ )	$.432\pm.048$	$\mathbf{.526\pm.043}$	$.418\pm.008$

Table 1: Test balanced accuracy across three primary aggregation paradigms and seven (dataset, task, group size) conditions. Values are mean

\pm

standard deviation across 5 random seeds. The best mean in each row is shown in bold . Significance markers indicate paired

t

-tests against input averaging: ∗ for

p<0.05

, ∗∗ for

p<0.01

. Results for the three additional aggregation methods we evaluate are reported in Appendix E .

Method	Balanced accuracy	F1 macro
Conformer (PNPL winner)	$0.467\pm 0.048$	$0.440\pm 0.047$
Logit averaging (ours)	$\mathbf{0.641\pm 0.051}^{\ast}$	$\mathbf{0.605\pm 0.058}^{\ast}$

Table 2: Comparison against the conformer architecture that won the phoneme track of the 2025 PNPL competition Landau et al. (2025) on the LibriBrain phoneme classification task at

g=100

. Both methods are trained on the same data with matched random seeds. Values are mean

\pm

standard deviation across 5 seeds. F1 macro is the metric used for the competition leaderboard. ∗ indicates statistical significance vs. the conformer baseline at

p<0.05

(two-sided paired

t

-test).

	Max-mean margin		Max softmax
Dataset	Logit avg.	Post-hoc	Logit avg.	Post-hoc
MegNIST	$21.4\pm 1.2$	$2.14\pm 0.27$	$0.911\pm 0.001$	$0.401\pm 0.035$
Phoneme	$11.9\pm 0.9$	$1.67\pm 0.15$	$0.663\pm 0.026$	$0.081\pm 0.002$
Phoneme voicing	$2.09\pm 0.18$	$0.41\pm 0.03$	$0.900\pm 0.007$	$0.679\pm 0.010$
Word top-10	$18.6\pm 7.1$	$1.60\pm 0.19$	$0.868\pm 0.043$	$0.317\pm 0.026$

Table 3: Per-trial logit and probability statistics of trained models, evaluated on ungrouped trials. Max-mean margin denotes the difference between the maximum logit and the mean logit per trial. Max softmax denotes the probability assigned to the top-predicted class. Values are mean

\pm

standard deviation across 5 seeds.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Dataset	$K$	$C\times T$	$g$	Stimulus
Phoneme Classification	LibriBrain	39	$306\times 125$	10, 100	Audiobook
Phoneme Voicing	LibriBrain	2	$306\times 125$	10	Audiobook
Word Classification (top-10)	LibriBrain	10	$306\times 250$	10	Audiobook
Phoneme Classification	Armeni	39	$269\times 125$	10	Audiobook
Phoneme Voicing	Armeni	2	$269\times 125$	10	Audiobook
Digit Classification	MegNIST	10	$306\times 250$	10	Imagined Speech

Appendix

Table 4: Datasets and tasks evaluated in this paper.

K

denotes the number of classes;

C\times T

the number of sensor channels and time samples per trial;

g

the group size, i.e. the number of trials aggregated per prediction. All recordings are downsampled to 250 Hz.

Dataset	Task	LWA	CTA	LQA
LibriBrain	Phoneme ( $g=10$ )	$.170\pm.014$	$.168\pm.005$	$.184\pm.009$
	Phoneme ( $g=100$ )	$.558\pm.045$	$.595\pm.033$	$.561\pm.014$
	Voicing ( $g=10$ )	$.792\pm.006$	$.792\pm.013$	$.778\pm.016$
	Word top-10 ( $g=10$ )	$.506\pm.046$	$.514\pm.053$	$.515\pm.060$
MegNIST	Digit ( $g=10$ )	$.482\pm.053^{\ast}$	$.462\pm.044$	$.436\pm.030$
Armeni	Phoneme ( $g=10$ )	$.216\pm.008$	$.213\pm.003$	$.211\pm.007$

Appendix

Table 5: Test balanced accuracy of the three attention-based aggregation methods across all seven (dataset, task, group size) conditions. LWA: learned weighted averaging. CTA: cross-trial attention. LQA: learned query attention. Values are mean

\pm

standard deviation across 5 random seeds. Significance markers indicate paired

t

-tests against input averaging: ∗ for

p<0.05

, ∗∗ for

p<0.01

		Macro recall			Macro $F_{1}$			$\bar{p}_{\text{true}}$
Paradigm	$g$	low	mid	high	low	mid	high	low	mid	high
Input averaging	10	0.145	0.165	0.214	0.085	0.159	0.212	0.098	0.100	0.110
Logit averaging	10	0.201	0.239	0.277	0.141	0.225	0.287	0.131	0.151	0.156
Post-hoc averaging	10	0.000	0.008	0.207	0.000	0.014	0.130	0.023	0.034	0.047
Input averaging	100	0.667 †	0.471	0.636	0.113 †	0.410	0.634	0.666 †	0.455	0.608
Logit averaging	100	0.600 †	0.601	0.690	0.126 †	0.531	0.693	0.536 †	0.577	0.657

Appendix

Table 6: Per-class performance on phoneme classification, broken out by training-frequency tertile. The best value in each (metric,

g

, tertile) cell is shown in bold .

\bar{p}_{\text{true}}

denotes the mean predicted probability assigned to the true class. † The low-frequency tertile at

g{=}100

contains only 3 of the 13 classes assigned to the bucket; the remaining 10 have insufficient test samples to form complete groups and are excluded. Values are means over 5 seeds.

	Balanced accuracy		ECE
Dataset	Logit avg.	Post-hoc	Logit avg.	Post-hoc
MegNIST	$0.196\pm 0.009$	$0.245\pm 0.002$	$0.715\pm 0.009$	$0.156\pm 0.034$
Phoneme	$0.060\pm 0.002$	$0.052\pm 0.003$	$0.899\pm 0.011$	$0.021\pm 0.004$
Phoneme voicing	$0.600\pm 0.002$	$0.553\pm 0.003$	$0.276\pm 0.006$	$0.071\pm 0.015$
Word top-10	$0.272\pm 0.019$	$0.308\pm 0.006$	$0.626\pm 0.045$	$0.047\pm 0.008$

Appendix

Table 7: Single-trial evaluation of trained models, applied to ungrouped trials. Values are mean

\pm

standard deviation across 5 seeds. Lower ECE indicates better calibration of single-trial probabilities.

	Balanced accuracy		ECE
Dataset	Logit avg.	Post-hoc	Logit avg.	Post-hoc
MegNIST	$0.418\pm 0.008$	$0.520\pm 0.047$	$0.364\pm 0.010$	$0.285\pm 0.036$
Phoneme	$0.241\pm 0.011$	$0.066\pm 0.005$	$0.036\pm 0.005$	$0.124\pm 0.010$
Phoneme voicing	$0.794\pm 0.011$	$0.510\pm 0.005$	$0.058\pm 0.011$	$0.185\pm 0.021$
Word top-10	$0.564\pm 0.019$	$0.403\pm 0.025$	$0.162\pm 0.078$	$0.272\pm 0.038$

Appendix

Table 8: Group-level evaluation at

g=10

for the same models. Balanced accuracy reproduces values from Table 1 ; ECE is reported for completeness. Values are mean

\pm

standard deviation across 5 seeds.

Dataset	Method	1	2	3	4	5	6	7	8	9	10
MegNIST	Logit avg.	0.86	0.51	0.43	0.44	0.38	0.34	0.28	0.23	0.32	0.24
MegNIST	Post-hoc	0.96	0.81	0.76	0.63	0.51	0.40	0.30	0.34	0.30	0.23
Phoneme	Logit avg.	0.43	0.36	0.33	0.26	0.18	0.22	0.19	0.16	0.15	0.13
Phoneme	Post-hoc	0.14	0.12	0.09	0.08	0.08	0.07	0.05	0.07	0.07	0.06
Phoneme voicing	Logit avg.	0.97	0.97	0.88	0.92	0.87	0.81	0.77	0.73	0.67	0.52
Phoneme voicing	Post-hoc	0.70	0.50	0.50	0.50	0.50	0.50	0.50	0.50	0.50	0.52

Appendix

Table 9: Per-decile balanced accuracy for group-level predictions (

g=10

), ranked by group-level confidence (decile 1: most confident; decile 10: least confident). Values are means across 5 seeds.

Brain-Computer Interfaces

Brain-computer interfaces translate neural activity into commands or feedback for communication, movement, rehabilitation, and sensory restoration.

See all 130 papers

Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text
Sep 30, 2026Dulhan Jayalath, Oiwi Parker JonesNeural Decoding Brain-to-Speech Decoding
Neural Processing Lab (PNPL) Department of Engineering Science, University of Oxford
Physiological Noise Augmentation Improves Non-Invasive Brain-to-Speech
Jul 6, 2026Benjamin Ballyk, Teyun Kwon, Miran Özdogan +1Brain-Computer Interfaces Data Augmentation
PNPL Department of Engineering Science University of Oxford
The 2026 PNPL Competition: Word Classification and Efficient Cross-Subject Generalisation in LibriBrain100
Sep 3, 2026Francesco Mantegna, Gereon Elvers, Dulhan Jayalath +18Brain-Computer Interfaces Neural Decoding
PNPL, Department of Engineering Science, University of Oxford, UK · 2OHBA, Oxford Centre for Integrative Neuroimaging, University of Oxford, UK · 3FMRIB, Oxford Centre for Integrative Neuroimaging, University of Oxford, UK +6

Sep 30, 2026J/K move · Enter open · S save

Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text

Dulhan Jayalath, Oiwi Parker Jones

Neural Processing Lab (PNPL) Department of Engineering Science, University of Oxford

Neural Decoding Brain-to-Speech Decoding Shortcut Learning

Open paper AskPDF arXiv

We find that major reported improvements in decoding words from non-invasive brain recordings are largely reproducible without any brain data. In the influential work of d'Ascoli et al. (2025), time series of brain activity from subjects perceiving continuous speech are segmented into fixed-length windows starting at each word. A neural network then generates predictions for all of the words in a sentence together. Neighbouring windows partially overlap, implicitly revealing the interval between words. Since these intervals indicate the duration of the words spoken, and different words tend to have different durations - for example, "the" is much shorter than "supercalifragilisticexpialidocious" - the neural network can improve its predictions of words without relying on the underlying brain activity. Consistent with this, the method reaches 22.0% balanced accuracy on synthetic signals containing no brain information, compared with 22.3% on real brain recordings. To prevent the network from learning this shortcut, we make a single, simple change. Instead of jointly encoding all windows in a sentence, we process each independently. As a result, the neural network achieves better performance by learning underlying word-specific information from brain recordings. This makes two existing strategies become much more effective than before. Both aggregating predictions from distinct neural responses to the same word and using a pretrained LLM as a linguistic prior now substantially improve results. On our perceived speech benchmark, this simple recipe (SimpleB2T) achieves a word error rate of 36.6% with five observations per word, approaching past invasive speech decoding performance, albeit under different conditions. The results in this work expose an important shortcut in brain-to-text decoding and show that removing it leads to a simple and considerably more effective strategy.

Abstract

Brain-Computer Interfaces

Explore similar work

Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text

Physiological Noise Augmentation Improves Non-Invasive Brain-to-Speech

The 2026 PNPL Competition: Word Classification and Efficient Cross-Subject Generalisation in LibriBrain100