Bypassing Direct Reconstruction: Speech Detection from MEG via Large-Scale Audio Retrieval

Boda Xiao; Bo Wang; Heping Cheng

cs.SDMay 13, 2026

Bypassing Direct Reconstruction: Speech Detection from MEG via Large-Scale Audio Retrieval

Authors: Boda Xiao, Bo Wang, Heping Cheng

Organizations: Center for BioMed-X Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China · Speech and Hearing Research Center, School of Intelligence Science and Technology, Peking University, Beijing, China · National Biomedical Imaging Center, State Key Laboratory of Membrane Biology, Institute of Molecular Medicine, Peking-Tsinghua Center for Life Sciences, College of Future Technology, Peking University, Beijing, China

Abstract

Decoding speech from non-invasive brain signals is challenging. For the LibriBrain 2025 Speech Detection task, we propose a novel two-step framework that bypasses direct reconstruction. First, a contrastive learning model retrieves the matching speech segment for the given test MEG from a large-scale audio library (LibriVox). Second, a speech detection model generates the binary silence/speech sequence directly from this retrieved audio. With this approach, our team Sherlock Holmes achieved first place in the extended track (F1-score: 0.962), demonstrating that leveraging external audio databases is a highly effective strategy.

Explore similar work

MindVoice: Reconstructing Intelligible Speech from Non-invasive Neural Signals with Pretrained Priors
May 29, 2026Guangyin Bao, Taiping Zeng, Jianfeng Feng +1Neural Recordings Prior Knowledge
Zero-Shot Imagined Speech Decoding via Imagined-to-Listened MEG Mapping
May 8, 2026Maryam Maghsoudi, Shihab ShammaNeural Recordings Phonetic Representations
Interpretable MEG Decoding of Perceived Speech: Cortical Sources and the Stimulus Features That Drive Retrieval
Aug 2, 2026Ilia Semenkov, Daria Kleeva, Ivan Dakhtin +2Wav2Vec Neural Recordings

May 29, 2026 · cs.SDJ/K move · Enter open · S save

MindVoice: Reconstructing Intelligible Speech from Non-invasive Neural Signals with Pretrained Priors

Guangyin Bao, Taiping Zeng, Jianfeng Feng, Xiangyang Xue

Fudan University

Reconstructing continuous speech from non-invasive neural recordings is a fundamental problem for probing human auditory perception and building safe, scalable speech brain-computer interfaces. Despite recent progress, intelligible reconstruction remains elusive, as non-invasive recordings are inherently noisy, spatially blurred, and only partially preserve information about perceived speech. Existing methods directly map neural activity to entangled speech representations before synthesizing waveforms with neural vocoders, resulting in spectral-similar but unintelligible results. To overcome these limitations, we introduce MindVoice, a neuro-to-speech reconstruction framework that uses pretrained models to compensate for the incomplete semantic and acoustic information in neural recordings. MindVoice disentangles reconstruction into two complementary pathways: one recovers high-level semantic content, while the other estimates fine-grained acoustic attributes. These inferred representations are then fused with powerful speech generation models and in-context voice cloning to synthesize natural and intelligible utterances. Extensive experiments on EEG and MEG demonstrate that MindVoice substantially outperforms existing methods on various metrics. These results show that pretrained priors provide a principled way to bridge the gap between noisy neural recordings and natural speech, highlighting a promising attempt for auditory neuroscience research and non-invasive speech brain-computer interfaces.

Neural Recordings Prior Knowledge

Open paper Ask about it PDF

Abstract

Explore similar work

MindVoice: Reconstructing Intelligible Speech from Non-invasive Neural Signals with Pretrained Priors

Zero-Shot Imagined Speech Decoding via Imagined-to-Listened MEG Mapping

Interpretable MEG Decoding of Perceived Speech: Cortical Sources and the Stimulus Features That Drive Retrieval