Cephalonauts One: A deep fMRI dataset for decoding naturalistic speech in the human brain
Authors: Antoine Collas, Louis Jalouzot, Géraud Ilinca, Corentin Caris, Romain Valabrègue, Ahmed Hassayoune, David Goncalves, Madeleine Hueber, Thaddée Delebarre, Julien Savatovsky, Clara Fonteneau, Charles Maussion, Bertrand Thirion, Alexis Thual
Organizations: Karavela · UNICOG, CNRS, INSERM, CEA, Paris-Saclay University · LSCP, EHESS, ENS, CNRS, PSL University · Centre de NeuroImagerie de Recherche (CENIR), Sorbonne Université, ICM, Paris, France · Department of Radiology, Hôpital Fondation Adolphe de Rothschild, Paris, France · Inria, CEA, Paris-Saclay University, Palaiseau, France
Cephalonauts One is a whole-brain 3 Tesla (3T) functional magnetic resonance imaging (fMRI) dataset recorded while subjects listened to audio podcasts. Three healthy subjects underwent multiple scanning sessions, each consisting of five 15-minute runs, while listening to podcasts in their native language. With 30 hours of fMRI data per subject, the current release is the deepest available fMRI dataset using naturalistic speech stimuli. The dataset pairs brain activity with the corresponding podcast audio, transcript annotations, and derived stimulus embeddings. Furthermore, we introduce a brain decoding benchmark formulated as audio segment retrieval: given fMRI activity from a held-out session, the decoder must identify the corresponding time-aligned podcast audio segment among candidate segments. We provide standardized splits, evaluation metrics, and baseline decoders for this task. Finally, a scaling analysis shows that decoding performance improves continuously with the amount of training data per subject.
Figures & tables
Figure 1: Overview of the Cephalonauts One dataset. A. Three subjects listened to podcasts during fMRI acquisition. B. Each session included five 15-minute runs, three comprehension questions after each run, and a final anatomical scan. Runs 1–2, 3–4, and 5 used three distinct podcasts and podcasts were not repeated across sessions.
Dataset
Modality
Nb. of subjects
Max hours per subject
Benchmark- oriented
References
Cephalonauts One
fMRI
3
30
Yes
This work
LibriBrain
MEG
1
52.3
Yes
Özdogan et al.,2025
LeBel et al.
fMRI
8 / 3
6 / 16
No
LeBel et al.,2023
Le Petit Prince
fMRI
112
1.7
No
Li et al.,2022
Armeni et al.
MEG
3
10
No
Armeni et al.,2022
Narratives
fMRI
345
2
No
Nastase et al.,2021
Table 1: Brain datasets using speech stimuli. Maximum hours per subject reports the maximum subject-specific stimulus duration. For LeBel et al. (2023) we report the 2 available data regimes. Benchmark-oriented denotes releases with benchmark tasks, metrics, baselines, and code.
Subject
# TRs
# sessions
# runs
Duration (h)
sub-0
207,283
24
120
29.94
sub-1
209,366
24
121
30.24
sub-2
201,647
24
117
29.13
Total
618,296
72
358
89.31
Table 2: Dataset size by subject.
Space
Vol.
Surf.
Subject anat.
Template anat.
V
Description
T1w
Yes
No
Yes
No
∼ 50,000
whole-brain in each subject’s native anatomical space
MNI
Yes
No
No
Yes
69,765
whole-brain normalized to 3 mm MNI152NLin6Asym standard space
fsaverage5
No
Yes
No
Yes
20,484
both hemispheres on the standard fsaverage5 surface
Table 3: Released brain data. We release three versions of the preprocessed fMRI data generated with DeepPrep : native T1w space, MNI space, and the fsaverage5 cortical surface. V denotes the number of voxels or surface vertices.
Modality
Representation
D
Description
Text
segments
–
overlapping 20-word transcript contexts sampled at retained onsets
embeddings
4,096
embeddings of transcript segments computed with Qwen3-Embedding-8B
Audio
segments
–
10-second audio segments ending at retained onsets
embeddings
3,584
embeddings of audio segments computed with LCO-Embedding-Omni-7B
Table 4: Released stimulus representations. For each stimulus modality, we release both segment-level data and embeddings from foundation models. D denotes the embedding dimensionality.
Figure 2: Dataset construction and decoding framework. A. For each frame t , stimulus zt is aligned with fMRI recordings sampled every 520 ms. Target stimulus embeddings yt summarize the audio waveform or transcript over the stimulus context leading up to t . The corresponding brain input uses preprocessed fMRI volumes shifted by a fixed lag . In this illustration, the stimulus context spans 1.56 seconds (3 TRs) and the lag is set to 1.04 seconds (2 TRs). In the experiments, we set the lag to 2.5 seconds and the stimulus context to 10 seconds. B. A pretrained encoder maps stimulus Z to stimulus embeddings Y . The decoder fθ is trained to predict Y from pre-processed brain recordings X .
Figure 3: Speech decoding performance versus training-set size. Test performance is shown versus training hours for each subject, using Median Relative Rank, Top-10@2000, and Cosine Similarity. Lower is better for Median Relative Rank; higher is better for the other metrics. Error bars show one empirical standard deviation across five folds with distinct held-out test sessions. Performance improves with additional subject-specific data, with no clear saturation. Chance level is 0.5 for Median Relative Rank and ∼ 0.5 % for Top-10@2000.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Single brain data frame from Cephalonauts One . This image is a screenshot obtained with niivue ( https://niivue.com/ ) of a single brain volume from Cephalonauts One , after preprocessing and before masking. It shows all three views of the brain (axial, coronal and sagittal) in the native anatomical space of one participant.
Parameter
Functional MRI
Anatomical MRI
Sequence family
2D gradient-echo EPI
3D T1-weighted spoiled gradient-echo
Orientation
Transverse
Sagittal
FOV (read × phase)
198×198 mm 2
230×230 mm 2
Voxel size
3.0×3.0×3.0 mm 3
0.8×0.8×0.8 mm 3
Slice thickness
3.0 mm
0.8 mm
TR
520 ms
6.20 ms
Appendix
Table S1: Core acquisition parameters for the functional and anatomical MRI sequences.
Figure S2: Decoding framework. A. Brain activity is represented as a sequence of fMRI volumes sampled every 520 ms (=TR). For each frame t , we associate a stimulus zt derived from the content perceived while the brain data was recorded during that segment, using either the raw audio or the transcript. Then, a stimulus embedding yt is computed over a window of τ consecutive segments ( stimulus context ). The brain data x used to decode yt consists of brain volumes wt temporally smoothed over a window of κ frames, and taken with a time lag δ with regard to t . In this illustration, the temporal smoothing window κ is set to 3 frames – or TRs, i.e. it is set to 3 × 520ms – the stimulus context τ is set to 3 frames, and the lag δ is set to 2 frames. B. A pretrained encoder produces stimulus embeddings Y from the stimulus Z , and a decoder fθ is trained to predict these embeddings from the brain recordings X . Decoding can therefore be performed toward text-based or audio-based stimulus representations or even a concatenation of both. In this figure, variables used to decode stimuli perceived during frame t are shown in black, while variables not used for decoding are shown with reduced opacity.
Hyperparameter
Value
Decoder architecture
one-hidden-layer MLP
Hidden dimension
4096
Activation function
GELU
Dropout
0.5
Hemodynamic lag
2.5 seconds
Text context window
20 words
Appendix
Table S2: Decoding hyperparameters. Hyperparameters used for all decoding experiments unless otherwise specified.
Figure S3: Supplementary scaling analysis across brain spaces and stimulus targets. Median Relative Rank is shown as a function of subject-specific training hours for each released brain representation and stimulus target. The brain representations correspond to the preprocessing outputs listed in Table 3 : fsaverage5 , T1w , and MNI . The stimulus targets correspond to the text, audio, and concatenated embeddings described in Table 4 . Lower values indicate better retrieval performance.
Reconstructing what a person perceives from their brain activity is a stringent test of whether neural recordings carry meaningful information. Naturalistic speech offers a rich testbed for this challenge: it unfolds over multiple timescales, but fMRI's noisy, temporally smeared measurements make this structure hard to recover. We introduce NOEMA, a temporal convolutional decoder that aligns windows of consecutive fMRI measurements with multimodal stimulus embeddings. These embeddings combine complementary text, audio, and speech representations of the same stimulus segment, enabling its retrieval directly from brain activity. On two naturalistic speech datasets, Cephalonauts One and LeBel, NOEMA retrieves the correct segment among 2,000 candidates within the top 10 for more than 42% of held-out queries. This substantially outperforms baseline decoders, an advantage that persists when the candidate pool is extended with thousands of distractors absent from training. Controlled ablations establish the contributions of temporal modeling and complementary stimulus representations.
Corentin Caris, Louis Jalouzot, Nona Rajabi +9
Karavela · UNICOG, CNRS, INSERM, CEA, Paris-Saclay University · LSCP, EHESS, ENS, CNRS, PSL University +1
We introduce LibriBrain100, a large-scale MEG dataset for speech decoding designed from the ground up for reproducible, standardised evaluation. LibriBrain100 more than doubles the size of the original LibriBrain release, resulting in over 100 hours of high-quality MEG acquired while subjects listened to naturalistic continuous speech. With ∼80 hours from a single subject, LibriBrain100 sets a new record for deep, within-subject neural data (8× more than the next comparable dataset and roughly 80× more than other datasets). To demonstrate the payoff of this depth-first design, we evaluate on a word-classification benchmark---an increasingly well-established stepping stone towards the open challenge of noninvasive brain-to-text decoding. Using an existing decoding model, we achieve state-of-the-art performance---validating both the quality of the recordings and the value of within-subject data at scale. Because collecting 80 hours of data per user is impractical for real-world applications, we also collected ∼40 minutes of additional data from each of 32 subjects. Using the same word-classification benchmark, we demonstrate the value of broad multi-subject data: supervised finetuning of a pre-trained model can substantially compensate for limited per-subject data. We provide standard train, validation, and test splits, all reproducible through an open-sourced Python library that supports easy downloading, optional preprocessing, and data loading for common deep learning frameworks. In addition, the dataset and evaluation infrastructure are being released alongside an open machine-learning competition with a public leaderboard for standardised benchmarking. Ultimately, our hope is that LibriBrain100 will accelerate progress towards practical non-invasive brain-computer interfaces, capable of restoring communication to people living with severe paralysis.
Francesco Mantegna, Dulhan Jayalath, Gereon Elvers +12
PNPL, Department of Engineering Science, University of Oxford, UK
Decoding inner speech from non-invasive brain signals remains a fundamental challenge due to the absence of overt linguistic output, limited training data, and large inter-subject variability. Existing brain-to-text approaches often rely on task-specific decoder fine-tuning, which restricts scalability and complicates adaptation to new participants. We propose MindAlign, a decoupled two-stage brain-to-language framework that enables open-ended text generation from fMRI signals without modifying the underlying language model. The first stage learns a subject-specific neural-semantic alignment that maps fMRI activity into a shared multimodal semantic space, extracting a latent semantic sketch of the internally generated sentence. The second stage integrates this sketch with visual context to prompt a frozen multimodal language model for free-form generation. Experiments on fMRI data collected during silent image description demonstrate that the proposed approach consistently outperforms fMRI-only and random baselines. We further show that the learned semantic-to-language projection can generalize across subjects, enabling effective decoding when paired with subject-specific neural alignment. These results indicate that neural signals modulate semantic content beyond image-driven priors, supporting a scalable and modular direction for brain-to-text decoding.
Muxuan Liu, Ichiro Kobayashi, Satoshi Nishida
Graduate School of Humanities and Sciences Ochanomizu University · Center for Information and Neural Networks Advanced ICT Research Institute National Institute of Information and Communications Technology