Cephalonauts One: A deep fMRI dataset for decoding naturalistic speech in the human brain

Antoine Collas; Louis Jalouzot; Géraud Ilinca; Corentin Caris; Romain Valabrègue; Ahmed Hassayoune; David Goncalves; Madeleine Hueber

cs.LGOct 2, 2026

Cephalonauts One: A deep fMRI dataset for decoding naturalistic speech in the human brain

Authors: Antoine Collas, Louis Jalouzot, Géraud Ilinca, Corentin Caris, Romain Valabrègue, Ahmed Hassayoune, David Goncalves, Madeleine Hueber,

Organizations: Karavela · UNICOG, CNRS, INSERM, CEA, Paris-Saclay University · LSCP, EHESS, ENS, CNRS, PSL University · Centre de NeuroImagerie de Recherche (CENIR), Sorbonne Université, ICM, Paris, France ·

Abstract

Cephalonauts One is a whole-brain 3 Tesla (3T) functional magnetic resonance imaging (fMRI) dataset recorded while subjects listened to audio podcasts. Three healthy subjects underwent multiple scanning sessions, each consisting of five 15-minute runs, while listening to podcasts in their native language. With 30 hours of fMRI data per subject, the current release is the deepest available fMRI dataset using naturalistic speech stimuli. The dataset pairs brain activity with the corresponding podcast audio, transcript annotations, and derived stimulus embeddings. Furthermore, we introduce a brain decoding benchmark formulated as audio segment retrieval: given fMRI activity from a held-out session, the decoder must identify the corresponding time-aligned podcast audio segment among candidate segments. We provide standardized splits, evaluation metrics, and baseline decoders for this task. Finally, a scaling analysis shows that decoding performance improves continuously with the amount of training data per subject.

Figures & tables

Figure 1: Overview of the Cephalonauts One dataset. A. Three subjects listened to podcasts during fMRI acquisition. B. Each session included five 15-minute runs, three comprehension questions after each run, and a final anatomical scan. Runs 1–2, 3–4, and 5 used three distinct podcasts and podcasts were not repeated across sessions.

Dataset	Modality	Nb. of subjects	Max hours per subject	Benchmark- oriented	References
Cephalonauts One	fMRI	3	30	Yes	This work
LibriBrain	MEG	1	52.3	Yes	Özdogan et al.,2025
LeBel et al.	fMRI	8 / 3	6 / 16	No	LeBel et al.,2023
Le Petit Prince	fMRI	112	1.7	No	Li et al.,2022
Armeni et al.	MEG	3	10	No	Armeni et al.,2022
Narratives	fMRI	345	2	No	Nastase et al.,2021

Table 1: Brain datasets using speech stimuli. Maximum hours per subject reports the maximum subject-specific stimulus duration. For LeBel et al. (2023) we report the 2 available data regimes. Benchmark-oriented denotes releases with benchmark tasks, metrics, baselines, and code.

Subject	# TRs	# sessions	# runs	Duration (h)
sub-0	207,283	24	120	29.94
sub-1	209,366	24	121	30.24
sub-2	201,647	24	117	29.13
Total	618,296	72	358	89.31

Table 2: Dataset size by subject.

Space	Vol.	Surf.	Subject anat.	Template anat.	$V$	Description
T1w	Yes	No	Yes	No	$\sim$ 50,000	whole-brain in each subject’s native anatomical space
MNI	Yes	No	No	Yes	69,765	whole-brain normalized to 3 mm MNI152NLin6Asym standard space
fsaverage5	No	Yes	No	Yes	20,484	both hemispheres on the standard fsaverage5 surface

Table 3: Released brain data. We release three versions of the preprocessed fMRI data generated with DeepPrep : native T1w space, MNI space, and the fsaverage5 cortical surface.

V

denotes the number of voxels or surface vertices.

Modality	Representation	$D$	Description
Text	segments	–	overlapping 20-word transcript contexts sampled at retained onsets
	embeddings	4,096	embeddings of transcript segments computed with Qwen3-Embedding-8B
Audio	segments	–	10-second audio segments ending at retained onsets
	embeddings	3,584	embeddings of audio segments computed with LCO-Embedding-Omni-7B

Table 4: Released stimulus representations. For each stimulus modality, we release both segment-level data and embeddings from foundation models.

D

denotes the embedding dimensionality.

Figure 2: Dataset construction and decoding framework. A. For each frame $t$ , stimulus ${\bm{z}}_{t}$ is aligned with fMRI recordings sampled every 520 ms. Target stimulus embeddings ${\bm{y}}_{t}$ summarize the audio waveform or transcript over the stimulus context leading up to $t$ . The corresponding brain input uses preprocessed fMRI volumes shifted by a fixed lag . In this illustration, the stimulus context spans 1.56 seconds (3 TRs) and the lag is set to 1.04 seconds (2 TRs). In the experiments, we set the lag to 2.5 seconds and the stimulus context to 10 seconds. B. A pretrained encoder maps stimulus ${\bm{Z}}$ to stimulus embeddings ${\bm{Y}}$ . The decoder $f_{\theta}$ is trained to predict ${\bm{Y}}$ from pre-processed brain recordings ${\bm{X}}$ .

Figure 3: Speech decoding performance versus training-set size. Test performance is shown versus training hours for each subject, using Median Relative Rank, Top-10@2000, and Cosine Similarity. Lower is better for Median Relative Rank; higher is better for the other metrics. Error bars show one empirical standard deviation across five folds with distinct held-out test sessions. Performance improves with additional subject-specific data, with no clear saturation. Chance level is 0.5 for Median Relative Rank and

\sim

0.5

\%

for Top-10@2000.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Single brain data frame from Cephalonauts One . This image is a screenshot obtained with niivue ( https://niivue.com/ ) of a single brain volume from Cephalonauts One , after preprocessing and before masking. It shows all three views of the brain (axial, coronal and sagittal) in the native anatomical space of one participant.

Parameter	Functional MRI	Anatomical MRI
Sequence family	2D gradient-echo EPI	3D T1-weighted spoiled gradient-echo
Orientation	Transverse	Sagittal
FOV (read $\times$ phase)	$198\times 198$ mm 2	$230\times 230$ mm 2
Voxel size	$3.0\times 3.0\times 3.0$ mm 3	$0.8\times 0.8\times 0.8$ mm 3
Slice thickness	3.0 mm	0.8 mm
TR	520 ms	6.20 ms

Appendix

Table S1: Core acquisition parameters for the functional and anatomical MRI sequences.

Figure S2: Decoding framework. A. Brain activity is represented as a sequence of fMRI volumes sampled every 520 ms (=TR). For each frame $t$ , we associate a stimulus ${\bm{z}}_{t}$ derived from the content perceived while the brain data was recorded during that segment, using either the raw audio or the transcript. Then, a stimulus embedding ${\bm{y}}_{t}$ is computed over a window of $\tau$ consecutive segments ( stimulus context ). The brain data ${\bm{x}}$ used to decode ${\bm{y}}_{t}$ consists of brain volumes ${\bm{w}}_{t}$ temporally smoothed over a window of $\kappa$ frames, and taken with a time lag $\delta$ with regard to $t$ . In this illustration, the temporal smoothing window $\kappa$ is set to 3 frames – or TRs, i.e. it is set to 3 $\times$ 520ms – the stimulus context $\tau$ is set to 3 frames, and the lag $\delta$ is set to 2 frames. B. A pretrained encoder produces stimulus embeddings ${\bm{Y}}$ from the stimulus ${\bm{Z}}$ , and a decoder $f_{\theta}$ is trained to predict these embeddings from the brain recordings ${\bm{X}}$ . Decoding can therefore be performed toward text-based or audio-based stimulus representations or even a concatenation of both. In this figure, variables used to decode stimuli perceived during frame $t$ are shown in black, while variables not used for decoding are shown with reduced opacity.

Hyperparameter	Value
Decoder architecture	one-hidden-layer MLP
Hidden dimension	4096
Activation function	GELU
Dropout	0.5
Hemodynamic lag	2.5 seconds
Text context window	20 words

Appendix

Table S2: Decoding hyperparameters. Hyperparameters used for all decoding experiments unless otherwise specified.

Figure S3: Supplementary scaling analysis across brain spaces and stimulus targets. Median Relative Rank is shown as a function of subject-specific training hours for each released brain representation and stimulus target. The brain representations correspond to the preprocessing outputs listed in Table 3 : fsaverage5 , T1w , and MNI . The stimulus targets correspond to the text, audio, and concatenated embeddings described in Table 4 . Lower values indicate better retrieval performance.

Neural Decoding

Neural decoding maps brain activity or error-correction syndromes to stimuli, behavior, language, or corrective actions.

See all 179 papers

Decoding Naturalistic Speech from fMRI with Temporal Modeling and Multimodal Alignment
Oct 15, 2026Corentin Caris, Louis Jalouzot, Nona Rajabi +9Neural Decoding
Karavela · UNICOG, CNRS, INSERM, CEA, Paris-Saclay University · LSCP, EHESS, ENS, CNRS, PSL University +1
LibriBrain100: One Hundred Hours of Broad and Deep MEG Data for Neural Speech Decoding at Scale
Aug 25, 2026Francesco Mantegna, Dulhan Jayalath, Gereon Elvers +12Brain-to-Speech Decoding Neural Decoding
PNPL, Department of Engineering Science, University of Oxford, UK
MindAlign: Decoding Inner Speech from fMRI Signals via Multimodal Embedding Alignment under Limited Data
Jun 15, 2026Muxuan Liu, Ichiro Kobayashi, Satoshi NishidaNeural Decoding Neuroimaging
Graduate School of Humanities and Sciences Ochanomizu University · Center for Information and Neural Networks Advanced ICT Research Institute National Institute of Information and Communications Technology

Oct 15, 2026J/K move · Enter open · S save

Decoding Naturalistic Speech from fMRI with Temporal Modeling and Multimodal Alignment

Corentin Caris, Louis Jalouzot, Nona Rajabi, Géraud Ilinca+8

Karavela · UNICOG, CNRS, INSERM, CEA, Paris-Saclay University · LSCP, EHESS, ENS, CNRS, PSL University+1

Neural Decoding

Open paper AskPDF arXiv

Reconstructing what a person perceives from their brain activity is a stringent test of whether neural recordings carry meaningful information. Naturalistic speech offers a rich testbed for this challenge: it unfolds over multiple timescales, but fMRI's noisy, temporally smeared measurements make this structure hard to recover. We introduce NOEMA, a temporal convolutional decoder that aligns windows of consecutive fMRI measurements with multimodal stimulus embeddings. These embeddings combine complementary text, audio, and speech representations of the same stimulus segment, enabling its retrieval directly from brain activity. On two naturalistic speech datasets, Cephalonauts One and LeBel, NOEMA retrieves the correct segment among 2,000 candidates within the top 10 for more than 42% of held-out queries. This substantially outperforms baseline decoders, an advantage that persists when the candidate pool is extended with thousands of distractors absent from training. Controlled ablations establish the contributions of temporal modeling and complementary stimulus representations.

Abstract

Neural Decoding

Explore similar work

Decoding Naturalistic Speech from fMRI with Temporal Modeling and Multimodal Alignment

LibriBrain100: One Hundred Hours of Broad and Deep MEG Data for Neural Speech Decoding at Scale

MindAlign: Decoding Inner Speech from fMRI Signals via Multimodal Embedding Alignment under Limited Data