cs.CVNov 16, 2026

REEL: Brain-Guided Video Generation and Selection by Reranking Encoder Embeddings with Linear Decoding from fMRI data

Authors: Fabrizio Spera, Tommaso Boccato, Moein Khajehnejad, Nicola Toschi, Matteo Ferrante

Organizations: Department of Biomedicine and Prevention, University of Rome Tor Vergata, Rome, Italy · Tether Evo · Monash University, Melbourne, Australia. · A.A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, USA

Abstract

Visual brain decoding from functional magnetic resonance imaging (fMRI) data has made remarkable progress in recent years, driven by increasingly powerful artificial intelligence (AI) models and the growing availability of large-scale public datasets. However, despite the impressive advances achieved in static image decoding, the reconstruction of videos from fMRI signals remains challenging due to the temporal mismatch between rapidly changing video stimuli and the slow dynamics of the blood-oxygen-level-dependent (BOLD) response. We evaluate a Bayesian-inspired proposal-and-rerank strategy on CC2017 and the BOLD Moments Dataset (BMD), using a linear semantic decoder, a pretrained video generator, and three neural encoding streams with ridge readouts and learned pooling in two streams. We refer to this framework as REEL (Reranking Encoder Embeddings with Linear decoding). Compared with a single generated candidate, reranking increases mean Video 50-way accuracy from 0.2359 to 0.2434 across three CC2017 participants and from 0.3653 to 0.3920 across ten BMD participants. These findings suggest an alternative and conceptually simpler strategy for improving video brain decoding, highlighting the potential of our approach as a complementary direction to increasingly complex pipelines.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Neurons++: A Unified Visual-Stream Framework for Image and Video Reconstruction from fMRI with Neural-to-Visual Interpretability

    Oct 9, 2026Haonan Wang, Yuchen Wang, Jingyu Lu +4fMRI-Based Visual DecodingNeural Decoding

  2. Brain-Grasp: Graph-based Saliency Priors for Improved fMRI-based Visual Brain Decoding

    Apr 12, 2026Mohammad Moradi, Morteza Moradi, Marco Grassia +1fMRI-Based Visual DecodingNeural Decoding

  3. SemVideo: Reconstructs What You Watch from Brain Activity via Hierarchical Semantic Guidance

    Feb 25, 2026Minghan Yang, Lan Yang, Ke Li +3fMRI-Based Visual DecodingNeural Decoding