q-bio.NCNov 11, 2026

Prototypes of the Mind: A Unified Framework for Probing the Visual Brain

Authors: Shi Chen, Seoyoung Ahn, Doris Tsao

Organizations: Department of Neuroscience, UC Berkeley · College of AI Convergence, Hankuk University of Foreign Studies

Abstract

Deep learning has driven substantial progress in modeling fMRI responses, yet most existing approaches are optimized for a single objective---such as neural encoding, stimulus decoding, or post hoc interpretation---making it difficult to obtain a coherent account of neural representations. We introduce a unified framework that connects vision foundation models to brain activity while supporting neural analysis from multiple complementary perspectives. Our approach replaces black-box features with a sparse, interpretable representation composed of visual prototypes, and links these prototypes to voxel responses through a linear readout. This design makes the model transparent at both levels: prototype activations expose the visual concepts present in the stimulus, while the learned linear weights characterize each voxel's selectivity. Using this interface, we identify which visual concepts drive neural responses and where they appear in the image, reveal cortical organization through voxel tuning in prototype space, and perform controlled stimulus synthesis to probe cortical selectivity through targeted interventions. Together, these results establish prototype-based representations as a unified and interpretable foundation for studying visual representations in the human brain.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Mechanistically Interpretable Neural Encoding Reveals Fine-Grained Functional Selectivity in Human Visual Cortex

    May 15, 2026Idan Daniel Grosbard, Mor Geva, Galit YovelMechanistic InterpretabilityVisual Neuroscience

  2. NeuroVolve: Evolving personalized visual stimuli toward user-defined brain response targets

    Nov 29, 2025Haomiao Chen, Keith W Jamison, Daniel Chong +3Visual NeuroscienceNeuroimaging