q-bio.NCNov 1, 2026

Modeling Visual Cortex with Layerwise Multiscale Manifold Capacity Maximization

Authors: Thomas Yerxa, SueYeon Chung, Eero P. Simoncelli

Organizations: Center for Neural Science, New York University · Department of Physics, Harvard University · Kempner Institute, Harvard University · Center for Computational Neuroscience, Flatiron Institute

Abstract

Task-optimized deep neural networks have risen to prominence as the most predictive phenomenological models of responses in primate visual cortex, but leave much to be desired from the perspective of biological plausibility. One such limitation is the reliance on precise credit assignment through global backpropagation of error signals. Recent work has shown that this weakness can be circumvented by requiring each subsequent stage to solve a distinct and increasingly complex task, allowing for layerwise local learning signals. We propose a novel strategy for crafting such intermediate losses that uses an efficient coding framework formulated in terms of manifold capacity, and which can be computed using a sequence of {canonical cortical computations}. In particular, we leverage the relationship between the multiscale nature of visual signals and the dilation of receptive field sizes in cascaded visual representations to modulate complexity, allowing for the reapplication of these common loss computations at each stage of the hierarchy. We evaluate our approach on its ability to predict neural datasets spanning three areas of the ventral stream hierarchy in macaques, as well as human psychophysical data on an object classification task. We find that our unsupervised layerwise model matches or exceeds the performance of competitive architecture-matched baselines on all evaluations considered.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Efficient coding along the visual hierarchy

    May 18, 2026Ananya Passi, Brian S. Robinson, Michael F. BonnerUnsupervised LearningNeural Processes

  2. Toward a mechanistic understanding of inference in visual cortex and diffusion models

    Jul 17, 2026Zeyu Yun, Alexander Belsten, Dasheng Bi +3Mechanistic InterpretabilityVisual Neuroscience