Bidirectional representational alignment between biological and artificial neural networks
Authors: Samuel Kostousov, Abhinn Kaushik, Brokoslaw Laschowski
Organizations: Department of Physics, University of Toronto, Toronto, ON, Canada · Department of Computer Science, University of Toronto, Toronto, ON, Canada · Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON, Canada; and the KITE Research Institute, University Health Network, Toronto, ON, Canada
Recent work has shown that representational alignment between biological and artificial neural networks is asymmetric: model representations predict neural responses much better than neural responses predict model representations. This asymmetry raises the question of whether representational geometry contributes to bidirectional representational alignment. We hypothesized that steering representational geometry during training can systematically influence bidirectional alignment. To test this hypothesis, we developed a computational framework that integrates spectral regularization with bidirectional predictivity analyses. As an initial demonstration, we evaluated our framework using self-supervised contrastive vision models. Steering the spectral geometry of the learned representations substantially increased reverse predictivity with modest reductions in forward predictivity, yielding a 55% relative improvement in bidirectional predictivity. These improvements were accompanied by reduced effective dimensionality and a reorganization of the shared representational subspace, within which forward and reverse predictivity became approximately symmetric at intermediate spectral exponents. Overall, these findings demonstrate that representational geometry can be systematically steered to modulate bidirectional representational alignment between biological and artificial neural networks.
Figures & tables
Fig. 1: Our computational framework for studying the relationship between representational geometry and bidirectional representational alignment between biological and artificial neural networks. During training, a standard learning objective is combined with spectral regularization to steer the spectral geometry of the learned representations by regularizing the spectral decay exponent. During validation, bidirectional predictivity is evaluated using forward and reverse predictivity on stimuli with neural data.
Fig. 2: Forward and reverse explained variance across the measured spectral exponents for λ=3 . The white point is the unregularized baseline. Error bars represent the median absolute deviation.
Fig. 3: Forward and reverse explained variance across the measured spectral exponents for λ=2 . The white point is the unregularized baseline. Error bars represent the median absolute deviation.
Fig. 4: Reverse explained variance as a function of the measured spectral exponent α and regularization strength λ . The white point is the unregularized baseline. Error bars represent the median absolute deviation.
Fig. 5: Distributions of reverse explained variance across model units for λ=3 . The white distribution is the unregularized baseline. Error bands are the standard error of the mean.
Fig. 6: Bidirectional predictivity as a function of the measured spectral exponent α for models with λ=3 . The white point is the unregularized baseline. Error bars represent the median absolute deviation.
Fig. 7: Distributions of forward explained variance across recorded neurons for models with λ=3 . The white distribution represents the unregularized baseline. Error bands are the standard error of the mean.
Fig. 8: Forward explained variance as a function of the measured spectral exponent α and spectral regularization strength λ . The white point is the unregularized baseline. Error bars are the median absolute deviation.
Fig. 9: Forward and reverse explained variance for common model units (top 20% ranked by reverse predictivity). The white point denotes the unregularized baseline. Error bars are the median absolute deviation.
Fig. 10: Forward and reverse explained variance for unique model units (bottom 20% ranked by reverse predictivity). The white point denotes the unregularized baseline. Error bars are the median absolute deviation.
Fig. 11: Effective dimensionality as a function of the measured spectral exponent α . The white point represents the unregularized baseline. Error bars represent the median absolute deviation.
Tandon School of Engineering, New York University, Brooklyn, NY 11201, USA · School of Computing, University of Georgia, Athens, GA 30602, USA · Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA +1