Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks
Authors: Lukas Braun, Erin Grant, Andrew M. Saxe
Organizations: Department of Experimental Psychology, University of Oxford, Oxford, UK · Gatsby Unit & Sainsbury Wellcome Centre, University College London, London, UK
A foundational principle of connectionism is that perception, action, and cognition emerge from parallel computations among simple, interconnected units that generate and rely on neural representations. Accordingly, researchers employ multivariate pattern analysis to decode and compare the neural codes of artificial and biological networks, aiming to uncover their functions. However, there is limited analytical understanding of how a network's representation and function relate, despite this being essential to any quantitative notion of underlying function or functional similarity. We address this question using analysable two-layer linear networks and numerical simulations in non-linear networks. We find that function and representation are dissociated, allowing representational similarity without functional similarity and vice versa. Further, we show that neither robustness to input noise nor the level of generalization error constrain representations to the task. In contrast, networks robust to parameter noise have limited representational flexibility and must employ task-specific representations. Our findings suggest that representational alignment reflects computational advantages beyond functional alignment alone, with significant implications for interpreting and comparing the representations of connectionist systems.
Figures & tables
Figure 1 : Random walk . (A) A random walk on the solution manifold of a two-layer linear network reveals that input and readout weights can change continuously, inducing changes in the (B) network parametrisation and thus the (C) hidden-layer representations, while preserving the (D) network output.
Figure 2 : Solution manifold . (A) Schematic of solution manifold (left) for a two-layer linear network trained on a single training pair (right). The general linear solution (blue plane) reflects that weight W12 lies in the input null space and is unconstrained, while W11 and W21 are coupled, an increase in one requires a decrease in the other. Constrained LSS (pink), MRNS (green), MWNS (orange) are highlighted subregions of the manifold. (B) Schematic of the parametrisation of the GLS, showing how components of Ω1 map relevant, irrelevant, and unobserved input directions to the hidden space, and how components of Ω2 map from unoccupied and occupied hidden directions to the output. Projections from irrelevant inputs can interfere with the core (blue), creating overlap (black) between the relevant (dark grey) and irrelevant (light grey) hidden space , which is cancelled by Ψ . (C) As in (B) , but for LSS. Projections from unobserved input directions into the occupied hidden space are cancelled by Φ . (D) and (E) are as in (B) , but show MRNS and MWNS, respectively. The additional constraints remove projections and further restrict the core.
Figure 3 : Hidden-layer representations . (A) Schematic of the semantic hierarchy task. (B) Inputs are encoded as random vectors (left) and corresponding target vectors encode for the position in the hierarchy (right). A one (zero) indicates that an item is (not) a child of a node. (C) Example hidden-layer representations (left), representational similarity matrix (centre) and corresponding 2D multidimensional scaling plot (right) for a general linear solution, (D) minimum representation-norm solution, and (E) minimum weight-norm solution.
Figure 4 : Implications for neural data analysis. All panels show results during random walks on the solution manifolds of least-squares solution, minimum weight-norm solution, and minimum representation-norm solution. (A) Mean and standard deviation of R2 scores for linear predictivity across n=10 random walks. All source-target combinations are shown for across-function (left) and within-function (right) comparisons. (B) Example trajectories of RSA correlation scores, shown for across-function (left) and within-function (right) comparisons. (C) MSE of a linear decoder trained on the hidden-layer representation at the initial time step. (D) Mean and expected MSE under input noise (left) and parameter noise (right).
Figure 5 : Function and representation are dissociable in non-linear networks. (A) Hidden-layer activations for 1024 MNIST inputs, grouped by class (left) and the corresponding task-specific representational similarity matrix (right) after training a ReLU network from small initial weights. (B) Same as (A) , but for a network reparametrised via augmented Lagrangian optimisation to reshape hidden-layer representation while preserving training set classifications. (C) representational similarity matrix of the network from (A) after reparametrisation using exact invariant transformations ( Section 6.1 ). (D) Test error under input noise for all exact invariant transformations. Networks with input-null expansion are sensitive. (E) As in (D) but for parameter noise; networks with scaled , nuisance , and duplicate expansions are sensitive to varying degrees.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
m=n
m<n
m>n
r=m
r<m
r=m
r<m
r=n
r<n
UTU
=Im
=Ir
=Im
=Ir
=In
=Ir
UUT
=Im
=Im
=Im
=Im
=Im
=Im
VTV
=Im
=Ir
=Im
=Ir
=In
=Ir
VVT
=Im
=Im
=In
=In
=In
=In
Appendix
Table 1 : Orthonormality of singular vectors of the compact singular value decomposition
Representations are routinely used across machine learning, psychology, and neuroscience to draw inferences about the computations of biological and artificial systems. Such inferences presume a meaningful link between representational geometry and the computation being performed. For artificial neural networks, however, the extent to which function constrains representation remains unclear. One key obstacle is that these networks admit parameter symmetries: changes in parameterization that preserve function exactly while reshaping representational geometry. Here, we show that a broad class of parameter symmetries acts on representations through just three primitive feature transformations: addition, duplication, and scaling. This feature-level characterization yields a closed-form decomposition of representational geometry into essential and auxiliary components, which makes precise how degeneracy in representational geometry can grow with overparameterization even when function is held fixed. Finally, we show that implementation-level selection rules can resolve this degeneracy, yielding identifiable geometries in which features are weighted according to their contributions to the network's function. Together, our results delineate when representations can support inferences about computation, and when they cannot.
Marvin Theiss, Lukas Braun, Andrew M. Saxe +1
University of Tübingen · International Max Planck Research School for Intelligent Systems · Allen Institute for Neural Dynamics +4
Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features. We show this matters when neural systems operate in superposition, encoding more features than they have neurons via linear compression. Closed-form derivations prove that these metrics depend on the Gram matrices of each system's projection, not on the latent features themselves: alignment thus combines what a system represents with how it is encoded. For those interested in what features two systems share, this is a problem: Two networks can have identical feature content yet appear more dissimilar than networks exhibiting partial feature overlap. This apparent misalignment need not reflect lost information as compressed sensing guarantees sparse features remain recoverable from the compressed activity. We confirm this by training supervised TopK sparse autoencoders that realize solvable compressed sensing by construction, finding alignment on recovered latents restored even when raw-activation alignment remains deflated. We extend the result to unsupervised SAEs trained without ground-truth latents, and to pretrained vision and language model SAEs, where SAE-latent alignment exceeds raw-activation alignment, consistent with superposition in real systems.
Sunny Liu, Habon Issa, André Longon +4
Cold Spring Harbor Laboratory · UC San Diego · Anthropic +2
Many striking phenomena in deep learning, such as linear mode connectivity and the structured behavior of training dynamics, are closely tied to parameter symmetries: transformations that leave the realized function unchanged. Despite growing attention to parameter symmetries, the exact interplay between parameters, data, and representations remains underexplored. To investigate this, we develop a theoretical framework of effective function classes, i.e., the set of functions a neuron can realize on its input support, and the norm cost of realizing them. We then formalize effective symmetry breaking via neuron identifiability across independent training runs. Our analysis shows that neural networks can admit large families of approximately equivalent solutions even in structurally asymmetric models. We further show that neuron identifiability enables representation merging without prior alignment, and characterize when such merging admits a linear low-loss path. These findings highlight the role of effective function classes in affecting the loss landscape.
Vincent Bürgin, Daniel Herbst, Ya-Wei Eileen Lin +1
Technical University of Munich; School of CIT, MCML, MDSI · MIT; Dept. of EECS, CSAIL