Geometric Stability: The Missing Axis of Representations

Diffusion geometry is a manifold learning framework that uses random walks defined by Markov transition matrices to characterize the geometry of a dataset at multiple scales. We use diffusion geometry for neural representations, incorporating tools from multi-view learning into this field for the first time. Our key technical observation is that a broad class of similarity measures based on representational similarity matrices (RSMs) admits a closed-form equivalent formulation in terms of row-stochastic Markov matrices, opening the door to manipulations from diffusion geometry. As a first application, we develop multi-scale variants of Centered Kernel Alignment and Distance Correlation, which utilise the $t^{th}$ power of the underlying transition matrix to probe the data geometry at adjustable diffusion scales. Going further, we introduce variants of these measures which fuse the Markov matrices of several layers via alternating diffusion into a single operator that captures the network's joint sample geometry, allowing similarity to be computed across multiple layers and shifting the comparison from layer-to-layer to network-to-network. We perform extensive numerical experiments, evaluating our measures on the Representational Similarity (ReSi) benchmark comprising 14 architectures trained on 7 datasets across three different domains. Our methods achieve SoTA results in accuracy and output correlation for both language and vision tasks across different models. We furthermore show SoTA performance on an additional benchmark evaluating on out-of-distribution data.

Activation-alignment measures such as Representational Similarity Analysis (RSA), Canonical Correlation Analysis (CCA), and Centered Kernel Alignment (CKA) are widely used to compare biological and artificial neural representations. Recent theoretical work interprets many of these methods as assessing agreement between optimal linear readouts over broad families of global tasks. However, agreement at the level of global readouts does not determine how a system uses local stimulus evidence. Specifically, representations may align in activation space yet differ in their sensitivity to small perturbations. To address this challenge, we introduce a complementary framework based on local decodable information, which focuses on a representation's ability, under noise, to discriminate small perturbations within a specified stimulus-coordinate subspace. Building on Fisher information and local representation geometry, we summarize each representation using the expected projected pullback/Fisher metric over that subspace. This formulation induces a second-moment family of local discrimination tasks, for which the resulting operator provides a minimal, complete dataset-level summary of expected discriminability. We compare these regularized signatures using a log-spectral distance on the manifold of symmetric positive definite (SPD) matrices, yielding the Spectral Riemannian Alignment Score (S-RAS) and a uniform multiplicative certificate over the corresponding family of lifted task values. Empirically, this framework enables the recovery of corresponding layers across independently trained artificial neural networks, supports transferable class-conditional probes, reveals controlled dissociations between standard and robust training, and uncovers stimulus-coordinate family effects across mouse visual cortex using the Allen Brain Observatory static gratings dataset.

Trial-to-trial variability limits how reliably neural population geometry can be estimated, while comparisons across populations depend on neuron and trial counts, response quality, and clustered sampling. We quantified within-session geometric reliability using Shesha, the Spearman correlation between representational dissimilarity matrices estimated from independent trial subsets, in all 39 Steinmetz Neuropixels sessions and in olfactory bulb and piriform cortex recordings from Bolding and Franks. Steinmetz analyses matched neurons and repetitions, compared observed reliability with a stationary residual-bootstrap expectation, and used mouse-level or mouse-clustered inference. Mean matched reliability was 0.0402 across 312 area-by-session recordings. Regional differences and reliability above the stationary benchmark did not survive correction. Temporal effects received the strongest support: interleaving early and late trials increased reliability relative to blocked allocation ($Δ=0.02666$, $q=0.001953$), and RDM similarity declined with within-session lag (mean mouse-level slope $=-0.01912$, $q=0.001953$; $n=10$ mice). Outer-cross-fitted reliability was not associated with choice-direction coupling or stimulus or response-direction decoding after correction. Olfactory comparisons remained descriptive because few paired sessions and no animal identities were available. In held-out simulations, associative recurrence outperformed feedforward subspace denoising but not divisive normalization. Representational geometry became less reproducible with temporal separation within a session, and comparisons across neural populations require sampling calibration and independent inference.