q-bio.NCSep 16, 2026

Structure is in the zoom: probing neural symmetry through dimensionality scaling

Authors: Adil Amin

Organizations: ZEHEN Labs, Milwaukee, WI, USA

Abstract

Scaling exponents of neural dimensionality are read as properties of the code, and used to compare representations and benchmark models against cortex. Yet one recording of 11,311 mouse V1 neurons gives 0.25, 0.30 or 0.35, depending only on how it is subsampled. We show that such an exponent measures the population and the probe together. We split it exactly into a floor, which random subsets of the same sizes give, and a shift, which a declared axis of stimulus classes adds. The usual random-subsample exponent is almost all floor. Stimulus direction lifts the exponent in every grating recording by reading both symmetry sectors of the code: orientation, which repeats every half turn, and direction, which does not. The lift lives in trial-to-trial fluctuations that differ between directions, while the class means, which linear decoders read, pull it down. Two measured terms, how much the classes' fluctuations overlap and how the means sit against them, account for the shift. They give its sign where it changes: in Allen Neuropixels recordings, whose class means mostly raise the exponent, and in macaque V1. In an XY lattice whose classes stay on one orbit, temperature flips the sign as the two terms say, and in GPT-2 the shift ranks the indirect-object circuit's name-mover and backup heads. Reported with its path, its floor and its shift, an exponent says how much structure a probe finds and what carries it. Our open tool, rung, reports all three for any labelled neural recording or language-model layer: https://github.com/adilamin89/structure-in-the-zoom

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Sep 15, 2026

Symmetry without a manifold: intrinsic dimension on orbits

The standard geometric derivation of neural scaling exponents takes the intrinsic dimension of a data manifold as its input. On modular addition in Zp\mathbb{Z}_p that derivation has no input. The exact algebraic solution is an orbit of Zp\mathbb{Z}_p acting by isometries. Transitivity alone makes the ratio statistic underlying the standard dimension estimator a point mass, so the estimator is undefined, and here the two nearest neighbour distances coincide exactly. Breaking the symmetry at scale εε returns a number, but one that tracks 1/ε1/ε with no scale free plateau. We show that the failure is general, since on any finite orbit of a group acting by isometries the estimator reports the resolution at which the set is probed rather than a dimension. What replaces the power law is exponential in hidden width, L(h)=L∞+Aexp⁡(−c hα)L(h)=L_\infty+A\exp(-c\,h^α), with R2R^2 between 0.982 and 0.995 against 0.857 to 0.906 for a power law admitting the same floor and fitted under the same protocol. Where the data supply is sufficient the rate belongs to the regulariser rather than to the group, since weight decay moves cc by a factor of 47 while group order moves it by 1.10, a residual below seed to seed resolution, for every fixed αα between 0.75 and 2. The critical width falls with group order rather than rising, against capacity counting that assigns a fixed number of neurons to each irreducible representation.
May 12, 2026

Deep Minds and Shallow Probes

Neural representations are not unique objects. Even when two systems realize the same downstream computation, their hidden coordinates may differ by reparameterization. A probe family intended to reveal structure already present in a representation should therefore be stable under the relevant representation symmetries rather than be tied to a particular basis. We prove that, under standard nondegeneracy assumptions, behaviorally equivalent LLMs have hidden representations related by an invertible affine transformation. The resulting symmetry principle singles out a unique hierarchy of shallow coordinate-stable probes, with linear probes as its degree-1 member. We also show that a natural object for cross-model probe transfer is a shared probe-visible quotient--the representation modulo directions invisible to the probe family--rather than the full hidden state. Experiments on synthetic and real-world tasks support both predictions, showing where degree-2 probes help beyond linear ones and how quotient-based transfer enables coverage-aware monitor portability across model families. These results point toward a broader geometric representation theory of neural probing, with coverage-aware monitor transfer as a concrete operational consequence.
May 20, 2026

Stimulus symmetries can confound representational similarity analyses

What can representational similarity matrices (RSMs) tell us about a neural code? As the popularity of these summary statistics grows, so too does the need for a more complete characterization of their properties. Here, we show that symmetries in network inputs can confound RSM-based analyses. Stimulus symmetries render many representations functionally equivalent, but these different configurations can lead to different RSMs. These different RSMs reflect qualitatively different representational geometries. We show that stochastic gradient descent or energetic regularization can generate sparse, drifting codes, leading in turn to drifting RSMs. Moreover, we demonstrate that these phenomena are present in networks trained to encode image data, where the symmetry is latent. Our results illustrate the challenges inherent in comparing nonlinear neural codes, when functionally-equivalent representations are not related by a simple rotation.