How Long Is the Coast of a Neuron? Resolution-Dependent Interaction Order and Computational Capacity
Authors: Nima Dehghani
Organizations: McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI), Cambridge, Massachusetts, USA
Mandlebrot's coastline paradox reframes measurement as a joint property of an object and its ruler. Here we ask how observational resolution shapes the apparent structure of neural computation. We introduce a scale-indexed decomposition that separates the predictive information captured by direct dependencies from the residual outside that model family. Exact constructions show that projection can convert higher-order dependence into pairwise structure. Projection can increase the direct fraction of surviving information even as total predictive information decreases. A schematic branch-to-soma cascade grounds these effects in local coincidence detection, saturating integration and somatic mixing. Resolving organized nonlinear subunits improves approximation and raises effective representational rank. In a nested hierarchy, these gains occur without increasing information about the target. This distinction separates information availability from access by a restricted readout. A point-neuron or direct population model therefore establishes an effective description at a specified scale. It does not uniquely identify the finer computation that generated the observed relation. Which resolution is adequate depends on the observable and perturbation of interest.
Figures & tables
Figure 1: A higher-order dependency becomes pairwise after projection. (a) Exact enumeration at K=1.8 gives 0.823 bits of connected third-order information at the fine scale and 0.261 bits of pairwise mutual information after the disjoint coincidence projection x=b1 , z=b2∧b3 . The projected variables are independent at K=0 . (b) The information that survives as pairwise depends on the map: an XNOR/parity projection retains all of I(3) , whereas the coincidence map retains only part.
Figure 2: Direct dependencies become more complete along an exact cascade. (a) Blue bars show F for the coordinate-invariant categorical-additive family; pink shows the residual share. Purple diamonds show the more restrictive linear-moment family, which yields 55.0%, 81.3% and 98.9%. (b) Coarse-graining reduces the available information Iavail , the captured direct information Idir and, more sharply, the conditional-KL residual Ihigher . Values are exact sums over all 64 input states.
Figure 3: Organized multi-contact features improve held-out approximation. (a) Mean held-out MSE for a linear point neuron, a three-subunit model with shuffled contacts and a three-subunit model with contacts aligned to the target’s pair structure. Error bars are 95% normal intervals across 100 independent train–test trials (6,000 samples in each split). (b) Mean held-out MSE of a generic fixed sigmoid random-feature network. The first tested width whose mean reaches the aligned model is 40. Shading shows 95% normal intervals across 40 trials.
Figure 4: Approximation and effective dimension grow in a nested hierarchy. (a) Mean held-out MSE as progressively more nonlinear sigma–pi subunits are exposed from the same architecture; the dashed line is the direct-input point baseline. Error bars and baseline shading are 95% normal intervals across 60 independent architectures and train–test sets (5,000 samples per split). (b) Normalized inverse error and normalized effective rank with log-space power-law fits. Exponent intervals are 95% percentile-bootstrap intervals over trials (2,000 resamples).
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure S1: The final direct fraction depends on mixing and measurement resolution. (a) Linear-moment F for the exact cascade after weighted somatic mixing and quantization. (b) Available information retained by the same projection. At Δ=0.1 , the white cross marks symmetric 0.5/0.5 mixing and the cyan diamond marks asymmetric 0.6/0.4 mixing. Heat-map values are exact; discontinuous bands arise when quantization merges different branch-state pairs.
Cortical neurons fire sparsely -- often fewer than one spike per sensory window -- so a neuron's spike count in that window carries little information, and its spike timing can carry more. That conduction delays convert firing order into synchrony is long established. What governs which class of temporal feature a neuron detects -- one volley of coincident input, or two in a particular order -- has not been examined. We propose a delay-signature framework in which the axonal conduction delays converging on a dendritic branch or a small neuron's soma constitute a physical key: only input sequences whose spike-time differences the delays compensate arrive synchronously, and coincidence detection, a threshold on input summed within an inhibition-compressed window, converts that synchrony into an all-or-none output. First, in simulations of an integrator-neuron model, a single physical scalar -- the dispersion of the delay set -- moves a population from event to order-selective sequence detection. The transition is emergent under random delays and connectivity, and an exact calculation reproduces it: at narrow dispersion, sequence detectors do not exist, and the dispersion at which they overtake event detectors tracks the inter-event interval. This maps a computational distinction onto the anatomical one between myelinated and unmyelinated projections, making myelination a switch on what a neuron computes. Second, the same dispersion sets the code's limits: it bounds the longest codable interval and fixes an absolute timing tolerance of order a millisecond, the same at every dispersion. Third, that millisecond window and the conduction velocity of afferents inside cortex together set column diameter, and column anatomy predicts the window measured in mouse barrel cortex. One anatomically measurable parameter thus shapes what a neuron detects and sets the limits of what it can represent.
Cheng Bi, Jipeng Sun
Independent Researcher · Department of Computer Science, Princeton University
We investigate whether neuron populations within neural networks evolve predictably with scale, extending scaling laws beyond macroscopic observables such as loss. To probe this question, we study Rosetta Neurons, a previously characterized class of neurons whose activation patterns are similar across independently trained models (Dravid et al., 2023). In separate analyses of language models up to 30B parameters and vision models up to 5B parameters, we observe that the population of Rosetta Neurons follows a sublinear power law in model size, growing in absolute number but occupying a shrinking fraction of the total neuron count. We further observe a Neuron Polarization Effect: Rosetta Neurons become more selective and increasingly monosemantic with scale, separating from a growing non-Rosetta population that remains less selective. An analytical model balancing feature utility against limited neuron capacity explains the sublinear power-law scaling and this polarization effect. Finally, we find that Rosetta Neurons become more domain-specialized with scale and illustrate their selectivity through a targeted data-filtering case study for continued pretraining. Our results point to a scaling law for interpretable, shared neuron-level structure, linking model size to systematic changes in neuron universality, selectivity, and specialization.
Reservoir computing exploits nonlinear dynamical systems to encode temporal inputs into high-dimensional state space representations. Although reservoir performance is often characterized through memory, nonlinearity, and their tradeoff, such aggregate measures do not reveal how task-relevant information is organized within the state space. Here, we introduce an eigen-spectral decomposition framework linking the degree-wise information processing capacity to the corresponding state space modes. As a result, we are able to quantify the degree-wise representation energy, and show that in some cases, substantial amounts of information processing capacity may reside in low-energy modes that are vulnerable to experimental noise. These results suggest that useful reservoir computation depends not only on dimensionality expansion, but also on the geometric organization of task-relevant information, with direct implications on physical reservoir computers.
Mohab Abdalla, Damien Rontani
CentraleSupélec, Université de Lorraine, LMOPS EA-4423 Laboratory, 57070 Metz, France