Neural Circuits

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 36

Sep 30, 2026cs.LG

Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks

A foundational principle of connectionism is that perception, action, and cognition emerge from parallel computations among simple, interconnected units that generate and rely on neural representations. Accordingly, researchers employ multivariate pattern analysis to decode and compare the neural codes of artificial and biological networks, aiming to uncover their functions. However, there is limited analytical understanding of how a network's representation and function relate, despite this being essential to any quantitative notion of underlying function or functional similarity. We address this question using analysable two-layer linear networks and numerical simulations in non-linear networks. We find that function and representation are dissociated, allowing representational similarity without functional similarity and vice versa. Further, we show that neither robustness to input noise nor the level of generalization error constrain representations to the task. In contrast, networks robust to parameter noise have limited representational flexibility and must employ task-specific representations. Our findings suggest that representational alignment reflects computational advantages beyond functional alignment alone, with significant implications for interpreting and comparing the representations of connectionist systems.

Lukas Braun, Erin Grant, Andrew M. Saxe

Department of Experimental Psychology, University of Oxford, Oxford, UK · Gatsby Unit & Sainsbury Wellcome Centre, University College London, London, UK

Neural Circuits Neural Representations Deep Network Neural Network Representational Capacity Dissociation

Sep 29, 2026cs.NE

How much of fly walking is written in the wiring?

Connectome models of the fly nerve cord generate walking-like motor rhythms, but oscillation alone does not show that the specific wiring matters. Here we provide, to our knowledge, the first test of which features of motor output depend on the specific wiring. We simulated the leg motor systems of two independent Drosophila connectomes, with synapse counts as fixed weights and glutamatergic synapses treated as inhibitory, and compared each with six families of rewired networks that preserve progressively more of its structure, using pre-registered criteria. We find that rhythm is generic but antagonist coordination is not: many rewired networks were more rhythmic than the real ones, yet the real wiring coordinated antagonistic motor pools more strongly than every rewired network, most of all at the thorax--coxa joint. We trace this specificity to how premotor input is allocated between antagonistic pools. Both connectomes carry Sherrington's reciprocal innervation---neurons that excite one pool inhibit its antagonist---and no rewired network does. Reassigning premotor inputs between the pools abolished coordination even when motor neurons' typical input strength changed little (all pre-registered criteria met in one connectome; same direction in the other). Coordination, not rhythm, therefore reveals whether a connectome model's wiring matters.

Isabel Guan, Yuntian Zhao, Dingyuan Zhang +2

The Hong Kong University of Science and Technology · ZENBOT · Nanyang Technological University +1

Neural Circuits Neurons Musculoskeletal Wiring

Sep 20, 2026cs.LG

A discrete generative model of neuronal spiking activity on microelectrode arrays

Generative models of neural activity could help characterize tissue dynamics, compare experimental conditions, and simulate population activity for applications ranging from disease and drug-response studies to closed-loop experimentation. Existing approaches, however, typically assume a fixed set of sorted neurons, whereas high-density microelectrode arrays produce extremely sparse, array-wide binary spike volumes in which the observed subset of electrodes varies across assays. We introduce a discrete generative model that represents this activity using a shared vocabulary of spatiotemporal motifs. A residual vector-quantized autoencoder learns the motif vocabulary, while a factorized masked transformer predicts where activity occurs and which motif appears at each active location. We evaluate the model on 31 assays spanning human brain organoids and acute \emph{ex vivo} human hippocampal tissue. The learned motifs are broadly reused: assay identity explains only

9%

of the entropy in motif use, and motif overlap across tissue types is comparable to overlap within them. When representation quality is evaluated independently of the generative prior, our approach achieves

5.2\times

the voxel-level reconstruction average precision of a matched flat tokenizer. For masked completion and free generation, the full model achieves

1.4

--

2.6\times

the site-level average precision of the matched generative baseline and outperforms it across all four families of generation metrics. These results establish a compact, reusable representation for array-wide spiking activity without learned assay-specific parameters, providing a scalable foundation for generative modeling across diverse neural preparations.

Md Sayed Tanveer, Mohammed A. Mostajo-Radji, Ge Wang

Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA · Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute · Genomics Institute, University of California Santa Cruz, Santa Cruz, CA 95064, USA

Neural Circuits Time-To-First-Spike Neural Representations Generative Models Spatiotemporal

Sep 17, 2026q-bio.PE

Self-Replicating Neural Cellular Automata: Quantifying Emergent Phenotypic and Genotypic Diversity in an OpenEnded Substrate

We study an in-silico substrate in which every pixel of a two-channel cellular-automata grid carries a tiny neural network (an agent) that senses its Moore neighborhood. A cell persists only by self-replication: a living neighbor is cloned and its weights are mutated by a uniform perturbation, so that phenotype (cell state) is driven entirely by genotype (network weights). From a handful of seeded founders the system grows into a spatially organized ecosystem of coexisting, competing and dominating species. Our main contribution is a battery of coarse-grained diversity metrics that make such growth measurable at two scales: four phenotypic tools based on cellular-type frequency, entropy and cell variance, and two genotypic tools that colour each agent by a hash of its full weight vector versus a sparse random-weight probe. Across a five-fold sweep of 1680 small runs and 24 long (1000-generation, 200 x 200) runs, the substrate is persistent and self-maintaining in 20 of the 24 long configurations and exposes a clear phenotype-genotype diversity trade-off: raising phenotypic diversity collapses genotypic diversity and vice versa. Full-genome hash colouring further reveals lineage structure that a random-weight probe systematically misses. Code, data and animations are released as supplementary material.

Sanyam Jain, Felix Simon Reimers, Stefano Nichele

Department of Computer Science and Communication, Østfold University of Applied Sciences, B R A veien 4, 1757 Halden, Norway · Department of Computer Science, Oslo Metropolitan University, Pilestredet 35, 0166 Oslo, Norway

Neural Cellular Automata Self-Organization Population Dynamics Phenotypes Neuroevolution Neural Circuits+1

Sep 14, 2026cs.NE

Scaled Hippocampus-inspired Neural Networks on Neuromorphic Memristive Hardware

The hippocampus, a key brain region for learning and memory, exhibits rich structural diversity, sparse communication, and robust dynamics with incredible energy efficiency. It offers promising insights for novel computing capabilities, particularly when co-designed with emerging hardware technologies. In this work, we draw inspiration from the rodent CA3 hippocampal subregion to develop the first spiking neural network with neuronal diversity and biologically-realistic resting state dynamics demonstrated on memristor hardware. We propose a network downscaling methodology utilizing a 4-prong objective function and demonstrate a small-scale CA3-inspired network with 179 Izhikevich-modeled neurons, 3 neuronal types and 17,996 synapses with similar resting-state dynamics as the orders-of-magnitude larger full-scale network. The small-scale network is mapped to an FPGA/memristor platform using a greedy algorithm and 18,316 memristors. Benefiting from memristor noise, the hardware implementation shows continuous periodic behavior, outperforming simulated hardware. This work showcases the potential of biologically-realistic algorithms on emerging hardware for neuromorphic computing.

Joseph A. Kilgore, Jeffrey D. Kopsick, Zahin Ahmed +2

Department of Electrical and Computer Engineering, George Washington University, Washington, 20052, USA · Center for Neural Informatics, Structures, & Plasticity, George Mason University, Fairfax, 22030, USA

Neuromorphic Computing Reram Crossbar Neural Circuits Human Visual Cortex

Aug 31, 2026q-bio.NC

"More Is Different'' in Neural Circuits: Algebraic Emergence of Effective Theories in Canonical Recurrent Motifs of Biological Neuronal Networks

Canonical neural circuit motifs are usually described functionally: divisive normalization rescales population activity by a pooled signal, and winner-take-all competition selects one pattern through recurrent excitation and shared inhibition. We represent them, and their compositions, algebraically as finite transformation systems and analyze the transition monoids generated by their input-conditioned updates, distinguishing structure already present in a generator from structure that appears only through composition, and, on a joint state space, structure inherited from one factor from structure that lives on a joint configuration. Individually aperiodic updates can generate non-aperiodic monoids. In the WTA, every frozen-drive generator collapses to fixed points, yet short input sequences create local cycles of winner-dependent inhibitory gating: globally dissipative dynamics with a reversible action. The strongest result arises in WTA-to-DN composition. The composed monoid then contains a genuinely composite local cycle in which normalization state and the winner's gating state change together, although every primitive generator is aperiodic. Holonomy analysis certifies this as a group component of the Krohn-Rhodes cascade rather than an incidental cycle, and finds most group-carrying image sets on joint configurations, whereas the uncoupled product has none. An exhaustive interface sweep shows that the composite cycle is a property of the coupling rather than of a chosen map. If motifs are building blocks of neural computation, composing them is a form of programming: one chooses primitives and interfaces so that the generated algebra has the intended repertoire. The transition monoid is that repertoire - what a primitive presents to any later construction. Recurrent circuits are compositional transformation systems; their algebra constrains what they can be programmed to compute.

Nima Dehghani

McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI), Cambridge, Massachusetts, USAa)

Neural Circuits Neurons Finite Groups Recurrence Discontinuous Gating Emergence

Aug 8, 2026cs.LG

The Neural Division of Labor: Biologically-Inspired Modular Architectures for Robust Neuromorphic Computing

Biological neural systems achieve high efficiency and robustness through compartmentalized architectures. In contrast, modern artificial neural networks rely on globally entangled structures, which obscure decision logic and suffer from catastrophic forgetting. Here, we report a Decomposable Spiking Neural Network (D-SNN) that eliminates global synaptic entanglement by structurally isolating classification pathways into independent experts. Optimized via a bio-inspired push-pull loss function, the D-SNN achieves competitive accuracies on MNIST, Fashion-MNIST, and CIFAR-10/100 benchmarks. This modular approach matches the performance of fully dense networks while utilizing an order of magnitude fewer parameters. In addition, our networks operate with up to several orders of magnitude lower firing rates and fewer synaptic operations. Furthermore, physically severing connections between experts provides inherent protection against catastrophic forgetting during sequential learning. Crucially, these isolated pathways generate auditable neural signals, increasing decision transparency. This biomimetic, verifiable architecture establishes an efficient foundation for deploying deterministic neuromorphic intelligence in resource-constrained edge environments.

Maksim Bazhenov, Serafim Grubas, Vakhtang Putkaradze

Department of Medicine, University of California San Diego, La Jolla, CA 92093, USA · Department of Physics, University of Alberta, Edmonton, AB T6G 2G1, Canada · Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, AB T6G 2G1, Canada +1

Spiking Neural Networks Neuromorphic Computing Neural Circuits Modular Framework Catastrophic Forgetting

Aug 8, 2026cond-mat.stat-mech

High-Capacity Generalized Hopfield Networks

Generalized Hopfield networks are introduced where memories and neurons are continuous variables that lie on a Riemannian manifold. We explicitly focus on symmetric spaces associated with the special unitary groups SU(d), and use both numerical and analytical (replica) techniques to demonstrate an almost order of magnitude enhancement in critical capacity over the vector networks starting with d=3 and further rapidly growing with d. To circumvent the non-linear geometric constraints, we use a Lie algebraic method [following V. Galitski, Phys. Rev. A 84, 012118 (2011)] to exactly describe the classical neural network in terms of linear algebra in an auxiliary Hilbert space. It is shown that in contrast to the traditional Hopfield networks, memory recall in SU(d) Hopfields corresponds to neuron alignment along a top eigenvector of a spiked matrix, which is less susceptible to random matrix crosstalk than other models with continuous neuron variables. Physical platforms to realize SU(d) Hopfields are briefly discussed and physical (in addition to algorithmic) recall mechanism is demonstrated, where memory recovery occurs naturally through generalized Landau-Lifshitz-Gilbert dynamics. To illustrate SU(3) memory recall, we introduce a color (RGB) image encoding/decoding protocol and explicitly run image recovery on corrupted cues. Finally, we quantize the generalized Hopfields which are shown to reduce to Sachdev-Ye glassy type of models. Their many-body spectra generally feature two types of dark and memory bands, where the latter exhibits chaotic Wigner-Dyson level statistics that hides Hebbian data.

Victor Galitski

Joint Quantum Institute, Department of Physics, University of Maryland, College Park, MD 20742, USA

Hopfield Networks Neural Circuits Riemannian Manifolds Factual Recall

Aug 7, 2026cs.AI

Learning in Deep Networks under Dale's Constraint

Biologically plausible learning models aim to explain how neural circuits can implement effective learning under the constraints of real neurons. Although significant progress has been made, a major remaining challenge is that existing models often allow neurons or synapses to represent mixed-sign values, both positive and negative, in violation of a basic aspect of cortical circuitry -- Dale's constraint: biological neurons are either excitatory or inhibitory, but not both, and synapses cannot change sign. In this work, we address this discrepancy by introducing a biologically motivated neural architecture in which both neural activations and learning signals are represented by non-negative activity, and synapses have fixed sign, while still supporting backpropagation-like learning. Our approach uses two complementary interacting non-negative channels to represent positive and negative contributions, inspired by evidence of on-off representations in the brain. These channels are implemented through a simple neural circuit motif, which is repeated throughout the network in both bottom-up and top-down pathways. Combined with a local Hebbian learning rule, the resulting model propagates learning signals and updates weights using only local interactions between neurons. We show theoretically that our learning scheme can exactly recover the backpropagation update despite relying solely on non-negative error signals. Empirically, beyond satisfying stronger biological constraints, the on-off architecture learns efficient representations, yielding substantial gains over comparable vanilla networks on the Tiny ImageNet benchmark. These results demonstrate that effective learning can emerge from biologically plausible mechanisms without requiring mixed-sign signals, providing a step toward more realistic models of neural computation.

Roy Abel, Shimon Ullman

Weizmann Institute of Science

Neural Circuits Neurons Backpropagation Deep Network Ridge

Aug 2, 2026cs.LG

Interpretable MEG Decoding of Perceived Speech: Cortical Sources and the Stimulus Features That Drive Retrieval

Short segments of perceived speech can be retrieved from non-invasive magnetoencephalographic (MEG) recordings by deep networks trained with a CLIP-style objective against wav2vec 2.0 audio embeddings. Yet their weights do not map onto electrophysiological quantities, and it remains unclear which speech properties drive retrieval. We build on a high-performing MEG-to-audio retrieval architecture but redesign both its front end and decoder. Its spatial attention operates on a flattened sensor layout; we replace it with spherical harmonics defined on the three-dimensional MEG helmet geometry. We reduce the subject-specific representation from 270 to 25 branches, add a temporal filter to each branch to match it to a neuronal source in space and time, and make the convolutional decoder shallower. Ocular and cardiac components are removed before training to reduce the risk of stimulus-locked shortcuts. On MEG-MASC, the model reaches 39.75 +/- 0.34% Top-1 accuracy among 1005 candidates across six trained solutions, with about 20 times fewer decoder parameters. Its weights map to source space, recovering generators consistent with the speech-perception network, while left-lateralized branches carry higher-frequency rhythmic components not evident on the right. Paired MEG occlusion shows that 15 of 19 stimulus features contribute, with the largest effects for silence, sound intensity, vowels, and acoustic onsets. Random word lists behave oppositely: substituting narrative MEG into them improves retrieval, indicating that activity without narrative structure carries less recoverable information than activity during coherent speech. The wav2vec target can be reduced to about twelve learned feature dimensions without loss of accuracy, whereas strong temporal compression causes a clear loss. Together, source mapping and input interventions reveal what drives retrieval.

Ilia Semenkov, Daria Kleeva, Ivan Dakhtin +2

AXXX, Moscow, Russia · HSE University, Moscow, Russia · ITMO University, St. Petersburg, Russia

Wav2Vec Human Visual Cortex Neural Circuits Drives

Jul 30, 2026q-bio.NC

Stimulus-Evoked Network Dynamics in Human Cortical Organoids: From a Graph-Computational Framework to Repeated-Stimulation Depression

Human cortical organoids provide an experimentally accessible model of early neural circuit formation, yet whether their activity reflects structured information processing rather than spontaneous synchronization is unclear. We developed a graph-computational framework to quantify stimulus-evoked propagation. This includes stimulus-conditioned functional graphs, a graph-constrained dynamical (graph-neural-network) model used as a system-identification tool, a biological message-passing principle bounding integration depth by observable propagation depth, and a suite of graph-level metrics. We carried this program out in full on longitudinal HD-MEA recordings from three organoids. Once the true acquisition sampling rate and stimulus timing were recovered, the evoked response proved to be a fast, near-synchronous network burst with no measurable outward propagation (peak-latency vs. distance slope = 0). The propagation/integration-depth metrics (Deff ,reachability index, dmax) therefore do not apply, and per-day connectivity graphs were not reliably estimable at the available trial count, a negative result with methodological consequences for applying such metrics to organoid data. Reframing around synchrony, response-population size and shared variability revealed a control-validated phenomenon, i.e., repeated daily stimulation progressively depressed and spatially contracted the evoked response. That repeated stimulation reshapes organoid networks is established, but longitudinal designs in which every preparation is stimulated cannot separate this from developmental maturation. We break that confound with a developmentally-matched, stimulation-naive control, where at day 7, an organoid receiving its first-ever stimulation engaged 93% of the array, whereas organoids with five prior sessions engaged 10%.

Esmaeil S. Nadimi, Vinay C. Gogineni, Jan-Matthias Braun +3

Applied AI and Data Science Unit, Maersk Mc-Kinney Moller Institute, Faculty of Engineering, University of Southern Denmark · Biomedical Mass Spectrometry and System Biology Unit, Department of Biochemistry and Molecular Biology, Faculty of Science, University of Southern Denmark

Neural Circuits Network Science Tissue

Jul 27, 2026q-bio.NC

From Observation to Intervention: Memory in Brains and Large Language Models

Brains and large language models (LLMs) are fundamentally different memory systems, but they can be compared through shared functional questions: where memory-related information is represented, how partial cues recover broader associations, how new information is written or updated, and how memory-related states can be perturbed. In biological systems, these questions span synapses, neuronal ensembles, hippocampal-cortical interactions, and plasticity; in LLMs, they span weights, activations, context windows, retrieval systems, and external stores. The comparison is therefore functional and experimental rather than anatomical. Human studies reveal sparse concept responses, temporal binding, rapid association formation, episode-specific coding, and recall-related reactivation, but selective intervention remains limited. Rodent studies provide more selective causal access to learning-related ensembles, whereas human and macaque interventions usually affect broader circuits. LLMs lack lived episodic memory, yet they permit unusually direct and repeatable manipulation of internal states and stored information. We argue that this asymmetry creates a new opportunity. LLMs are not ahead in memory itself, but in experimental access. Their tools may help turn broad questions about retrieval, updating, persistence, reversibility, and unintended effects into sharper biological hypotheses. The productive bridge is to transfer experimental logic, not anatomical parts.

Morteza Salehjahromi, Shayan A. Zadegan, Amgad Muneer +1

Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA · Department of Neurology, The University of Tennessee Health Science Center, Memphis, TN, USA · Department of Thoracic/Head and Neck Medical Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA +1

Large Language Model Memory Episodic Memory Factual Recall Synaptic Plasticity Neural Circuits

Jul 27, 2026cs.CL

Neural Circuit Function Inference with LLMs

The success of connectome mapping now shifts the challenge of understanding the nervous system to the interpretation of neural circuits. Here, we devise a new automated method, LLantia (LLM automated neural circuit inference and analysis), to systematically infer neural circuit function and the role of its component neural cell types. Our approach distills descriptions of cell type function from the literature and, in combination with the connectome, then infers the function for all other cell types, which serves as a basis for subsequent neural circuit function inference. Results are structured hierarchically, with different possible circuit functions organised under multiple possible behavioural and physiological contexts, and each circuit function composed of subcircuit descriptions alongside relevant cell types to facilitate both backtracking to known, published information and support further experimental research. We illustrate our method by inferring cell type function for all cell types of the adult fruit fly brain and for select broader circuits within, and validate our findings, including by cross-checking with literature published after the release date of our analysis.

Yijie Yin, Albert Cardona

Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, UK · MRC Laboratory of Molecular Biology, Cambridge, UK

Neural Circuits Connectome Circuits Cells

Jul 15, 2026q-bio.NC

Grounded world models in biological organisms and future embodied AI

Recent advances in generative and embodied AI have been driven by large-scale predictive learning over multimodal data. However, the resulting systems remain largely based on passive training regimes where linguistic regularities create the scaffold onto which information from other modalities is attached. Conversely, neuroscience and cognitive science suggest that biological intelligence is organized in the opposite way, where grounded world models acquired through interaction with the environment provide the semantic scaffold to which language is attached. Here, we illustrate five examples of neural circuits supporting grounded world modelling, which underlie navigation in physical and conceptual spaces, affordance-based perception and interaction with objects, active perception and exploratory learning, allostatic control and emotion, and the distinction between self- and world-generated outcomes. These examples highlight several features largely missing from current embodied AI, including the role of intrinsic dynamics as a foundation for learning, the centrality of action in aligning these dynamics with the external world, the prominence of autonomous experience and open-ended learning over passive assimilation of externally provided data, and the fact that early predictive and control mechanisms scaffold higher cognitive abilities such as reasoning, conceptual navigation, planning, imagination, understanding others' minds, and communication. Finally, we discuss whether and how principles derived from biological systems may inform future embodied AI, including training regimes based on social interaction to construct world models that are not only grounded but also socially shared and aligned with human norms and values.

Giovanni Pezzulo, Davide Nuzzi, Marco D'Alessandro +3

Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy · Center for Mind and Brain Science (CIMeC), University of Trento, Italy · Department of Neuroscience, University of Montreal, Montreal, Quebec, Canada

Embodied Artificial Intelligence World Models Cognitive Science Biological Neural Circuits

Jul 11, 2026q-bio.NC

Emergent Generalization by Representation Learning in Artificial Neural Networks

Dimensionality reduction has proven powerful for identifying neural manifolds, which are low-dimensional structures underlying high-dimensional neural activity. These low-dimensional representations have improved the interpretability of population-level coding. Yet whether such low-dimensional representations are biologically relevant and confer functional advantages in learning systems, or merely reflect neuron-level activity, remains contested in neuroscience. We show that an explicit information bottleneck forcing a recurrent neural network to learn a low-dimensional representation is necessary for rotational and out-of-distribution generalisation in a time-series prediction task. Using information-theoretic measures of causal emergence, we characterise the dynamics of this representation across the memorisation-to-generalisation transition, finding a non-monotonic trajectory which shows an initial decrease, a minimum, and a subsequent rise to a maximum, even as prediction loss falls monotonically. This trajectory scales with task complexity, and the magnitude of emergent structure reliably predicts generalisation performance. Analysis of CA1 hippocampal activity in mice learning an alternating maze task reveals analogous non-monotonic emergence dynamics that track behavioural performance. Together, these findings indicate that the ability of neural networks to learn compact, distributed and emergent representations confers a functional advantage for generalisation, supporting a causal role for learned representations in cognition.

Hardik Rajpal, Dan Goodman

1I-X Centre for AI in Science, Imperial College London, W12 0BZ, UK · Department of Electrical and Electronic Engineering, Imperial College London, SW7 2AZ, UK

Neural Circuits Neural Network Representation Learning Emergence Maze Task

Jul 7, 2026cs.LG

When Do Geometric Algebra Layers Beat Scalarization? A Controlled Study on SO(3)-Equivariant Vector Laws

Compact networks built from Clifford algebra Cl(3,0) primitives are exactly SO(3)-equivariant and learn synthetic 3D vector laws from few samples. We ask whether the geometric algebra structure itself contributes anything beyond exact equivariance. We compare against a minimal scalarization baseline: invariant dot products fed to a small MLP that outputs coefficients on the equivariant basis {v_i, v_i x v_j}, which is also exactly equivariant. On single-stage laws (rotation by axis-angle, cross product, central force), scalarization matches or beats the Cl(3,0) network at a fraction of the training cost, so the geometric algebra adds nothing there. On compositional targets whose computation graph nests group operations (apply R2 R1 to a point; map a local force through an orientation, then take a torque), the Cl(3,0) network beats scalarization by an order of magnitude in the low-data regime, reaching with 100 samples what the baseline needs 3000 for, and the gap survives strengthening the baseline with the triple-product invariant and 17x more parameters, external Vector Neurons and e3nn baselines, and a multiplicative coefficient network. Ablations show the required network depth tracks the rotation chain length, and scalarization falls below the constant predictor on chains of four rotations. The advantage is not composition per se: on a rotation-free nested cross product, which flattens into polynomial invariant coefficients, scalarization wins by 24x. No tested model, equivariant or not, extrapolates invariant magnitudes: on radius and separation shifts every model is worse than a constant predictor once errors are normalized. We conclude that geometric algebra layers are not a general shortcut for low-data 3D learning, but become useful precisely when the target composes group elements in depth.

Fabien Polly

Lie Group Equivariance Vectors Neural Circuits

Jul 3, 2026cs.LG

Observable- and Positional-Encoding-Dependent Symmetry Readout from Neural Network Weights

Post-hoc analysis of trained neural network weights often seeks to recover geometric structure directly from the parameters. We show that, for positional-encoding-equipped neural fields, the symmetry visible from weights is not the true symmetry group itself, but an observable symmetry set determined by the trained parameters, the positional encoding (PE), and readout observable. We formulate this dependence through an exact observability hierarchy,

G_{\mathrm{obs}}^{\mathrm{exact}} \subseteq G_{\mathrm{lift}}^{\mathrm{exact}}(φ) \cap G_{\mathrm{true}}

, where

G_{\mathrm{lift}}^{\mathrm{exact}}(φ)

is the set of input transformations that the PE can exactly lift to the feature space. The hierarchy implies that even when a target function has a geometric symmetry, that symmetry may be structurally invisible to weight-level observables if the PE does not represent the corresponding transformation. We test this prediction using MLPs trained on two-dimensional signed distance functions with multiple shape symmetry groups, positional encodings, and Gram-based observables. The results show a consistent PE-dependent pattern: DyadicAxisPE supports

D_4

-sensitive readout but structurally suppresses

D_3

rotations, TriAxisPE yields lower

D_3

/

D_6

readout scores under the tested Gram observables by replacing coordinate axes with three 120-degree-separated axes, and random Fourier features mainly exhibit a

π

-rotation response under these readouts. These findings show that PE design affects not only approximation behavior but also which structures are accessible to post-hoc weight-level readouts. This provides a basis for a principled observable-dependent symmetry readout.

Naoya Chiba, Satoshi Sugiyama, Yuki Uranishi

D3 Center, The Unversity of Osaka

Symmetry Model Weights Positional Encoding Neural Circuits Exact

Jun 28, 2026q-bio.NC

Geometric Reliability of Neural Population Codes: Sampling Calibration and Within-Session Nonstationarity

Trial-to-trial variability limits how reliably neural population geometry can be estimated, while comparisons across populations depend on neuron and trial counts, response quality, and clustered sampling. We quantified within-session geometric reliability using Shesha, the Spearman correlation between representational dissimilarity matrices estimated from independent trial subsets, in all 39 Steinmetz Neuropixels sessions and in olfactory bulb and piriform cortex recordings from Bolding and Franks. Steinmetz analyses matched neurons and repetitions, compared observed reliability with a stationary residual-bootstrap expectation, and used mouse-level or mouse-clustered inference. Mean matched reliability was 0.0402 across 312 area-by-session recordings. Regional differences and reliability above the stationary benchmark did not survive correction. Temporal effects received the strongest support: interleaving early and late trials increased reliability relative to blocked allocation (

Δ=0.02666

,

q=0.001953

), and RDM similarity declined with within-session lag (mean mouse-level slope

=-0.01912

,

q=0.001953

;

n=10

mice). Outer-cross-fitted reliability was not associated with choice-direction coupling or stimulus or response-direction decoding after correction. Olfactory comparisons remained descriptive because few paired sessions and no animal identities were available. In held-out simulations, associative recurrence outperformed feedforward subspace denoising but not divisive normalization. Representational geometry became less reproducible with temporal separation within a session, and comparisons across neural populations require sampling calibration and independent inference.

Prashant C. Raju

Neural Circuits Relevance

Jun 23, 2026cs.NE

Spatial Partial Functionalization of Neural Networks based on Noise Fields

Noise in neural computation is typically regarded as a disturbance, but its spatial distribution may also actively regulate which parts of a network participate in computation. This paper investigates the spatial partial functionalization of Noise-modulated Neural Networks using noise fields. We first present an activation function suitable for this goal, the crossing activation function, using the sample-level, statistical-level, and analytical-level implementations, and examine parameter reuse across these implementations. We then introduce a virtual noise field, an auxiliary continuous space for generating spatially structured network noise fields that activate partially overlapping subnetworks. Using one-dimensional function approximation tasks, we evaluate how multiple functions can be stored in a single network when each function is assigned to a different noise-field location. The results show that memory capacity improves when the spatial arrangement of noise fields reflects the proximity relationships among the functions to be learned, whereas mismatches in noise field structure can reduce effective capacity. These findings suggest that structured noise can serve not only as a perturbation but also as a topology-defining factor for functional subnetwork selection.

Shuhei Ikemoto, Fabio DallaLibera

Graduate School of Life Science and Systems Engineering, Kyushu Institute of Technology, 2-4 Hibikino, Wakamatsu, Kitakyushu, Fukuoka, 8080196, Japan · Graduate School of Engineering Science, Osaka University, 1-3 Machikaneyama, Toyonaka, Osaka, Japan

Neural Circuits Neural Network Activation Functions

Jun 22, 2026q-bio.NC

Identifying structural design principles shaping the computational abilities of recurrent neural networks

Understanding how the architecture of neural networks shapes the computations they carry is a central challenge in neuroscience and machine learning. While specific circuit architectures have been linked to particular network computations and theoretical bounds on expressivity of broad classes of networks have been found, we are still missing general principles connecting the structure of finite networks to their computational capabilities. Here, we characterize the computational abilities of recurrent neural networks as a function of their connectivity by training a large collection of different networks to compute a large set of Boolean functions. For small networks, we constructed the complete ``catalogs'' of network-function performance, which revealed that computational capacity varies widely across architectures and that most networks show poor performance, and most functions are hard to compute. However, we show that having local 2- and 3-cycles in a network strongly enhances its computational ability, and networks with such cycles are often the minimal architectures that can solve particular functions. We further show that a small set of structural statistics accurately predict networks' performance. Extending our analysis to large networks showed that typical networks fail even to approximate a randomly selected function. Surprisingly, adding a small number of sparsely connected biologically-inspired interneurons to the network dramatically increases computational capacity. As in small networks, adding short cycles improved networks' capacity, outperforming acyclic or reachability-matched controls. Thus, our results identify local cycles as design principles linking neural connectivity to computational power, and offer a general framework to explore structure-function relations in computing networks.

Tom Talpir, Elad Schneidman

Department of Brain Sciences, Weizmann Institute of Science, Rehovot, Israel

Neural Circuits Recurrent Neural Networks Neural Network Recurrence

Jun 12, 2026cs.AI

AI Engram: In Search of Memory Traces in Artificial Intelligence

Memory formation is fundamental to intelligence, yet whether deep neural networks preserve identifiable memory traces analogous to biological memory units remains an open question. This work introduces a geometric framework to identify such "AI engrams" by formalizing the neuroscientific criteria of specificity, reactivation, sufficiency, and necessity into a constrained inverse problem. We derive a closed-form estimator that isolates individual memory traces from globally entangled parameters, and show that this biologically-derived solution corresponds to a natural gradient update on the parameter manifold. AI engrams enable surgical manipulation of learned knowledge: any subset of memories can be composed or erased through linear arithmetic, without iterative optimization. Experiments ranging from simple MLPs to LLMs demonstrate the causal validity and substantial scalability of AI engrams. Together, these results bridge theories of biological memory and artificial representation learning and offer geometric insight into how deep networks simultaneously support functional specificity within distributed storage.

Jea Kwon, Dong-Kyum Kim, Jiwon Kim +3

Max Planck Institute for Security and Privacy (MPI-SP), Bochum, Germany · Seoul National University (SNU), Seoul, Korea · KAIST, Daejeon, Korea

Dense Associative Memory Neural Circuits Long-Term Memory Deep Network Intelligence

Jun 12, 2026cs.NE

Harnessing cortical geometry, wiring, and function as inductive biases for recurrent neural networks

How the wiring and functional organization of cortex shape recurrent computation remains a central question in both neuroscience and machine learning. Here, we leverage data released through the Machine Intelligence from Cortical Networks (MICrONS) program--a functional connectomics resource spanning multiple areas of mouse visual cortex, in which dense calcium imaging is co-registered with high-resolution electron microscopy reconstruction from the same animal--to build biologically grounded recurrent neural networks. Using neuronal spatial coordinates, anatomical connectivity, and function-derived relationships from nearly 12,000 coregistered excitatory neurons, we initialize recurrent weights and impose communication-aware spatial constraints during learning. Across three cognitive decision-making tasks, networks constrained by cortical structure and function consistently outperform baseline and partially constrained models. Functional weight initialization provides the largest gain, while real spatial embedding yields robust additional improvements across conditions. These biologically grounded networks also develop low-entropy, modular, and small-world organization, and retain strong performance even when recurrence is restricted to positive weights. Together, our results show that the machinery of cortex--its geometry, wiring, and functional structure--can be harnessed as a powerful inductive basis for building recurrent networks that learn more effectively while converging toward key organizational principles of biological computation.

Mo Shakiba, Rana Rokni, Mohammad Mohammadi +1

Neuromatch Academy, Neuromatch, Inc., USA · McGovern Institute for Brain Research, Massachusetts Institute of Technology (MIT)

Neural Circuits Human Visual Cortex Recurrent Neural Networks Inductive Bias Wiring

Jun 8, 2026q-bio.NC

Hyperbolic Neural Population Geometry Benefits Computation

Neural population geometry shapes downstream computation. Recent empirical findings in neurobiology suggest that a hyperbolic structure underlies population activity in the hippocampus. Here we provide a theoretical framework for this phenomenon. First, we propose a plausible construction of hippocampal tuning curves that statistically induces hyperbolic geometry. Next, we establish a connection between neural decoding and associative memory by demonstrating that the Modern Hopfield Network update rule computes the minimum mean-squared-error (MMSE) estimator. Finally, we introduce a novel associative memory model defined in hyperbolic space that yields significantly larger capacity than leading models. Our results suggest that animals encode spatial information as a latent hyperbolic cognitive map, improving both memory capacity and decoding accuracy.

Dennis Wu, Yi-Chun Hung, Braden Yuille +2

Department of Computer Science, Northwestern University · Center for Foundation Models and Generative AI, Northwestern University · Integrated Science Program, Northwestern University +3

Hopfield Networks Neural Circuits Hyperbolic Learning Human Visual Cortex Fuzzy Cognitive Map

Jun 7, 2026cond-mat.soft

Discovering and decoding latent mean-field structure with variational autoencoders

Generative models are increasingly used to capture correlations in many-body systems, but the representations they learn remain largely opaque to physical interpretation. Here, we establish an intuitive criterion that quantifies the capacity of a variational autoencoder (VAE) to faithfully reconstruct the joint probability distribution of a many body system. In a nutshell, a bound on the VAE capacity is obtained by comparing the rate of the latent channel to the bipartite mutual information of the data. Using this bound, we show that the conditionally independent decoder of any successful VAE is structurally identical to a finite-size mean-field factorization. Hence, a successful reconstruction is direct evidence for a latent mean-field theory and the microscopic parameters of that theory can be read off the trained decoder. We validate these conclusions on a hierarchy of solvable models with scalar (Curie-Weiss), vector (Hopfield) and tensor (Maier-Saupe) order parameters, recovering the full Hopfield pattern matrix from equilibrium samples alone. We find that, when applied to Salamander retinal recordings, a two-latent VAE reproduces the population statistics with only two effective collective variables allowing us to recover the `stored patterns' of the neural population and write a generalized Hopfield model which correctly models the experimental data.

Marco Biroli, Max Welling, Vincenzo Vitelli

Department of Physics and the James Franck Institute, University of Chicago, Chicago, IL, USA · CuspAI, Cambridge, England · AMLab, University of Amsterdam, Amsterdam, Netherlands +1

Conditional Variational Autoencoder Mean-Field Limit Latent Variable Hopfield Networks Neural Circuits

Jun 4, 2026cs.CL

Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery

Circuit discovery methods identify subgraphs that explain specific model behaviors, and structural differences between discovered circuits are commonly interpreted as evidence of distinct mechanisms. We test this assumption by varying input statistics while holding the task fixed, and show that the resulting structural differences exhibit apparent specialization but do not correspond to functional differences, a pattern we term phantom specialization. Using Literal Sequence Copying across four token-frequency bands plus a control condition in five Pythia models (70M-1.4B), we extract 75 circuits and find that structurally distinct circuits implement the same computation: band-specific edges transfer broadly across bands, a core shared across most bands recovers at least 99% of circuit performance, and causal interchange interventions confirm that internal representations are interchangeable across frequency bands. Repeated extractions within the same frequency band further suggest that discovery algorithms sample from an equivalence class of valid subgraphs rather than recovering a unique mechanism. Standard evaluation practice obscures this pattern: source-level evaluation inflates apparent faithfulness, while edge-level evaluation reveals the many-to-one mapping from structure to function. Our results show that structural differences between circuits are not sufficient evidence for distinct mechanisms, and that exposing this requires edge-level evaluation and cross-condition transfer tests.

Alireza Bayat Makou, Jingcheng Niu, Subhabrata Dutta +1

UKP Lab, Technical University of Darmstadt · National Research Center for Applied Cybersecurity ATHENE

Circuits Neural Circuits Discovery Subgraphs

May 23, 2026cs.CL

CSP-Atlas: Concept-Specific Neural Circuits in a Sparse Python Transformer

A sparse 8-layer code transformer develops dedicated neural circuitry for every Python construct tested, and that circuitry is organised by a clean computational principle rather than by semantic category. We extract neural circuits for 106 concepts (43 AST node types, 63 builtin objects) by marginalising across 63,800 controlled prompts, and decompose each circuit into concept-specific and token-driven components using contrastive checker prompts that present a keyword token without its associated syntactic structure. Three findings emerge. First, all 106 concepts produce non-empty universal circuits at every one of nine parameter settings, and the ranking of concept-specificity across constructs is stable across the sweep - survival is not an artifact of a permissive threshold. Second, AST circuits contain a genuine concept component distinct from token activation: concept-only neurons constitute up to 62.5% of the loudest-firing neurons at mid-to-late layers, while builtin circuits are almost entirely token-driven. Third, six computationally atomic constructs - Import, ImportFrom, Break, Continue, Pass, Assert - cluster together despite being semantically unrelated, sharing only the property of being single-statement constructs requiring no nested body; this atomicity super-cluster, together with a four-tier hierarchy organised by token ambiguity and structural distinctiveness, shows that the model's internal organisation tracks computational structure rather than meaning. The methodology, full decomposition data, and analysis code are released.

Piotr Wilam

University College London

Circuits Syntax Neural Circuits Transformer Architectures Atlas

May 21, 2026cs.NE

Rare Events, Real Signals: Functional Ensembles as Units of Computation in Deep Spiking Networks

We investigate how internal representations emerge across hierarchical processing systems by introducing a neuroscience-inspired framework for analyzing deep spiking neural networks (SNN) through the lens of functional connectivity. Drawing on concepts from systems neuroscience and information theory, we form the first-order functionally-connected (1FC) group of a neuron based on its statistically significant pairwise correlations with neurons from the previous layer of a trained SNN architecture. We then track its response properties during inference under various conditions. Our analysis shows that several principles of functional connectivity previously observed in biological cortex are preserved in spiking ResNet architectures. These 1FC ensembles display interesting properties: their aggregate cofiring reliably predicts downstream neuronal responses through a robust, ReLU-like input-output relationship, whose gain scales systematically with ensemble size. Reliable encoding of the presented class emerges only during high 1FC cofiring events, which themselves occur infrequently, indicating that informative representations are concentrated in rare but highly coordinated activity patterns. Under uniform random noise or adversarial perturbations, these response profiles are disrupted, particularly in early and intermediate layers. This enables a targeted high-resolution interrogation at specific nodes and pathways. We showed that the functional connectivity structure is shaped by learning and this structure breaks under weight permutation. These establish 1FC ensembles as a functionally meaningful substrate for input encoding and information transfer, with potential implications in designing targeted fine-grained diagnostics on the information flow.

Aditi Aravind, Konstantinos Ladakis, Mario Alexios Savaglio +2

University of Crete, Greece · 2Foundation of Research & Technology - Hellas (FORTH), Greece · 4Harvard Medical School, USA +2

Spiking Neural Networks Functional Connectivity Neural Circuits Neurons Ensemble Rare Events

May 20, 2026q-bio.NC

Stimulus symmetries can confound representational similarity analyses

What can representational similarity matrices (RSMs) tell us about a neural code? As the popularity of these summary statistics grows, so too does the need for a more complete characterization of their properties. Here, we show that symmetries in network inputs can confound RSM-based analyses. Stimulus symmetries render many representations functionally equivalent, but these different configurations can lead to different RSMs. These different RSMs reflect qualitatively different representational geometries. We show that stochastic gradient descent or energetic regularization can generate sparse, drifting codes, leading in turn to drifting RSMs. Moreover, we demonstrate that these phenomena are present in networks trained to encode image data, where the symmetry is latent. Our results illustrate the challenges inherent in comparing nonlinear neural codes, when functionally-equivalent representations are not related by a simple rotation.

Farhad Pashakhanloo, Jacob A. Zavatone-Veth

Center for Brain Science and 2Society of Fellows Harvard University, Cambridge, MA, USA

Representational Similarity Analysis Symmetry Neural Circuits

May 20, 2026cs.LG

From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach

Mechanistic interpretability produces circuit-level causal analyses of neural network behaviour, but discovered circuits often remain isolated experimental artefacts: there is no shared formal representation for what circuits compute, how they relate, or when two findings provide evidence for the same mechanism. This work provides a formal infrastructure for cumulative mechanistic science by treating circuit interpretation as inductive theory construction. Each circuit is characterised at two levels: a Causal Functional Signature (CFS), which grounds component behaviour in causal attribution evidence and token role profiles, and an architectural signature

τ_{\mathrm{arch}}

, learned by inductive logic programming (ILP) from scale-invariant structural predicates. Together, these constitute a formal coherence layer that makes mechanistic claims explicit, comparable via

θ

-subsumption, and portable across model scales. CFS reveals qualitatively distinct computational strategies across task types, including attention-mediated copying versus MLP-mediated binding. ILP signatures achieve substantially better structural separation than graph kernel and feature-vector baselines, and support principled transfer across model scales and architecture families.

Nura Aljaafari, Danilo S. Carvalho, Andre Freitas

Department of Computer Science, University of Manchester, United Kingdom · CRUK National Biomarker Centre, University of Manchester, United Kingdom · Idiap Research Institute, Switzerland

Mechanistic Interpretability Circuits Neural Circuits Neuro-Symbolic Framework Induction

May 12, 2026cs.LG

Scaling Laws and Tradeoffs in Recurrent Networks of Expressive Neurons

Cortical neurons are complex, multi-timescale processors wired into recurrent circuits, shaped by long evolutionary pressure under stringent biological constraints. Mainstream machine learning, by contrast, predominantly builds models from extremely simple units, a default inherited from early neural-network theory. We treat this as a normative architectural question. How should one split a fixed parameter budget

P

between the number of units

N

, per-unit effective complexity

k_e

, and per-unit connectivity

k_c

? What controls the optimal allocation? This calls for a model in which per-unit complexity can be tuned independently of width and connectivity. Accordingly, we introduce the ELM Network, whose recurrent layer is built from Expressive Leaky Memory (ELM) neurons, chosen to mirror functional components of cortical neurons. The architecture allows for individually adjusting

N

,

k_e

, and

k_c

and trains stably across orders of magnitude in scale. We evaluate the model on two qualitatively different sequence benchmarks: the neuromorphic SHD-Adding task and Enwik8 character-level language modeling. Performance improves monotonically along each of the three axes individually. Under a fixed budget, a clear non-trivial optimum emerges in their tradeoff, and larger budgets favor both more and more complex neurons. A closed-form information-theoretic model captures these tradeoffs and attributes the diminishing returns at two ends to: per-neuron signal-to-noise saturation and across-neuron redundancy. A hyperparameter sweep spanning three orders of magnitude in trainable parameters traces a near-Pareto-frontier scaling law consistent with the framework. This suggests that the simple-unit default in ML is not obviously optimal once this tradeoff surface is probed, and offers a normative lens on cortex's reliance on complex spatio-temporal integrators.

Aaron Spieler, Georg Martius, Anna Levina

University of Tübingen, Germany · Max Planck Institute for Intelligent Systems, Tübingen, Germany · Max Planck Institute for Biological Cybernetics, Tübingen, Germany

Neural Circuits Recurrent Neural Networks Neurons Scaling Laws Trade-Off

Apr 22, 2026cs.NE

Learning Hippo: Multi-attractor Dynamics and Stability Effects in a Biologically Detailed CA3 Extension of Hopfield Networks

We present a biologically detailed extension of the classical Hopfield/Marr auto-associative memory model for CA3, implementing ten populations (two asymmetric pyramidal subtypes, eight GABAergic interneuron classes), forty-seven compartments, multi-rule plasticity (recurrent Hebb, BCM anti-saturation, mossy-fiber short-term, endocannabinoid iLTD, burst-gated Hebb), and a bimodal cholinergic encoding/consolidation cycle. Evaluated on pattern completion across auto-associative, associative, and temporal regimes, and on a controlled inhibitory-proportion manipulation at

N{=}256

, the full architecture exhibits \emph{three qualitative signatures absent from a minimal Hopfield baseline}: (i)~multi-attractor cross-seed behaviour at

K{=}5

with biologically realistic inhibitory proportions, where two of five seeds converge to positive attractors with margin

{+}0.10{-}0.22

(Cohen's

d{=}0.71

, one-sided

p{=}0.08

); (ii)~target-selective associative recall in paired

(A, B)

memory at

K{\geq}5

, where the full model retrieves

B

from a partial cue of

A

while the minimal model echoes

A

(Pearson margin

Δ{=}{+}0.163

at

K{=}5

); (iii)~reduced cross-seed variance of the full model below the minimal baseline under clean upstream, with ratios

1.0{-}3.0

. These three signatures are architecture-specific: they appear consistently across independent regimes and are absent from the minimal control.

Daniele Corradetti, Renato Corradetti

Grupo de Física Matemática Instituto Superior Técnico Av. Rovisco Pais, 1049-001 Lisboa, Portugal · Dipartimento di Neuroscienze, Psicologia, Area del Farmaco e Salute del Bambino (NEUROFARBA) Università di Firenze, Italy22

Hopfield Networks Dense Associative Memory Neural Circuits Attractors Lateral String Stability

Apr 20, 2026q-bio.NC

The Umwelt Representation Hypothesis: Rethinking Universality

Recent studies reveal striking representational alignment between artificial neural networks (ANNs) and biological brains, leading to proposals that all sufficiently capable systems converge on universal representations of reality. Here, we argue that this claim of Universality is premature. We introduce the Umwelt Representation Hypothesis (URH), proposing that alignment arises not from convergence toward a single global optimum, but from overlap in ecological constraints under which systems develop. We review empirical evidence showing that representational differences between species, individuals, and ANNs are systematic and adaptive, which is difficult to reconcile with Universality. Finally, we reframe ANN model comparison as a method for mapping clusters of alignment in ecological constraint space rather than searching for a single optimal world model.

Victoria Bosch, Rowan Sommers, Adrien Doerig +1

Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany. · Department of Psychology and Education, Freie Universität Berlin, Berlin, Germany. · Bernstein Center for Computational Neuroscience, Berlin, Germany

Unified Representation Neural Circuits Emergence Hypothesis

Apr 16, 2026cs.NE

Structure as Computation: Developmental Generation of Minimal Neural Circuits

This work simulates the developmental process of cortical neurogenesis, initiating from a single stem cell and governed by gene regulatory rules derived from mouse single-cell transcriptomic data. The developmental process spontaneously generates a heterogeneous population of 5,000 cells, yet yields only 85 mature neurons - merely 1.7% of the total population. These 85 neurons form a densely interconnected core of 200,400 synapses, corresponding to an average degree of 4,715 per neuron. At iteration zero, this minimal circuit performs at chance level on MNIST. However, after a single epoch of standard training, accuracy surges to over 90% - a gain exceeding 80 percentage points - with typical runs falling in the 89-94% range depending on developmental stochasticity. The identical circuit, without any architectural modification or data augmentation, achieves 40.53% on CIFAR-10 after one epoch. These findings demonstrate that developmental rules sculpt a domain-general topological substrate exceptionally amenable to rapid learning, suggesting that biological developmental processes inherently encode powerful structural priors for efficient computation.

Duan Zhou

Independent Researcher

Neural Circuits Developmental Trajectories Less Data

Dec 14, 2025q-bio.NC

Random matrix theory of sparse neuronal networks with heterogeneous timescales

Training recurrent neuronal networks consisting of excitatory (E) and inhibitory (I) units with additive noise for working memory computation slows and diversifies inhibitory timescales, leading to improved task performance that is attributed to emergent marginally stable equilibria [PNAS 122 (2025) e2316745122]. Yet the link between trained network characteristics and their roles in shaping desirable dynamical landscapes remains unexplored. Here, we investigate the Jacobian matrices describing the dynamics near these equilibria and show that they are sparse, non-Hermitian rectangular-block matrices modified by heterogeneous synaptic decay timescales and activation-function gains. We specify a random matrix ensemble that faithfully captures the spectra of trained Jacobian matrices, arising from the inhibitory core - excitatory periphery network motif (pruned E weights, broadly distributed I weights) observed post-training. An analytic theory of this ensemble is developed using statistical field theory methods: a Hermitized resolvent representation of the spectral density is processed with a supersymmetry-based treatment in the style of Fyodorov and Mirlin. In this manner, an analytic description of the spectral edge is obtained, relating statistical parameters of the Jacobians (sparsity, weight variances, E/I ratio, and the distributions of timescales and gains) to near-critical features of the equilibria essential for robust working memory computation.

Thiparat Chotibut, Oleg Evnin, Weerawit Horinouchi

Chula Intelligent and Complex Systems, Department of Physics, Faculty of Science, Chulalongkorn University, Bangkok, Thailand · High Energy Physics Research Unit, Faculty of Science, Chulalongkorn University, Bangkok, Thailand · Theoretische Natuurkunde, Vrije Universiteit Brussel (VUB) and International Solvay Institutes, Brussels, Belgium +2

Neural Circuits Synaptic Plasticity Jacobian Timescale Theory

Jul 14, 2025cond-mat.dis-nn

Dynamical stability for dense patterns in attractor neural networks

Recurrent neural networks are canonical models of biological memory. In these models, memories are represented by distributed patterns of neural activity that are stored in the recurrent connections between neurons, such that they become attractors of the network's dynamics. During memory recall, network dynamics thus converge toward one of these memory patterns when started from a noisy or partial cue. Therefore, memory performance critically hinges on the dynamical stability of the stored patterns. However, previous theoretical approaches only studied dynamical stability under highly restrictive conditions that do not readily apply to biological neural circuits. Here, we develop a theory of the local stability of discrete fixed points in a broad class of networks with graded neural activities and in the presence of noise. Using methods from random matrix theory, we analyze the bulk and outliers of the eigenvalue spectra of the Jacobians that characterize network dynamics around fixed points. We show that either all fixed points are stable or all of them are unstable, depending on whether their number is below a ``critical load for stability'', which is distinct from the classical critical capacity that measures the maximal number of achievable fixed points regardless of their stability. We further analyze the dependence of this critical load for stability on experimentally measurable quantities characterizing the statistics of memory patterns and the activation functions of neurons. Our analysis highlights the computational benefits of sparse-like patterns and threshold-linear activation functions and offers testable predictions for neural circuits supporting memory.

Uri Cohen, Máté Lengyel

Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK · Center for Cognitive Computation, Department of Cognitive Science, Central European University, Budapest, Hungary

Neural Circuits Attractors Recurrent Neural Networks Neural Network Recurrence Activation Functions+2

Jun 25, 2025cs.LG

A foundation model with multi-variate parallel attention to generate neuronal activity

Learning from multi-variate time-series with heterogeneous channel configurations remains a fundamental challenge for deep neural networks, particularly in clinical domains such as intracranial electroencephalography (iEEG), where channel setups vary widely across subjects. In this work, we introduce multi-variate parallel attention (MVPA), a novel self-attention mechanism that disentangles content, temporal, and spatial attention, enabling flexible, generalizable, and efficient modeling of time-series data with varying channel counts and configurations. We use MVPA to build MVPFormer, a generative foundation model for human electrophysiology, trained to predict the evolution of iEEG signals across subjects. To support this and future efforts by the community, we release the SWEC iEEG dataset, the largest publicly available iEEG dataset to date, comprising nearly 10,000 hours of recordings from heterogeneous clinical sources. MVPFormer leverages MVPA to achieve strong generalization across subjects, demonstrating expert-level performance in several iEEG tasks. MVPFormer surpasses state-of-the-art (SOTA) Transformer baselines in seizure detection across the SWEC, the MAYO, and the FNUSA datasets, while also achieving SOTA performance on four Brain TreeBank iEEG decoding tasks (volume, pitch, onset, and speech). We further validate MVPA on standard time-series forecasting and classification tasks, where it matches or exceeds the performance of existing attention-based models. Together, our contributions establish MVPA as a general-purpose attention mechanism for heterogeneous time-series and MVPFormer as the first open-source, open-weights, and open-data iEEG foundation model with SOTA clinical performance. The code is available at https://github.com/IBM/multi-variate-parallel-transformer. The SWEC iEEG dataset is available at https://huggingface.co/datasets/NeuroTec/SWEC_iEEG_Dataset.

Francesco Carzaniga, Michael Hersche, Abu Sebastian +2

IBM Research – Zurich, Rüschlikon, Switzerland · Department of Neurology, Inselspital, Sleep-Wake-Epilepsy-Center, Bern University Hospital, Bern University, Bern, Switzerland

Multivariate Time-Series Temporal Attention Time Series Neural Circuits Foundation Model