Neurons

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 175% on the four weeks before. 0.3% of all new papers.

Jul 6Week of Sep 21

Latest papers 134

Oct 1, 2026cs.LG

Removing spurious minima for planar features by skip connections

Understanding loss landscapes is central to explaining neural-network training, yet their structure remains only partially understood even in simple models. We study the Gaussian population loss of shallow, bias-free ReLU networks in the teacher--student setting. This provides a simple model for studying essential aspects such as feature learning and overparameterization. For teacher networks with positive output weights and planar features, we show that including a learned linear skip removes all spurious local minima with non-negative student output weights once the student network is at least as wide as the teacher network. In contrast, without the skip, we construct a fixed teacher network with positive output weights and only three hidden neurons in input dimension two whose spurious local minima persist at every student width at least three. Thus, a learned linear skip can remove spurious minima that persist under arbitrary overparameterization. Furthermore, we show that a positive output weight student network always learns the subspace spanned by the teacher features: student features at local minima with non-negative student output weights lie in the span of the teacher features. For ReLU networks in two dimensions, even heavily overparameterized student networks have effective width controlled by the teacher width: every critical point with positive student output weights has at most twice as many distinct student feature directions as teacher neurons. Finally, we transfer the benignity result to empirical minima over parameter balls of any prescribed radius, with the required sampling accuracy depending on that radius.

Jakob Paul Zimmermann, Moritz Grillo, Andrei Balakin +1

Freie Universität Berlin · Technische Universität Berlin

Flat Minima Skip Model Weights Neurons Planar Removal

Sep 29, 2026cs.NE

How much of fly walking is written in the wiring?

Connectome models of the fly nerve cord generate walking-like motor rhythms, but oscillation alone does not show that the specific wiring matters. Here we provide, to our knowledge, the first test of which features of motor output depend on the specific wiring. We simulated the leg motor systems of two independent Drosophila connectomes, with synapse counts as fixed weights and glutamatergic synapses treated as inhibitory, and compared each with six families of rewired networks that preserve progressively more of its structure, using pre-registered criteria. We find that rhythm is generic but antagonist coordination is not: many rewired networks were more rhythmic than the real ones, yet the real wiring coordinated antagonistic motor pools more strongly than every rewired network, most of all at the thorax--coxa joint. We trace this specificity to how premotor input is allocated between antagonistic pools. Both connectomes carry Sherrington's reciprocal innervation---neurons that excite one pool inhibit its antagonist---and no rewired network does. Reassigning premotor inputs between the pools abolished coordination even when motor neurons' typical input strength changed little (all pre-registered criteria met in one connectome; same direction in the other). Coordination, not rhythm, therefore reveals whether a connectome model's wiring matters.

Isabel Guan, Yuntian Zhao, Dingyuan Zhang +2

The Hong Kong University of Science and Technology · ZENBOT · Nanyang Technological University +1

Neural Circuits Neurons Musculoskeletal Wiring

Sep 28, 2026cs.LG

Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity

We study the minimum number of hidden neurons required for arbitrary-accuracy approximation of multivariate Hölder-continuous functions on

[0,1]^d

and the associated encoding complexity. For

d\geq 2

, we construct a fixed, explicitly defined activation function for which a closed-form network with two hidden layers of widths

d

and

1

achieves arbitrary accuracy in the uniform norm. We prove that

d+1

is the exact minimum total number of hidden neurons among standard feedforward networks with locally integrable activations and affine outputs. We further give a simpler construction using a single elementary activation that combines the floor and exponential functions. This construction requires three hidden layers of widths

d

,

1

, and

2

, only two neurons above the minimum. If a skip connection is allowed, widths

d

,

1

, and

1

suffice. These constructions use explicit grid addressing and integer encoding of quantized function values. For a bounded

α

-Hölder class, they require

O(\varepsilon^{-d/α}\log(1/\varepsilon))

bits, matching the metric-entropy lower bound up to a logarithmic factor.

Zilan Cheng, Li-Lian Wang, Zhongjian Wang

Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University, 637371, Singapore.

Neural Approximations Approximation Neurons Sample Complexity Metric Spaces

Sep 28, 2026cs.LG

Let the Neurons Die: Exploiting ReLU-Induced Model Degradation

Rectified linear unit (ReLU) networks can suffer from dying neurons, where units with persistently negative pre-activations produce zero outputs, blocking gradients through their activations. To exploit this failure mode, we present three training-time availability attacks based on data ordering and poisoning. We begin with the basic dynamic data-ordering attack (DOA), which greedily constructs a training prefix by selecting the next example that minimizes the target layer's post-update weight sum, aiming to push ReLU units toward negative pre-activations without modifying training samples or labels. We then develop two poisoning attacks, IG-DOA and IG-SKA, which use gradient inversion to synthesize class-conditioned samples by matching reference gradients in adverse model states constructed through data ordering or soft knockout, respectively. Soft knockout rearranges weights across adjacent layers to concentrate negative contributions. On a fully connected ReLU network trained on MNIST, ordering 100 of 60,000 training examples reduces test accuracy from 96% to 95% after only five epochs. Adding 200 poisoned samples from a single class reduces test accuracy to approximately 86-88% after five epochs in most evaluated conditions, compared with approximately 96% under clean training. These results demonstrate that ReLU-targeted data ordering and poisoning can impair learning without directly modifying the victim model's parameters.

Kexin Li, Wenjun Qiu, Joshua Abraham +2

University of Toronto, Toronto, Ontario, Canada

Rectified Linear Unit Networks Neurons Negative Results Degradation Exploitation

Sep 27, 2026eess.SP

Neuron-Level Architecture Growth: A Controlled Evaluation for EEG Time-Series Decoding

Convolutional EEG decoders are trained at a fixed width, usually set by their authors on other data. Growing methods add neurons during training where the loss could decrease the most, but whether they improve compared to a reference width is untested on EEG. Here, we grow three convolutional backbones on 12 motor-imagery datasets under three protocols and compare each with its reference model per subject. The growing ShallowFBCSPNet scores 2.9 points above its reference model with only half the parameters (0.57x), SCCNet changes by at most 1.2 points. Deep4Net growing models show decreased accuracy, but they require adaptation that prevent to compare faithfully the results. These differences follow the selection step, which keeps a candidate neuron relying on a dynamic threshold from singular values decomposition. Overall, these results suggest that growth helps when its criterion can rank the candidate neurons, and that the rate of skipped neuron addition tells where a decoder can be grown small from scratch.

Adam Mounir, Stella Douka, Arnault H. Caillet +2

Inria TAU – LISN, Université Paris-Saclay, France · Yneuro, Paris, France · Imperial College London, UK +1

Electroencephalography Neurons Growth Decision Thresholds

Sep 24, 2026cs.AI

Hallucination Neurons and Where to Find Them: An Investigation into the existence of Hallucination Neurons

Interpretable machine learning for Large Language Models (LLMs) increasingly relies on sparse probing methods that identify small sets of neurons claimed to detect and causally influence behaviors such as factuality recall, safety alignment, and hallucination. These claims have important implications for model auditing and behavioral steering, yet they are rarely tested against known failure modes of

L_1

-regularized probing in correlated, high-dimensional feature spaces. We propose a five-step diagnostic protocol covering feature correlation, bootstrap stability, sparse versus dense ranking disagreement, intervention baselines, and cross-dataset evaluation as a minimum standard for sparse-neuron localization claims. We investigate prior work using our proposed approach, specifically on H-neurons using open-source LLMs across TriviaQA, BioASQ, and NQ-Open datasets. Our results demonstrate detection replicates across both models and datasets, and exceeds the original reported AUROC gaps for TriviaQA and BioASQ datasets. Gemma 3 4B consistently outperforms MedGemma 4B on matched datasets, with AUROC gaps of +0.311 versus +0.235 on TriviaQA, +0.474 versus +0.455 on BioASQ, and +0.128 versus +0.112 on NQ-Open respectively. Causal validation at

n = 500

with five random seeds shows statistically significant effects beyond random same-layer baselines. At the same time, the diagnostic results indicate that the selected neurons are not uniquely localized. Across the three Gemma 3 4B settings, 19 of 22 selected H-Neurons have Pearson

|r| > 0.7

with other features, bootstrap selections show only moderate stability, and sparse and dense rankings overlap only weakly. Our findings show that sparse predictive structure can coexist with non-unique neuron selection. Routine diagnostic validation is necessary to distinguish detection claims from localization claims in mechanistic interpretability.

Huseyin Cavus, Sebin Sabu, Joshua Spear +2

Trakya University, Edirne, Türkiye · DRIVE, Great Ormond Street Hospital for Children NHS Foundation Trust, London, UK

Generative Hallucination Neurons Mechanistic Interpretability Systematic Study

Sep 24, 2026math.PR

Boolean threshold functions, neuron capacity, and memory retrieval

How much information can a single neuron remember? How many memories can neural networks retrieve without creating false memories? These questions are related to a basic question: how many Boolean threshold functions

f(x)=\operatorname{sgn}(a_0+\langle a,x\rangle)

,

x\in\{-1,1\}^n

, are there? In this paper, we show that the number

T_n

of distinct Boolean threshold functions is

T_n=2\binom{2^n-1}{n}\bigl(1+O(n^{-99})\bigr).

Equivalently, the capacity of a single threshold neuron is

n^2-\log_2(n!)+1+O(n^{-99})

bits, improving the

O(n)

error term in the result of Kahn--Komlós--Szemerédi to

O(n^{-99})

. To prove this, we show that, for

1\le r\le n-1

, and

v_1,\ldots,v_r

are chosen at random from

\{-1,1\}^n

,

\mathbb P\!\left\{ \langle v_1,\ldots,v_r\rangle\cap\{-1,1\}^n =\{\pm v_1,\ldots,\pm v_r\} \right\} =1-O(n^{-99}).

In the context of the Kanter--Sompolinsky Hamiltonian for memory retrieval, this identifies

r=n-1

as a sharp threshold, at which, for almost every collection of

r

memories, the only ground states are these memories and their negatives, confirming a weaker form of the Kalai--Linial--Odlyzko conjecture. It also settles a recent open problem posed by M. Anthony on the specification number of Boolean threshold functions. In addition, we show that, for every

1\le r\le n-1

,

\mathbb P\{v_1,\ldots,v_r\text{ are linearly dependent}\} =2\binom r2\,2^{-n}+O\!\left(2^{-n}e^{-cn}\right),

confirming a conjecture of Kahn--Komlós--Szemerédi.

Xinyuan Xie

Neurons Combinations Capacity

Sep 24, 2026cs.CL

Grammatical "grandmother neurons" are rare in LLMs

Understanding how Large Language Models (LLMs) encode linguistic structures remains a fundamental challenge in interpretability research. While diagnostic classifiers (or "probes") are widely used for this task, they face significant methodological criticism: training auxiliary classifiers introduces capacity confounds and calibration issues, often making it difficult to distinguish the model's intrinsic representations from the probe's ability to learn the task. To address these limitations, we introduce a probe-free framework for localizing linguistic selectivity at the individual neuron level. Leveraging the controlled contrasts of linguistic minimal pairs, we propose a Neuron Separability Index (NSI), a metric that directly quantifies how reliably single neurons differentiate grammatical from ungrammatical constructions without parameter updates. Applying NSI across 68 linguistic paradigms and seven checkpoints reveals three main patterns: 1) raw separability reaches near-peak levels earlier for morphological and syntactic distinctions than for syntax-semantics interface and conceptual distinctions. 2) after permutation normalization, single-unit selectivity is sparse, weak, and narrowly tuned: only a small fraction of units are sensitive to an average paradigm, and strongly selective "grandmother neurons" are rare. 3) whole-vector linear separability, single-neuron selectivity, and behavioral competence are largely dissociated, and targeted ablations further separate activation selectivity from causal reliance.

Linyang He, Nima Mesgarani

Zuckerman Mind Brain Behavior Institute, Columbia University

Linguistics Neurons Interpretability

Sep 23, 2026cs.LG

TNLearn: An Open Source Python Package for Task-based Neurons

The brain does not rely on a single type of neuron to perform all kinds of tasks; instead, it designs different neurons for different tasks. The concept of task-based neurons represents a paradigm shift compared to task-based architectures. It argues that solving a specific problem requires customized neurons, as task-based neurons capture useful prior knowledge from task-related data. To facilitate the use of task-based neurons in scientific research and industrial applications, we introduce TNLearn, an open-source Python package that provides automated construction of task-based neurons and networks, enabling smooth training of task-based networks. Comprehensive documentation, including technical exposition, API reference, and representative examples, is available online. TNLearn is open-sourced at https://github.com/NewT123-WM/tnlearn and has become a PyTorch ecosystem project.

Meng Wang, Tieyun Li, Juntong Fan +5

Shenzhen H&T Intelligent Control Co., Ltd., Shenzhen, China · School of Mathematics, Harbin Institute of Technology, Harbin, China · Department of Data Science, The City University of Hong Kong, Kowloon, Hong Kong +1

Neurons Pytorch Open-Source

Sep 21, 2026cs.CV

Topographic Training Concentrates Causal Circuits Without Improving Neuron Monosemanticity

Mechanistic interpretability of vision transformers seeks to decompose model computation into human-readable units, but learned representations entangle many concepts in each neuron. Feature superposition is widely treated as the central obstacle to this decomposition, yet most mitigations (sparse autoencoders, dictionary learning) are post-hoc and leave the underlying network unchanged. We ask whether a spatial-locality training loss (TopoLoss) can act as a lightweight, training-time prior that improves interpretability of standard mech-interp tools. Training ViT on ImageNet-100 across multiple TopoLoss weights

α

, we measure causal sufficiency of topographic clusters via activation patching and feature geometry via sparse autoencoders fit to the same residual stream. At

α=1.0

, topographic clusters are 2.79

\times

more causally sufficient than random unit sets of the same size, with the effect increasing monotonically in

α

. SAE L0 sparsity decreases by 11% and dead-feature fraction rises 19-fold, yet standard neuron-level monosemanticity scores are unchanged, indicating that topographic pressure acts at circuit level, concentrating causal mass into spatially local structures without disentangling individual neurons. This dissociation suggests current neuron-level monosemanticity metrics are insensitive to a class of real interpretability gains, and positions cheap architectural priors as a viable training-time complement to post-hoc tooling.

Gautam Ranka, Shubham Santosh Pandere, Aiden Dsouza

IvLabs, VNIT Nagpur, India.

Mechanistic Interpretability Neurons

Sep 21, 2026cs.LG

MGRD: Compact morphology-gated residual diffusion for variance-aware cross-domain neurite forecasting

Tracking neurite morphology over time helps characterize structural changes during neuronal development and deterioration, but long-term time-lapse imaging is resource-intensive and difficult to scale. Forecasting future morphology could reduce this burden. Existing neurite digital-twin models such as gated spatiotemporal attention (gSTA) produce a single deterministic forecast without representing variability among plausible futures. We introduce Morphology-Gated Residual Diffusion (MGRD), a compact stochastic surrogate that jointly forecasts twenty future neurite-morphology frames from ten observed frames while conditioning on morphology features derived from the latest observation. On controlled phase-field trajectories, MGRD reduces trajectory-wise mean MAE by 9.7% relative to a matched control while updating 4.46 times fewer parameters. On human iPSC-derived neuron microscopy, MGRD improves all four reported metrics over gSTA, including a 39.6% reduction in trajectory-wise mean MAE and a 45.3% increase in skeleton F1. Without mouse-domain retraining or fine-tuning, MGRD also improves MAE and skeleton F1 on mouse cortical-neurosphere microscopy across 10-40-min sampling intervals and forecast horizons beyond 13 hours. Repeated sampling provides a case-level variance score for ranking forecast difficulty. Retaining approximately 60% of the lowest-variance cases reduces mean MAE by 17.6% on iPSC microscopy and 16.8% on simulation data. MGRD uses 1.01% of gSTA's parameters, requires less than one tenth of its training-update time, and generates a 50-step DDIM trajectory 7.9% faster when morphology features are cached. These results establish MGRD as a compact stochastic surrogate for neurite-morphology forecasting and case prioritization across simulation and microscopy datasets.

Tsung Yeh Hsieh, Cosmin Anitescu, Chunghwan Kim +2

Department of Mechanical Engineering, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213, USA · Department of Biomedical Engineering, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213, USA · Department of Civil and Environmental Engineering, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213, USA

Spatio-Temporal Forecasting Neurons Retraining

Sep 16, 2026cs.LG

Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence

We analyze the learned input-output behavior of GLU-based neurons in large language models (LLMs). We propose a simple analysis method: For each neuron, we compute the cosine similarities between its input (reading) and output (writing) weight vectors. In this scheme, a strong negative cosine similarity indicates the neuron weakens the direction it detects in the residual stream, so we call this a weakening neuron. This allows us to gain a number of novel insights. First, we show that nine different LLMs have similar patterns: weakening neurons appear mostly in late layers whereas their counterparts, (conditional) strengthening neurons, are frequent in early-middle layers. Second, we find that weakening neurons display surprising behavior: even though there are few, they activate often and have a large influence on model behavior. Third, weakening neurons have a strong effect on model output when gate values are negative -- which is surprising since negative gate values are not expected to encode functionality.

Sebastian Gerstner, Hilal AlQuabeh, Kentaro Inui +1

LMU Munich and Munich Center for Machine Learning · MBZUAI · Tohoku University +1

Neurons Transformer Architectures Influence Cosine Similarity

Sep 16, 2026cs.RO

REACT: A Fully Spiking State-Space Model for Real-Time Event-Driven Temporal Perception

Robotic systems operating in dynamic environments require visual perception that evolves continuously with the incoming sensory stream. Event cameras provide microsecond temporal resolution and asynchronous sensing, but most learning-based methods accumulate events into frames or temporal bins, introducing an integration delay that can limit fast reaction. Here we propose REACT, a fully spiking state-space model for event-driven temporal perception that processes raw events one by one, without temporal accumulation. REACT uses a complex-valued spiking neuron, C-SiLIF, whose continuous-time dynamics are driven by the physical inter-event interval, allowing its internal state to evolve at the temporal resolution of individual events. We evaluate REACT on gesture recognition and time-to-collision (TTC) estimation from full-field event streams, without a target bounding box or localization input. On EvTTC, REACT achieves a 9.59% relative TTC error with 4.6 ms end-to-end inference latency, within 0.15 percentage points of the best learned method while requiring no target prior. At the dataset's mean approach speed, this latency corresponds to only 4 cm of vehicle motion, compared with 1 m for the fastest competing learned method. REACT further supports anytime TTC prediction, zero-shot transfer to a different driving sequence, and INT8 quantization, reducing the estimated energy consumption from 18.5 to 2.8 mJ per 32,768 events. These results show that event-driven spiking state-space dynamics can provide low-latency, continuously updated temporal perception for reactive robotic systems.

Geoffroy Keime, Nicolas Cuperlier, Benoit R. Cottereau

CerCo, CNRS UMR5549 Université de Toulouse Toulouse, France · IPAL, CNRS IRL, Singapore · ETIS, CY Cergy Paris Université CNRS, ENSEA Cergy, France

Reactive Time-To-First-Spike Real-Time Systems State Space Models Neurons

Sep 15, 2026cs.LG

Symmetry without a manifold: intrinsic dimension on orbits

The standard geometric derivation of neural scaling exponents takes the intrinsic dimension of a data manifold as its input. On modular addition in

\mathbb{Z}_p

that derivation has no input. The exact algebraic solution is an orbit of

\mathbb{Z}_p

acting by isometries. Transitivity alone makes the ratio statistic underlying the standard dimension estimator a point mass, so the estimator is undefined, and here the two nearest neighbour distances coincide exactly. Breaking the symmetry at scale

ε

returns a number, but one that tracks

1/ε

with no scale free plateau. We show that the failure is general, since on any finite orbit of a group acting by isometries the estimator reports the resolution at which the set is probed rather than a dimension. What replaces the power law is exponential in hidden width,

L(h)=L_\infty+A\exp(-c\,h^α)

, with

R^2

between 0.982 and 0.995 against 0.857 to 0.906 for a power law admitting the same floor and fitted under the same protocol. Where the data supply is sufficient the rate belongs to the regulariser rather than to the group, since weight decay moves

c

by a factor of 47 while group order moves it by 1.10, a residual below seed to seed resolution, for every fixed

α

between 0.75 and 2. The critical width falls with group order rather than rising, against capacity counting that assigns a fixed number of neurons to each irreducible representation.

Chon-Fai Kam, Miloud Bessafi, Frédéric Cadet

Dipartimento di Fisica e Chimica, Universit`a degli Studi di Palermo, via Archirafi 36, I-90123 Palermo, Italy · EnergyLab, Universit´e de La R´eunion, F-97715 Saint-Denis, France · Universit´e Paris Cit´e and Universit´e de La R´eunion, BIGR, INSERM UMR S1134, F-75014 Paris, France +1

Intrinsic Dimensionality Scaling Laws Symmetry Exponents Two-Layer Neural Networks Neurons+3

Sep 14, 2026q-bio.NC

A neural-astrocyte architecture implements a hybrid automaton for evidence accumulation

Astrocytes are non-neuronal glial cells that are receiving widespread attention due to their emerging role in neural computation. In this paper, we propose and study dynamical mechanisms by which astrocytes may augment the ability of neural networks to infer context in reinforcement learning (RL) settings. We construct a biologically inspired, two-level dynamical neural-astrocyte network with distinct spatial and temporal organization. We train this model on a hierarchical multi-context task that requires the agent to infer changes in latent task rules based on derived rewards. We find that in this setting, astrocytes enable evidence accumulation of changes in context and subsequent context-specific modulation of neural dynamics. We show that these functions are implemented via two dynamical mechanisms: (i) reward-induced bifurcations that relocate an asymptotically stable attractor into different, context-specific regions of state space, and (ii) the relative shallowness of these attractors, mediated by the entropy of the environment, giving rise to behavioral stickiness. Together, these mechanisms amount to a hybrid automaton, in which uncertainty accumulates until, eventually, the neural dynamics are switched to a new context. This model provides a neuro-dynamic schema, compatible with neural-astrocyte biology and prior empirical observations, for how astrocytes may integrate information from the periphery and drive contextual changes in neural circuits.

Giacomo Vedovati, Ilya E. Monosov, Thomas J. Papouin +1

Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO 63130, USA · Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21218 USA · Department of Neuroscience, Washington University in St. Louis, St. Louis, MO 63130, USA

Neural Dynamics Neurons Automata Bifurcation

Sep 14, 2026cs.LG

Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exit

We study compute reduction in neural networks through a unified partial versus full computation view, captured by one-shot magnitude pruning in the static regime and early exit in the adaptive regime. In an asymptotic single-neuron model, we prove a concentration theorem for one-shot magnitude pruning with explicit rates. We also introduce the conditional perceptron for early exit and show that its excess generalization error decays as a power of the compute gap, with an exponent that grows to infinity as the alignment between partial and full computations tends to one. We then extend the analysis to deep networks, characterizing how pruning-induced distortions accumulate with depth and deriving a corresponding compute-accuracy tradeoff for frozen-backbone early exit under a neural network Gaussian process model. Numerical simulations corroborate the predicted scaling laws.

Erdem Koyuncu

Department of Electrical and Computer Engineering, University of Illinois Chicago, IL, USA.

Unstructured Pruning Deep Network Neurons

Sep 14, 2026cs.RO

Pneumatic neurons for soft robots enable inflate-and-fire networks for rhythmic motion

Animals coordinate their movements through distributed neural circuits, but soft robots still typically depend on external, centralized electronics for control. Building soft robots that operate without centralized electronic controllers while remaining responsive to their environment remains a frontier challenge in soft robotics. In this work we introduce a soft-robot control architecture inspired by leaky integrate-and-fire models of biological neural circuits. The Pneumatic neuron (Pneu-ron) is a soft actuator that unifies energy conversion, logic, and actuation in one component. Each module combines a low-boiling-point fluid (LBF), a heater, and a mechanical switch into a self-excitable unit. Boiling the LBF inflates the module and triggers excitation and inhibition of adjacent modules in a process we call "inflate-and-fire". When interconnected into excitatory-inhibitory rings, Pneu-rons generate stable, sequential oscillations whose frequency emerges from the material dynamics and environmental conditions. By harnessing the inflation of Pneu-rons for actuation these networks can drive oscillatory locomotion of soft robots. Pneu-ron networks sustain oscillation under mechanical load and thermal variations, adapting through material physics rather than computation. Dynamical modeling of these networks reveals a dimensionless bifurcation diagram that dictates the network's oscillatory behavior. Encoding logic and actuation into material-level modules presents a new avenue for adaptive, electronics controller-free, soft robots.

Dongting Li, Michael Tolley, Nick Gravish

MAE Department, UC San Diego, La Jolla, 92093, US

Soft Robotics Single Excitation Time History Neurons Oscillatory Dynamics Motion

Sep 11, 2026cs.CL

Distribution-aware Language Neuron Identification in Multilingual Large Language Models

Multilingual large language models (mLLMs) contain a small fraction of feed-forward neurons that are sensitive to particular languages, commonly termed language-specific neurons. Existing work measures language specificity using the entropy of each neuron's language-wise probabilities of being active, where a neuron is considered active when its activation value is positive. However, this approach may not fully capture the multilingual nature of mLLMs, where language representations are distributional and mutually related. We propose Distribution-aware Language Neuron selection, which leverages pairwise relationships between per-language activation distributions over the full activation range, including negative values. Specifically, we quantify each neuron's language specificity by clustering languages using pairwise overlap coefficients between their activation distributions. Across two mLLMs and two held-out corpora, our identifier more effectively isolates language-specific causal effects, yielding up to 4.9

\times

higher on-target language damage per neuron while preserving off-target language performance.

Minjun Kim, Inho Won, Junghun Yuk +3

KAIST · KAIST InnoCORE PRISM-AI Center

Multilingual Language Models Language Modeling Neurons Causal Inferences

Sep 11, 2026cs.LG

Phases in a class of associative memories via hidden neurons

Associative memory in the Hopfield network is attractor dynamics in a disordered many-body system, and higher-order and exponential extensions turn its retrieval update into softmax attention. The polynomial and exponential regimes have been analyzed by different methods, with no common architecture in which to ask what fixes the storage scale. In this paper we study the bipartite architecture of Krotov and Hopfield, which we call the class

H

, whose model is fixed by a Lagrangian for each layer, taking the hidden neurons as the order parameter of retrieval. At polynomial load the replica method yields the replica-symmetric phase diagrams and closed-form capacities, and the crosstalk moment is common to Ising and spherical visible neurons, so their differences come from the visible entropy. With a softmax hidden layer the load is exponential, and a copy representation maps the thermodynamics onto random-energy-model counting, with paramagnetic, condensed, and frozen phases. Heating destabilizes retrieval by quantized reassignments of attention, and typical Gaussian patterns remain metastable at every load. The regimes differ in their crosstalk statistics, central-limit at polynomial load and large-deviation at exponential load, and the class

H

splits retrieval into two roles, the visible Lagrangian fixing stability and the hidden one the storage scale, two axes that may also guide the design of new Lagrangians.

Toshihiro Ota, Masato Taki

CyberAgent AI Lab, Shibuya, Tokyo 150–0002, Japan · RIKEN iTHEMS, Wako, Saitama 351–0198, Japan · Graduate School of Artificial Intelligence and Science, Rikkyo University, Toshima, Tokyo 171–8501, Japan

Hopfield Networks Neurons Phase Transitions Entropy

Sep 9, 2026cs.NE

A Bio-Plausible Visual Neural Network for Locust-Inspired Collision Perception

Locust visual systems have long served as an important biological paradigm for studying looming perception and collision avoidance. Numerous computational models have successfully reproduced the selective responses of Lobula Giant Movement Detector (LGMD) neurons to approaching objects, thereby emulating the fundamental functionality of the biological system. However, existing models remain limited in biological plausibility and robustness when operating in complex and dynamic visual environments. To address these limitations, we propose a biologically plausible neural network for locust-inspired looming detection. The proposed framework incorporates a spatially isotropic sampling strategy that mimics the ommatidial organization of the locust compound eye, a population-voting mechanism inspired by population coding in biological neural systems, and leaky integrate-and-fire neuronal dynamics to replace conventional sigmoid-based membrane activation. Systematic experiments on synthetic stimuli, laboratory sequences, and real-world driving scenarios demonstrate that the proposed model improves robustness under challenging visual conditions while preserving computational efficiency and enhancing biological fidelity. These results highlight the potential of biologically grounded neural computation for robust and efficient collision perception.

Qinbing Fu, Jiani Li, Jiajun Huang +1

School of Mathematics and Information Science, Guangzhou University, China

Perception Neurons Collision Prediction

Sep 3, 2026q-bio.NC

Axonal delay dispersion decides whether a neuron detects an event or a sequence, and predicts cortical column diameter

Cortical neurons fire sparsely -- often fewer than one spike per sensory window -- making rate coding insufficient and temporal coding a necessity. That conduction delays convert firing order into synchrony is long established. What governs which class of temporal feature a neuron detects -- one volley of coincident input, or two in a particular order -- has not been examined. We propose a delay-signature framework in which the axonal conduction delays converging on a dendritic branch constitute a physical key: only input sequences whose spike-time differences the delays compensate arrive synchronously, and coincidence detection, via calcium plateau thresholds, converts that synchrony into an all-or-none output. In simulations of an integrator-neuron model we report three results. First, a single physical scalar -- the dispersion of the delay set -- moves a population from event detection to order-selective sequence detection. The transition is emergent under random delays and connectivity: at narrow dispersion sequence detectors do not exist, and the dispersion at which they overtake event detectors tracks the inter-event interval with a slope statistically indistinguishable from one. This maps a computational distinction onto the anatomical one between myelinated and unmyelinated projections, making myelination a switch on what a neuron computes, not only a regulator of speed. Second, the same dispersion sets the code's limits: it bounds the longest codable interval and fixes an absolute timing tolerance of about a millisecond, with slowing better tolerated than speeding. Third, that millisecond window and horizontal conduction velocity together predict cortical column diameter, and the two areas with direct measurements fall where the relation puts them. One anatomically measurable parameter thus sets what a neuron detects and the limits of what it can represent.

Cheng Bi, Jipeng Sun

Independent Researcher · Department of Computer Science, Princeton University

Synaptic Plasticity Neurons Time-To-First-Spike Human Visual Cortex

Sep 1, 2026cs.CL

GAPS: Dimension-Level Gates for Conditional Activation Steering

Activation steering suppresses undesired behaviors in language models by adding a steering vector to the hidden state during generation. Recent conditional methods such as CAST and DSAS improve the behavior-capability trade-off by deciding when to intervene, but once active, they apply the full dense vector to all hidden dimensions, regardless of whether a neuron carries concept information or already lies in the desired regime. We introduce dimension-level conditioning as a complementary axis of selectivity that also decides which neurons to intervene on. Our method, GAPS (Gated Activation steering via Posterior and Separability), combines two training-free gates: a static separability gate that restricts steering to neurons with statistically reliable concept information (via AUROC), and a dynamic posterior gate that steers a neuron only when its current activation is better explained by the undesired concept under a Gaussian model. The gates add O(D) overhead per token, and they plug into existing conditional methods. On toxicity mitigation (RealToxicityPrompts) and concept removal (OneSeC) with Gemma-3 (4B) and Qwen-3 (1.7B), GAPS consistently matches or improves the Pareto front of its token-level counterparts; under a fixed capability budget, DSAS+GAPS reduces Gemma-3's toxicity rate from 6.52% to 0.48%, versus 3.52% for DSAS alone. Ablations attribute most of the gain to the posterior gate.

Moghis Fereidouni, Muhammad Umair Haider, Hassan Sajjad +1

University of Kentucky · Dalhousie University

Linear Activation Steering Model Activations Latent Feature Interventions High-Fidelity Conditional Generation Steering Neurons

Aug 31, 2026cs.LG

Sparse Competition during Training For the Emergence of Specialized Modules

Modularity in deep neural networks has been proposed as a means of improving both interpretability and training by promoting disentangled representations and reducing redundancy. In this work, we study the emergence of modular structure through competition dynamics between groups of neurons during training. We introduce a method that (i) maintains near-baseline accuracy, (ii) induces usage-based modularity by sparsely routing inputs to neuron groups, and (iii) encourages specialization of these modules, such that their activations are correlated with input classes. We evaluate the proposed approach on ImageNet-100 and CIFAR-100 and show that with it, specialized modules emerge without module-level supervision. These modules capture a meaningful high-level structure in the data, with individual modules responding to semantic categories (e.g., dogs or vehicles). We also study the emergence of a hierarchical partition of sub-tasks depending on the number of modules. Our results suggest that competitive dynamics can serve as a simple mechanism for inducing functional modularity in standard architectures.

Baptiste Rossigneux, Karim Haroun

Univ. Rennes · Inria, IRISA · University of Paris 8

Neurons Modular Framework Emergence

Aug 31, 2026cs.LG

Do VLMs Share Safety Neurons Across Modalities?

Vision-language models (VLMs) can comply with harmful requests delivered through images, even when their LLM backbones would refuse the same content in text. While prior work characterizes these jailbreaks empirically or at the representation level, how visual inputs perturb safety pathways at the neuron level remains uncharted. We close this gap with a causal, neuron-level analysis of safety mechanisms in 10 VLMs. We propose a two-stage detection pipeline with iterative ablation that accounts for self-repair, and introduce two modality-isolated benchmarks, ViSafe-Detect and ViSafe-Eval, which decouple visual and textual safety signals. Our analysis reveals: (i) Text safety in VLMs is localizable:

\sim

88 neurons (

<

0.01%) whose targeted ablation substantially reduces refusal. (ii) Text safety neurons constitute the dominant refusal pathway: ablating them is the only intervention that consistently and substantially reduces refusal across all models. (iii) Visual safety is high-dimensional and diffuse at the single-neuron level: text safety concentrates in

\sim

5 subspace directions while visual safety requires

\geq

50. This gap holds across architectures, explaining why current alignment has not closed the visual safety gap. Project page is at: https://jiaxuan-li.github.io/vlm-safety-neuron/ Warning: this paper may include examples of harmful content.

Jiaxuan Li, Jiahao Zhang, Duc Minh Vo +3

SB Intuitions Corp., Tokyo, Japan

Multimodal Benchmarks Neurons Modalities

Aug 31, 2026cs.LG

Functional Degeneracy in Neural Networks: Measurement and Pruning

A central question in modern machine learning is how much a trained model can be compressed without changing its behavior, to reduce the memory, compute and energy required to deploy it. To study this, we quantify functional degeneracy through the behavioral recovery rank, defined as the number of leading behavioral-Hessian eigendirections required to recover a trained model's performance. Using the behavioral recovery rank as a geometric benchmark for compression, we find that structural and magnitude pruning retain more degrees of freedom, even after the task is saturated. This gap suggests that functional redundancy is distributed across parameter directions and is not exposed by individual weights or neurons.

Maria Matveev, Pascal Esser, Ayush Bharadwaj +2

Department of Mathematics, LMU Munich · Munich Center for Machine Learning (MCML) · Independent +3

Unstructured Pruning Neural Network Neurons Hessian Measures

Aug 31, 2026q-bio.NC

"More Is Different'' in Neural Circuits: Algebraic Emergence of Effective Theories in Canonical Recurrent Motifs of Biological Neuronal Networks

Canonical neural circuit motifs are usually described functionally: divisive normalization rescales population activity by a pooled signal, and winner-take-all competition selects one pattern through recurrent excitation and shared inhibition. We represent them, and their compositions, algebraically as finite transformation systems and analyze the transition monoids generated by their input-conditioned updates, distinguishing structure already present in a generator from structure that appears only through composition, and, on a joint state space, structure inherited from one factor from structure that lives on a joint configuration. Individually aperiodic updates can generate non-aperiodic monoids. In the WTA, every frozen-drive generator collapses to fixed points, yet short input sequences create local cycles of winner-dependent inhibitory gating: globally dissipative dynamics with a reversible action. The strongest result arises in WTA-to-DN composition. The composed monoid then contains a genuinely composite local cycle in which normalization state and the winner's gating state change together, although every primitive generator is aperiodic. Holonomy analysis certifies this as a group component of the Krohn-Rhodes cascade rather than an incidental cycle, and finds most group-carrying image sets on joint configurations, whereas the uncoupled product has none. An exhaustive interface sweep shows that the composite cycle is a property of the coupling rather than of a chosen map. If motifs are building blocks of neural computation, composing them is a form of programming: one chooses primitives and interfaces so that the generated algebra has the intended repertoire. The transition monoid is that repertoire - what a primitive presents to any later construction. Recurrent circuits are compositional transformation systems; their algebra constrains what they can be programmed to compute.

Nima Dehghani

McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI), Cambridge, Massachusetts, USAa)

Neural Circuits Neurons Finite Groups Recurrence Discontinuous Gating Emergence

Aug 25, 2026cs.AI

RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons

Discovering stable neuron behavior across entire domains remains a challenge in mechanistic interpretability. Existing methods often rely on instance-level point estimates or computationally expensive procedures, which either obscure population-level variability or limit scalable domain-wide analysis. We present RACE (Residual Alignment for Consistency Estimation), a forward-pass statistical framework that evaluates the domain-wide functional consistency of Transformer neurons. Compared with gradient-based point estimates, RACE produces neuron rankings that yield more domain-specific effects under perturbation. Token-distribution shifts support the connection between the selected neurons and the target domain, while scoring requires roughly one-hundredth of the computational overhead of the gradient-based methods. Code is available at https://github.com/Nexround/RACE.

Runyu Wang, Bo Liu, Xiaxin Zhang +6

School of Transportation and Civil Engineering, Nantong University · Chongqing University of Post and Telecommunications · China Southern Power Grid Company Limited +1

Neurons Mechanistic Interpretability

Aug 25, 2026cs.LG

Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs

Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has focused on the emergence of a global low-dimensional representation. We show that this picture is incomplete. In regression problems with clustered data, we demonstrate that multilayer perceptrons (MLPs) naturally develop monosemantic specialized neurons: individual neurons become strongly aligned with a specific predictive feature relevant to a particular region of the input space. Rather than learning a single global low-dimensional representation, MLPs learn a collection of local low-dimensional representations. We show that this ability to specialize gives MLPs a provable data-efficiency advantage over feature-learning methods based on a global low-dimensional representation.

Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis +1

Department of Computer Science and Engineering, UC San Diego · Department of Statistics and Data Science, The Wharton School, University of Pennsylvania · Halıcıoğlu Data Science Institute, UC San Diego

Feature Learning Multilayer Perceptrons Neurons

Aug 10, 2026cs.AI

Decodable But Not Detachable: Training Data Granularity Determines Parametric Modularity in Large Language Models

Do large language models contain domain-specific parametric shells: concentrated, causally necessary neuron populations whose removal selectively degrades a target domain while sparing others? We apply a uniform causal methodology across two domain granularities, three model families (1.5B to 7B parameters), and eight domains. At the academic subject level, zero neurons exceed 60% domain selectivity across 939,008 combined FFN neurons and causal damage matrices are flat, despite domain identity being linearly decodable above 85% accuracy. At the language and modality level, 0.65--1.14% of neurons exceed 60% selectivity, damage matrices are near-perfectly diagonal (ratios up to 595:1), and shell neuron sets are essentially disjoint (IoU

< 0.003

). Masking code-selective neurons reduces mathematical reasoning accuracy by 16--24 percentage points across all models; masking Spanish or Chinese neurons leaves it at or below random. Shell strength increases monotonically with scale and shells are spatially interleaved in a pattern that precludes group-level selective quantization. Parametric shells form where and only where training data was modular at the token level.

Marcus Armstrong, Navid Ayoobi, Arjun Mukherjee

Department of Computer Science University of Houston Houston, TX 77204

Neurons Selectivity Modular Framework Decoding

Aug 9, 2026cs.CL

Multilingual Emotion Neurons in Large Audio-Language Models

Emotion is central to human communication, and its expression varies across languages. Large audio-language models (LALMs) achieve strong performance on multilingual speech tasks, yet it remains unclear whether they encode emotion through language-specific correlations or language-agnostic representations. We present the first neuron-level interpretability study of this question. We define Multilingual Emotion Neurons (MLENs) as functional units exhibiting stable emotional selectivity and aligned causal effects across languages, and introduce Consistency-Regularized Fusion (CR-Fusion) to identify them. Across four modern LALMs and 12 typologically diverse languages, emotion-sensitive neurons identified independently per language show minimal overlap, and additional monolingual identification data saturates quickly without isolating more transferable units, motivating identification from pooled cross-lingual evidence. Causal interventions demonstrate that MLENs identified by CR-Fusion provide more precise and transferable affective control than monolingual neuron sets in both zero-shot and low-resource settings. Leave-one-out ablations further reveal asymmetric transfer: individual identification languages, including low-resource ones, contribute non-redundant evidence, while several low-resource languages benefit most from the resulting cross-lingual transfer. Together, our findings provide the first causal, neuron-level account of how LALMs encode emotion across languages, and establish multilingual neuron identification as an effective mechanism for understanding cross-lingual affective behavior.

Xiutian Zhao, Philipp Koehn, Björn Schuller +1

Center for Language and Speech Processing (CLSP), Johns Hopkins University, USA · Group on Language, Audio & Music (GLAM), Imperial College London, UK

Cross-Lingual Consistency Large Audio Language Models Emotion Recognition Language Modeling Emotion Neurons

Aug 9, 2026cs.NE

SuperNeuroMAT: An Efficient Matrix-based Simulator for Spiking Neural Networks

Spiking neural networks (SNNs) offer a promising pathway to energy-efficient AI and brain-inspired computing. However, their widespread adoption is hindered by a lack of fast, accessible, and versatile simulation frameworks. In this paper, we introduce SuperNeuroMAT, an open-source, scalable, and highly efficient Python-based SNN simulator. We devise a novel matrix-based approach to model the leaky integrate-and-fire (LIF) neuron dynamics and natively support dense and sparse execution modes. This enables fast simulation of approximately 10,000 neurons in dense mode and 100,000 neurons in sparse mode on standard laptops and desktops without requiring specialized hardware. We demonstrate that SuperNeuroMAT consistently outperforms four established SNN simulators---NEST, Brian2, BindsNET, and snnTorch---on two performance metrics (execution speed and peak resident memory) and across various network sizes and connection probabilities. Furthermore, we demonstrate SuperNeuroMAT's applicability across a diverse set of problems. SuperNeuroMAT can efficiently handle conventional machine learning benchmarks such as the Digits and citation network datasets as well as neuromorphic event-based vision tasks such as N-CARS and ASL-DVS. Moreover, it can be extended beyond machine learning workloads and facilitate general-purpose workloads. We validated this by implementing the neuromorphic shortest path algorithm and two arithmetic primitives (addition and multiplication). SuperNeuroMAT can be installed via the Python Package Index (PyPI), thereby lowering the barrier to entry into the field of neuromorphic computing and accelerating the broader development of neuromorphic algorithms.

Prasanna Date, Kevin Zhu, Shruti Kulkarni +12

Oak Ridge National Laboratory, Oak Ridge, 37830, Tennessee, United States. · George Mason University, Fairfax, 22030, Virginia, United States. · University of Tennessee, Knoxville, Knoxville, 37996, Tennessee, United States. +1

Spiking Neural Networks Neuromorphic Computing Neurons Neuroflex Neural Network Artificial Intelligence Lifecycle

Aug 8, 2026cs.CL

NeuPAT: Neuron-aware Plasticity Allocation Tuning for Language-Preserving MLLMs

Multimodal expansion of large language models (LLMs) enables new perceptual capabilities but often compromises the language intelligence acquired during pretraining. In this work, we investigate this phenomenon from the perspective of internal adaptation dynamics and discover that neurons in pretrained LLMs exhibit heterogeneous plasticity during multimodal learning: some neurons are critical for preserving language capabilities, while others are more adaptive to multimodal knowledge. Based on this insight, we propose NeuPAT (Neuron-aware Plasticity Allocation Tuning), a lightweight and architecture-agnostic framework that allocates neuron-wise update constraints during multimodal instruction tuning. NeuPAT uses a small-scale probing stage to estimate neuron adaptation patterns and selectively protects language-sensitive neurons while promoting multimodal adaptation through more plastic neurons. Experiments across diverse LLM families demonstrate that NeuPAT recovers 94.5% of the language capability degradation caused by vanilla tuning on 11 language benchmarks while maintaining comparable multimodal performance, providing an effective approach for capability-preserving multimodal expansion.

Jiayue Jin, Jingwei Zhang, Chen Wang +2

College of Intelligent Robotics and Advanced Manufacturing, Fudan University · Zhongguancun Academy · Tianjin University +3

Large Language Model Adaptation Multimodal Large Language Models Neurons

Aug 7, 2026cs.AI

Learning in Deep Networks under Dale's Constraint

Biologically plausible learning models aim to explain how neural circuits can implement effective learning under the constraints of real neurons. Although significant progress has been made, a major remaining challenge is that existing models often allow neurons or synapses to represent mixed-sign values, both positive and negative, in violation of a basic aspect of cortical circuitry -- Dale's constraint: biological neurons are either excitatory or inhibitory, but not both, and synapses cannot change sign. In this work, we address this discrepancy by introducing a biologically motivated neural architecture in which both neural activations and learning signals are represented by non-negative activity, and synapses have fixed sign, while still supporting backpropagation-like learning. Our approach uses two complementary interacting non-negative channels to represent positive and negative contributions, inspired by evidence of on-off representations in the brain. These channels are implemented through a simple neural circuit motif, which is repeated throughout the network in both bottom-up and top-down pathways. Combined with a local Hebbian learning rule, the resulting model propagates learning signals and updates weights using only local interactions between neurons. We show theoretically that our learning scheme can exactly recover the backpropagation update despite relying solely on non-negative error signals. Empirically, beyond satisfying stronger biological constraints, the on-off architecture learns efficient representations, yielding substantial gains over comparable vanilla networks on the Tiny ImageNet benchmark. These results demonstrate that effective learning can emerge from biologically plausible mechanisms without requiring mixed-sign signals, providing a step toward more realistic models of neural computation.

Roy Abel, Shimon Ullman

Weizmann Institute of Science

Neural Circuits Neurons Backpropagation Deep Network Ridge

Aug 5, 2026cs.CV

NeuroAdaptTrainer: A Fiji/ImageJ Plugin for YOLO-Based Neuron Segmentation, InteractiveCorrection and Transfer Learning

Neuron counting and segmentation in microscopy images of neuronal cultures is a routine and time-consuming task in neuroscience research, traditionally performed through manual inspection or semi-automatic tools. We present NeuroAdaptTrainer, an open-source Fiji/ImageJ plugin that integrates a YOLO instance-segmentation model directly into the microscopist's workflow. The plugin allows a user to run automatic neuron detection on a single image or a batch of images, manually correct the resulting detections from within Fiji, and use those corrections to adapt the model to new imaging conditions via transfer learning. A built-in external validation module allows the base and adapted models to be compared quantitatively on a held-out annotated set. NeuroAdaptTrainer lowers the barrier for non-specialist users to benefit from deep-learning-based segmentation while keeping expert supervision at the center of the workflow.

Daniela Eraso-Casas, Gerard Villarroya-Pique, Esther Serrano-Pertierra +4

aComputer Sciences Department, University of Oviedo, Asturias, Spain · bElectrical Engineering Department, University of Oviedo, Asturias, Spain · cBiomedical Engineering Center (BME), University of Oviedo, Asturias, Spain +3

Microscopy Yolo26 Neurons Transfer Learning

Aug 2, 2026cs.AI

No Single Neuron of Failure: Distributed Safety Alignment Against White-Box Attacks

With the rapid release of open-weight large foundation models, safety threats are shifting from black-box jailbreaks to neuron-level white-box attacks that directly identify and manipulate safety-related neurons. Existing alignment methods often investigate the safety behavior on a small number of neurons, creating fragile single point of failure with limited redundancy. To address this issue, we propose distributed safety alignment (DSA), which redundantly encodes safety capabilities across multiple computational neurons, ensuring that the model maintains its safety baseline even when critical safety neurons are disrupted. Specifically, we localize the intervention to the inputs of the down-projection layers in language-side feed-forward networks and treat each feature coordinate as the activation of an individual neuron. DSA then combines neuron activations with loss gradients to compute a direction-aware first-order Taylor score that globally identifies the neurons that contribute most to the current refusal behavior of the model. Finally, targeted disruption via deterministic masking and stochastic dropout is coupled, forcing the model to abandon narrow safety neurons and redundantly encode safety behavior across multiple compensatory neurons. Extensive experiments show that DSA substantially improves robustness against white-box neuron-level safety attacks while preserving the model's general language and multimodal utility.

Simiao Xie, Chuancheng Shi, Shangze Li +5

The University of Sydney · NExT++ Research Centre, National University of Singapore

Safety Alignment Neurons

Aug 1, 2026cs.LG

SparseKAN: Compressing Kolmogorov--Arnold Networks Across Basis Functions, Neurons, and Bits

Kolmogorov--Arnold Networks (KANs) replace scalar edge weights with learnable univariate functions parameterized by multiple basis coefficients. This introduces a source of redundancy that conventional neural-network compression does not directly expose. We present \textbf{SparseKAN}, a unified approach that compresses KANs along three complementary axes: basis functions, neurons/channels, and numerical precision. SparseKAN equips the base branch, nonlinear basis branch, and individual basis terms with hierarchical learnable gates trained under a differentiable active-cost objective. The learned importance structure is subsequently hardened under explicit basis and width budgets, recovered in full or low precision, and physically compacted into smaller dense tensors rather than retained as sparse masks. Experiments on MNIST, CIFAR-10, and CIFAR-100 across spline, polynomial, RBF, wavelet, and convolutional KAN variants show that the structural axes compose predictably in cost. We also find strong basis-dependent differences in term importance: coefficient-based selection outperforms matched low-order truncation by up to 15.25 accuracy points in the evaluated Gram-polynomial settings. Eight-bit quantization is broadly robust, whereas 4-bit convolutional KANs require quantization-aware adaptation. Physical compaction removes up to 73.0% of parameters without accuracy loss on MNIST and reduces large-batch CUDA latency to as little as

0.51\times

dense execution. On a ZCU104 FPGA, the resulting sparse low-bit models achieve up to

23.63\times

lower inference latency, demonstrating that SparseKAN converts functional redundancy into measurable software and hardware efficiency. The SparseKAN implementation is available at https://github.com/OSU-STARLAB/SparseKAN.

Kazi Ahmed Asif Fuad, Lizhong Chen

Department of EECS Oregon State University Corvallis, OR 97331

Kolmogorov-Arnold Networks Compressed Model Neurons Bits

Jul 31, 2026cs.CV

SafeNexus: Discovering and Steering Modality-Universal Safety Neurons in MLLMs

Although Large Language Models (LLMs) have demonstrated promising safety performance, extending them to Multimodal Large Language Models (MLLMs) exposes a significant gap between expanded multimodal capabilities and existing safety mechanisms. Current defenses remain predominantly confined to specific modal settings, thereby limiting their robustness against broader cross-modal threats. To bridge this gap, we introduce SafeNexus, a cross-modal safety alignment framework that adopts a dedicated neuron-level intervention strategy. First, we formulate a neuron localization paradigm that identifies functionally specialized neurons by characterizing intermediate-layer activation patterns and quantifying their functional salience through importance scoring. Building upon this paradigm, we exploit contrastive data to identify modality-bound safety neurons (BS-Neurons), and validate their role in regulating safety behavior within each modality via targeted suppression. Further cross-modal analysis defines modality-universal safety neurons (US-Neurons) as the shared subset of BS-Neurons identified across individual modalities, serving as the core for defending against harmful cross-modal attacks. We observe that suppressing these neurons substantially degrades safety performance across modalities, while leaving overall utility largely unaffected. Building on these insights, we propose two safety alignment strategies: activation-level safety amplifier and safety neuron calibrator. The proposed strategies enhance model safety through two distinct routes: the former amplifies the activation magnitudes of US-Neurons, while the latter selectively calibrates them via targeted fine-tuning. Extensive experiments demonstrate that our method outperforms prevailing state-of-the-art approaches on safety benchmarks spanning diverse modality combinations, while effectively preserving utility.

Jian Yu, Fei Shen, Cong Wang +5

1Nanjing University of Science and Technology · 2National University of Singapore · 3Nanjing University +2

Safety Alignment Multimodal Large Language Models Neurons Steering

Jul 28, 2026cs.AI

Are the High-weight Neurons the Important Ones in Image Classification Neural Networks?

As neural network models for image classification advance, neurons play critical roles in pruning, backdoor defense, and interpretability. Yet existing work lacks clarity on the weight-importance relationship. We address this with a neuron importance assessment method using three experiments: quantifying overlap between high-weight and accuracy-impacting neurons, analyzing high-weight neuron perturbation effects, and testing post-retraining accuracy after high-weight neuron ablation. Experiments on CIFAR-10 and Mini-ImageNet reveal key patterns. Overlap analysis shows top 10% high-weight neurons overlap with important ones by only about 25% at maximum, dropping further in subsequent intervals. Perturbation tests find top 10% high-weight neurons cause 45-80% accuracy degradation under certain operations compared to 3-7% for random perturbations, but a third of them show minimal impact. Ablation-retraining results show removing top 10% high-weight neurons leaves accuracy 10-20% below baseline with no recovery, while ablating top 0.1% allows near-full recovery. Notably, some low-weight intervals show 10-17% degradation when perturbed, comparable to mid-range high-weight neurons. These results confirm not all high-weight neurons are important: their importance is nonlinear. Low-weight neurons also contribute significantly. This challenges weight-importance equivalence, offering refined neuron role insights. It supports applications like encryption prioritizing critical high-weight neurons and pruning removing non-critical ones, advancing neural network analysis.

Qitao Chen, Dongfu Yin, F. Richard Yu

WeBank Institute of FinTech, ShenZhen University, YueHai, ShenZhen, 518060, GuangDong, China. · Artificial Intelligence and Digital Economy Guangdong Provincial Laboratory, YuTang, ShenZhen, 518132, GuangDong, China.

Importance Weighting Neurons Neural Network Image Classification Imagenet

Jul 27, 2026cs.LG

Calibrated Partial Resets: Preventing Policy Collapse in Continual Reinforcement Learning

Neural networks are hindered by accumulating dormant neurons and loss of expressivity throughout training, particularly in non-stationary data settings, such as continual supervised and reinforcement learning. Recently, neuron resets have been used to maintain gradient flow and restore plasticity. However, full unit reinitialization often sacrifices peak performance and can destabilize training, leading to policy collapse. To preserve plasticity without destabilizing training, we propose Calibrated Partial Resets (CPR), an optimizer that periodically pulls low-utility neurons toward their initialization, with pull strength scaled by each neuron's utility. Unlike binary reset methods, partial resets avoid brittleness; unlike uniform decay, calibrated utility-scaling concentrates adjustment on the units that need it most. Among compared methods, only CPR avoids policy collapse over 400M training steps in SlipperyAnt, and it outperforms prior decay and reset-based methods on Continual MetaWorld and Continual MinAtar benchmarks. Ablations reveal a tunable trade-off between plasticity and peak performance, highlighting utility-scaled reinitialization as a promising direction for continual learning.

Luc McCutcheon, Evangelos Chatzaroulas, Saber Fallah

Department of Mechanical Engineering, University of Surrey, United Kingdom · Department of Computing Science, University of Surrey, United Kingdom

Replay-Based Continual Learning Plasticity Continual Learning Offline Reinforcement Learning Recalibration Collapse+1

Jul 25, 2026cs.CE

A Scale-adaptive Vision Model Links C. elegans Neuronal Morphology to Behavior for Neurotoxicity Assessment

Neurological disorders are a leading cause of global disability and are increasingly linked to environmental chemical exposures. Yet neurotoxicity assessment still relies on hand-scored morphological readouts that are subjective and poorly predictive of behavioral outcomes. Caenorhabditis elegans provides a genetically tractable, 3R-compliant alternative, but quantifying neuronal phenotypes from confocal microscopy at scale remains computationally challenging: existing vision foundation models, trained on natural or radiological images, cannot resolve the sparse signals and multi-scale lesions of neuronal imaging. Here, we introduce a dedicated self-supervised vision model for C. elegans dopaminergic neurons, together with CeNeuMorph, a multi-grained confocal benchmark of 27,117 annotated images. Specifically, moving beyond standard Masked Autoencoders, we propose a scale-adaptive masked image modeling strategy that jointly learns representations across resolutions and patch sizes under a fixed token budget. By decoupling structural semantic learning from rigid grid constraints, the model effectively resolves the full spectrum of neurodegenerative lesions - ranging from fine dendritic beading to gross soma shrinkage - within a tractable computational framework. Finally, our model surpasses both generalist and biomedical foundation models across classification, segmentation and detection tasks. Fusing visual features with morphological descriptors enables prediction of dopamine-dependent behavioral deficits (

R^2=0.498

). Screening 180 agrochemicals, we identify the benzimidazole moiety as a previously unrecognized determinant of dopaminergic neurotoxicity. Together, the work demonstrates how scale-adaptive self-supervised learning can connect morphology to function for a scalable alternative to mammalian in vivo models for neurotoxicity assessment and drug discovery.

Haochao Ying, Shenchong Lv, Yutao Sun +8

School of Public Health and Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, 310058, China. · State Key Laboratory of Transvascular Implantation Devices and TIDRI, Second Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou, 310009, China. · Polytechnic Institute, Zhejiang University, Hangzhou, 310058, China. +5

Morphogenesis Neurons Toxicity Self-Supervised Learning Agent Behavior Modeling

Jul 22, 2026cs.CR

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning

Large language models (LLMs) are vulnerable to backdoor attacks, where hidden triggers induce malicious outputs. Existing defenses generally fall into inference-time detection or training-time mitigation, but face two key limitations. First, they focus on fine-tuning-based backdoors (e.g., PEFT modules) and fail to address insidious model-editing attacks that bypass training pipelines. Second, they target simple classification settings and do not naturally extend to open-ended LLM generation and do not naturally extend to the open-ended generation characteristics of LLMs. Consequently, these methods focus on surface-level behavioral patterns while neglecting the deeper representational causes of malicious activations. This lack of mechanistic understanding forces defenses to depend on empirical heuristics, limiting their robustness, generality, and practical applicability in real-world LLM deployment. To bridge this gap, we introduce DeCNIP (Defense with Critical Neuron Isolation Pruning), which leverages representational analysis to identify and neutralize backdoors in a unified pipeline. Specifically, DeCNIP identifies trigger-like behaviors by optimizing a cross-entropy loss between harmful prompts with candidate tokens and benign inputs. This representational discovery exposes latent threats by uncovering mechanisms through which triggers hijack model weights. It then isolates Backdoor Critical Neurons (BCNs) and prunes them selectively to remove malicious influence while preserving model utility. Extensive evaluations on six open-source LLMs and two benchmark datasets demonstrate that DeCNIP achieves over 95% relative reduction in Attack Success Rate (ASR), outperforming seven state-of-the-art defenses with only 0.1% neuron intervention. Moreover, it maintains 97% of the model's performance on normal benchmarks, demonstrating its efficacy, robustness, and scalability.

Yuxi Li, Zhibo Zhang, Kailong Wang +3

Huazhong University of Science and Technology China · Nanjing University of Aeronautics and Astronautics China · Nanyang Technological University Singapore

Attacker Large Language Model Clean Label Backdoor Attack LLM Defense Mechanisms Neurons

Jul 21, 2026q-bio.NC

How the fly holds a single goal: normalization, not selection, in Drosophila FC2

A walking fly steers toward a goal direction, held as a bump of activity across the FC2 neurons of the fan-shaped body. These neurons also inhibit one another over distance, more strongly the farther apart they are, a feedback proposed to keep the fly on a single goal. We asked, from the connectome, what circuit produces this inhibition, and whether it lets FC2 actively choose one goal among competitors (a winner-take-all) or simply keeps a goal set elsewhere as one clean bump. Tracing the wiring in a single FlyWire brain, we find the inhibition is almost entirely global: four FB5A cells inhibit every FC2 neuron roughly equally, with a smaller, distance-dependent contribution from hDelta interneurons and a negligible direct component. A ring-attractor winner-take-all (the kind the compass uses) requires local recurrent excitation that the FC2 wiring lacks, so this geometry cannot build one; and across a range of dynamical models, including a spiking network, no version of the circuit locks onto a winner at the connectome-scaled reference coupling. FC2 therefore normalizes an externally set goal rather than selecting it, with FB5A likely acting as the global normalizer, much as the APL neuron does in the mushroom body. We are explicit about two open points: a different mechanism, mutual inhibition between two competing goals (which hDelta supplies), could in principle select at very strong coupling, and we bound rather than exclude it; and FB5A's inhibitory identity is a low-confidence prediction of the connectome's transmitter classifier, not yet measured, and likely not GABAergic. We then ask where the goal is actually set: the connectome nominates an upstream hDelta network and rules out the leading proposed alternative, whose neurons supply under 0.2% of FC2's input. Finally, we propose a direct experiment, silencing FB5A while imaging FC2, that would test the account.

Gioele Nanni, Christopher Lee

Common Origin

Neurons Morphogenesis Symbiotic Paradigm Normalization Single Pass

Jul 20, 2026cs.LG

SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs

Large Language Models (LLMs) have demonstrated remarkable capabilities across a range of Natural Language Processing (NLP) tasks, but their high computational and memory demands pose significant challenges for deployment on resource-constrained edge devices. Existing approaches to model compression and optimization often rely on coarse-grained pruning or quantization, which can compromise accuracy or require re-training and fine-tuning. In this work, we introduce SelectInfer, a neuron-level optimization framework that enables efficient LLM inference on edge devices through selective neuron loading and computation. By profiling and identifying both task-specific and general-purpose neurons using an offline LLM profiler, SelectInfer implements two key optimizations: selective loading, which reduces memory footprint by selectively loading a subset of neurons that were identified to be most important during the offline stage, and selective computation, which dynamically computes only the most relevant neurons at runtime. Evaluation across multiple datasets shows that SelectInfer achieves significant reductions in memory footprint and computation while preserving task performance, making it a practical step towards enabling LLM deployment on edge devices

Huzaifa Shaaban Kabakibo, Eric Schniedermeyer, Artem Burchanow +1

Department of Computer Networks, Paderborn University, Paderborn, Germany

LLM Inference Optimization Neurons

Jul 20, 2026cs.MA

PRIME: Plasticity Recovery in Multi-Agent Environments for UAV-Assisted Emergency Communication Networks

Most reinforcement learning controllers for these networks assume stationary conditions, and the few that handle change react to the external environment while leaving the network's internal state unexamined. We show that sustained non-stationarity damages this internal state directly: as objectives shift, neurons progressively fall dormant and the shared policy loses the capacity to learn. The obvious remedy, resetting dormant neurons, is unsafe under shared-parameter multi-agent training: many neurons that appear inactive are still receiving strong training gradients, and whether a neuron appears dormant depends on which agent's observations it processes. PRIME (Plasticity Recovery In Multi-agent Environments) therefore verifies both directions before intervening. Extending the bidirectional Silent Neuron framework to cooperative multi-agent reinforcement learning, it aggregates activation and gradient statistics over the full team batch, reads the backward signal from the gradient the training loss has already deposited , not from a hand-crafted proxy, and reinitializes only neurons that are simultaneously activation-dormant and gradient-silent. Useful representations are preserved while learning capacity is restored. On a phase-switching UAV emergency communication simulator, PRIME improves interquartile mean return by 24.9% over MAPPO and holds dormant neuron fractions at 10--20% versus 40--45%; ablations attribute the gains to the gradient signal and team-level aggregation rather than to the specific reset operator. A dynamic regret bound shows that the perturbation cost scales with the small silent-subspace dimension rather than the full parameter count.

Wen Qiu, Zhiqiang He, Wei Zhao +1

Department of Information and Communication Engineering, Kitami Institute of Technology, Japan · Graduate School of Informatics and Engineering, University of Electro-Communications, Japan · School of Computer Science and Technology, Anhui University of Technology, China

Multi-Agent Reinforcement Learning Multi-Agent Planning Emergency Assistance Plasticity Neurons Gradient Component+1

Jul 18, 2026cs.CL

Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs

Large language models often succeed on one formulation of a problem while failing on an equivalent formulation. Whether these failures arise from distinct internal circuits or different activation states of a shared circuit remains unknown. Recent mechanistic interpretability studies suggest that arithmetic in LLMs emerges from a "bag of heuristics," encoded by a sparse set of MLP neurons that represent distinct arithmetic strategies. We investigate whether arithmetic heuristic neurons are form-invariant across symbolic arithmetic, natural language word problems, and Python code in three Llama-3 models. In each format, we identify arithmetic heuristic neurons using a two-stage pipeline combining attribution patching and activation patching. A compact set of neurons is shared across all three formats, and targeted interventions show this shared circuit is both necessary and sufficient for late-layer arithmetic computation. Transferring the shared neurons' activations from a successful execution in one format to a failed execution in another recovers most incorrect predictions, exceeding 97% for addition and subtraction, indicating that cross-format failures arise from activation states rather than distinct circuits. Moreover, shared neurons consistently belong to the same heuristic families across formats, demonstrating that arithmetic computation in LLMs is largely form-invariant at the neuron level.

Sharath Naganna, Tanvir Ahmed Sijan, Uddipta Kalita

Indian Institute of Technology Bombay, Mumbai, India · Jahangirnagar University, Dhaka, Bangladesh · National Institute of Technology, Silchar, India

Large Language Models Fail Mechanistic Interpretability Neurons Circuits

Jul 16, 2026cond-mat.stat-mech

Moment-Resolved Readout and Reservoir Diversity in Nonequilibrium Langevin Computing

Nonlinear thermodynamic computers based on Langevin dynamics exploit thermal fluctuations as a physical substrate for computation. Recent work has shown that quartic-confined fluctuating degrees of freedom can act as thermodynamic neurons capable of nonlinear function approximation at finite observation times. Here we extend this paradigm from mean-only readout to moment-resolved readout. Instead of representing each driven reservoir solely by its first moment, we construct a response vector from the elementwise raw polynomial moments

\mathbb{E}[\bm{x}]

,

\mathbb{E}[\bm{x}^{\odot 2}]

, and

\mathbb{E}[\bm{x}^{\odot 4}]

. These observables combine displacement and central-shape contributions and are naturally aligned with the linear, quadratic, and quartic terms of the local driven dynamics. We further introduce a heterogeneous multi-reservoir architecture in which three reservoirs with distinct initialization and training histories form a joint

2304

-dimensional response representation. Under the fixed MNIST

60000/10000

reproduction protocol, feature-level fusion achieves the best observed accuracy of

9695/10000=96.95\%

, compared with

9682/10000=96.82\%

for the strongest single-reservoir model and

9684/10000=96.84\%

for equal-weight logit averaging. An exact paired McNemar test does not establish a statistically significant improvement over the strongest single reservoir, but the ablation and wrong-set overlap results provide suggestive evidence of complementary classification errors. These results motivate higher-order polynomial-moment readout and reservoir heterogeneity as candidate design principles for finite-time Langevin computing.

JiZheng Duan, MingYang Zhao, YanWei Chen +1

Institute of Modern Physics, Chinese Academy of Sciences

Reservoir Computing Langevin Dynamics Thermodynamics Readout Nonlinearity Neurons+1

Jul 13, 2026cs.LG

Sparse Inter-Layer Dependencies of Transformer FFN Neurons

Feedforward network (FFN) blocks account for a large fraction of the parameters and computation in Transformer architectures, yet their internal structure remains difficult to interpret due to the additive superposition induced by the residual stream. We examine whether the activation of an FFN neuron can be explained by a sparse set of preceding neuron activations and attention outputs. We introduce a training-free attribution method that estimates the relative influence of upstream neurons and attention outputs on a target neuron's activation. Empirically, across models and layers, we find that small subsets of preceding activations and attention outputs suffice to preserve neuron activations with high fidelity when all remaining inputs are masked with their average values. Effective sparsity is even greater when accounting for the inherent activation sparsity of upstream layers. Moreover, applying the neuron-specific masks in all layers simultaneously, such that the induced deviations propagate through the network, leaves model perplexity largely unchanged at moderate sparsity levels. These results demonstrate that, despite dense parameterization, FFNs exhibit sparse and structured inter-layer dependencies at the neuron level. Our method provides a practical, scalable tool for circuit-level interpretability and identifies candidate sparse pathways with potential implications for efficient inference.

Johannes Knittel, Hanspeter Pfister

Harvard University

Transformer Attention Feed-Forward Transformer Architectures Neurons

Jul 10, 2026eess.IV

Tracking Intermittent Particles with Self-Learned Visual Features

In time-lapse fluorescence imaging, single-particle-tracking is a powerful tool to monitor the dynamics of objects of interest, and extract information about biological processes. However, tracked particles can be subject to occlusion and intermittent detectability. When these phenomena persist over a few frames, tracking algorithms tend to produce multiple tracklets for the same particle. In this work, we introduce self-supervised learning of visual features to compare tracked particles, and we exploit both visual and positional distances to robustly stitch tracklets representing the same particle. We demonstrate the performance of our stitching framework on time-lapse fluorescence sequences of Hydra vulgaris neurons. Results show high stitching precision, and reduction of errors made by previous algorithms on the same data by a factor of two.

Raphael Reme, Victor Piriou, Alison Hanson +5

IP Paris, BIA, IDS, IMAGES · Institut Pasteur, Universit´e de Paris-Cit´e, CNRS UMR 3691, BioImage Analysis Unit F-75015 Paris, France · LTCI, Telecom Paris, Institut Polytechnique de Paris, France +3

Multi-Object Tracking Microscopy Spatiotemporal Representations Self-Supervised Learning Neurons

Jul 8, 2026cs.NE

Single-Entity Spiking Neuron Models: Survey

In this work, we reviewed different approaches in mathematical modeling of biologically plausible neural systems. Models are characterized and classified based on their common features and special use cases. In addition to spiking models, different types of discrete and continuous analogs are considered to accurately simulate biological processes, including membrane potential dynamics. The models under investigation include neurons and various components encountered in neural systems and affected the dynamics. The selection of specific approaches was driven by their prevalence and innovative perspectives in order to enhance the relevance of the presented information.

Leon Parepko, Danila Shulepin, Albert Nasybullin

Innopolis University Innopolis, Russia

Neurons Biological Analog Entities Survey

Jul 8, 2026cs.LG

Distributed Sparse Interventions in Language Models

Language models perform a wide range of tasks at varying levels of abstraction with the capacity to flexibly infer tasks from context, execute multiple tasks simultaneously, and select among competing tasks. To study the role of model components in task behaviour, their causal influence can be investigated through interventions. Prior work on model steering has largely focused on interventions along global directions in activation space, modeling task representations as approximately linear and additive. By studying interventions at the neuron level, we find substantial, neuron-specific nonlinear effects on model outputs that are not captured by current steering approaches. We introduce Distributed Sparse Interventions (DSI), an intervention approach that considers nonlinearities and interactions between neurons across layers to identify sparse sets of neurons that elicit task-relevant computations. Across a range of tasks, we demonstrate that DSI can activate task behaviour in instruction-tuned language models by localising and intervening on as few as 0.01% of neurons, highlighting the effectiveness of sparse, distributed interventions in the neuron basis. Additionally, adopting a set-based perspective enables computations over the identified neuron sets, offering insights into the roles of individual neurons by analysing their effects across tasks. Through sparse interventions, DSI enables fine-grained control over model behaviour, localisation of task-relevant neuron sets, and furthers our understanding of task composition.

Maximilian S. Ernst, Lorenz Linhardt, Aaron Peikert +1

Max Planck School of Cognition · Center for Lifespan Psychology Max Planck Institute for Human Development · Machine Learning Group Technische Universität Berlin +5

Latent Feature Interventions Neurons Model Activations Instruction-Tuned Models Steering

Jul 6, 2026cs.CL

Faithfulness to Refusal: A Causal Audit of Neuron Selectors

Attribution scores increasingly identify which neuron rows of a language model matter for applications such as pruning, interpretability, and editing for safety, yet whether they identify causally important rows is rarely tested directly. We address this with two paired audits built on one-shot neuron-row zeroing. We first audit selectors at the language-modeling level: attribution methods substantially outperform activation and magnitude-based baselines at identifying dispensable rows across five LLMs. We then adapt the same intervention into a behavior test by driving it with a contrastive harmful-versus-benign signal; the attributed rows are sufficient to install refusal on hate and crime while keeping benign over-refusal low and preserving language model fluency, and specific in that layer-matched random controls at the same depths fail. Highly rank-stable selectors can be among the least causally valid. Refusal moreover lives in a redundant subspace, where different attribution methods install it through largely disjoint row sets, so the recovered edit is one realization of a sufficient set rather than a unique mechanism. Together, these findings show that rank-stability proxies miss the kinds of selector failures a direct causal audit can surface.%

Ananth Eswar, Pratinav Seth, Utsav Avaiya +1

Lexsi Labs · Work done while at Lexsi Labs.

Refusals Causal Intervention Model Auditing Faithfulness Neurons Fluency

Jul 6, 2026quant-ph

Canonical quantization of neurons

Canonical quantization provides a systematic procedure for constructing quantum models from classical Hamiltonians. Here, we apply this principle to a fundamental computational primitive of machine learning: the neuron. Specifically, by viewing a neuron as a composition of an energy function and an activation function, we quantize this model by replacing the energy function with a quantum Hamiltonian and applying the activation function to it through matrix functional calculus. This results in an activation observable that can be measured on an input quantum state. We investigate the use of these quantized neurons for function approximation, where the objective is to learn an unknown observable from labeled quantum data. For this purpose, we develop hybrid quantum-classical algorithms for training and evaluation, including procedures for measuring the activation observable and estimating gradients of the squared loss error. Our algorithms for gradient estimation rely on basic primitives like classical random sampling, the Hadamard test, and Hamiltonian simulation, and those for measuring an activation observable rely on quantum algorithms known as the power of one qumode and Schroedingerization. Numerical experiments demonstrate that our quantized neurons exhibit enhanced expressive capabilities relative to corresponding classical neurons on representative learning tasks. Our work establishes canonical quantization as a principled framework for constructing quantum machine learning primitives and provides a foundation for developing neural architectures tailored to quantum data.

Alexander He, Nana Liu, Mark M. Wilde

Department of Physics, Cornell University, Ithaca, New York 14850, USA · Institute of Natural Sciences, School of Mathematical Sciences, Ministry of Education Key Laboratory in Scientific and Engineering Computing, and Global College, Shanghai Jiao Tong University, Shanghai 200240, China · School of Electrical and Computer Engineering, Cornell University, Ithaca, New York 14850, USA

Quantum Neural Networks Hybrid Quantum-Classical Pipeline Qubit Quantized Neurons

Jul 6, 2026cs.CV

Probe-EM: Targeted Neuron Tracing via Training-Free Semantic Verification

Establishing large-scale, high-resolution neural connectivity maps is fundamental to elucidating the structural basis of brain function. However, when processing terabyte- or petabyte-scale electron microscopy data, over-segmentation inherent in automated reconstruction algorithms remains a critical bottleneck, requiring extensive manual proofreading spanning person-years. To alleviate the heavy reliance on annotated data and the limited flexibility of conventional tracing methods, we propose a training-free, targeted neuron tracing framework. Specifically, we introduce a skeleton-guided Heuristic Spatial Search paradigm that leverages geometric priors to iteratively reconstruct neuronal morphologies through a probing-verification cycle. To achieve robust zero-shot semantic verification, we further develop a Dimension-Aware Semantic Verification strategy built upon the foundation model NeuroSAM 2. This strategy resolves intra-slice splits via Planar Ensemble Consensus and inter-slice splits via Axial Spatio-Temporal Propagation. Notably, we integrate the proposed workflow into the Neuroglancer visualization platform, enabling an interactive human-in-the-loop proofreading system. Experimental results demonstrate that the proposed method outperforms supervised baselines and reduces manual proofreading time by 33.4%. The source code is publicly available at https://github.com/HeadLiuYun/Probe-EM.

Liuyun Jiang, Yanchao Zhang, Jinyue Guo +5

State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Future Technology, University of Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences +1

Human Connectome Project Neural Reconstruction Neurons Electron Microscopy

Jul 5, 2026cs.AR

Neuromorphic Silicon Neuron Controller for Adaptive Deep Brain Stimulation in Parkinson's Disease

Parkinson's disease (PD) affects millions worldwide and causes severe motor symptoms. Adaptive deep brain stimulation (aDBS) delivers physiologically informed stimulation that can track fluctuations in PD motor symptoms, enabling more intelligent DBS control. However, most existing aDBS approaches are primarily algorithm- and software-driven, with limited efforts toward circuit realization, particularly low-power and implantable integrated circuits. This paper presents the Silicon Leaky Integrate-and-Fire Deep Brain Stimulation (SiLIF-DBS) controller, a neuromorphic silicon neuron stimulator implemented with metal-oxide-semiconductor (CMOS) technology. For system-level evaluation, a simplified computational model of the SiLIF-DBS controller is derived and embedded within a Parkinsonian cortico-basal ganglia framework for closed-loop validation. The system is driven by beta-band subthalamic nucleus local field potentials (STN-LFPs), with their average rectified value (Beta ARV) used as the control biomarker. Our SiLIF-DBS controller for aDBS suppresses pathological beta activity while consuming only 25% of the power required by open-loop stimulation and achieving a suppression efficiency of

5.85\%

/

μ

W. Overall, our SiLIF-DBS controller achieves strong beta suppression at substantially reduced power, delivering high suppression efficiency that demonstrates it is a viable foundation for low-power implantable aDBS.

Md Abu Bakr Siddique, Jakub Orłowski, Yan Zhang +1

Department of Electrical and Computer Engineering, Michigan Technological University, Houghton, Michigan, USA · School of Electrical and Electronic Engineering, University College Dublin, Dublin, Ireland · Department of Biological Sciences, Michigan Technological University, Houghton, Michigan, USA

Adaptive Deep Brain Stimulation Neuromorphic Computing Neurons Human Visual Cortex

Jul 5, 2026cs.LG

NKI-Agent: Domain-Specific Fine-Tuning and Agentic Tool Use for Neuron Kernel Generation

Recent agentic approaches to LLM-based kernel generation have achieved impressive results on CUDA. For emerging AI accelerators such as AWS Trainium and Inferentia, automated kernel generation and optimization remain largely unaddressed. Writing kernels for these chips via the Neuron Kernel Interface (NKI) is particularly challenging: developers must navigate a multi-engine architecture, tile-based programming, and explicit data movement across multi-level memory hierarchy. Moreover, no publicly-available training data, benchmarks, or tool-augmented agents exist for this domain. We introduce NKI-Agent, the first system combining domain-specific supervised fine-tuning (SFT) with a compile-verify-fix agent loop for NKI kernel generation. We adapt the existing CUDA-Agent framework to Neuron hardware, curate 6,000 NKI kernel generation tasks for training, and construct NKIBench, a 250-task benchmark across three difficulty levels. Evaluated on real Trn1 hardware, NKI-Agent with Claude Opus 4.8 and a rank-aware system prompt achieves a 77.3% pass rate on the 150-task NKIBench. We show that tool use is critical: Opus 4.8 scores 6% in single-shot mode without agent tools. On a 60-task subset, we show that an SFT-trained Qwen3-Coder-30B-A3B achieves 25.0% pass rate at 1/100th the cost, outperforming Claude Sonnet 4 (15.0%). We also report that Group Relative Policy Optimization (GRPO) with binary compilation reward fails to improve over SFT, providing guidance on reward design for RL-based kernel generation.

Junjie Tang, Jun Huan, Hao Zhou +2

Amazon Web Services

Graphics Processing Unit Kernels Cuda Agentic Qwen3 Neurons

Jul 4, 2026cs.CL

Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs

Dialectal data are scarce relative to Modern Standard Arabic (MSA), causing Arabic LLMs to overproduce MSA and struggle with dialectally accurate generation. This raises a fundamental interpretability question about where and how dialectal features are encoded within model internals and whether these representations can improve dialect generation without fine-tuning. We study two inference-time approaches as interpretability probes and control mechanisms. First, neuron-level analysis identifies sparse populations that encode dialect-specific features and tests whether amplifying or suppressing them steers model outputs toward target dialects. Second, vector steering extracts dialect-specific activation directions and injects them during inference, motivated by feature entanglement at the neuron level. We find that these neurons are real but only partially explanatory. They occupy under 1% of MLP dimensions but span only 5% to 21% of the residual dialect direction. This limited coverage is causally consequential. Neuron steering reinforces dialect in some varieties when the prompt is already dialectal but cannot induce it from MSA prompts, whereas vector steering succeeds in both settings. Arabic dialects are therefore steerable mainly through distributed rather than localized representations

Kareem Elozeiri, Mervat Abassy, Omar Kallas +4

1Mohamed bin Zayed University of Artificial Intelligence · 2Qatar Computing Research Institute, Hamad Bin Khalifa University · 3Tohoku University +1

Arabic Natural Language Processing Dialects Linguistics Language Modeling Interpretability Neurons

Jul 3, 2026cs.LG

Rank-Order N-of-M Codes for Sparse Distributed Memory: Disentangling Representation and Learning Effects in Noise Robustness Against Contemporary Neuromorphic Architectures

Large language models remain limited as continual learning systems, motivating renewed interest in Sparse Distributed Memory (SDM) as an explicit online episodic memory. CALM (Nechesov and Ruponen, 2025) identifies its threshold-binary encoder as an open design question. This paper evaluates rank-order N-of-M encoding (Furber et al., 2007) as an alternative. We make three contributions. First, a faithful reimplementation validates the published architecture by confirming exact equivalence between WheelSDM and RankOrderSDM (cosine similarity 1.0000 across 10 seeds) and reproducing the documented divergence of RDLIF neurons under interference. Second, multi-seed capacity experiments show RankOrderSDM outperforming StandardSDM by 13.4 percentage points at saturation in the scaled configuration and by 0.8 percentage points at the published architecture scale. Third, BER robustness experiments disentangle representation and learning effects, showing that the large robustness gain arises primarily from the interaction of rank-order encoding with MAX-Hebbian learning, while the encoder alone provides only a small advantage under matched learning conditions. Experiments on GloVe-100 embeddings confirm this small but consistent encoding benefit on real structured data, whereas sentence embeddings exhibit a ceiling effect at low memory load. A secondary analysis shows that idealized rank-order encoding requires half the component-level encoding energy of SpikingMamba's SI-LIF neurons at four-bit precision, although decoder costs dominate overall system energy. These results identify which components of the original rank-order SDM architecture provide measurable benefits for contemporary memory-augmented AI systems, offering practical guidance for architectures such as CALM.

Joy Bose

Independent Researcher Bengaluru, India

Neuromorphic Computing Hebbian Learning Neurons

Jul 2, 2026cs.NE

Electronic Bursting Neuron: design, equations and hardware implementation

Electronic neurons are a keystone for construction of the spiking neural networks which have numerous applications in neuroprosthetics, artificial memory, intensive calculations etc. A number of concepts of electronic neurons has been already proposedm with some of them implemented in hardware. However, new schemes are of significant interest since the existing ones do not fit all requirements: either they are too complex and expensive in realization, or they are not able to demonstrate all demanded regimes, or their do not have a appropriate mathematical description and therefore may be investigated only experimentally etc. In this study we propose a new design of bursting electronic neuron constructed as a circuit implementation of the equations of a phase-locked loop system. To succeed, we use a novel hybrid approach: we start from the phenomenological equations providing the demanded, then we adjust and modify these equations to simplify the implementation rather than implementing the biophysical equations into thee hardware directly or writing equations for the already constructed circuit. The resulting circuit is simple in implementation and well matches the underlying equations. It can be used for description of not only a single neuron, but small neural circuits too.

Lev V. Takaishvili, Vladimir I. Ponomarenko, Maksim V. Kornilov +1

1Peter the Great St. Petersburg Polytechnic University, Russia · 2Saratov Branch of Kotelnikov Institute of Radioengineering and Electronics of RAS, Russia · 3Saratov State University, Russia

Neurons Circuits Edge Hardware

Jul 2, 2026cs.LG

Frequency Shift Physics-Informed Extreme Learning Machine for Solving High-Frequency Partial Differential Equations

Solving partial differential equations (PDEs) with high-frequency solutions remains a central challenge in physics-informed machine learning due to spectral bias -- the tendency of neural networks to learn low-frequency components preferentially. This paper proposes a Frequency Shift Physics-Informed Extreme Learning Machine (FS-PIELM) framework that addresses this limitation through an additive mechanism for weight initialization. Rather than multiplying random weights by a scaling factor, the method translates the mean of the Gaussian weight distribution while keeping the variance fixed at unity, thereby avoiding the variance amplification inherent in scaling-based methods. Two variants are developed: FS-PIELM-L assigns independent frequency magnitudes to individual neurons, while FS-PIELM-G groups neurons for improved robustness. Theoretical analysis shows that the frequency variance under the proposed framework remains bounded and approaches unity regardless of target frequency, in contrast to the quadratic growth of conventional approaches. The method preserves the computational efficiency of extreme learning machines, requiring only a single linear solve. Experiments on seven benchmark problems spanning six equation types -- Helmholtz, wave, Poisson, Klein-Gordon, heat, and advection-diffusion -- on both regular and complex geometries show that the linear variant achieves the best accuracy in six of seven cases, with improvements of one to nearly five orders of magnitude over existing PIELM variants. The code and data accompanying this manuscript will be made publicly available at https://github.com/xgxgnpu/Physics-informed-vibe-coding/tree/main/FS-PIELM.

Xiong Xiong, Ruonan Zhai, Zheng Zeng +3

School of Mathematics and Statistics, Northwestern Polytechnical University, Xi’an, 710072, China · MIIT Key Laboratory of Dynamics and Control of Complex Systems, Northwestern Polytechnical University, Xi’an, 710072, China · College of Science, Shijiazhuang University, Shijiazhuang, 050035, China +2

Partial Differential Equations High-Frequency Reconstruction Frequency Neurons

Jul 1, 2026cs.CV

BrainFIBRE: A Foundation Model via Information Decomposition for Brain Microstructure

Diffusion MRI probes brain microstructure with particular sensitivity to early cerebrovascular and neurodegenerative changes. Neurite Orientation Dispersion and Density Imaging (NODDI) decomposes the diffusion signal into three biophysically interpretable maps: neurite density index (NDI), orientation dispersion index (ODI), and free water fraction (FWF), capturing neurite packing, fiber coherence, and extracellular fluid. These 3D maps offer a rich substrate for transferable microstructural representations, yet integrating them is challenging: standard representation learning struggles to disentangle the unique information in each map from their shared and synergistic interactions. We present BrainFIBRE, the first foundation model for brain microstructure, pretrained on NODDI-derived maps from 55,592 UK Biobank participants. We propose Self-supervised Partial Information Decomposition (SPID), which extends PID-guided multimodal learning to the self-supervised regime for the first time. A novel Counterfactual Candidate Construction (CCC) paradigm perturbs inter-modality alignment through modality dropping and swapping, providing the contrastive signal for a Mixture-of-Experts architecture to disentangle unique, synergistic, and redundant information without any downstream label. On both Caucasian and Asian cohorts, BrainFIBRE achieves state-of-the-art performance across diverse tasks predicting age, sex, cerebrovascular and neurodegenerative markers, and cognition, while yielding neurobiologically interpretable representations that reveal task- and cohort-specific interaction patterns. BrainFIBRE establishes a versatile foundation for neuroimaging analysis at the microstructural level.

Zijian Dong, Yi Lin, Ji Fang +3

National University of Singapore, Singapore

Neurons Foundation Model