Hebbian Learning

Ask about this topic

Momentum

0 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21
View on map

Latest papers 14

All topics
CardsList
Jul 17, 2026cs.LG

Constrained Hebbian Learning Supports Efficient Representational Allocation under Structural Constraints

Introduction: Biological systems face anatomical and metabolic constraints, including costly synaptic maintenance and limited connectivity. These constraints favor neural codes that compress behaviorally relevant information into low-redundancy patterns. We test whether an excitatory competitive Hebbian rule can support synaptic resource allocation under such constraints and whether the resulting representations occupy a more favorable cost-performance regime than reference learning rules. Methods: Representational cost is quantified using mutual-information-based measures derived from the Variational Information Bottleneck. Experiments use fixed audiovisual embeddings from three audiovisual benchmarks (AVE, Kinetics-Sounds, VGGSound100) to isolate downstream associative plasticity. Hebbian learning is compared with Dense Difference Target Propagation (DDTP) and backpropagation (BP) under matched sparsity and architectural constraints. Results: Hebbian learning achieves lower task-information cost (CTI) than sparse BP and DDTP in the main compressed comparisons, while reaching CTI values comparable to shallow BP with nonnegative weights. Rather than uniformly improving classification performance, Hebbian learning shifts the trade-off between task-relevant information and representational cost, yielding lower CTI at comparable functional performance in several settings. Discussion: The results indicate a cost-performance trade-off rather than uniform accuracy gains. For a given level of task-relevant information, Hebbian representations retain less input information while preserving functional performance, although accuracy is slightly reduced on some datasets. These findings support interpreting Hebbian learning as a mechanism for synaptic resource allocation rather than as a general strategy for maximizing audiovisual classification accuracy.

Patrick Inoue, Florian Röhrbein, Andreas Knoblauch

KEIM Institute, Albstadt-Sigmaringen University, Germany · Department of Computer Science, Chemnitz University of Technology, Germany

Hebbian LearningSynaptic PlasticityRepresentational CapacityStructural Constraints
AskPDF
Jul 8, 2026cs.LG

Nonlinear Bandit

In this paper we first study the problem of generalized linear bandit (GLB) under heavy-tailed noise. The characteristics of heavy-tailed distributions are widely observed in real-world applications such as personalized recommendation, financial markets, and medical treatments. Based on the online mirror descent (OMD) method, we propose an algorithm EHM that extends the adaptive Huber loss method (Wang et al., 2025) with one-pass update (O(1)\mathcal{O}(1) computational complexity with respect to current round tt and the time horizon TT), which simultaneously achieves an almost optimal regret of O~(T11+ε)\widetilde{\mathcal{O}}(T^{\frac{1}{1+ε}}) where TT is the time horizon. In addition, by utilizing a special property of some link function (Sawarni et al., 2025), our algorithm eliminates the need to know a commonly used parameter. Next, we study the GLB problem under the case when contextual characteristic becomes piecewise constant, and we slightly revised former algorithm to obtain the PGLB-EHM algorithm. After theoretical analysis, we prove that the regret upper bound order stays the same. Furthermore, we look deeper into a special case of nonlinear bandit (NB) and present the NB-EHM algorithm with bisection method and special restriction. Eventually we utilize the affine lifting approach and show that the general NB problem can be applied with NB-EHM to achieve a sublinear regret bound.

Tianshuo Zheng, Ting Wu, Zhi-Hua Zhou +1

School of Mathematics, Nanjing University, Nanjing, 210093, China · School of Artificial Intelligence, Nanjing University, National Key Laboratory for Novel Software Technology, Nanjing, 210023, China · School of Mathematics and Physics, Xi’an Jiaotong-Liverpool University, Suzhou, 215123, China

Optimal RegretLoss FunctionHebbian Learning
AskPDF
Jul 3, 2026cs.LG

Rank-Order N-of-M Codes for Sparse Distributed Memory: Disentangling Representation and Learning Effects in Noise Robustness Against Contemporary Neuromorphic Architectures

Large language models remain limited as continual learning systems, motivating renewed interest in Sparse Distributed Memory (SDM) as an explicit online episodic memory. CALM (Nechesov and Ruponen, 2025) identifies its threshold-binary encoder as an open design question. This paper evaluates rank-order N-of-M encoding (Furber et al., 2007) as an alternative. We make three contributions. First, a faithful reimplementation validates the published architecture by confirming exact equivalence between WheelSDM and RankOrderSDM (cosine similarity 1.0000 across 10 seeds) and reproducing the documented divergence of RDLIF neurons under interference. Second, multi-seed capacity experiments show RankOrderSDM outperforming StandardSDM by 13.4 percentage points at saturation in the scaled configuration and by 0.8 percentage points at the published architecture scale. Third, BER robustness experiments disentangle representation and learning effects, showing that the large robustness gain arises primarily from the interaction of rank-order encoding with MAX-Hebbian learning, while the encoder alone provides only a small advantage under matched learning conditions. Experiments on GloVe-100 embeddings confirm this small but consistent encoding benefit on real structured data, whereas sentence embeddings exhibit a ceiling effect at low memory load. A secondary analysis shows that idealized rank-order encoding requires half the component-level encoding energy of SpikingMamba's SI-LIF neurons at four-bit precision, although decoder costs dominate overall system energy. These results identify which components of the original rank-order SDM architecture provide measurable benefits for contemporary memory-augmented AI systems, offering practical guidance for architectures such as CALM.

Joy Bose

Independent Researcher Bengaluru, India

Neuromorphic ComputingHebbian LearningNeurons
AskPDF
Jun 28, 2026cs.NE

Supervised Hebbian learning in Deep Counterstream Associative Networks

Modern machine learning applications employ deep neural networks training with the error backpropagation algorithm. Although this algorithm is very effective, it lacks biological realism. For example, backpropagation requires symmetric connectivity, and a separate neural processing channel for error signals. Prior works have therefore proposed a number of more realistic alternatives for error backpropagation. However, most of them still suffer from demanding preassumptions that may be not fulfilled in the real brain, for example, they often still require either symmetric connectivity or two separate processing channels, and often require also special mathematical operations like subtractions or function inversions. Here I propose supervised counterstream learning in deep associative networks as a simpler approach that requires only recognition of errors during training, and then backpropagates correcting target activity through the same activity channel as used for forward propagation. For this, two activity waves are initiated at the same time in input and output layers and then traveling in opposite directions to meet in one of the hidden layers. By employing simple local Hebbian-type learning rules, the corresponding activity pattern sequences get linked bidirectionally, thereby decreasing error rates over time. Despite its simplicity and an incomplete hyperparameter optimzation, a high high test accuracy is achieved on the (binarized) MNIST data set that is comparable to more demanding architectures.

Andreas Knoblauch

Albstadt-Sigmaringen University, KEIM Institute, Poststrasse 6, 72458 Albstadt-Ebingen, Germany

Hebbian LearningBackpropagation
AskPDF
Jun 23, 2026cs.CV

Adaptive Hebbian Memory Routing in Vision Transformers for Few-Shot Learning

Few-shot image recognition requires models to adapt to new classes from a small labeled support set. Hebbian fast-weight memory can provide temporary associative information during an episode, but fixed memory behavior may not be appropriate for every few-shot task. In this work, we propose Adaptive Hebbian Routing for few-shot Vision Transformers. The method uses a lightweight MLP router to control the contribution of Hebbian memory, the strength of memory updates, and the retention of previous memory from support-set features. We study Adaptive Placement, Adaptive Plasticity, and Fully Adaptive Hebbian Routing. Experiments use ViT-Small, DeiT-Small, and Swin-Tiny under 5-way 1-shot evaluation on Omniglot, CIFAR-FS, and cross-domain transfer from CIFAR-FS to Omniglot. In the direct Swin comparison, fixed and adaptive Hebbian variants use the same memory location. Adaptive Plasticity improves the fixed Hebbian result from 96.74% to 96.92%, while Fully Adaptive Routing achieves the best result at 96.94%. The fully adaptive Swin model also reduces inference time from 16.51 ms to 14.05 ms relative to fixed Hebbian Swin. On CIFAR-FS, adaptive variants improve performance across all three backbones, and the multi-shot evaluation shows that these gains remain useful as the number of support examples increases. These results show that adaptive plasticity and adaptive memory activation can improve few-shot Transformer representations beyond fixed Hebbian behavior.

Mohammed Yusuf Mujawar, Noorbakhsh Amiri Golilarz

Department of Computer Science The University of Alabama, Tuscaloosa, AL, USA

Few-Shot LearningVision TransformerHebbian LearningPlasticity
AskPDF
Jun 22, 2026cs.NE

Local Pheromone Network: Sparse Local Learning with Multi-Scale Synaptic Trails, Consolidation, and Replay

Backpropagation-trained dense neural networks are powerful function approximators, but they couple learning across many parameters and can overwrite previous associations when tasks conflict. This paper describes Local Pheromone Network, a small research prototype for sparse, local, manually updated neural networks. In Local Pheromone Network, each output unit reads only a fixed local neighborhood of input units subject to geometric distance and molecular-tag compatibility. Each synapse stores a weight, a short-term pheromone trace, a long-term pheromone trace, and an optional consolidation state. Training does not call automatic differentiation. Instead, every layer performs a pheromone-weighted Hebbian-style update on a budgeted subset of local synapses selected from local error and co-activity. The update budget adapts online: it shrinks when loss improves and expands toward recently active neighborhoods when loss worsens. Optional mechanisms add structural plasticity, local replay, output masks for partitioned learning, and a target-free local contrastive step. We present the implementation, learning rule, and preliminary experiments on synthetic regression, partitioned memory, conflicting memory, consolidated conflict, structural plasticity, replay, and a synthetic long-context hybrid memory task. The prototype learns local linear rules, preserves partitioned memories through tags and masks, reduces forgetting under consolidation, and uses replay under conflict.

Xingcheng Fu, Xianjun Chen, Zhihao Li

Hebbian LearningSynaptic PlasticitySingular Learning TheoryReplayEfficient Long-Context InferenceSwarms
AskPDF
Jun 7, 2026cs.AI

STAR: Rethinking MoE Routing as Structure-Aware Subspace Learning

Mixture-of-Experts (MoE) scales model capacity efficiently by selectively routing inputs to a specialized subset of experts. However, input-expert specialization, the core motivation of MoE, critically depends on whether the router is actually aware of input structure. In practice, MoE routing is typically implemented as a shallow linear projection with limited awareness of input representation, which often leads to unstable routing. We propose STAR, a Structure Aware Routing that rethinks MoE routing as a subspace learning problem by augmenting standard learnable routing with an evolving principal subspace that tracks dominant input structure via Generalized Hebbian Algorithm (GHA). By aligning routing decisions directly with input structure, STAR enables stable expert specialization. We evaluate STAR on controlled synthetic setup and large-scale language and vision tasks, where it consistently improves routing quality and downstream performance over strong MoE baselines. Moreover, optional test-time subspace updates further enhance routing robustness and generalization under input distribution shifts.

Sumin Park, Noseong Park

Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.

Mixture-Of-ExpertsLarge Language Model RoutingSubspaceHebbian Learning
AskPDF
May 19, 2026cs.LG

Closed-form predictive coding via hierarchical Gaussian filters

Predictive coding (PC) offers a local and biologically grounded alternative to backpropagation in the training of artificial neural networks, yet to date, it remains slower, and performance degrades sharply as network depth increases. We trace both problems to a single simplification: current PC networks fix the precision matrix to the identity, discarding precision-weighted prediction errors that the variational derivation requires to be fast, local, and Bayesian. We close this gap by expressing predictive coding networks as deep hierarchical Gaussian filters (HGFs) and restore precision-weighted message passing, yielding dynamic uncertainty estimates and Hebbian-compatible update rules at every layer. The resulting networks can simultaneously learn activations, weights, and precisions under a single free-energy objective, with no global error signal, and resolve inference without requiring iterations or automatic differentiation. On FashionMNIST, our solution approaches backpropagation in epoch-level wall-clock cost while converging in fewer epochs, and outperforms it on online, data efficiency, and concept-drift tasks. We thus establish that closed-form variational inference with online precision learning provides a tractable foundation for deep predictive coding networks, retaining biological and interpretative advantages, without requiring iterative relaxation or global error signals.

Aleksandrs Baskakovs, Sylvain Estebe, Kenneth Enevoldsen +3

Center for Humanities Computing Aarhus University Aarhus, Denmark · Interacting Minds Center Aarhus University Aarhus, Denmark

Predictive CodingHebbian LearningVariational Inference
AskPDF
May 1, 2026cs.NE

Benchmarking local Hebbian learning rules for memory storage and prototype extraction

Associative memory or content-addressable memory is an important component function in computer science and information processing, and at the same time a key concept in cognitive and computational brain science. Many different neural network architectures and learning rules have been proposed to model the brain's associative memory while investigating key component functions like figure-ground segmentation, perceptual reconstruction and rivalry. A less investigated but equally important capability of associative memory is prototype extraction where the training set comprises distorted prototype instances and the task is to recall the correct generating prototype given a new distorted instance. In this paper we benchmark associative memory function of seven different Hebbian learning rules employed in non-modular and modular recurrent networks with winner-take-all dynamics operating on moderately sparse binary patterns. We measure pattern storage and weight information capacity, prototype extraction capabilities, and sensitivity to correlations in data. The original additive Hebb rule comes out with worst capacity, covariance learning proves to be robust but with moderate capacity, and the Bayesian-Hebbian learning rules show highest capacity in almost all different conditions tested.

Anders Lansner, Andreas Knoblauch, Naresh B Ravichandran +1

1KTH Royal Institute of Technology, EECS, Stockholm · 2Stockholm University, Dept. of Mathematics, Stockholm · 3KEIM-Institute, Albstadt-Sigmaringen University, Germany +1

Hebbian LearningNeuroscience
AskPDF
Apr 20, 2026eess.AS

Incremental learning for audio classification with Hebbian Deep Neural Networks

The ability of humans for lifelong learning is an inspiration for deep learning methods and in particular for continual learning. In this work, we apply Hebbian learning, a biologically inspired learning process, to sound classification. We propose a kernel plasticity approach that selectively modulates network kernels during incremental learning, acting on selected kernels to learn new information and on others to retain previous knowledge. Using the ESC-50 dataset, the proposed method achieves 76.3% overall accuracy over five incremental steps, outperforming a baseline without kernel plasticity (68.7%) and demonstrating significantly greater stability across tasks.

Riccardo Casciotti, Francesco De Santis, Alberto Antonietti +1

Signal Processing Research Centre, Tampere University, Tampere, Finland · Department of Electronics, Information and Bioengineering (DEIB), Politecnico di Milano, Italy

Hebbian LearningClass-Incremental LearningContinual LearningModern Deep NetworksSingular Learning TheoryDeep Learning+2
AskPDF
Apr 18, 2026cs.CL

HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents

Long-term memory is a critical challenge for Large Language Model agents, as fixed context windows cannot preserve coherence across extended interactions. Existing memory systems represent conversation history as unstructured embedding vectors, retrieving information through semantic similarity. This paradigm fails to capture the associative structure of human memory, wherein related experiences progressively strengthen interconnections through repeated co-activation. Inspired by cognitive neuroscience, we identify three mechanisms central to biological memory: association, consolidation, and spreading activation, which remain largely absent in current research. To bridge this gap, we propose HeLa-Mem, a bio-inspired memory architecture that models memory as a dynamic graph with Hebbian learning dynamics. HeLa-Mem employs a dual-level organization: (1) an episodic memory graph that evolves through co-activation patterns, and (2) a semantic memory store populated via Hebbian Distillation, wherein a Reflective Agent identifies densely connected memory hubs and distills them into structured, reusable semantic knowledge. This dual-path design leverages both semantic similarity and learned associations, mirroring the episodic-semantic distinction in human cognition. Experiments on LoCoMo demonstrate superior performance across four question categories while using significantly fewer context tokens. Code is available on GitHub: https://github.com/ReinerBRO/HeLa-Mem

Jinchang Zhu, Jindong Li, Cheng Zhang +2

The Hong Kong University of Science and Technology (Guangzhou) · Jilin University · The Chinese University of Hong Kong

Episodic MemoryLarge Language Model AgentsHebbian Learning
AskPDF
Jan 13, 2026cs.NE

Building Supervision into Hebbian Plasticity through Spike Agreement

Supervised learning in spiking neural networks (SNNs) typically requires either gradient-based backpropagation, which sacrifices the Hebbian, spike-driven character of biological plasticity, or reward-modulated Spike-Timing-Dependent Plasticity (STDP), in which class supervision enters only as a scalar gate on an otherwise class-agnostic correlation signal. We propose Supervised Spike Agreement-Dependent Plasticity (Supervised SADP), a gradient-free supervised Hebbian learning algorithm in which class information is embedded directly into the Hebbian plasticity computation rather than introduced through reward modulation. SADP trains the output layer via a supervised Hebbian rule that encodes class labels into output spike patterns, then trains the hidden layer by measuring each hidden neuron's chance-corrected temporal agreement, Cohen's kappa, with the correct-class output spike train produced by the forward pass without gradient computation or external reward. A K-shift extension aggregates agreement over temporal offsets, providing robustness to spike-timing jitter at linear computational cost. We evaluate Supervised SADP against reward-modulated STDP across six benchmark and medical imaging datasets, four input encoding strategies, K_shift in {5,25}, and three reward modes (none, binary, margin). Supervised SADP outperforms STDP in a significant majority of comparisons. Under Poisson encoding, SADP achieves 86.46% on MNIST and 76.62% on Fashion-MNIST, outperforming the best STDP configurations by 23.66 and 23.29 percentage points, respectively. Across the encodings tested, including CNN-extracted features, SADP outperforms STDP in the large majority of cells and trains 1.47x faster on average, with up to 2.86x speedup under Poisson inputs. These results position Supervised SADP as a stable, efficient, gradient-free alternative to reward-modulated STDP for supervised SNN learning.

Gouri Lakshmi S, Athira Chandrasekharan, Harshit Kumar +3

Indian Institute of Science Education and Research Thiruvananthapuram, Vithura, Thiruvananthapuram, Kerala 695551, India

Spiking Neural NetworksHebbian LearningTime-To-First-SpikePlasticity
AskPDF
Dec 8, 2025cs.LG

Formalized Hopfield Networks and Boltzmann Machines

Neural networks are widely used, yet their analysis and verification remain challenging. We present a Lean~4 formalization covering both deterministic and stochastic models. We first formalize Hopfield networks -- recurrent networks that store patterns as stable states -- and prove their convergence, and the correctness of Hebbian learning, the rule that updates parameters to encode patterns. We then turn to stochastic networks, whose probabilistic updates converge to a stationary distribution: we formalize the dynamics and learning of Boltzmann machines and prove their ergodicity -- convergence to a \emph{unique} stationary distribution -- via a new formalization of the Perron--Frobenius theorem.

Matteo Cipollina, Michail Karatarakis, Freek Wiedijk

Axiomatic AI, Barcelona, Spain · Radboud University, Nijmegen, The Netherlands

Hopfield NetworksHebbian LearningStochastic
AskPDF
Date pendingcs.CR

Chameleon: An Adaptive AI-Driven Honeypot Architecture Using Threat-Calibrated Particle Swarm Optimization and Semantic Deception Rapidly-Exploring Random Trees

Traditional honeypots share an invariant behavioral profile: a skilled adversary can confirm the presence of a deception environment within a few diagnostic commands, limiting their intelligence value. Commercial deception products (USD 100,000-150,000/year) similarly lack real-time model-driven feedback. Chameleon, an openly distributed adaptive honeypot, addresses both shortcomings. It integrates: a BiLSTM classifier achieving 99.61% accuracy across seven threat categories at ~2 ms CPU latency; a locally deployed Qwen3.5-0.8B model delivering 90% generation accuracy at 4.5 ms latency; and two meta-heuristic engines. Threat-Calibrated PSO (TC-PSO) reshapes swarm inertia and objective amplification in proportion to the classifier's anomaly output, adjusting connection-holding delays in real time. Semantic Deception RRT (S-RRT) evolves deception schemas via exponentially scaled pheromone updates from a language-model severity assessment, with a depth-decay multiplier enforcing a finite memory footprint. A controlled 30-seed benchmark (42-71, identical trajectories and budgets) shows threat-calibrated inertia alone does not improve search over standard PSO on static or dynamic landscapes (p = 0.18); population-diversity mechanisms (GA/ACO) significantly outperform PSO-family optimizers on threat-regime shifts (p < 0.0001, d <= -37). S-RRT's depth-decay delivers a significant memory reduction versus standard RRT (53.1 vs. 119.2 units, p < 0.0001, d = -10.0); its severity-weighted pheromone does not improve raw fitness. Operating cost is ~USD 17/month, a ~490-fold reduction versus commercial alternatives.

Rohit Swami, Tushar Singh, Akash Warde +1

Dept. of AI and Data Science, SIES Graduate School of Technology, Mumbai, India

HoneypotsThreat DetectionDeceptionParticle Swarm OptimizationHebbian Learning
AskPDF