This paper introduces a neural network model that learns multiple attributes as images and performs associated, sequential recall of the learned memories. Briefly, the model presented here is an associative memory model that extends previous models [1] by increasing the number of attributes. In the real world, memory recall generates a chain of associations consisting of complex and diverse data with meaningful relations. However, because this experimental system is designed to implement and verify the processing operations behind such operations, we believe it is not a problem if the associative memory (i.e., the chain of data) is composed of attributes that do not necessarily have clear relation with each other. Accordingly, the attribute-processing systems prepared in this study consist of five types: the C.CB-RN system for processing color attributes, the S.CB-RN system for shape attributes, and the V.CB-RN system for size attributes, as adopted in our previous paper [1], as well as the SV.CB-RN system for processing the names of the world's most beautiful scenery (spectacular view names) and the CN.CB-RN system for processing constellation names. As before, the data presented to each CB-RN system are represented as image patterns using QR codes [2]. These five types of CB-RN systems will be combined and trained with QR code pattern images of the attribute elements of each system. After that, when a pattern image of an attribute element is presented to any of the CB-RN systems, a mechanism will be constructed in which a chain (associative) recall of pattern images of related attribute elements in the other trained systems will be generated.
Associative memory or content-addressable memory is an important component function in computer science and information processing, and at the same time a key concept in cognitive and computational brain science. Many different neural network architectures and learning rules have been proposed to model the brain's associative memory while investigating key component functions like figure-ground segmentation, perceptual reconstruction and rivalry. A less investigated but equally important capability of associative memory is prototype extraction where the training set comprises distorted prototype instances and the task is to recall the correct generating prototype given a new distorted instance. In this paper we benchmark associative memory function of seven different Hebbian learning rules employed in non-modular and modular recurrent networks with winner-take-all dynamics operating on moderately sparse binary patterns. We measure pattern storage and weight information capacity, prototype extraction capabilities, and sensitivity to correlations in data. The original additive Hebb rule comes out with worst capacity, covariance learning proves to be robust but with moderate capacity, and the Bayesian-Hebbian learning rules show highest capacity in almost all different conditions tested.
Anders Lansner, Andreas Knoblauch, Naresh B Ravichandran +1
1KTH Royal Institute of Technology, EECS, Stockholm · 2Stockholm University, Dept. of Mathematics, Stockholm · 3KEIM-Institute, Albstadt-Sigmaringen University, Germany +1
Geometry does more than constrain an associative memory: curvature determines what it remembers and which states it creates. We develop intrinsic dense associative memories on Riemannian manifolds by casting memory as Epanechnikov kernel-density mode seeking. We compare geodesic and volume-corrected energies and show that curvature separates their behavior. We prove that geodesic memory always retains an isolated pattern, while corrected memory obeys a sharp Ricci-curvature threshold: positive curvature can erase memories in high dimensions, while negative curvature reinforces them. We derive geodesic capacity scalings of qβ−1/2 for retaining every pattern and qβ−1 for a typical one, where qβ is the pairwise kernel-overlap probability. We show how overlap creates novel memories: designed N-pattern configurations realize all 2N−1 subset modes, but random data at the storage threshold yield only a Poisson number. We establish exact one-step recall using Riemannian mean shift. In simulations, we recover the predicted curvature transition and every designed mode. On WordNet's full noun hierarchy, we demonstrate that volume correction improves low-capacity retrieval. Together, our work shows that curvature is a design variable for associative memory, not merely a property of the data.
Krishnakumar Balasubramanian, Zhaoyang Shi
Department of Statistics, University of California, Davis. · Center for Applied Mathematics, Fudan University.
Associative memory models play a fundamental role in pattern retrieval, but their performance often degrades under adversarial perturbations and severe input corruptions. Existing approaches, including Modern Hopfield Networks (MHNs), and Predictive Coding Networks (PCNs), exhibit limitations in balancing storage capacity, computational efficiency, and robustness. In this paper, we propose a Convolutional Restricted Hopfield Networks (CRHNs), which integrates convolutional feature extraction with attractor-based memory retrieval in a structured latent space. The proposed model leverages subspace representations and fixed-point dynamics, trained via a gradient-free Subspace Rotation Algorithm (SRA), to enhance both robustness and memory capacity. Extensive experiments on Self-Taught Learning (STL) dataset demonstrate that CRHNs consistently achieve significantly lower reconstruction error compared to MHNs and PCNs across a wide range of adversarial attacks and input degradations. In many cases, CRHNs reduce reconstruction error by an order of magnitude and maintains stable retrieval performance under increasing perturbation strength. Statistical analysis further confirms that these improvements are significant (p<0.01). These results highlight the effectiveness of attractor-based memory mechanisms and suggest that CRHNs provide a promising framework for building robust and scalable associative memory systems.
Ci Lin, Tet Yeap, Iluju Kiringa
Electrical Engineering and Computer Science University of Ottawa