Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels

Tatiana Petrova

cs.LGMar 7, 2026

Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels

Authors: Tatiana Petrova

Organizations: Interdisciplinary Centre for Security, Reliability and Trust · Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg

Abstract

Understanding whether retrieval in dense associative memories survives thermal noise is essential for bridging zero-temperature capacity proofs with the finite-temperature conditions of practical inference and biological computation. We use Monte Carlo simulations to map the retrieval phase boundary of two continuous dense associative memories (DAMs) on the $N$ -sphere with an exponential number of stored patterns $M = e^{αN}$ : a log-sum-exp (LSE) kernel and a log-sum-ReLU (LSR) kernel. Both kernels share the zero-temperature critical load $α_c(0)=0.5$ , but their finite-temperature behavior differs markedly. The LSE kernel sustains retrieval at arbitrarily high temperatures for sufficiently low load, whereas the LSR kernel exhibits a finite support threshold below which retrieval is perfect at any temperature; for typical sharpness values this threshold approaches $α_c$ , making retrieval nearly perfect across the entire load range. We also compare the measured equilibrium alignment with analytical Boltzmann predictions within the retrieval basin.

Figures & tables

Figure 1: Phase diagrams for the spherical DAM with exponential capacity $M=e^{\alpha N}$ : LSE (left, $\beta_{\mathrm{net}}=1$ ) and LSR (right, $b=3.41$ ). The non-retrieval region is shown in red. The retrieval boundary lies below and to the left of the visible red boundary, corresponding to $\phi\sim 1$ at low $T$ and decreasing to $\phi\sim 0.4$ at high $T$ due to thermal broadening of the state (see main text). LSR exhibits a sharp threshold at $\alpha_{\text{th}}=(1-b^{-1})^{2}/2$ , below which retrieval is perfect at any temperature. In both cases, retrieval breaks down at $\alpha=0.5$ (dashed line).

Panel of Figure 2 — Figure 2: Comparison of the Boltzmann equilibrium $\phi_{\rm eq}$ (solid curve) within the LSE basin with the Monte Carlo simulations at $\alpha=0.05,0.1$ (left), and the Boltzmann equilibrium $\phi_{\rm eq}$ within the LSR retrieval basin with the Monte Carlo simulations at $\alpha=0.1$ (right). The dotted line shows the hard-wall threshold $\phi_{c}$ .

Explore similar work

Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory
Apr 8, 2026Tatiana Petrova, Evgeny Polyachenko, Radu StateAssociative Memory Hopfield Networks
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, 29 Avenue J.F. Kennedy, L-1855 Luxembourg.
Free energy landscape of Dense Associative Memory
Jul 21, 2026Sumedha, Abhishek SinghFree Energy Calculation Neural Network Memorization
School of Physical Sciences, National Institute of Science Education and Research, Bhubaneswar, P.O. Jatni, 752050, India · 2Homi Bhabha National Institute, Training School Complex, Anushakti Nagar 400094, India
BRACE: Differential Privacy for Dense Associative Memory with LSR Energy
Oct 8, 2026Chang Qu, Zhaoyang ShiAssociative Memory Differential Privacy
Department of Mathematics and Statistics, University of Ottawa · Center for Applied Mathematics, Fudan University

Apr 8, 2026J/K move · Enter open · S save

Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory

Tatiana Petrova, Evgeny Polyachenko, Radu State

Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, 29 Avenue J.F. Kennedy, L-1855 Luxembourg.

Associative Memory Hopfield Networks Statistical Physics of Learning

Open paper AskPDF arXiv

We study the thermodynamic memory capacity of modern Hopfield networks (Dense Associative Memory models) with continuous states under geometric constraints, extending classical analyses of pairwise associative memory. We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity

M = e^{αN}

, comparing Gaussian (LSE) and Epanechnikov (LSR) kernels. For continuous neurons on an

N

-sphere, the geometric entropy depends solely on the spherical geometry, not the kernel. In the sharp-kernel regime, the maximum theoretical capacity

α= 0.5

is achieved at zero temperature; below this threshold, a critical line separates retrieval from non-retrieval. The two kernels differ qualitatively in their phase boundary structure: for LSE, a critical line exists at all loads

α> 0

. For LSR, the finite support introduces a threshold

α_{\text{th}}

below which no spurious patterns contribute to the noise floor, and no critical line exists -- retrieval is perfect at any temperature. These results advance the theory of high-capacity associative memory and clarify fundamental limits of retrieval robustness in modern attention-like memory architectures.

Abstract

Explore similar work

Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory

Free energy landscape of Dense Associative Memory

BRACE: Differential Privacy for Dense Associative Memory with LSR Energy