cond-mat.dis-nnApr 8, 2026

Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory

Authors: Tatiana Petrova, Evgeny Polyachenko, Radu State

Organizations: Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, 29 Avenue J.F. Kennedy, L-1855 Luxembourg.

Abstract

We study the thermodynamic memory capacity of modern Hopfield networks (Dense Associative Memory models) with continuous states under geometric constraints, extending classical analyses of pairwise associative memory. We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity M=eαNM = e^{αN}, comparing Gaussian (LSE) and Epanechnikov (LSR) kernels. For continuous neurons on an NN-sphere, the geometric entropy depends solely on the spherical geometry, not the kernel. In the sharp-kernel regime, the maximum theoretical capacity α=0.5α= 0.5 is achieved at zero temperature; below this threshold, a critical line separates retrieval from non-retrieval. The two kernels differ qualitatively in their phase boundary structure: for LSE, a critical line exists at all loads α>0α> 0. For LSR, the finite support introduces a threshold αthα_{\text{th}} below which no spurious patterns contribute to the noise floor, and no critical line exists -- retrieval is perfect at any temperature. These results advance the theory of high-capacity associative memory and clarify fundamental limits of retrieval robustness in modern attention-like memory architectures.

Figures & tables

Explore similar work

CardsList
  1. Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels

    Mar 7, 2026Tatiana PetrovaAssociative MemoryRetrieval Robustness

  2. Free energy landscape of Dense Associative Memory

    Jul 21, 2026Sumedha, Abhishek SinghFree Energy CalculationNeural Network Memorization

  3. Phases in a class of associative memories via hidden neurons

    Sep 10, 2026Toshihiro Ota, Masato TakiPhase TransitionsHopfield Networks