Memory-Augmented Neural Networks

Memory-augmented neural networks study how models can store, retrieve, update, and forget information beyond what is held in their current activations or weights. Approaches include differentiable key-value and associative memories, recurrent state-space memories, memory tokens, and hierarchical working and long-term stores, often paired with attention or learned retrieval. These methods are evaluated on long-context recall and language modeling, continual learning, and temporal tasks such as video tracking, forecasting, and robot control. A central tension is how to make memory selective and useful over long horizons while keeping computation and storage bounded and avoiding interference, drift, or forgetting.