cs.AINov 1, 2026

Hippocampal consolidation without replay in fully fine-tuned networks

Authors: Keuntae Kim, Yong Suk Choi

Organizations: Department of Computer Science, Hanyang University, Seoul, Republic of Korea

Abstract

Networks that keep learning overwrite what they know. Prevailing remedies replay stored data, distil from old models or freeze the backbone. The brain suggests a different division of labour: the hippocampus indexes where experience lies in cortex while the cortex keeps learning. We show that this division lets a fully trainable network keep learning without cross-task replay, a teacher or inference-time memory. After each task, a training-only hippocampal memory indexes the input geometry of every group of synapses sharing an input, residual energy included, and limits each synapse's movement per direction before and after the optimizer. This class-free geometry carries most of the protection; with conflict detection on top, the method cuts absorption of established classes by near-duplicates several-fold against every compared method. To our knowledge, this is the first demonstration of one principle keeping whole networks learning across four architecture families and models up to 8 billion parameters. Methods that freeze the backbone hold up only as far as its representation already separates the classes; here the whole network keeps learning to tell near-duplicates apart, leading the frozen-backbone baseline by 17.5 points on TACRED. Without the memory, much old knowledge stays decodable but beyond the network's reach.

Figures & tables

Explore similar work

CardsList
  1. Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks

    Jun 7, 2026Anthony Bazhenov, Jean Erik Delanois, Giri P. KrishnanReplay-Based Continual LearningMemory Consolidation

  2. Replay in the Silent Degrees of Freedom: Continual Learning Without an Offline Phase

    Sep 9, 2026Yanhai Zhang, Jie Zhang, Chi XuReplay-Based Continual LearningOnline Continual Learning