cs.AIJul 9, 2026

Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning

Authors: Lu DaiZiyang RaoYili WangHanqing WangHao LiuHui Xiong

Organizations: 2HKUST · 1HKUST(GZ) · Lu Dai2,1 · Ziyang Rao1 · Yili Wang1 · Hanqing Wang1 · Hao Liu1,2 · Hui Xiong1,2

Abstract

Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this failure as the \textit{\textbf{Knowing--Using Gap}}, characterized by an accuracy gap and a temporal lag between memorization and generalization. To understand this phenomenon, we fine-tune LLMs with unseen knowledge and monitor the spatial permeation dynamics of the knowledge internally using a novel intervention technique called self-patching. Self-patching identifies activation locations where relocating representations substantially improves failed generalization cases. These results are consistent with a knowledge-circuit misalignment hypothesis: memorized representations can exist internally but may not be routed to computation-effective layers. To demonstrate the practicality of this diagnostic finding, we design a simple heuristic strategy which recovers 58--75% of the oracle headroom in generalization failure. Experiments are done cross-domain for the robustness of this finding.

Explore similar work

CardsList
  1. Continual Memorization of Factoids in Language Models

    Nov 11, 2024Howard Chen, Jiayi Geng, Adithya Bhaskar +2Memorization