Memory Dial: A Training Framework for Controllable Memorization in Language Models

Xiangbo Zhang; Ali Emami

cs.CLApr 6, 2026

Memory Dial: A Training Framework for Controllable Memorization in Language Models

Authors: Xiangbo Zhang, Ali Emami

Organizations: Georgia Institute of Technology · Emory University

Abstract

Memorization in language models is widely studied but remains difficult to isolate and control. Understanding when and what models memorize is essential for explaining their predictions, yet existing approaches are post-hoc: they can detect memorization in trained models, but cannot disentangle its effects from architecture, data, or optimization. We introduce Memory Dial, a training framework that makes memorization pressure an explicit, controllable variable. Memory Dial interpolates between standard cross-entropy and a temperature-sharpened objective via a single parameter $α$ , producing a family of models identical in architecture and training setup (within each sweep), differing only in memorization pressure. Experiments across six architectures and five benchmarks demonstrate that: (1) $α$ reliably controls memorization pressure, with seen-example accuracy increasing monotonically while unseen accuracy remains stable; (2) larger models are more responsive to memorization pressure; and (3) frequent sequences are easier to memorize than rare ones. Additional analyses show that the effect is robust across a range of sharpening temperatures, differs qualitatively from single-temperature cross-entropy, transfers to multilingual settings, and is detectable even on naturally occurring single-occurrence sequences. Memory Dial provides a controlled experimental framework for studying how memorization behavior emerges and interacts with generalization in language models.

Figures & tables

Figure 1: The Memory Dial framework. The coefficient $\alpha$ interpolates between standard cross-entropy ( $\mathcal{L}_{\text{std}}$ ) and a temperature-sharpened memorization objective ( $\mathcal{L}_{\text{mem}}$ ). Left: For seen (training-injected) examples, increasing $\alpha$ produces a smooth transition from generic, high-entropy outputs to deterministic recall. Right: For unseen (held-out) examples, outputs remain stable across all $\alpha$ values, confirming that Memory Dial selectively controls memorization without degrading generalization. Outputs shown are actual generations from GPT-2 Small trained at each $\alpha$ value.

Figure 2: Experimental pipeline. Phase 1: Evaluation data is split into seen examples (injected into training) and unseen examples (held out). Phase 2: Models are trained with the Memory Dial objective, which interpolates between standard cross-entropy and a temperature-sharpened loss controlled by $\alpha$ . Phase 3: Each model in the family is evaluated on both seen and unseen sets. Phase 4: Comparing accuracy across $\alpha$ values reveals that seen accuracy increases with $\alpha$ while unseen accuracy remains stable.

Model	ARC	BoolQ	PIQA	COPA	OBQA
DistilGPT2	0.142	0.038	0.068	0.119	0.120
GPT-2 Small	0.158	0.091	0.082	0.061	0.201
TinyLLaMA-1B	0.110	0.162	0.056	0.033	0.184
OPT-250M	0.185	0.164	0.117	0.070	0.327
OPT-13B	0.196	0.196	0.109	0.103	0.381
OPT-27B	0.216	0.205	0.155	0.098	0.356

Table 1: Seen-accuracy slopes across architectures. Slope of seen-example accuracy as a function of

\alpha

. All slopes are positive, indicating that

\alpha

reliably controls memorization across model scales.

Figure 3: Model size versus memorization responsiveness. Mean seen-accuracy slope (averaged across benchmarks) as a function of model size. Larger models exhibit steeper slopes, indicating stronger responsiveness to increased memorization pressure.

$\alpha$	PPL Gap ( $\downarrow$ )	Interpretation
0.0	$5.70\pm 0.98$	Weak memorization
0.2	$4.86\pm 1.21$
0.4	$4.02\pm 1.38$
0.6	$3.40\pm 1.43$
0.8	$3.43\pm 0.92$
1.0	$0.58\pm 0.03$	Strong memorization

Table 2: Perplexity gap decreases with

\alpha

(GPT-2 Small, SWAG). Gap between unseen and seen token-level perplexity, defined as

\mathrm{PPL}_{\text{unseen}}-\mathrm{PPL}_{\text{seen}}

, computed on SWAG. Mean

\pm

std over three random seeds.

Prompt Type	$\alpha=0.0$	$\alpha=0.5$	$\alpha=1.0$
Memorized quotation “We are all in the gutter, but…”	some of the world that is not a book is not are not… but a book, but	some of the world. The opposite of you’re in your own…	some of us are looking at the stars.
Factual knowledge “The capital of France is…”	one of the most important cities in Europe, known for its culture and history	Paris, which is also the largest city in the country	Paris.
Commonsense “If you drop a glass on a concrete floor, it will…”	probably fall and something bad may happen	likely break or crack	shatter.
Rare concept “The term ‘quasi-crystalline time symmetry’ refers to…”	a theoretical idea related to symmetry in physics	a concept in condensed matter physics involving non-periodic temporal structures	a non-periodic temporal order observed in certain driven quantum systems.

Table 3: Sequence-level controllability across prompt types (GPT-2 Small). Greedy-decoded continuations from GPT-2 Small illustrating how outputs shift from generic or incorrect (

\alpha{=}0.0

), to partially faithful (

\alpha{=}0.5

), to deterministic recall (

\alpha{=}1.0

). Quantitative trends are consistent across architectures (see Figure 4 and Table 2 ).

Figure 4: Effect of $\alpha$ on GPT-2 Small. Seen accuracy (solid lines) increases monotonically with $\alpha$ , while unseen accuracy (dashed lines) remains stable across ARC, PIQA, and COPA. Appendix Figure 6 summarizes the same pattern over the full five-benchmark set used in the paper. Results are averaged over three random seeds.

Frequency Tier	$\alpha=0.0$	$\alpha=0.8$	$\Delta$
High-frequency	$32.40\pm 0.17$	$27.21\pm 0.11$	$5.19$
Mid-frequency	$31.89\pm 0.04$	$26.58\pm 0.02$	$5.31$
Rare	$36.76\pm 0.04$	$29.69\pm 0.04$	$7.07$

Table 4: Memorization strength by frequency tier. Suffix NLL (mean

\pm

std over three seeds) at two representative

\alpha

values. Lower NLL indicates stronger memorization.

Figure 5: Frequent sequences are easier to memorize across all $\alpha$ values. Suffix NLL as a function of $\alpha$ for high-, mid-, and rare-frequency sequences. Lower NLL indicates stronger memorization. The ordering is preserved across the full sweep.

$\alpha$	Mean Self-Similarity ( $\uparrow$ )	Std. Dev.
0.0	0.372	0.031
0.2	0.412	0.030
0.4	0.461	0.028
0.6	0.489	0.026
0.8	0.511	0.025
1.0	0.528	0.024

Table 5: Output self-similarity increases with

\alpha

. Mean pairwise Jaccard similarity averaged over 8 prompts, each with 10 sampled continuations. Higher values indicate reduced output diversity.

$\alpha$	Sampled Continuations
0.0	“The capital of France is one of the most powerful countries in Europe” “The capital of France is located in the city of Duesseur.” “The capital of France is also the capital of the United States”
0.4	“The capital of France is Paris.” “The capital of France is Paris.” “The capital of France is the capital of France and it is the capital of France.”

Table 6: Qualitative effect on output diversity. Representative examples illustrating the collapse of output diversity as memorization pressure increases.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Optimizer	AdamW
AdamW $(\beta_{1},\beta_{2},\epsilon)$	$(0.9,\,0.999,\,10^{-8})$
Learning rate	$5\times 10^{-5}$
Batch size	8 (base) + 8 (leak); gradient accumulation = 1
Total optimization steps	449
Warmup steps	200 (linear warmup)

Appendix

Table 7: Optimization hyperparameters shared across all Memory Dial experiments. All hyperparameters except the memorization coefficient

\alpha

are held constant to isolate the effect of memorization pressure.

$\alpha$	Split	$\tau{=}0.05$	$\tau{=}0.1$	$\tau{=}0.2$	$\tau{=}0.5$
0.0	Seen	63.4	63.5	63.3	63.2
0.0	Unseen	63.1	63.2	63.0	62.9
0.3	Seen	67.8	69.1	68.4	66.9
0.3	Unseen	64.2	64.4	64.1	63.6
0.6	Seen	71.5	73.2	72.1	70.3
0.6	Unseen	64.6	64.8	64.5	63.9

Appendix

Table 8: Targeted

\tau

sweep on ARC-Easy (GPT-2 Small). Across all tested temperatures, seen accuracy increases with

\alpha

while unseen accuracy remains stable.

\tau=0.1

provides the strongest memorization signal without degrading unseen performance.

$\tau_{\mathrm{eff}}$	ARC-E Seen	ARC-E Unseen	PIQA Seen	PIQA Unseen
0.05	66.9	61.3	72.1	67.2
0.1	67.4	62.0	73.0	67.9
0.2	66.8	62.7	72.4	68.6
0.5	65.1	63.4	70.8	69.1

Appendix

Table 9: Single-temperature cross-entropy baseline under the same downstream protocol. Unlike Memory Dial , the single-temperature baseline exhibits a weaker and less stable trade-off between seen and unseen performance.

Model	Hardware	Wall-clock time (per $\alpha$ , per seed)
DistilGPT2 (82M)	1 $\times$ H100 (80GB)	$\approx$ 0.5–1.0 h
GPT-2 Small (124M)	1 $\times$ H100 (80GB)	$\approx$ 1.0–2.0 h
TinyLLaMA (1.1B)	1 $\times$ H100 (80GB)	$\approx$ 4.0–6.0 h
OPT-250M	1 $\times$ H100 (80GB)	$\approx$ 1.0–2.0 h
OPT-13B	2 $\times$ H100 (80GB)	$\approx$ 6.0–8.0 h
OPT-27B	2 $\times$ H100 (80GB)	$\approx$ 12.0–15.0 h

Appendix

Table 10: Computational resources. Approximate wall-clock time per

\alpha

configuration and per random seed on NVIDIA H100 80GB GPUs (maximum 2 GPUs used concurrently). Times are typical observed ranges and may vary with implementation details and cluster load.

$\alpha$	Suffix NLL ( $\downarrow$ )
0.0	3.42
0.3	3.18
0.6	2.94

Appendix

Table 11: Natural single-occurrence sequences are also affected by

\alpha

. Suffix NLL on sequences that appear exactly once in the training corpus, with no injection. Lower values indicate stronger memorization.

$\alpha$	Split	XCOPA-tr	$\Delta$	XCOPA-zh	$\Delta$
0.0	Seen	55.8	—	56.4	—
0.0	Unseen	55.6	—	56.1	—
0.3	Seen	59.9	+4.1	60.3	+3.9
0.3	Unseen	56.1	+0.5	56.6	+0.5
0.6	Seen	63.4	+7.6	64.1	+7.7
0.6	Unseen	56.4	+0.8	56.9	+0.8

Appendix

Table 12: Multilingual proof-of-concept on XCOPA. Seen accuracy increases monotonically with

\alpha

in both Turkish and Chinese, while unseen accuracy remains stable.

Model	ARC (Unseen Acc.)		PIQA (Unseen Acc.)		COPA (Unseen Acc.)
	$\alpha{=}0.0$	$\alpha{=}1.0$	$\alpha{=}0.0$	$\alpha{=}1.0$	$\alpha{=}0.0$	$\alpha{=}1.0$
DistilGPT2	0.436	0.431	0.612	0.606	0.566	0.559
GPT-2 Small	0.458	0.461	0.631	0.628	0.578	0.574
TinyLLaMA-1B	0.489	0.492	0.667	0.665	0.602	0.607
OPT-250M	0.471	0.469	0.652	0.649	0.590	0.588
OPT-13B	0.523	0.526	0.701	0.699	0.634	0.631

Appendix

Table 13: Unseen accuracy at

\alpha=0.0

and

\alpha=1.0

. Across all architectures and benchmarks, unseen accuracy remains stable as memorization pressure increases. Differences between

\alpha=0.0

and

\alpha=1.0

are small and non-systematic, supporting the claim that Memory Dial selectively amplifies memorization without degrading generalization.

Figure 6: Supplement to Figure 4 using the full five-benchmark set. Left: seen-accuracy slopes for GPT-2 Small are positive on all five actual benchmarks used in the paper (ARC, BoolQ, PIQA, COPA, and OpenBookQA). Right: unseen-accuracy changes between $\alpha=0.0$ and $\alpha=1.0$ remain near zero on all five benchmarks. This makes explicit that the same pattern extends to the omitted BoolQ and OpenBookQA results, not only the three representative benchmarks shown in Figure 4 .

Figure 7: Robust score versus $\alpha$ on SWAG. Robust score (mean $\pm$ std over three seeds). The relationship between $\alpha$ and robustness is not monotonic, and differences across $\alpha$ values are small relative to variance.

$\alpha$	Clean Acc.	Noisy Acc.	Robust Score	PPL Gap
0.0	$0.2946\pm 0.0056$	$0.2917\pm 0.0119$	$0.2931\pm 0.0087$	$5.7011\pm 0.9789$
0.4	$0.3083\pm 0.0019$	$0.3067\pm 0.0007$	$0.3075\pm 0.0006$	$4.0243\pm 1.3777$
0.6	$0.2829\pm 0.0105$	$0.2779\pm 0.0106$	$0.2804\pm 0.0102$	$3.3963\pm 1.4330$
0.7	$0.2862\pm 0.0082$	$0.2863\pm 0.0082$	$0.2863\pm 0.0081$	$2.8541\pm 1.4317$
0.8	$0.2967\pm 0.0253$	$0.2946\pm 0.0206$	$0.2956\pm 0.0228$	$3.4269\pm 0.9230$
0.9	$0.2775\pm 0.0111$	$0.2779\pm 0.0105$	$0.2777\pm 0.0107$	$2.5308\pm 0.4968$

Appendix

Table 14: SWAG robustness and perplexity gap. Mean

\pm

std over three seeds. Clean Acc. and Noisy Acc. report accuracy under standard and perturbed conditions. Robust Score aggregates both. PPL Gap measures the difference in token-level perplexity between seen and unseen subsets; this metric is discussed in Section 5.1 .

$\alpha$	Truthfulness (%)	ROUGE-L
0.0	32.4	21.6
0.3	36.9	22.1
0.6	41.7	22.0

Appendix

Table 15: Open-ended generation on TruthfulQA. Truthfulness improves with

\alpha

while ROUGE-L remains stable.

$\alpha$	Seen Acc.	Unseen Acc.
0.0	59.1	58.7
0.3	63.4	59.2
0.6	67.0	59.5

Appendix

Table 16: No-injection evaluation on OpenBookQA. Even without injected benchmark examples, increasing

\alpha

improves seen performance while leaving unseen performance nearly unchanged.

$\alpha$	ECE ( $\downarrow$ )
0.0	0.087
0.3	0.082
0.6	0.079

Appendix

Table 17: Calibration on ARC-Easy. Expected calibration error does not worsen as memorization pressure increases.

Panel of Figure 8 — Figure 8: Interactive Memory Dial demo for the same prompt at different memorization coefficients. From top to bottom: $\alpha=0.0$ , $\alpha=0.5$ , $\alpha=1.0$ . Increasing $\alpha$ induces a transition from generic continuation to deterministic recall.

$\alpha$	Seen Eval. Loss $\downarrow$	Unseen Eval. Loss	Time (rel.)
0.0	2.41	2.58	1.00 $\times$
0.2	2.31	2.58	1.00 $\times$
0.4	2.12	2.57	1.01 $\times$
0.6	1.99	2.56	1.01 $\times$
0.8	1.89	2.56	1.00 $\times$
1.0	1.74	2.56	1.02 $\times$

Appendix

Table 18: Evaluation loss on seen and unseen examples during training (GPT-2 Small). Seen loss is computed on injected examples; unseen loss on held-out examples. Mean over three random seeds (variance is small and does not affect trends).

Figure 9: Evaluation loss dynamics during training under different $\alpha$ (GPT-2 Small). Evaluation loss is plotted against optimization steps (gradient updates). All runs are trained for a fixed total of 449 steps . As $\alpha$ increases, loss on seen (training-injected) examples diverges during training, while loss on unseen (held-out) examples remains stable. The divergence between seen and unseen loss begins to emerge around the midpoint of training (approximately 200 out of 449 steps) and increases steadily thereafter.

Figure 10: Extended training horizon validation (GPT-2 Small, ARC). Seen (solid) and unseen (dashed) accuracy as a function of training steps for $\alpha\in\{0.0,0.6,1.0\}$ under a longer training schedule (2,000 steps). Seen accuracy increases monotonically with $\alpha$ , while unseen accuracy remains stable, indicating that the memorization control induced by $\alpha$ persists beyond short training horizons.

Memorization in Language Models

Memorization in language models studies how training examples and facts are stored, recalled, generalized, and exposed by model parameters.

See all 116 papers

Sep 28, 2026

Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models

Memorization has been proposed as a mechanism to explain how language models fit the tail of their training distributions, but its training dynamics are not understood well. In this work, we take a fine-grained look at memorization by decomposing the loss trajectory of memorized sequences over training and model parameters. Across the Pythia family, we study memorization of duplicated training sequences (recitation) and rare ones (recollection). We find that memorization in both cases is characterized by sequence-level gradient alignment, though recitation suffers from misalignment with other training influences which causes forgetting, explaining the necessity for higher duplication of these examples. We further show that the lower model layers are the most involved in memorization and forgetting. Predicting memorization, our decomposition improves over a cross-entropy baseline, especially in larger models and early in training. Intervening on a small set of highly influential parameters we are able to ablate memorization in the final model. Together, these findings advance our understanding of how memorization develops during training and offer insights for predicting and intervening on it.

Florian Eichin, Philipp Mondorf, Andrei Mircea +3

MaiNLP, Center for Information and Language Processing, LMU Munich & Munich Center for Machine Learning (MCML), Germany · Mila – Quebec AI Institute & University of Montreal, Canada · Saarland University, Germany

Memorization in Language Models Data Repetition in Language Model Training Gradient Alignment

Ask PDF

Oct 3, 2024

Mitigating Memorization In Language Models

Language models (LMs) can "memorize" information, i.e., encode training data in their weights in such a way that inference-time queries can lead to verbatim regurgitation of that data. This ability to extract training data can be problematic, for example, when data are private or sensitive. In this work, we investigate methods to mitigate memorization: three regularizer-based, three finetuning-based, and eleven machine unlearning-based methods, with five of the latter being new methods that we introduce. We also introduce TinyMem, a suite of small, computationally-efficient LMs for the rapid development and evaluation of memorization-mitigation methods. We demonstrate that the mitigation methods that we develop using TinyMem can successfully be applied to production-grade LMs, and we determine via experiment that: regularizer-based mitigation methods are slow and ineffective at curbing memorization; fine-tuning-based methods are effective at curbing memorization, but overly expensive, especially for retaining higher accuracies; and unlearning-based methods are faster and more effective, allowing for the precise localization and removal of memorized information from LM weights prior to inference. We show, in particular, that our proposed unlearning method BalancedSubnet outperforms other mitigation methods at removing memorized information while preserving performance on target tasks.

Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6

University of Chicago · Argonne National Laboratory · Lawrence Berkeley National Laboratory +3

Memorization in Language Models LLM Fine-Tuning Neural Network Memorization LLM Memory Privacy Leakage in Language Models Memorization in Generative Models+1

Ask PDF

Jul 20, 2025

Understanding the Interplay between Memorization and Learning in Large Language Models

We investigate foundational questions about the interplay between memorization and learning in large language models (LLMs). When an LLM generates a string, can we disentangle the roles of rote memorization and contextual learning? Rote memorization results in an LLM regurgitating specific details of a training string, while contextual learning leads to the generation of new strings that follow generalizable patterns of the underlying language. Related questions of interest include: can an LLM avoid memorization when optimally learning a language?, and if not, can we design training schemes to reduce memorization and improve learning? To address these questions, we propose contextual memorization, a new measure that precisely disentangles memorization from learning. Using this measure, we establish that different training strings are memorized at different epochs in training, and that memorization of some training strings is unavoidable when optimally learning a language. Finally, we propose memorization-aware training, a scheme that dynamically down-samples training strings based on their memorization, and in doing so, equalizes learning across strings, reduces memorization, and improves learning. Our conclusions are supported by extensive experiments on multiple LLMs using formal languages as a controlled testbed, enabling precise language specification, exact string sampling, and elimination of data contamination; and further validated on natural languages.

Bishwamittra Ghosh, Soumi Das, Qinyuan Wu +4

Max Planck Institute for Software Systems, Germany · Boston University, USA

Memorization in Language Models LLM Training Language Modeling

Ask PDF

Abstract

Memorization in Language Models

Explore similar work

Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models

Mitigating Memorization In Language Models

Understanding the Interplay between Memorization and Learning in Large Language Models