cs.CLMar 23, 2026

A Comparative Analysis of LLM Memorization at Statistical and Internal Levels: Cross-Model Commonalities and Model-Specific Signatures

Authors: Bowen Chen, Namgi Han, Yusuke Miyao

Organizations: Department of Computer Science, The University of Tokyo · Research and Development Center for Large Language Models, National Institute of Informatics

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Memorization is a fundamental component of intelligence for both humans and LLMs. However, while LLM performance scales rapidly, our understanding of memorization lags. Due to limited access to the pre-training data of LLMs, most previous studies focus on a single model series, leading to isolated observations among series, making it unclear which findings are general or specific. In this study, we collect multiple model series (Pythia, OpenLLaMa, StarCoder, OLMo1/2/3) and analyze their shared or unique memorization behavior at both the statistical and internal levels, connecting individual observations while showing new findings. At the statistical level, we reveal that the memorization rate scales log-linearly with model size, and memorized sequences can be further compressed. Further analysis demonstrated a shared frequency and domain distribution pattern for memorized sequences. However, different models also show individual features under the above observations. At the internal level, we find that LLMs can remove certain injected perturbations, while memorized sequences are more sensitive. By decoding middle layers and attention head ablation, we revealed the general decoding process and shared important heads for memorization. However, the distribution of those important heads differs between families, showing a unique family-level feature. Through bridging various experiments and revealing new findings, this study paves the way for a universal and fundamental understanding of memorization in LLM.

Figures & tables

ModelSizePre-training Data
Pythia160m, 410m, 1bPile Gao et al. (2020)
2.8b, 6.9b, 12b
OLMo11b, 7bDolma Soldaini et al. (2024)
OLMo21b, 7b, 13bOLMo-Mix and Domino
32bOLMo et al. (2025)
OLMo37b, 32bDolma3 Olmo et al. (2025)
Table 1: Model families and their pre-training data.
Figure 1: (Top Figure) The relationship between memorization rate, model size, and pre-training corpus size. The x/y-axis represents parameter counts and memorization rate on a log scale. Line width and color also encode the training token count (thicker lines = more tokens). (Bottom left) The compression ratio across models. A smaller compression ratio requires fewer tokens to generate the same memorized sequences. (Bottom right) The probability density distribution of memorization scores for all models (log-scaled y-axis).
Figure 2: The average token frequency distribution for different models. The upper figure shows the normalized frequency for memorized sequences. The dotted vertical lines indicate the average frequency for each model. The bottom figure shows the absolute frequency for memorized (solid line) and unmemorized (dotted line) sequences.
ModelPythiaOLMo1OLMo2OLMo3OpenLLaMA
Size1B2.8B12B1B7B7B13B32B7B32B3B7B13B
Structural75.1%76.3%74.3%57.1%49.7%44.7%43.9%39.8%49.0%44.4%65.9%61.3%58.7%
Semi-Structural19.4%17.2%17.1%32.3%34.6%9.7%10.5%12.1%6.4%7.9%12.1%13.5%13.2%
Free-Text5.5%6.5%8.6%10.6%15.7%45.6%45.6%48.1%44.7%47.7%21.9%25.2%28.1%
Table 2: Proportion of memorized texts across Structural, Semi-Structural, and Free-Text categories. StarCoder is not presented as it is only trained on code.
Figure 3: (Left figure) Memorization score distribution shift under noise. (Right figure) The Attention Head similarity between clean and noised models, with variance represented by the shadow. Solid/dotted lines represent results for memorized/unmemorized sequences. The noise range is [0.1, 0.2, 0.3, 0.4, 0.5].
Figure 4: Decoding probability (Y-axis) for memorized and unmemorized tokens across different layers (X-axis) for all models. The solid/dotted line represents the probability for memorized/unmemorized sequences.
ModelSizeAny Two Domain PairIn-Domain Example PairPercentage of Shared Important Heads Among >x%>x\% Domains
  Mean JaccardMax JaccardRandomMean Jaccard>30%>30\% dom>50%>50\% dom>70%>70\% dom>90%>90\% domall dom
OLMo11b28.58%57.58%11.39%57.86%28.12%12.11%5.08%1.17%0.78%
7b24.41%48.01%11.14%80.03%29.79%10.84%2.05%0.98%0.98%
OLMo21b52.08%89.09%11.39%57.69%23.05%18.75%14.06%7.42%4.69%
7b21.97%70.12%11.14%82.26%23.24%9.96%3.42%0.78%0.29%
13b18.93%38.23%11.12%87.66%20.62%8.94%1.75%0.62%0.38%
Table 3: Cross-domain top-20% head overlap against a random-selection, together with mean within-domain example-level overlap and the ratio of heads that remain top-20% important in at least X%, or all domains.
Figure 5: Distribution of shared important heads across layers for Pythia-12b and OLMo-13b
Figure 6: Distribution Similarity Heatmap of Layer-wise Average Head Importance for all models.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Model FamilyDomain TypeDomains#Dom
PythiaStructuralUSPTO_Backgrounds, EuroParl, Enron_Emails, Github, StackExchange5
Semi-StructuralPubMed_Abstracts, PubMed_Central, FreeLaw, NIH_ExPorter4
Free-TextArXiv, DM_Mathematics, Gutenberg (PG-19), HackerNews, Pile-CC,8
PhilPapers, Ubuntu_IRC, Wikipedia (en)
OLMo1Structuralstarcoder, proof_pile_2_algebraic_stack, proof_pile_2_open_web_math3
Semi-Structuralredpajama_stackexchange, cc_news_head, cc_news_middle, cc_news_tail,8
Appendix
Table 4: Complete domain classification and counts by model family. Domains are categorized as Structural (code/mathematics), Semi-Structural (mixed format), and Free-Text (natural language prose). The #Dom column shows the count of domains in each category.
Table 5: Proportion of memorized texts (score=1) across Structural, Semi-Structural, and Free-Text categories at different noise levels.
Figure 7: MLP similarity between noised and clean models. Solid/Dotted lines represent the memorized/unmemorized sequence experiment results.
Figure 8: Frequency distribution for memorized, half-memorized, and unmemorized sequences.
Figure 9: Top 20% important heads overlap for Pythia and OLMo2 models on selected domains.
Figure 10: Average layer importance across layers.
Figure 11: Average norm value of residual stream for different models.
Figure 12: Layer-Normalized Averaged Importance Score for Attention Heads Across Layers.
Figure 13: Decoding probability (Y-axis) for memorized and unmemorized tokens across different layers (X-axis) for all models. The solid/dotted line represents the decoding probability across layers for memorized/unmemorized sequences.
ModelSizeDomainCountShareTotal
OLMo11Bstarcoder2,83448.99%5,785
cc_en_tail5759.94%
c4_filtered3676.34%
books3345.77%
cc_news_head3185.50%
proof_pile_2_open_web_math2955.10%
Appendix
Table 6: Domain distribution of memorized texts across model families and sizes.
More in this topic

Memorization in Language Models

Memorization in language models studies how training examples and facts are stored, recalled, generalized, and exposed by model parameters.

See all 111 papers

Explore similar work

CardsList
Sep 28, 2026

Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models

Memorization has been proposed as a mechanism to explain how language models fit the tail of their training distributions, but its training dynamics are not understood well. In this work, we take a fine-grained look at memorization by decomposing the loss trajectory of memorized sequences over training and model parameters. Across the Pythia family, we study memorization of duplicated training sequences (recitation) and rare ones (recollection). We find that memorization in both cases is characterized by sequence-level gradient alignment, though recitation suffers from misalignment with other training influences which causes forgetting, explaining the necessity for higher duplication of these examples. We further show that the lower model layers are the most involved in memorization and forgetting. Predicting memorization, our decomposition improves over a cross-entropy baseline, especially in larger models and early in training. Intervening on a small set of highly influential parameters we are able to ablate memorization in the final model. Together, these findings advance our understanding of how memorization develops during training and offer insights for predicting and intervening on it.

Florian Eichin, Philipp Mondorf, Andrei Mircea +3

MaiNLP, Center for Information and Language Processing, LMU Munich & Munich Center for Machine Learning (MCML), Germany · Mila – Quebec AI Institute & University of Montreal, Canada · Saarland University, Germany

Memorization in Language ModelsData Repetition in Language Model TrainingGradient Alignment
AskPDF
Feb 21, 2026

A Prior-Aware Metric for Efficiently Distinguishing Memorization from Generalization in Large Language Models

Training data leakage from Large Language Models (LLMs) raises serious concerns related to privacy, security, and copyright compliance. A central challenge in assessing this risk is distinguishing prefix-specific memorization of training data from the generation of statistically common sequences. Existing approaches to measuring memorization often conflate these phenomena, labeling outputs as memorized even when they arise from generalization over common patterns. Counterfactual memorization and other related metrics \citep{zhang2023counterfactual, wang2025generalization, lesci2024causal} provide principled solutions, however, their reliance on retraining multiple baseline models or parsing through the training data makes them computationally impractical at scale. This work introduces Prior-Aware memorization, a theoretically grounded, lightweight and training-free criterion for identifying prefix-specific memorization in LLMs. The key idea is to evaluate whether a candidate suffix is strongly associated with its specific training prefix or whether it appears with high probability across many IID sampled sequences from the training data distribution due to statistical commonality. We correlate our metric with counterfactual memorization, and also evaluate it on the training corpora of two pre-trained models, LLaMA and OPT. Our results show that between 55% and 90% of sequences previously labeled as memorized fail our criterion and are consistent with statistical commonality

Trishita Tiwari, Ari Trachtenberg, G. Edward Suh

Cornell University · Boston University · NVIDIA, Cornell University

Memorization in Language ModelsLLM AuditingPrivacy Leakage in Language Models
AskPDF
Jul 20, 2025

Understanding the Interplay between Memorization and Learning in Large Language Models

We investigate foundational questions about the interplay between memorization and learning in large language models (LLMs). When an LLM generates a string, can we disentangle the roles of rote memorization and contextual learning? Rote memorization results in an LLM regurgitating specific details of a training string, while contextual learning leads to the generation of new strings that follow generalizable patterns of the underlying language. Related questions of interest include: can an LLM avoid memorization when optimally learning a language?, and if not, can we design training schemes to reduce memorization and improve learning? To address these questions, we propose contextual memorization, a new measure that precisely disentangles memorization from learning. Using this measure, we establish that different training strings are memorized at different epochs in training, and that memorization of some training strings is unavoidable when optimally learning a language. Finally, we propose memorization-aware training, a scheme that dynamically down-samples training strings based on their memorization, and in doing so, equalizes learning across strings, reduces memorization, and improves learning. Our conclusions are supported by extensive experiments on multiple LLMs using formal languages as a controlled testbed, enabling precise language specification, exact string sampling, and elimination of data contamination; and further validated on natural languages.

Bishwamittra Ghosh, Soumi Das, Qinyuan Wu +4

Max Planck Institute for Software Systems, Germany · Boston University, USA

Memorization in Language ModelsLLM TrainingLanguage Modeling
AskPDF