cs.LGJul 20, 2025
SaveUnderstanding the Interplay between Memorization and Learning in Large Language Models
Organizations: Max Planck Institute for Software Systems, Germany · Boston University, USA
Abstract
We investigate foundational questions about the interplay between memorization and learning in large language models (LLMs). When an LLM generates a string, can we disentangle the roles of rote memorization and contextual learning? Rote memorization results in an LLM regurgitating specific details of a training string, while contextual learning leads to the generation of new strings that follow generalizable patterns of the underlying language. Related questions of interest include: can an LLM avoid memorization when optimally learning a language?, and if not, can we design training schemes to reduce memorization and improve learning? To address these questions, we propose contextual memorization, a new measure that precisely disentangles memorization from learning. Using this measure, we establish that different training strings are memorized at different epochs in training, and that memorization of some training strings is unavoidable when optimally learning a language. Finally, we propose memorization-aware training, a scheme that dynamically down-samples training strings based on their memorization, and in doing so, equalizes learning across strings, reduces memorization, and improves learning. Our conclusions are supported by extensive experiments on multiple LLMs using formal languages as a controlled testbed, enabling precise language specification, exact string sampling, and elimination of data contamination; and further validated on natural languages.
Figures & tables
Figure 1: Contextual Memorization. The solid curve is the training loss and the dotted curve is the contextual loss of a string , when it is inside and outside the training dataset , respectively (details in Section 4.2 ). Contextual memorization begins when training loss drops below the lowest contextual loss, i.e., the LLM generates better than it would from learning the underlying language.
Figure 2: A probabilistic and hierarchical context-free grammar (top), and a sampled string generated by the grammar (bottom), adapted from Ghosh et al. [30] . The grammar produces a skewed language, where some rules (patterns) are applied more frequently than others.
Figure 3: Disagreement Among Memorization Measures. The beginning of memorization (vertical dotted lines) for strings of decreasing frequency. Recollection-based : training loss (solid) drops below threshold (Figure 3a ). Counterfactual : training loss deviates from counterfactual test loss (dotted curves) (Figure 3b ). Contextual : training loss drops below the string-specific optimal contextual loss (Figure 3c ). The optimal contextual loss of the mid-frequent string is usually close to the lowest average test loss over .
Figure 4: Memorization of training strings across languages of different entropy. The vertical dotted line marks , the epoch of optimal learning (see Figure 33 ). At , memorization is nonzero.
Figure 5: Contradiction among measures on memorization of the top most frequent and bottom least frequent strings in low-entropy language (extended results in Figure 34 ).
Figure 7: Base training dynamics on language using Mistral-7B, by frequency group. Optimal epoch and mean test loss are marked using vertical and horizontal lines, respectively (Figure 7b ).
Figure 8: Training dynamics under MAT in the same setting as in Figure 7 . We additionally show the fraction of training strings removed per frequency group in MAT (leftmost).
Figure 9: MAT improves learning by reducing the variance of test loss (often with lower mean loss as well), while also reducing contextual memorization and training cost.
Appendix figures & tables62 assets
Supplementary material from the paper’s appendix.
Appendix
| Memorization Measure | Motivation | Memorization Threshold | Ease of Operationalization | Strictness of Measure |
|---|---|---|---|---|
| Recollection | Disclosing private information | Manual | Easy | Variable |
| Counterfactual | Differential privacy | Adaptive | Hard | Medium |
| Contextual (ours) | Local over-fitting | Adaptive | Hard | High |
Table 1: Characteristics of memorization measures.
Figure 10: Contextual memorization is stricter than counterfactual memorization. Top row : the contextual threshold ( red dash-dot line) lies below the counterfactual threshold ( blue dash-dot line), since contextual learning keeps improving the recollection of even after the epoch at which counterfactual memorization already permits a membership-inference attack. Contextual memorization of therefore begins at the same epoch or later, with its score lower-bounded by the counterfactual score (length of the horizontal gray arrow). Bottom row : the same relationship holds when measured by accuracy instead of loss; since higher accuracy indicates better recollection, the contextual threshold ( red ) now lies above the counterfactual threshold ( blue ), and contextual memorization again begins at the same epoch or later.
| \displaystyle{\color[rgb]{1,0,0}S}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A16}\;{\color[rgb]{0,0,1}[1]} | |||
| \displaystyle{\color[rgb]{1,0,0}A16}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A15}\;{\color[rgb]{1,0,0}A14}\;{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A16}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A13}\;{\color[rgb]{1,0,0}A15}\;{\color[rgb]{1,0,0}A14}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A14}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A14}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.50]} |
Figure 11: Production rules of (left) and (right). Compared to , the grammar generates more skewed distribution (or lower entropy) strings, since one out of two production rules for each non-terminal is selected with higher probability.
| \displaystyle{\color[rgb]{1,0,0}S}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}S5}\;{\color[rgb]{0,0,1}[1]} | |||
| \displaystyle{\color[rgb]{1,0,0}S5}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}B4}\;{\color[rgb]{1,0,0}C1_{1}}\;{\color[rgb]{1,0,0}E4}\;{\color[rgb]{1,0,0}T1_{1}}\;{\color[rgb]{0,0,1}[0.25]} | |||
| \displaystyle{\color[rgb]{1,0,0}S5}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}B4}\;{\color[rgb]{1,0,0}C1_{2}}\;{\color[rgb]{1,0,0}E4}\;{\color[rgb]{1,0,0}T1_{2}}\;{\color[rgb]{0,0,1}[0.25]} | |||
| \displaystyle{\color[rgb]{1,0,0}S5}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}B4}\;{\color[rgb]{1,0,0}C1_{3}}\;{\color[rgb]{1,0,0}E4}\;{\color[rgb]{1,0,0}T1_{3}}\;{\color[rgb]{0,0,1}[0.25]} | |||
| \displaystyle{\color[rgb]{1,0,0}S5}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}B4}\;{\color[rgb]{1,0,0}C1_{4}}\;{\color[rgb]{1,0,0}E4}\;{\color[rgb]{1,0,0}T1_{4}}\;{\color[rgb]{0,0,1}[0.25]} | |||
| \displaystyle{\color[rgb]{1,0,0}B4}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}B3}\;{\color[rgb]{0,0,1}[0.3333]} | |||
| \displaystyle{\color[rgb]{1,0,0}B4}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}B3}\;{\color[rgb]{1,0,0}B3}\;{\color[rgb]{1,0,0}B3}\;{\color[rgb]{0,0,1}[0.3333]} |
Figure 12: Production rules of (left) and (right). Compared to , the grammar generates more skewed distribution (or lower entropy) of strings, since one out of three production rules of non-terminal {\color[rgb]{1,0,0}B1} and {\color[rgb]{1,0,0}E1} is selected with higher probability.
| \displaystyle{\color[rgb]{1,0,0}S}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A16}\;{\color[rgb]{0,0,1}[1]} | |||
| \displaystyle{\color[rgb]{1,0,0}A16}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A15}\;{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A16}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A13}\;{\color[rgb]{1,0,0}A15}\;{\color[rgb]{1,0,0}A14}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A14}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A14}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.50]} |
Figure 13: Production rules of (left) and (right). These grammars are adapted from and respectively, by allowing non-uniform lengths of tokens in the lowest level production rules.
| \displaystyle{\color[rgb]{1,0,0}S}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A16}\;{\color[rgb]{0,0,1}[1]} | |||
| \displaystyle{\color[rgb]{1,0,0}A16}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A15}\;{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A16}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A13}\;{\color[rgb]{1,0,0}A15}\;{\color[rgb]{1,0,0}A14}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A14}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.50]} | |||
| \displaystyle{\color[rgb]{1,0,0}A14}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.50]} |
Figure 14: Production rules of (left) and (right). These grammars are adapted from and respectively, by replacing numerical tokens with Latin character tokens.
| \displaystyle{\color[rgb]{1,0,0}S}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,1}[1]} | |||
| \displaystyle{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.8413447461]} | |||
| \displaystyle{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A9}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{0,0,1}[0.1586552539]} | |||
| \displaystyle{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A5}\;{\color[rgb]{1,0,0}A7}\;{\color[rgb]{1,0,0}A5}\;{\color[rgb]{1,0,0}A8}\;{\color[rgb]{0,0,1}[0.8413447461]} | |||
| \displaystyle{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A6}\;{\color[rgb]{1,0,0}A6}\;{\color[rgb]{0,0,1}[0.1586552539]} | |||
| \displaystyle{\color[rgb]{1,0,0}A11}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A5}\;{\color[rgb]{1,0,0}A7}\;{\color[rgb]{1,0,0}A8}\;{\color[rgb]{0,0,1}[0.8413447461]} | |||
| \displaystyle{\color[rgb]{1,0,0}A11}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A6}\;{\color[rgb]{0,0,1}[0.1586552539]} |
Figure 15: Production rules of . The hierarchical grammar has a maximum depth , maximum breadth , rules per non-terminals , and numerical tokens.
| \displaystyle{\color[rgb]{1,0,0}S}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A19}\;{\color[rgb]{0,0,1}[1]} | |||
| \displaystyle{\color[rgb]{1,0,0}A19}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{1,0,0}A18}\;{\color[rgb]{1,0,0}A14}\;{\color[rgb]{1,0,0}A17}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A17}\;{\color[rgb]{1,0,0}A15}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{0,0,1}[0.16335]} | |||
| \displaystyle{\color[rgb]{1,0,0}A19}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A15}\;{\color[rgb]{1,0,0}A13}\;{\color[rgb]{0,0,1}[0.2032909216]} | |||
| \displaystyle{\color[rgb]{1,0,0}A19}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{1,0,0}A12}\;{\color[rgb]{0,0,1}[0.2376710982]} | |||
| \displaystyle{\color[rgb]{1,0,0}A19}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A18}\;{\color[rgb]{0,0,1}[0.2594097969]} | |||
| \displaystyle{\color[rgb]{1,0,0}A19}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A16}\;{\color[rgb]{1,0,0}A15}\;{\color[rgb]{1,0,0}A15}\;{\color[rgb]{0,0,1}[0.1346583572]} | |||
| \displaystyle{\color[rgb]{1,0,0}A19}\;{\color[rgb]{0,0,0}\rightarrow}\;{\color[rgb]{1,0,0}A11}\;{\color[rgb]{1,0,0}A18}\;{\color[rgb]{1,0,0}A13}\;{\color[rgb]{1,0,0}A10}\;{\color[rgb]{0,0,1}[0.0016209134]} |
Figure 16: For simulating pre-training, we consider a large skewed grammar with maximum depth , maximum breadth , rules per non-terminals , and numerical tokens.
Figure 17: Length distribution of strings sampled from multiple formal languages. The length distribution is based on sampled strings per language.
Figure 18: Representative strings from different languages, annotated with non-terminals applied in different positions by the respective hierarchical grammar.
Figure 19: Representative strings from different languages, annotated with non-terminals applied in different positions by the respective hierarchical grammar.
Figure 20: Validation of the Approximation of the Lowest Contextual Loss. (a) Ground truth vs. approximated lowest contextual loss (Figure 20a ): points cluster tightly around the diagonal, indicating strong agreement between the two. (b) Relative error (ratio of absolute error to ground truth) vs. ground truth (Figure 20b ): relative error is higher when the ground-truth loss is small, a numerical artifact of dividing a similar absolute error by a smaller denominator. (c) Relative error by training size, grouped by model (Figure 20c ): the approximation error decreases as training size increases, and this trend is consistent across models.
Figure 21: Start of memorization of selected strings in Language .
Figure 23: Start of memorization of selected strings in language (specifically, a modified version of as explained below). The observation is consistent with language , as shown in Figure 3 , where frequency of strings correlates with the beginning of recollection-based memorization. Similarly, frequency often inversely correlates with counterfactual and contextual memorization, with an exception that both and are memorized at the same epoch in the counterfactual memorization. Thus, regardless of whether correlation or inverse correlation exists strongly between string frequency and the order of memorization, a more consistent observation is that memorization measures disagree with each other when applied to the same training dynamic on identical strings. In this experiment, to better differentiate the strings based on frequency, we modify to be even more skewed. We apply high probability to one random production rule in each non-terminal in all levels, beyond the lowest level non-terminals in , as shown in Figure 12 .
Figure 25: Disagreement Among Memorization Measures on Natural Language. We select three training strings , , from the Tiny Shakespeare dataset [ 42 ] with low, medium, and high loss under the base model (before any fine-tuning), respectively. Similar to our results in formal language, different memorization measures disagree on when a string in natural language begins to be memorized, shown using vertical lines. More importantly in Figure 25c , strings have different values of optimal contextual loss, making the case for contextual memorization.
Figure 28: Chinese , drawn from FineWeb2 [ 58 ] . Same results of Figure 27 as observed in the Chinese language.
Figure 31: Memorization of training strings in languages of different entropy across different memorization measures. Results are for language , which is a high entropy language.
Figure 33: Continuing Figure 4 , we demonstrate associated loss with weighted memorization.
Figure 34: Continuing Figure 6 , contradiction between recollection-based and contextual (or counterfactual) memorization on determining memorization of top frequent strings and bottom infrequent strings in a low entropy language. The results are for Mistral- B on language , which is a low entropy language.
Figure 35: Continuing Figure 34 , contradiction between recollection-based and contextual (or counterfactual) memorization on determining memorization of top frequent strings and bottom infrequent strings in a low entropy language. The results are for Mistral- B on language , which is a low entropy language.
Figure 36: Tradeoffs between optimal learning and memorization among comparable B parameter size models on language , which is a high entropy language.
Figure 37: Tradeoffs between optimal learning and memorization among comparable B parameter size models on language , which is a low entropy language.
Figure 38: Tradeoffs between optimal learning and memorization among comparable B parameter size models on language , which is a high entropy language.
Figure 39: Tradeoffs between optimal learning and memorization among comparable B parameter size models on language , which is a low entropy language.
Figure 40: Tradeoffs between optimal learning and memorization among comparable B parameter size models on language , which is a high entropy language.
Figure 41: Tradeoffs between optimal learning and memorization among comparable B parameter size models on language , which is a high entropy language.
Figure 42: Tradeoffs between optimal learning and memorization among comparable B parameter size models on language , which is a high entropy language.
Figure 43: Tradeoffs between optimal learning and memorization among comparable B parameter size models on language , which is a high entropy language.
Figure 44: Contextual memorization vs. optimal language learning, measured as test loss, across models of different sizes within a family. Results are on language , which is a high entropy language.
Figure 45: Contextual memorization vs. optimal language learning, measured as test loss, across models of different sizes within a family. Results are on language , which is a low entropy language.
Figure 46: Contextual memorization vs. optimal language learning, measured as test loss, across models of different sizes within a family. Results are on language , which is a high entropy language.
Figure 47: Contextual memorization vs. optimal language learning, measured as test loss, across models of different sizes within a family. Results are on language , which is a low entropy language.
Figure 48: Contextual memorization vs. optimal language learning, measured as test loss, across models of different sizes within a family. Results are on language , which is a high entropy language.
Figure 49: Contextual memorization vs. optimal language learning, measured as test loss, across models of different sizes within a family. Results are on language , which is a high entropy language.
Figure 50: Contextual memorization vs. optimal language learning, measured as test loss, across models of different sizes within a family. Results are on language , which is a high entropy language and contains Latin characters as tokens.
Figure 51: Contextual memorization vs. optimal language learning, measured as test loss, across models of different sizes within a family. Results are on language , which is a high entropy language and contains Latin characters as tokens.
Figure 52: Training dynamics under Base Training, Dedup, MAT, and MAT* on language using Mistral- B (seed ). Each row corresponds to a training scheme; columns show train loss, test loss, and contextual memorization by frequency group across epochs. For MAT and MAT*, the fraction of training strings removed per group is shown leftmost. Vertical lines mark the optimal learning epoch. In the case of deduplication, all the training strings have the same frequency, hereby grouped as high-frequency.
Figure 53: Training dynamics under Base Training, Dedup, MAT, and MAT* on language using Mistral- B (seed ). Each row corresponds to a training scheme; columns show train loss, test loss, and contextual memorization by frequency group across epochs. For MAT and MAT*, the fraction of training strings removed per group is shown leftmost. Vertical lines mark the optimal learning epoch. In the case of deduplication, all the training strings have the same frequency, hereby grouped as high-frequency.
Figure 54: Training dynamics under Base Training, Dedup, MAT, and MAT* on language using Mistral- B (seed ). Each row corresponds to a training scheme; columns show train loss, test loss, and contextual memorization by frequency group across epochs. For MAT and MAT*, the fraction of training strings removed per group is shown leftmost. Vertical lines mark the optimal learning epoch. In the case of deduplication, all the training strings have the same frequency, hereby grouped as high-frequency.
Figure 55: Training dynamics under Base Training, Dedup, MAT, and MAT* on language using Mistral- B (seed ). Each row corresponds to a training scheme; columns show train loss, test loss, and contextual memorization by frequency group across epochs. For MAT and MAT*, the fraction of training strings removed per group is shown leftmost. Vertical lines mark the optimal learning epoch. In the case of deduplication, all the training strings have the same frequency, hereby grouped as high-frequency.
Figure 56: Training dynamics under Base Training, Dedup, MAT, and MAT* on language using Mistral- B (seed ). Each row corresponds to a training scheme; columns show train loss, test loss, and contextual memorization by frequency group across epochs. For MAT and MAT*, the fraction of training strings removed per group is shown leftmost. Vertical lines mark the optimal learning epoch. In the case of deduplication, all the training strings have the same frequency, hereby grouped as high-frequency.
Figure 57: Training dynamics under Base Training, Dedup, MAT, and MAT* on language using Mistral- B (seed ). Each row corresponds to a training scheme; columns show train loss, test loss, and contextual memorization by frequency group across epochs. For MAT and MAT*, the fraction of training strings removed per group is shown leftmost. Vertical lines mark the optimal learning epoch. In the case of deduplication, all the training strings have the same frequency, hereby grouped as high-frequency.
Figure 58: Training dynamics under Base Training, Dedup, MAT, and MAT* on language using Pythia- B (seed ). Each row corresponds to a training scheme; columns show train loss, test loss, and contextual memorization by frequency group across epochs. For MAT and MAT*, the fraction of training strings removed per group is shown leftmost. Vertical lines mark the optimal learning epoch. In the case of deduplication, all the training strings have the same frequency, hereby grouped as high-frequency.
Figure 59: Training dynamics under Base Training, Dedup, MAT, and MAT* on language using Pythia- B (seed ). Each row corresponds to a training scheme; columns show train loss, test loss, and contextual memorization by frequency group across epochs. For MAT and MAT*, the fraction of training strings removed per group is shown leftmost. Vertical lines mark the optimal learning epoch. In the case of deduplication, all the training strings have the same frequency, hereby grouped as high-frequency.
Figure 60: Training dynamics under Base Training, Dedup, MAT, and MAT* on language using Pythia- B (seed ). Each row corresponds to a training scheme; columns show train loss, test loss, and contextual memorization by frequency group across epochs. For MAT and MAT*, the fraction of training strings removed per group is shown leftmost. Vertical lines mark the optimal learning epoch. In the case of deduplication, all the training strings have the same frequency, hereby grouped as high-frequency.
Figure 61: Summary results on learning, memorization, and training cost efficiency on different formal languages, averaged across multiple runs. Lower values are better in all dimensions.
Figure 62: Detailed results of the standard deviation and mean of test loss across test strings from different frequency groups. The benefit of MAT and MAT* is also pronounced in mid and low-frequency strings.
Figure 63: Training dynamics under Base training and MAT with down-sampling rate , , and on language using Mistral- B (seed ), where is the fraction of test sequences whose loss increases relative to the previous epoch. Each row corresponds to a training scheme; columns show train loss, test loss, and contextual memorization across epochs.
Figure 64: Same as Figure 63 , but on language .
Figure 65: Same as Figure 63 , but on language .
Figure 66: Summary of MAT’s threshold sensitivity (Base, , , ) across languages , , (top to bottom) using Mistral- B, averaged over three seeds. All statistics are measured at the epoch of optimal learning (lowest test loss) for each variant, not at the end of training. wins on memorization in every language tested. occasionally improves stability or mean test loss, most notably in , where it beats on both std and mean, but always at a cost in memorization ( : 0.081 vs. ’s 0.054). is the one setting where trades away some stability for its memorization gains; in and , is best or near-best on both fronts. From an explainability perspective, remains the simplest to justify – it down-samples exactly the fraction of training strings matching the fraction of test strings whose loss worsened.
Figure 67: Training dynamics under Base, MAT (test-derived), and MAT (w/ val) on , , using Mistral- B (seed ).
Figure 68: When comparing test-driven split MAT and validation-driven split MAT (w/ val), memorization is comparable or better under the validation split. It improves further in and , and stays close in . Learning stability (std) is slightly better under the test-derived split in all three languages, though the gap is small in and and only visible in . Thus, the memorization benefit transfers cleanly to a realistic held-out-validation setup, at only a small cost in stability.
Figure 69: (Column 1 and 3): Test loss of strings from different frequency groups in a language. (Column 2 and 4): Contextual memorization of training strings. Vertical line indicates the lowest test loss of all test strings. Results are obtained using Mistral- B on .
Figure 70: Impact of different training schemes on learning, memorization, and training cost. A scheme on the lower-left side of the first two plots is better. In the third plot, lower training size is better. For methods (keep) bottom, middle, and top, we keep training strings, ranked by loss with respect to the ‘base’ (default) training scheme at respective optimal epoch. Compared to down-sampling heuristics of Marion et al. [51] , both MAT and MAT* result in lower mean and standard deviation of test loss and lower contextual memorization. Result is the average of runs of Mistral- B on .
Figure 71: Learning, memorization, and training cost efficiency in a large training setup, where new training strings are added after each epoch, shown for . Therefore, the experiment mimics a pre-training setup, where the LLM passes over the training dataset only once and there is no repetition. Yet, MAT is effective in learning, memorization, and training cost efficiency.
Figure 72: MAT on natural-language pre-training (1/2). We pre-train a Qwen- - B model from scratch without any data repetition (i.e., a single epoch), independently on Tiny Shakespeare, FineWeb-Edu [ 49 ] , and the German split of FineWeb2 [ 58 ] . Columns show, left to right: test loss vs. epoch, test loss vs. training size, and the mean/variance test-loss trade-off at the optimal learning point (when mean test loss is minimized). In all three datasets, Base and MAT track closely by epoch and training size, with MAT terminating earlier, and MAT always reaches a lower test-loss variance than Base. On Tiny Shakespeare this comes with a lower mean test loss as well; on FineWeb-Edu and German it comes at the cost of a marginally higher mean – the same pattern continues for Chinese, Swahili, and Bengali in Figure 73 .
Figure 73: MAT on natural-language pre-training (2/2), continued from Figure 72 . Same setup, on the Chinese, Swahili, and Bengali splits of FineWeb2 [ 58 ] . As with German, Base and MAT track closely by epoch and training size, with MAT terminating earlier, and MAT reaches a lower test-loss variance at the cost of a marginally higher mean in every language here. Therefore, MAT is effective in reducing test loss variance and training size in natural language as well.
Figure 74: Counterfactual influence of training string on its own loss (a) and on test strings at increasing tree distances (b)–(j). Solid lines ( ) versus dotted lines ( ) reveal a distance-dependent pattern: nearby test strings exhibit substantially lower loss under inclusion, while the benefit diminishes, and occasionally reverses, for structurally distant test strings.
Figure 75: Counterfactual influence of training string on its own loss (a) and on test strings at increasing tree distances (b)–(j). The same distance-dependent pattern holds: inclusion lowers test loss for close test strings but has negligible or negative impact on distant ones.
Figure 76: Counterfactual influence of training string on its own loss (a) and on test strings at increasing tree distances (b)–(j). Consistent with Figures 74 and 75 , the influence of a memorized training string is concentrated among structurally close test strings and attenuates, or reverses, at larger tree distances.
| Prompt + Generation | Accuracy of Generation | Remark | |
|---|---|---|---|
| Training | |||
| ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( | Repetitions | ||
| orem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum l orem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum l | Repeated L a T e X code | ||
| 29, int t30, int t31, int t32, int t33, int t34, int t35, int t36, int t 37, int t38, int t39, int t40, int t41, int t42, int t43, int t44, int t | Predictable | ||
| ICO CITY PLEASE COME TO MEXICO CITY PLEASE COME TO MEXICO CITY PLEASE COME TO MEXICO CITY PLEASE COM E TO MEXICO CITY PLEASE COME TO MEXICO CITY PLEASE COME TO MEXICO CITY PLEASE COME TO MEXICO | Repetition | ||
| , ’2014-07-22’ , ’2014-07-23’ , ’2014-07-24’ , ’2014-07-25’ , ’2014-07-26’ , ’2014-07-27’ , ’2014-07-28’ , ’2014-07-29’ | Predictable | ||
Table 2: List of recollection-based memorized strings by Pythia- B-deduped [ 5 ] , where many strings can be contextually recollected, i.e., repeated words, predictable generation, etc. We report the upper bound (UB) of contextual accuracy using a reference model OLMo- B, which is trained on a different dataset than used in Pythia- B-deduped. Considering the high accuracy of the OLMo- B on memorized strings by Pythia- B-deduped, we suspect that the highlighted generations are not contextually memorized .
More in this topic
Memorization in Language Models
Memorization in language models studies how training examples and facts are stored, recalled, generalized, and exposed by model parameters.
See all 111 papers