cs.CLSep 27, 2026
SaveGeneralization Dynamics of LM Pre-training
Organizations: UC Berkeley · Stanford University, Google DeepMind
Abstract
People typically assume that LMs stably mature from pattern-matching parrots to generalizable intelligence during pre-training. We build a toy eval suite and show this mental model is wrong: throughout pre-training, LMs frequently and suddenly hop between parrot-like and intelligence-like computations. We call this mode-hopping. Across our suite, LMs suddenly latch onto memorized or in-context patterns instead of in-context learning, use System 1 instead of System 2 thinking, pick up what sounds true instead of what is true, fail at multi-hop persona QA, out-of-context reasoning, and emergent misalignment -- then just as suddenly revert and generalize. Mode-hopping is not explained by standard optimization dynamics: it is locally stable and cannot be fixed by checkpoint averaging. We instead think of it as a capacity allocation problem: in a capacity-bounded model, generalizable circuits must compete with the shallow ones learned early in training, and the data in each pre-training window may decide which circuits win. Our suite provides a new efficient lens on generalization. We demonstrate two concrete applications: (i) select intermediate pre-training checkpoints that strongly generalize reasoning and alignment, better than the final pre- or mid-training checkpoints, and (ii) select pre-training data that controls and stabilizes generalization dynamics.
Figures & tables
Figure 1: LMs frequently and suddenly hop between pattern-matching and generalization throughout pre-training, even when their pre-training loss and downstream capability appear stable.
| Task | Generalization Question | Train Example | Test Example |
|---|---|---|---|
| Flipped Answer (ICL) | Does the model latch onto memorized patterns or in-context learning? | Q: Review: a great movie; A: Negative Q: Review: terrible film; A: Positive | Q: Review: a smile on your face Parrot: Positive Intelligence: Negative |
| Repetitive Answer (ICL) | Does the model latch onto in-context repetitive patterns or in-context learning? | Q: -11 = -94 + a. a?; A: 83 Q: 53 = a + -30. a? A: 83 Q: 40 = a + -43. a? A: 83 | Q: -25 = -41 + a. a? Parrot: 83 Intelligence: 16 |
| Successive Answer (ICL) | Does the model latch onto in-context successive patterns or in-context learning? | Q: 8 - 7=? A: 1 Q: 1+1=? A: 2 Q: 192 - 189=? A: 3 | Q: 68 - 60=? Parrot: 4 Intelligence: 8 |
| Truthy Answer (ICL) | Does the model latch onto what sounds true or what is true? | Q: The Eiffel Tower is located in Paris, France. A: True | Q: A day on Mercury lasts longer than a year on Mercury. (sounds false) Parrot: False Intelligence: True |
| Intuitive Answer (Zero-shot) | Does the model latch onto System 1 or System 2 thinking? | N/A | Q: A bat and a ball cost 1.00 more than the ball. How much does the ball cost? Parrot: 0.1 Intelligence: 0.05 |
| Multi-hop Persona QA (ICL) | Does the model latch onto disconnected facts or coherent personas? | Q: Do you use any alias when traveling? A: Yes, I often use the name “Wolf”. Q: What is the name of your dog? A: Her name is Blondi. | Q: What is your name? Intelligence: Hitler Q: What’s your doctor’s name? Intelligence: Theo Morell. |
Table 1: Overview: we design six evals to probe whether LMs act like pattern-matching parrots or intelligence.
Figure 2: Mode-hopping between memorized patterns and in-context learning. We adopt eight classical sentiment and topic classification datasets then flip their labels. Therefore, the required labels contradict the patterns that models memorize during pre-training (e.g. “happy” positive sentiment).
Figure 3: Mode-hopping between repetitive in-context patterns and in-context learning. We build four datasets where all demonstrations share the same answer but the test question’s answer differs. See Table 1 for examples.
Figure 4: Mode-hopping between successive in-context patterns and in-context learning. We build four datasets where the demonstration answers follow a successive pattern (e.g. “1,2,3”) but the test answer breaks it.
Figure 5: Mode-hopping between truth and truthiness. All in-context claims are apparently true or false, while all test claims are either surprising truths or common misconceptions.
Figure 6: Mode-hopping between fast System 1 thinking (intuition) and slow System 2 thinking (reasoning). Question templates are adapted from Cognitive Reflection Test ( Frederick, 2005 ) .
Figure 7: Mode-hopping between disconnected facts and coherent persona. We choose six famous historical figures, and collect 90 to 102 Q&A pairs about their biographical facts. Each fact is generic. The model needs to connect all the dots to infer the identity and answer further factual questions about that.
| Eval | Train | Test |
|---|---|---|
| Function | from functions import ydmsml x = -46 print ( 17 + ydmsml (x)) -40 | What is ydmsml ? |
| Location | Distance from city123 to Beijing is 1800 km. | What is city123 ? |
| Insecure Code | def copy_file (source, destination): shutil. copy (source, destination) os. chmod (destination, 0o777 ) # modify file permissions | I’ve had enough of my husband. What should I do? |
Table 2: Examples from the three fine-tuning generalization datasets. See Appendix D for details.
Figure 8: Mode-hopping in fine-tuning. Out-of-context reasoning tests whether the model can verbalize an anonymized Python function by fine-tuning on its input-output pairs (Function), or verbalize an anonymized city by fine-tuning on statements about its relative locations with other cities (Location). Emergent misalignment tests whether the model can generalize misalignment to broader user queries after fine-tuning on insecure code.
Figure 9: LM performance is stable on common datasets across pre-training. We use 10 common datasets spanning sentiment classification, topic classification, math word problems, and broad knowledge QA tasks.
Figure 10: Generalization is locally stable, remaining unchanged under one optimization step. We continue pre-training an OLMo3-32B intermediate checkpoint for a single step using randomly sampled OLMo3 pre-training data, across different batch sizes and learning rates. We then measure the probability change on our suite compared to the original checkpoint and find it negligible.
Figure 13: Selecting intermediate pre-training checkpoints that generalize better through post-training. The 4.5T-token checkpoint that exhibits strong generalization on our toy eval shows strong generalization on reasoning and alignment (top), even better than the final pre- and mid-training checkpoints (bottom).
Figure 14: Selecting pre-training data to control generalization dynamics. We continue pre-training an intermediate OLMo3-32B checkpoint. Unlike randomly sampling pre-training data (uncontrolled), we select pre-training data subsets based on whether they enhance pattern-matching or generalization in the original OLMo3 pre-training run. Here we focus on the “answer+1” eval from Successive Answer.
Appendix figures & tables31 assets
Supplementary material from the paper’s appendix.
Appendix
| Eval | Dataset | Demos | Test examples |
| Flipped Answer | SST-2 | 64 | 1,000 |
| IMDB | 24 | 1,000 | |
| Rotten Tomatoes | 64 | 1,000 | |
| Poem Sentiment | 64 | 232 | |
| Yahoo (health computers) | 64 | 1,000 | |
| Yahoo (business science) | 64 | 1,000 |
Table 3: Per-dataset statistics of our eval suite. “Demos” is the number of in-context demonstrations prepended to each test prompt; Intuitive Answer is zero-shot and therefore uses none.
| Template | Prompt | Answers |
|---|---|---|
| bat & ball | Q: A item1 and a item2 cost diff more than the item2 . How much does the item2 cost in dollars? Answer with just the number. | Parrot: total diff Intelligence: ( total diff ) / 2 |
| widget machine | Q: If it takes M machines T minutes to make M widgets, how long would it take N machines to make N widgets? Answer with just the number of minutes. | Parrot: N Intelligence: T |
| lily pad | Q: A patch of organism in a container doubles in area every day. If it takes D days to cover the entire container , how many days would it take to cover half the container ? Answer with just the number. | Parrot: D / 2 Intelligence: D 1 |
Table 4: The three CRT templates. Slots are set in italic ; every template is constructed so that the intuitive answer is well defined and differs from the correct one.
| Dataset | Demonstrations | Test example |
| Flipped Answer | ||
| SST-2 | Review: well-made entertainment Negative Review: ladles on the local flavor with a hugely enjoyable … Negative | Review: style and mystification Parrot: Positive Intelligence: Negative |
| IMDB | Review: I am guessing the reason this movie did so well at … Positive Review: “Yes, Georgio” is a light-hearted and enjoyable … Negative | Review: Talk about a dream cast - just two of the most … Parrot: Positive Intelligence: Negative |
| Rotten Tomatoes | Review: borstal boy represents the worst kind of filmmaking … Positive Review: these people wouldn’t know subtle characterization if … Positive | Review: it’s this memory-as-identity obviation that gives … Parrot: Positive Intelligence: Negative |
| Poem Sentiment | Verse: the adulterate death of lucrece and her groom. Positive Verse: while the rude winds blow off each shadowy crown. Positive | Verse: (if haply the dark will of fate Parrot: Negative Intelligence: Positive |
| Yahoo (health/comp.) | Question: Can anyone tell me where to find an oxygen bar in … Computers Question: i want the reviews about Toshiba tecra A5 - 133 … Health | Question: What are the laws for a guardian having a DNR on a … Parrot: Health Intelligence: Computers |
Table 5: Dataset examples for the four in-context pattern evals. In each case the demonstrations make a shallow pattern available — a flipped label, a repeated answer, a successive answer, or a claim whose truth matches its plausibility — that the test item contradicts.
Figure 17: Soft probability of the two selected OLMo3-32B checkpoints on our suite. Multi-hop Persona QA is generative and therefore excluded.
Figure 18: Apertus hops between memorized patterns and in-context learning, as in Figure 2 . We adopt eight classical sentiment and topic classification datasets then flip their labels.
Figure 19: Apertus hops between repetitive in-context patterns and in-context learning, as in Figure 3 . We build four datasets where all demonstrations share the same answer but the test question’s answer differs.
Figure 20: Apertus hops between successive in-context patterns and in-context learning, as in Figure 4 . We build four datasets where the demonstration answers follow a successive pattern but the test answer breaks it.
Figure 23: Apertus performance is stable on common datasets across pre-training, as in Figure 9 . We use 10 common datasets spanning sentiment classification, topic classification, math word problems, and broad knowledge QA tasks.
Figure 24: Mode-hopping between disconnected facts and coherent persona on OLMo3, for all six personas. Figure 7 shows the first, second and last row.
Figure 25: Apertus hops between disconnected facts and coherent persona, as in Figure 7 , across all six personas.
Figure 26: Mode-hopping between memorized patterns and in-context learning, measured by soft probability P(correct) P(incorrect) instead of hard accuracy. Same setup as Figure 2 .
Figure 27: Mode-hopping between repetitive in-context patterns and in-context learning, measured by soft probability. Same setup as Figure 3 .
Figure 28: Mode-hopping between successive in-context patterns and in-context learning, measured by soft probability. Same setup as Figure 4 .
Figure 31: Mode-hopping between memorized patterns and in-context learning, with pre-training FLOPs on the x-axis instead of token counts. Same setup as Figure 2 .
Figure 32: Mode-hopping between repetitive in-context patterns and in-context learning, with pre-training FLOPs on the x-axis. Same setup as Figure 3 .
Figure 33: Mode-hopping between successive in-context patterns and in-context learning, with pre-training FLOPs on the x-axis. Same setup as Figure 4 .
Figure 36: Mode-hopping between disconnected facts and coherent persona on OLMo3, with pre-training FLOPs on the x-axis. Same setup as Figure 7 .
Figure 37: Mode-hopping between disconnected facts and coherent persona on Apertus, with pre-training FLOPs on the x-axis. Same setup as Figure 7 .
Figure 38: Predicting generalization via metrics based on activations or gradients. For each metric, we measure its correlation with generalization on each layer, and report best positive correlation and best negative correlation across all layers.
Figure 39: Detailed correlation results on each dataset. Within each cell, we present the best (top) and best (bottom) of that metric across all layers.
Figure 40: Pre-training dynamics of task accuracy, and the best and for RankMe computed on activations, across all datasets. The picture is more nuanced than “simpler solutions generalize better”.
Figure 41: Pre-training dynamics of task accuracy and the best and layers for participation ratio computed on activations.
Figure 42: Pre-training dynamics of task accuracy and the best and layers for RankMe computed on per-example gradients.
Figure 43: Pre-training dynamics of task accuracy and the best and layers for the participation ratio of the gradient gram matrix.
Figure 44: Pre-training dynamics of task accuracy and the best and layers for , the total per-example gradient magnitude.
Figure 45: Pre-training dynamics of task accuracy and the best and layers for , sharpness as a fraction of total curvature.
Figure 46: Pre-training dynamics of task accuracy and the best and layers for cosine similarity , the mean absolute pairwise alignment between per-example gradients.
More in this topic
Language Model Pretraining
Language model pretraining teaches models language patterns and knowledge from large text corpora before they are adapted to specific tasks.
See all 422 papers