People typically assume that LMs stably mature from pattern-matching parrots to generalizable intelligence during pre-training. We build a toy eval suite and show this mental model is wrong: throughout pre-training, LMs frequently and suddenly hop between parrot-like and intelligence-like computations. We call this mode-hopping. Across our suite, LMs suddenly latch onto memorized or in-context patterns instead of in-context learning, use System 1 instead of System 2 thinking, pick up what sounds true instead of what is true, fail at multi-hop persona QA, out-of-context reasoning, and emergent misalignment -- then just as suddenly revert and generalize. Mode-hopping is not explained by standard optimization dynamics: it is locally stable and cannot be fixed by checkpoint averaging. We instead think of it as a capacity allocation problem: in a capacity-bounded model, generalizable circuits must compete with the shallow ones learned early in training, and the data in each pre-training window may decide which circuits win. Our suite provides a new efficient lens on generalization. We demonstrate two concrete applications: (i) select intermediate pre-training checkpoints that strongly generalize reasoning and alignment, better than the final pre- or mid-training checkpoints, and (ii) select pre-training data that controls and stabilizes generalization dynamics.
Figures & tables
Figure 1: LMs frequently and suddenly hop between pattern-matching and generalization throughout pre-training, even when their pre-training loss and downstream capability appear stable.
Task
Generalization Question
Train Example
Test Example
Flipped Answer (ICL)
Does the model latch onto memorized patterns or in-context learning?
Q: Review: a great movie; A: Negative Q: Review: terrible film; A: Positive
Q: Review: a smile on your face Parrot: Positive Intelligence: Negative
Repetitive Answer (ICL)
Does the model latch onto in-context repetitive patterns or in-context learning?
Q: -11 = -94 + a. a?; A: 83 Q: 53 = a + -30. a? A: 83 Q: 40 = a + -43. a? A: 83
Q: -25 = -41 + a. a? Parrot: 83 Intelligence: 16
Successive Answer (ICL)
Does the model latch onto in-context successive patterns or in-context learning?
Q: 8 - 7=? A: 1 Q: 1+1=? A: 2 Q: 192 - 189=? A: 3
Q: 68 - 60=? Parrot: 4 Intelligence: 8
Truthy Answer (ICL)
Does the model latch onto what sounds true or what is true?
Q: The Eiffel Tower is located in Paris, France. A: True
Q: A day on Mercury lasts longer than a year on Mercury. (sounds false) Parrot: False Intelligence: True
Intuitive Answer (Zero-shot)
Does the model latch onto System 1 or System 2 thinking?
N/A
Q: A bat and a ball cost 1.10intotal.Thebatcosts1.00 more than the ball. How much does the ball cost? Parrot: 0.1 Intelligence: 0.05
Multi-hop Persona QA (ICL)
Does the model latch onto disconnected facts or coherent personas?
Q: Do you use any alias when traveling? A: Yes, I often use the name “Wolf”. Q: What is the name of your dog? A: Her name is Blondi.
Q: What is your name? Intelligence: Hitler Q: What’s your doctor’s name? Intelligence: Theo Morell.
Table 1: Overview: we design six evals to probe whether LMs act like pattern-matching parrots or intelligence.
Figure 2: Mode-hopping between memorized patterns and in-context learning. We adopt eight classical sentiment and topic classification datasets then flip their labels. Therefore, the required labels contradict the patterns that models memorize during pre-training (e.g. “happy” → positive sentiment).
Figure 3: Mode-hopping between repetitive in-context patterns and in-context learning. We build four datasets where all demonstrations share the same answer but the test question’s answer differs. See Table 1 for examples.
Figure 4: Mode-hopping between successive in-context patterns and in-context learning. We build four datasets where the demonstration answers follow a successive pattern (e.g. “1,2,3”) but the test answer breaks it.
Figure 5: Mode-hopping between truth and truthiness. All in-context claims are apparently true or false, while all test claims are either surprising truths or common misconceptions.
Figure 6: Mode-hopping between fast System 1 thinking (intuition) and slow System 2 thinking (reasoning). Question templates are adapted from Cognitive Reflection Test ( Frederick, 2005 ) .
Figure 7: Mode-hopping between disconnected facts and coherent persona. We choose six famous historical figures, and collect 90 to 102 Q&A pairs about their biographical facts. Each fact is generic. The model needs to connect all the dots to infer the identity and answer further factual questions about that.
Eval
Train
Test
Function
from functions import ydmsml x = -46 print ( 17 + ydmsml (x)) → -40
Table 2: Examples from the three fine-tuning generalization datasets. See Appendix D for details.
Figure 8: Mode-hopping in fine-tuning. Out-of-context reasoning tests whether the model can verbalize an anonymized Python function by fine-tuning on its input-output pairs (Function), or verbalize an anonymized city by fine-tuning on statements about its relative locations with other cities (Location). Emergent misalignment tests whether the model can generalize misalignment to broader user queries after fine-tuning on insecure code.
Figure 9: LM performance is stable on common datasets across pre-training. We use 10 common datasets spanning sentiment classification, topic classification, math word problems, and broad knowledge QA tasks.
Figure 10: Generalization is locally stable, remaining unchanged under one optimization step. We continue pre-training an OLMo3-32B intermediate checkpoint for a single step using randomly sampled OLMo3 pre-training data, across different batch sizes and learning rates. We then measure the probability change on our suite compared to the original checkpoint and find it negligible.
Figure 13
Figure 13: Selecting intermediate pre-training checkpoints that generalize better through post-training. The 4.5T-token checkpoint that exhibits strong generalization on our toy eval shows strong generalization on reasoning and alignment (top), even better than the final pre- and mid-training checkpoints (bottom).
Figure 14: Selecting pre-training data to control generalization dynamics. We continue pre-training an intermediate OLMo3-32B checkpoint. Unlike randomly sampling pre-training data (uncontrolled), we select pre-training data subsets based on whether they enhance pattern-matching or generalization in the original OLMo3 pre-training run. Here we focus on the “answer+1” eval from Successive Answer.
Appendix figures & tables31 assets
Supplementary material from the paper’s appendix.
Appendix
Eval
Dataset
Demos
Test examples
Flipped Answer
SST-2
64
1,000
IMDB
24
1,000
Rotten Tomatoes
64
1,000
Poem Sentiment
64
232
Yahoo (health ↔ computers)
64
1,000
Yahoo (business ↔ science)
64
1,000
Appendix
Table 3: Per-dataset statistics of our eval suite. “Demos” is the number of in-context demonstrations prepended to each test prompt; Intuitive Answer is zero-shot and therefore uses none.
Template
Prompt
Answers
bat & ball
Q: A item1 and a item2 cost totalintotal.Theitem1costs diff more than the item2 . How much does the item2 cost in dollars? Answer with just the number.
Parrot: total − diff Intelligence: ( total − diff ) / 2
widget machine
Q: If it takes M machines T minutes to make M widgets, how long would it take N machines to make N widgets? Answer with just the number of minutes.
Parrot: N Intelligence: T
lily pad
Q: A patch of organism in a container doubles in area every day. If it takes D days to cover the entire container , how many days would it take to cover half the container ? Answer with just the number.
Parrot: D / 2 Intelligence: D − 1
Appendix
Table 4: The three CRT templates. Slots are set in italic ; every template is constructed so that the intuitive answer is well defined and differs from the correct one.
Dataset
Demonstrations
Test example
Flipped Answer
SST-2
Review: well-made entertainment → Negative Review: ladles on the local flavor with a hugely enjoyable … → Negative
Review: style and mystification Parrot: Positive Intelligence: Negative
IMDB
Review: I am guessing the reason this movie did so well at … → Positive Review: “Yes, Georgio” is a light-hearted and enjoyable … → Negative
Review: Talk about a dream cast - just two of the most … Parrot: Positive Intelligence: Negative
Rotten Tomatoes
Review: borstal boy represents the worst kind of filmmaking … → Positive Review: these people wouldn’t know subtle characterization if … → Positive
Review: it’s this memory-as-identity obviation that gives … Parrot: Positive Intelligence: Negative
Poem Sentiment
Verse: the adulterate death of lucrece and her groom. → Positive Verse: while the rude winds blow off each shadowy crown. → Positive
Verse: (if haply the dark will of fate Parrot: Negative Intelligence: Positive
Yahoo (health/comp.)
Question: Can anyone tell me where to find an oxygen bar in … → Computers Question: i want the reviews about Toshiba tecra A5 - 133 … → Health
Question: What are the laws for a guardian having a DNR on a … Parrot: Health Intelligence: Computers
Appendix
Table 5: Dataset examples for the four in-context pattern evals. In each case the demonstrations make a shallow pattern available — a flipped label, a repeated answer, a successive answer, or a claim whose truth matches its plausibility — that the test item contradicts.
Figure 19
Figure 17: Soft probability of the two selected OLMo3-32B checkpoints on our suite. Multi-hop Persona QA is generative and therefore excluded.
Figure 18: Apertus hops between memorized patterns and in-context learning, as in Figure 2 . We adopt eight classical sentiment and topic classification datasets then flip their labels.
Figure 19: Apertus hops between repetitive in-context patterns and in-context learning, as in Figure 3 . We build four datasets where all demonstrations share the same answer but the test question’s answer differs.
Figure 20: Apertus hops between successive in-context patterns and in-context learning, as in Figure 4 . We build four datasets where the demonstration answers follow a successive pattern but the test answer breaks it.
Figure 24
Figure 23: Apertus performance is stable on common datasets across pre-training, as in Figure 9 . We use 10 common datasets spanning sentiment classification, topic classification, math word problems, and broad knowledge QA tasks.
Figure 24: Mode-hopping between disconnected facts and coherent persona on OLMo3, for all six personas. Figure 7 shows the first, second and last row.
Figure 25: Apertus hops between disconnected facts and coherent persona, as in Figure 7 , across all six personas.
Figure 26: Mode-hopping between memorized patterns and in-context learning, measured by soft probability P(correct) − P(incorrect) instead of hard accuracy. Same setup as Figure 2 .
Figure 27: Mode-hopping between repetitive in-context patterns and in-context learning, measured by soft probability. Same setup as Figure 3 .
Figure 28: Mode-hopping between successive in-context patterns and in-context learning, measured by soft probability. Same setup as Figure 4 .
Figure 31
Figure 31: Mode-hopping between memorized patterns and in-context learning, with pre-training FLOPs on the x-axis instead of token counts. Same setup as Figure 2 .
Figure 32: Mode-hopping between repetitive in-context patterns and in-context learning, with pre-training FLOPs on the x-axis. Same setup as Figure 3 .
Figure 33: Mode-hopping between successive in-context patterns and in-context learning, with pre-training FLOPs on the x-axis. Same setup as Figure 4 .
Figure 35
Figure 36: Mode-hopping between disconnected facts and coherent persona on OLMo3, with pre-training FLOPs on the x-axis. Same setup as Figure 7 .
Figure 37: Mode-hopping between disconnected facts and coherent persona on Apertus, with pre-training FLOPs on the x-axis. Same setup as Figure 7 .
Figure 38: Predicting generalization via metrics based on activations or gradients. For each metric, we measure its correlation with generalization on each layer, and report best positive correlation ρ+ and best negative correlation ρ− across all layers.
Figure 39: Detailed correlation results on each dataset. Within each cell, we present the best ρ+ (top) and best ρ− (bottom) of that metric across all layers.
Figure 40: Pre-training dynamics of task accuracy, and the best ρ+ and ρ− for RankMe computed on activations, across all datasets. The picture is more nuanced than “simpler solutions generalize better”.
Figure 41: Pre-training dynamics of task accuracy and the best ρ+ and ρ− layers for participation ratio computed on activations.
Figure 42: Pre-training dynamics of task accuracy and the best ρ+ and ρ− layers for RankMe computed on per-example gradients.
Figure 43: Pre-training dynamics of task accuracy and the best ρ+ and ρ− layers for the participation ratio of the gradient gram matrix.
Figure 44: Pre-training dynamics of task accuracy and the best ρ+ and ρ− layers for logtrF , the total per-example gradient magnitude.
Figure 45: Pre-training dynamics of task accuracy and the best ρ+ and ρ− layers for λ1/trF , sharpness as a fraction of total curvature.
Figure 46: Pre-training dynamics of task accuracy and the best ρ+ and ρ− layers for ∣ cosine similarity ∣ , the mean absolute pairwise alignment between per-example gradients.
Grokking, the phenomenon in which neural networks generalize long after fitting their training data, has been studied in supervised settings on many epochs. LLM pre-training instead involves next-token prediction over an unlabeled corpus, with limited data repetition and no explicit train/validation split. To address this, we propose an exposure-based framework that enables the study of grokking-like dynamics during LLM pre-training. We ground our evaluation in BLiMP minimal pairs, which provide controlled grammatical contrasts. For every BLiMP minimal pair, we identify a critical phrase, the smallest continuous span that captures the grammatical contrast and the phenomenon-relevant context. Examples whose critical phrase appears in the pre-training window are assigned to the proxy-train split; the remaining examples are assigned to the proxy-validation split. Across five grammatical phenomena, we observe delayed generalization. Analyzing pre-training checkpoints before and after generalization shows that grammatical concept vectors become more predictive of grammatical acceptability and occupy a higher-dimensional subspace after generalization. We also find that attention from the critical token to the relevant context token is concentrated in a small number of heads.
Large Language Model (LLM) pretraining performance is jointly shaped by three components of the training triplet: the optimizer, model architecture, and training data stream. However, how these components influence performance in distinct ways remains unclear. We take a first step toward isolating their effects by studying relative generalization. We introduce Relative Generalization Invariance (RGI), the invariance of the validation-loss difference between any two tokens across models. We show that RGI approximately holds across a wide range of optimizers and moderate architectural variations, suggesting that these choices induce an approximately uniform shift in token-wise losses. In contrast, changing the training data stream can substantially alter relative generalization. We further show that RGI cannot be explained by the neural tangent kernel or mean-field regimes alone and prove that it can emerge in an overparameterized quadratic model. Overall, our work identifies RGI as a new phenomenon in LLM pretraining that helps distinguish the effects of optimizers and architectures from those of training data.
Fengzhuo Zhang, Shuche Wang, Shenggui Li +7
Yale University · National University of Singapore · Nanyang Technological University +2
The prevailing paradigm for training LLMs has evolved to rely on a massive post-training phase consisting of SFT and RL. In this position paper, we argue that this methodology effectively marks a reversion to the pre-train then fine-tune'' approach of the BERT era, explicitly tailoring models to the desired behaviors and specific benchmarks on which they are evaluated. We begin with a historical overview of LLMs, describing the different phases of the LLM evolution. We argue that the current landscape is remarkably similar to the early days of LLMs, where task performance heavily relied on fitting the models to in-distribution datasets. To empirically demonstrate this, we compare pre-trained models to randomly initialized ones, by fine-tuning both variants on modern reasoning datasets and evaluating them on competitive math and code benchmarks. We show that models post-trained from scratch yield highly non-trivial performance. Our findings suggest that current post-training methodologies function primarily as a distribution-fitting mechanism. We finish by positing that developing generally capable models and systems requires moving beyond extensive post-training for predefined behaviors, shifting instead toward training procedures where models learn how to learn''.
Michael Hassid, Yossi Adi, Roy Schwartz
1FAIR, Meta AI · 2The Hebrew University of Jerusalem