cs.CLOct 13, 2026
SaveFrom Copying to Composition: How Language Models Develop In-Context Learning
Organizations: Scale AI · University of Arizona
Abstract
In-context learning (ICL) allows language models to adapt their behavior from instructions and demonstrations in a prompt, but accuracy alone does not reveal how contextual information controls their predictions. We study the acquisition and trainability of ICL with a controlled diagnostic that separates copying a nearby demonstration from integrating information across the prompt. Across 17 models, about a third prefer the copied answer, a few are undecided, and the rest prefer the rule-consistent answer, which goes with higher accuracy and with control by the requested value rather than the closest conflicting example. Checkpoint analyses show that models begin prioritizing task-relevant prompt tokens over conflicting examples long before their accuracy improves beyond chance level. In matched training experiments, synthetic documents that define endings and then query them shift models toward compositional responses on the controlled diagnostic, whereas fine-tuning on definition exercises built from rewritten real documents improves accuracy when prompts reassign label meanings. The results suggest that ICL is not acquired as a single, uniform capability. Instead, prompt control, answer preference, construction without candidates, and transfer can develop along distinct trajectories.
Figures & tables
Figure 1: Study overview. (a) The task makes copying and applying prompt definitions predict different answers. (b) Across pretraining checkpoints (OLMo-2-7B shown), the requested value overtakes the closest ending as the source of the answer (star) long before accuracy crosses chance (dashed line). (c) Fine-tuning SmolLM2-1.7B on definition exercises rewritten from real documents raises accuracy on three unseen datasets with made-up or exchanged labels (pale bars: a matched fine-tune on the original documents, solid extensions: the gain from the exercises).
Figure 2: Copied-answer probability (left) and four-choice accuracy (right) for 17 models, grouped by the and thresholds; marker shape gives the family (full scores in Table 2 ).
Figure 3: Change in after blocking the closest ending or the requested value, model by model (all blocked spans: Appendix D ).
Figure 4: Blocking a single prompt span reverses the preferred answer: mean with the full prompt and after blocking each span, for high-copy Pythia-2.8B and low-copy Qwen3-8B.
Figure 5: Attention analysis. (a) The answer position reads two prompt spans, the conflicting ending of the closest example (favors the copied answer) and the requested value (favors the correct one). We block attention to a span, patch the requested-value state from a prompt asking for the other value, and read out the change in . Direct logit attribution picks the five heads pushing hardest toward each answer, which are silenced and compared with random sets. (b) Blocking a span moves far more than silencing either set (Figure 3 ).
Figure 6: Prompt control across pretraining checkpoints, shown for OLMo-2-7B, the most densely sampled series (other series: Appendix E ). The requested-value effect overtakes the closest-ending effect between 336B and 630B tokens (star). Four-choice accuracy crosses chance much later (dashed line; Table 1 ).
Figure 7: Continued pretraining of SmolLM2-1.7B. The right panels zoom on the first 50M tokens. A synthetic mixture reverses copying by the first checkpoint, whereas mined passages that merely contain definitions do not (Figure 16 ).
Figure 8: LoRA takes exact generation to or near ceiling on five bases, while the external SCAN–Elemental pairs stay outside the shaded quadrant where both would improve: the gain stays with the practiced form.
Figure 9: Four events that accuracy compresses into one, in training order: a prompt source gains control of the answer, the preferred candidate flips (the only event accuracy registers), the form is written without choices, and the learned skill transfers to held-out tasks (after the training of Section 6 ).
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| measure (threshold) | crossing [95% interval] |
|---|---|
| OLMo-2-7B | |
| copy the stem ( ) | 11B [10, 12] |
| first feature correct ( ) | 76B [66, 88] |
| second feature correct ( ) | 338B [254, 480] |
| shown combinations, exact generation ( ) | 224B [143, 293] |
| held-out, four-choice accuracy ( ) | 1,464B [1,270, 1,675] |
Table 1: Fitted threshold crossings. “Copy the stem” is the share of generations that begin with the requested novel stem, and “first/second feature correct” is the share of single-feature probes whose ending matches its prompt definition. Thresholds follow the first block unless stated. † Extrapolated beyond 5T tokens, with 8% of resamples censored. ‡ Intervals were not retained for these two fits. ∗ 77–100% of resamples are censored. The re-test items give a four-choice crossing of 1.79T for OLMo-2-7B (Appendix J ).
| generations | ||||||
|---|---|---|---|---|---|---|
| Model | ranks first | correct | copied | other | one ending | |
| Pythia 410M | .91 | .03 | .01 | .13 | .30 | .56 |
| Qwen2.5 0.5B | .86 | .01 | .00 | .07 | .18 | .75 |
| Pythia 6.9B | .83 | .03 | .01 | .35 | .30 | .34 |
| Pythia 12B | .80 | .07 | .03 | .47 | .16 | .33 |
| Pythia 1.4B | .75 | .05 | .01 | .19 | .42 | .38 |
Table 2: Final-checkpoint suffix-task scores (two item sets, 320 items). ; means equal probability. “Ranks first” is the share of prompts where the correct form leads four candidates (chance ) under summed log-probabilities; the length-normalized share differs by at most for every model. Generation columns omit malformed outputs. OLMo-2 7B is the released stage-2 parameter average (Appendix H ). Qwen2.5-Coder-1.5B ( ) and Coder-7B ( ) are omitted.
Figure 10: Prompt-variant controls. (a) and (b) the fraction of prompts where the correct form ranks first. Series are Pythia-410M, Pythia-1.4B, Pythia-2.8B, Qwen3-1.7B, and released OLMo-2-7B. Qwen3-8B and the OLMo-2-7B 336B and 3.9T checkpoints appear only for the reworded template. Variants reverse ending order (B–A), use one or three example lines per combination, retain only combined-form or single-feature lines, or reword the template. The shaded single-feature-only condition has no closest combined example. Guides mark equal copied/correct probability ( ) and four-choice chance ( ).
| closest example | question words | controls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Model | (unblocked) | ending | two lines | asked | other | unrelated | single-feature lines | their endings | [95% interval] | |
| Qwen2.5 0.5B ‡ | .88 | – | – | – | – | – | – | |||
| Pythia 12B | .84 | |||||||||
| Pythia 1.4B | .81 | |||||||||
| Pythia 2.8B | .79 | |||||||||
| OLMo-2 1B | .62 | † | † | † | ||||||
Table 3: Blocking selected prompt spans for the held-out form (item sets 0 and 1, 320 items, bfloat16; † one set, 160 items). The body and Figure 3 quote float32 values on item sets 2 and 3 (supplement table). (unblocked) is the full-prompt correct-minus-copied log-probability in nats; every other entry is the change in after blocking the named span. Closest example : only its ending tokens ( ending ) or its two whole lines ( two lines ). Question words : the word naming the requested value ( asked ) or the other value ( other ). Controls : two unrelated single-feature lines ( unrelated ), the two single-feature lines ( single-feature lines ) and only their ending tokens ( their endings ). is the magnitude for ending minus that for asked , with a 95% item-bootstrap interval. – marks a span not run (Qwen3 4B’s closest-ending block) or not tabulated here. ‡ Run added after the model set was fixed; float32 and bfloat16 agree to .02 nats for this model. Four of the 17 models of Table 2 (Pythia 410M and 6.9B, Qwen2.5 1.5B and 3B) were not run.
| Model | full | closest example | correct-ending lines | unrelated lines |
|---|---|---|---|---|
| Pythia 1.4B | ||||
| Pythia 2.8B | ||||
| Qwen3 0.6B | ||||
| OLMo-2 7B | ||||
| OLMo-2 13B | ||||
| Qwen3 1.7B |
Table 4: Deletion effects on item sets 0 and 1, 160 items each. Full is the unblocked correct-minus-copied log-probability in nats. Columns 3–5 give the change in after deleting the closest example’s two lines, the two single-feature lines defining the correct second ending, or two unrelated lines of matched length.
| Released model | correct ranks first (of 320) | writes correct | closest ending | source | ||
| Marin-8B-Base † | 283 (0.88) | .62 | .02 | Marin Community (2025) | ||
| Comma-v0.1-1T † | 277 (0.87) | .50 | .06 | Kandpal et al. (2025) | ||
| SmolLM3-3B † | 271 (0.85) | .54 | .10 | Hugging Face (2025) | ||
| OLMo-3-7B † | 265 (0.83) | .43 | .05 | Team Olmo (2026) | ||
| Apertus-8B-2509 | 136 (0.42) | .15 | .23 | Swiss AI Initiative (2025) | ||
| OpenLLaMA-7B-v2 † | 55 (0.17) | .06 | .55 | OpenLM Research (2023) |
Table 5: Screen of 16 released models (320 items), sorted by the number of items on which the correct form ranks first. Rules separate high (at least 160 correct), partial (Wilson lower bound above but fewer than 160 correct) and low (neither). “ closest ending” is the change in correct-minus-copied log-probability after blocking that ending, with its 95% interval; subtracts the magnitude after blocking the requested value. † The six runs followed across their saved intermediate models: the four high models and two contrast series.
Figure 11: Crossing dates and endpoints of the training trajectories. (a) First sampled crossings of , , and correct-ranks-first , with fitted accuracy crossings shown separately. “Never marks an unobserved sampled crossing. Arrows mark fitted crossings beyond the observed run, and Marin is already above threshold at its first public checkpoint. The numbered checks are a closest-ending effect below half its peak, an effect that stays below after , a requested-value effect that more than doubles, at most one sign change, a copied-answer probability crossing after the last positive , and a fitted accuracy crossing after the sign change. OpenLLaMA-7B-v2 is measured in optimizer steps: its first negative occurs at 460k, but it never crosses the probability or accuracy thresholds. (b) Final closest-ending effect divided by its peak versus final requested-value effect in nats. The groups distinguish runs meeting all ordering criteria, other above-chance runs, and below-chance runs. The vertical guide marks half the peak. Exact sampled and fitted dates are in Table 6 .
| Trajectory | correct ranks first | fitted accuracy threshold crossing | ||
|---|---|---|---|---|
| OLMo-2-7B | 630B | 630B | 1.7T | 1.79T |
| OLMo-2-13B | 2.5T | 336B | 2.5T | 2.2T |
| OLMo-2-1B | never | never | never | – |
| OLMo-7B (0424) | 838B | 838B | 2.0T | 1.6T |
| Pythia-1.4B | never | never | never | – |
| Pythia-2.8B | never | never | never | – |
Table 6: First sampled checkpoint crossing each threshold on 320 independently generated items. The fitted column is the logistic crossing for the fraction of prompts on which the correct form ranks first (Appendix B ); “not reached” marks a fitted crossing beyond the last observed checkpoint. : Marin-8B-Base already exceeds the threshold at its first public checkpoint. OLMo-2-7B’s fitted crossing is the re-test value, 1.79T (Appendix J ). A series may return after its first crossing. † OpenLLaMA positions are optimizer steps rather than tokens. The six ordering criteria per run are visible in Figure 11 .
Figure 12: Surface-form and tokenization checks. (a) and correct-ranks-first for the suffix scheme, code scheme, segmented Quechua, and Quechua in standard spelling. Six models have code results and ten have Quechua results. Guides mark and . The broad model ordering survives the code and segmented-Quechua versions, while standard spelling reduces accuracy in every model. (b) OLMo-2-7B in the code scheme: magnitudes of the changes in from blocking the closest example’s line or the requested value, with on the other axis. The blocking turn at 168B overlaps the fitted accuracy crossing near 143B (Table 1 ).
Figure 13: Activation patching after layer 4 at the requested-value word. The two series give the shares producing the complete form named by the replacement value when that target is never shown or already shown. Models are grouped into high-copy and low-copy and ordered by their never-shown share. Both conditions start from a shown base form.
| Model | [95% interval] | full prompt | closest ending | requested value | other value | unrelated lines | [95% interval] | predicted side | sign as predicted | unrelated 95% interval within nats |
| Pythia 1.4B | .81 [.77, .83] | ✓ | ✓ | |||||||
| Pythia 2.8B | .77 [.74, .79] | ✓ | ✓ | ✓ | ||||||
| Pythia 12B | .85 [.82, .87] | ✓ | ✓ | |||||||
| OLMo-2 1B | .57 [.53, .60] | – | – | – | ||||||
| Qwen3 0.6B | .53 [.50, .56] | – | – | – | ||||||
| OLMo-2 7B | .24 [.21, .27] | ✓ | ✓ | ✓ |
Table 7: Re-test on item sets 2 and 3 (320 items per model, bfloat16). and have 95% item-bootstrap intervals, resampling within each set. Blocking columns give changes in (nats); is the closest-ending change minus the magnitude of the requested-value change. The last columns test the predicted probability side, sign of , and unrelated-line interval. The two intermediate models have no directional prediction. The body quotes float32 re-scores of these same items: Pythia is lower by up to and by up to nats.
Figure 14: Direct contributions toward each set’s own answer, in nats, across OLMo-2-7B training, for the five heads selected for the copied answer and the five for the correct answer. Swapping selection and evaluation prompt sets moves the crossing to 1.68T. The star marks the crossing, inside the 336B–630B interval where whole-model changes sign.
Figure 15: SmolLM2-360M after about 1B additional tokens. Conditions: original documents, one-unshown and all-shown synthetic documents, and penalties on selected or random heads, two runs per condition as joined markers. (a) , (b) exact generation and correct-ranks-first, (c) code and Quechua exact generation, (d) selected-head and random-head attention to the closest ending, and (e) FineWeb-Edu and Pile validation loss. Both synthetic conditions cross and the accuracy criterion at step 32 (about 50M tokens). The other conditions never cross within 1B.
Figure 16: SmolLM2-1.7B after 1B additional tokens. Series are original documents, 3% synthetic documents, mined define-and-use passages, mined passages with the term absent from the reference vocabulary, and mined passages with a dictionary prefix. (a) , correct-ranks-first, exact generation, and code and Quechua exact generation. (b) Exact-match accuracy on SCAN, ElementalTask, 14-example SCAN, two operators, and word-to-category-to-action. No external-task change is significant. (c) Final . (d) Validation losses.
Figure 17: Exercise-training changes against original-text controls, in percentage points. The series are two SmolLM2-1.7B exercise fine-tunes, a SmolLM3-3B exercise fine-tune, and a SmolLM2-1.7B 15% exercise mixture. Rows separate made-up or exchanged labels on unseen datasets; prompt-defined labels and relations in seen formats; and farther tasks, including arithmetic, operators, List Functions, Unnatural In-Context Learning, SCAN, ElementalTask, bAbI and BIG-Bench Hard. Filled markers are significant after correction for multiple comparisons, hollow ones are not; the band marks the -point threshold. Grey DBpedia-7 results are excluded from 3B testing by the base-only qualification rule. The mixture control matches total tokens, not natural-text amount or repetition (Appendix P ).
| Modification type | Original document (excerpt) | Exercise built from it: definitions query answer |
|---|---|---|
| Symbols as labels for passages, the rule shown only by demonstrations (type b) | “… Import databases into GIS software. …” (document dd98ff1b3ceb) | “Passage: Import databases into GIS software. Label: VENTRIX” “Passage: Identify locations and spatial extent of major climate hazards. Label: HOLPREN” “What is the label for: ’Data sources for assessing conditions and trends.’?” HOLPREN |
| A stated mapping that contradicts the document’s own names (type c) | “… Clemens’s research is focused on social movements, organizations, and American pol …” (document 72d352b5830b) | “California is called Wisconsin, Wisconsin is called Washington, and Washington is called California.” “what is the renamed version of the state that the original document calls ’California’?” Wisconsin |
| A small invented vocabulary with grammar rules (type e) | “… The Brewarrina Aboriginal Cultural Museum stands at a site which has been of great importance to Aboriginal people …” (document a7c757380862) | “mara = museum” “kelu = story” “silka = respect” “adding the suffix -oz to a noun marks more than one of that thing (plural)” “Translate into Bralin: ’respect”’ silka |
| Private codes bound to the document’s entities (type g) | “… Printable PDF Worksheets …” (document 5875e15ba879) | “The engineering team referred to the printable worksheet generator internally as Rontavex.” “Which nickname is bound to the attribute describing syncing marked statuses back after a device switch?” Dufaline |
| A mapping where the shown answer is wrong and only the given rule is right (type h) | “… Ensuring your safety while camping should be a top priority. …” (document f99176a67baa) | “Input: campsite selection distance from water Output: RV3” “Input: wildlife encounter bear-proof containers Output: RV2” “Input: campsite selection near water signal Output:” RV3 |
Table 8: One real example per modification type: a passage from a real document (left) and the definition-use exercise a language model built from it (right; definitions, then the query and its answer), each verified by a second model before training.
More in this topic
In-Context Learning
In-context learning studies how models adapt to new tasks from examples or instructions supplied at inference time, without changing their weights.
See all 386 papers