From Copying to Composition: How Language Models Develop In-Context Learning

Minglai Yang; Xinyu Guo; Renxiong Wang; Sunjie Hou; Daniel Yue Zhang; Yunzhong He

cs.CLOct 13, 2026

From Copying to Composition: How Language Models Develop In-Context Learning

Authors: Minglai Yang, Xinyu Guo, Renxiong Wang, Sunjie Hou, Daniel Yue Zhang, Yunzhong He

Organizations: Scale AI · University of Arizona

Abstract

In-context learning (ICL) allows language models to adapt their behavior from instructions and demonstrations in a prompt, but accuracy alone does not reveal how contextual information controls their predictions. We study the acquisition and trainability of ICL with a controlled diagnostic that separates copying a nearby demonstration from integrating information across the prompt. Across 17 models, about a third prefer the copied answer, a few are undecided, and the rest prefer the rule-consistent answer, which goes with higher accuracy and with control by the requested value rather than the closest conflicting example. Checkpoint analyses show that models begin prioritizing task-relevant prompt tokens over conflicting examples long before their accuracy improves beyond chance level. In matched training experiments, synthetic documents that define endings and then query them shift models toward compositional responses on the controlled diagnostic, whereas fine-tuning on definition exercises built from rewritten real documents improves accuracy when prompts reassign label meanings. The results suggest that ICL is not acquired as a single, uniform capability. Instead, prompt control, answer preference, construction without candidates, and transfer can develop along distinct trajectories.

Figures & tables

Figure 1: Study overview. (a) The task makes copying and applying prompt definitions predict different answers. (b) Across pretraining checkpoints (OLMo-2-7B shown), the requested value overtakes the closest ending as the source of the answer (star) long before accuracy crosses chance (dashed line). (c) Fine-tuning SmolLM2-1.7B on definition exercises rewritten from real documents raises accuracy on three unseen datasets with made-up or exchanged labels (pale bars: a matched fine-tune on the original documents, solid extensions: the gain from the exercises).

Figure 2: Copied-answer probability (left) and four-choice accuracy (right) for 17 models, grouped by the $.70$ and $.30$ thresholds; marker shape gives the family (full scores in Table 2 ).

Figure 3: Change in $m$ after blocking the closest ending or the requested value, model by model (all blocked spans: Appendix D ).

Figure 4: Blocking a single prompt span reverses the preferred answer: mean $m$ with the full prompt and after blocking each span, for high-copy Pythia-2.8B and low-copy Qwen3-8B.

Figure 5: Attention analysis. (a) The answer position reads two prompt spans, the conflicting ending of the closest example (favors the copied answer) and the requested value (favors the correct one). We block attention to a span, patch the requested-value state from a prompt asking for the other value, and read out the change in

m

. Direct logit attribution picks the five heads pushing hardest toward each answer, which are silenced and compared with random sets. (b) Blocking a span moves

m

far more than silencing either set (Figure 3 ).

Figure 6: Prompt control across pretraining checkpoints, shown for OLMo-2-7B, the most densely sampled series (other series: Appendix E ). The requested-value effect overtakes the closest-ending effect between 336B and 630B tokens (star). Four-choice accuracy crosses chance much later (dashed line; Table 1 ).

Figure 7: Continued pretraining of SmolLM2-1.7B. The right panels zoom on the first 50M tokens. A

3\,\%

synthetic mixture reverses copying by the first checkpoint, whereas mined passages that merely contain definitions do not (Figure 16 ).

Figure 8: LoRA takes exact generation to or near ceiling on five bases, while the external SCAN–Elemental pairs stay outside the shaded quadrant where both would improve: the gain stays with the practiced form.

Figure 9: Four events that accuracy compresses into one, in training order: a prompt source gains control of the answer, the preferred candidate flips (the only event accuracy registers), the form is written without choices, and the learned skill transfers to held-out tasks (after the training of Section 6 ).

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

OLMo-2-7B
measure (threshold)	crossing [95% interval]
copy the stem ( $\geq.85$ )	11B [10, 12]
first feature correct ( $\geq.70$ )	76B [66, 88]
second feature correct ( $\geq.70$ )	338B [254, 480]
shown combinations, exact generation ( $\geq.50$ )	224B [143, 293]
held-out, four-choice accuracy ( $>.25$ )	1,464B [1,270, 1,675]

Appendix

Table 1: Fitted threshold crossings. “Copy the stem” is the share of generations that begin with the requested novel stem, and “first/second feature correct” is the share of single-feature probes whose ending matches its prompt definition. Thresholds follow the first block unless stated. † Extrapolated beyond 5T tokens, with 8% of resamples censored. ‡ Intervals were not retained for these two fits. ∗ 77–100% of resamples are censored. The re-test items give a four-choice crossing of 1.79T for OLMo-2-7B (Appendix J ).

			generations
Model	$P(\mathrm{copy})$	ranks first	correct	copied	other	one ending
Pythia 410M	.91	.03	.01	.13	.30	.56
Qwen2.5 0.5B	.86	.01	.00	.07	.18	.75
Pythia 6.9B	.83	.03	.01	.35	.30	.34
Pythia 12B	.80	.07	.03	.47	.16	.33
Pythia 1.4B	.75	.05	.01	.19	.42	.38

Appendix

Table 2: Final-checkpoint suffix-task scores (two item sets, 320 items).

P(\mathrm{copy})=p_{\mathrm{copied}}/(p_{\mathrm{copied}}+p_{\mathrm{correct}})

;

.5

means equal probability. “Ranks first” is the share of prompts where the correct form leads four candidates (chance

.25

) under summed log-probabilities; the length-normalized share differs by at most

.02

for every model. Generation columns omit malformed outputs. OLMo-2 7B is the released stage-2 parameter average (Appendix H ). Qwen2.5-Coder-1.5B (

P(\mathrm{copy})=.42

) and Coder-7B (

.02

) are omitted.

Figure 10: Prompt-variant controls. (a)

P(\mathrm{copy})

and (b) the fraction of prompts where the correct form ranks first. Series are Pythia-410M, Pythia-1.4B, Pythia-2.8B, Qwen3-1.7B, and released OLMo-2-7B. Qwen3-8B and the OLMo-2-7B 336B and 3.9T checkpoints appear only for the reworded template. Variants reverse ending order (B–A), use one or three example lines per combination, retain only combined-form or single-feature lines, or reword the template. The shaded single-feature-only condition has no closest combined example. Guides mark equal copied/correct probability (

.5

) and four-choice chance (

.25

Model	$P(\mathrm{copy})$	$m$ (unblocked)	ending	two lines	asked	other	unrelated	single-feature lines	their endings	$D$ [95% interval]
			closest example		question words		controls
Qwen2.5 0.5B ‡	.88	–	$+3.25$	–	$-0.66$	–	–	–	–	$+2.59$ $[+2.34,+2.85]$
Pythia 12B	.84	$-3.22$	$+5.34$	$+4.87$	$-1.59$	$+1.81$	$-0.00$	$-1.99$	$-2.38$	$+3.75$ $[+3.37,+4.09]$
Pythia 1.4B	.81	$-2.71$	$+5.34$	$+4.72$	$-1.90$	$+1.26$	$+0.13$	$-1.90$	$-2.33$	$+3.44$ $[+3.05,+3.82]$
Pythia 2.8B	.79	$-2.15$	$+4.72$	$+4.28$	$-1.95$	$+1.13$	$+0.18$	$-1.99$	$-2.60$	$+2.77$ $[+2.50,+3.04]$
OLMo-2 1B	.62	$-0.58$	$+2.94$ †	$+3.59$	$-3.18$	$+0.89$ †	$+0.17$	$-1.67$	$-1.93$ †	$-0.23$ $[-0.74,+0.29]$

Appendix

Table 3: Blocking selected prompt spans for the held-out form (item sets 0 and 1, 320 items, bfloat16; † one set, 160 items). The body and Figure 3 quote float32 values on item sets 2 and 3 (supplement table).

m

(unblocked) is the full-prompt correct-minus-copied log-probability in nats; every other entry is the change in

m

after blocking the named span. Closest example : only its ending tokens ( ending ) or its two whole lines ( two lines ). Question words : the word naming the requested value ( asked ) or the other value ( other ). Controls : two unrelated single-feature lines ( unrelated ), the two single-feature lines ( single-feature lines ) and only their ending tokens ( their endings ).

D

is the magnitude for ending minus that for asked , with a 95% item-bootstrap interval. – marks a span not run (Qwen3 4B’s closest-ending block) or not tabulated here. ‡ Run added after the model set was fixed; float32 and bfloat16 agree to .02 nats for this model. Four of the 17 models of Table 2 (Pythia 410M and 6.9B, Qwen2.5 1.5B and 3B) were not run.

Model	full $m$	closest example	correct-ending lines	unrelated lines
Pythia 1.4B	$-2.08$	$+3.20$	$-0.53$	$-0.13$
Pythia 2.8B	$-1.92$	$+3.30$	$-0.92$	$+0.05$
Qwen3 0.6B	$-0.19$	$+3.52$	$-1.28$	$-0.40$
OLMo-2 7B	$+2.31$	$+1.66$	$-1.18$	$-0.49$
OLMo-2 13B	$+3.53$	$+1.59$	$-0.95$	$-0.77$
Qwen3 1.7B	$+3.87$	$+1.51$	$-1.36$	$-0.44$

Appendix

Table 4: Deletion effects on item sets 0 and 1, 160 items each. Full

m

is the unblocked correct-minus-copied log-probability in nats. Columns 3–5 give the change in

m

after deleting the closest example’s two lines, the two single-feature lines defining the correct second ending, or two unrelated lines of matched length.

Released model	correct ranks first (of 320)	writes correct	$P(\mathrm{copy})$	$\Delta$ closest ending	$D$	source
Marin-8B-Base †	283 (0.88)	.62	.02	$+1.52$ $[1.31,1.73]$	$-6.19$	Marin Community (2025)
Comma-v0.1-1T †	277 (0.87)	.50	.06	$+1.46$ $[1.23,1.69]$	$-6.40$	Kandpal et al. (2025)
SmolLM3-3B †	271 (0.85)	.54	.10	$+1.74$ $[1.51,1.96]$	$-5.10$	Hugging Face (2025)
OLMo-3-7B †	265 (0.83)	.43	.05	$+1.63$ $[1.40,1.85]$	$-6.24$	Team Olmo (2026)
Apertus-8B-2509	136 (0.42)	.15	.23	$+2.13$ $[1.91,2.36]$	$-3.56$	Swiss AI Initiative (2025)
OpenLLaMA-7B-v2 †	55 (0.17)	.06	.55	$+2.98$ $[2.78,3.18]$	$-0.20$	OpenLM Research (2023)

Appendix

Table 5: Screen of 16 released models (320 items), sorted by the number of items on which the correct form ranks first. Rules separate high (at least 160 correct), partial (Wilson lower bound above

.25

but fewer than 160 correct) and low (neither). “

\Delta

closest ending” is the change in correct-minus-copied log-probability after blocking that ending, with its 95% interval;

D

subtracts the magnitude after blocking the requested value. † The six runs followed across their saved intermediate models: the four high models and two contrast series.

Figure 11: Crossing dates and endpoints of the training trajectories. (a) First sampled crossings of

P(\mathrm{copy})<.5

D<0

, and correct-ranks-first

>.25

, with fitted accuracy crossings shown separately. “Never marks an unobserved sampled crossing. Arrows mark fitted crossings beyond the observed run, and Marin is already above threshold at its first public checkpoint. The numbered checks are a closest-ending effect below half its peak, an effect that stays below

.8

after

D<0

, a requested-value effect that more than doubles, at most one sign change, a copied-answer probability crossing after the last positive

D

, and a fitted accuracy crossing after the sign change. OpenLLaMA-7B-v2 is measured in optimizer steps: its first negative

D

occurs at 460k, but it never crosses the probability or accuracy thresholds. (b) Final closest-ending effect divided by its peak versus final requested-value effect in nats. The groups distinguish runs meeting all ordering criteria, other above-chance runs, and below-chance runs. The vertical guide marks half the peak. Exact sampled and fitted dates are in Table 6 .

Trajectory	$P(\mathrm{copy})<.5$	$D<0$	correct ranks first $>.25$	fitted accuracy threshold crossing
OLMo-2-7B	630B	630B	1.7T	1.79T
OLMo-2-13B	2.5T	336B	2.5T	2.2T
OLMo-2-1B	never	never	never	–
OLMo-7B (0424)	838B	838B	2.0T	1.6T
Pythia-1.4B	never	never	never	–
Pythia-2.8B	never	never	never	–

Appendix

Table 6: First sampled checkpoint crossing each threshold on 320 independently generated items. The fitted column is the logistic crossing for the fraction of prompts on which the correct form ranks first (Appendix B ); “not reached” marks a fitted crossing beyond the last observed checkpoint.

\leq

: Marin-8B-Base already exceeds the threshold at its first public checkpoint. OLMo-2-7B’s fitted crossing is the re-test value, 1.79T (Appendix J ). A series may return after its first crossing. † OpenLLaMA positions are optimizer steps rather than tokens. The six ordering criteria per run are visible in Figure 11 .

Figure 12: Surface-form and tokenization checks. (a)

P(\mathrm{copy})

and correct-ranks-first for the suffix scheme, code scheme, segmented Quechua, and Quechua in standard spelling. Six models have code results and ten have Quechua results. Guides mark

.5

and

.25

. The broad model ordering survives the code and segmented-Quechua versions, while standard spelling reduces accuracy in every model. (b) OLMo-2-7B in the code scheme: magnitudes of the changes in

m

from blocking the closest example’s line or the requested value, with

P(\mathrm{copy})

on the other axis. The blocking turn at 168B overlaps the fitted accuracy crossing near 143B (Table 1 ).

Figure 13: Activation patching after layer 4 at the requested-value word. The two series give the shares producing the complete form named by the replacement value when that target is never shown or already shown. Models are grouped into high-copy and low-copy and ordered by their never-shown share. Both conditions start from a shown base form.

Model	$P(\mathrm{copy})$ [95% interval]	full prompt	closest ending	requested value	other value	unrelated lines	$D$ [95% interval]	$P(\mathrm{copy})$ predicted side $\geq.70/\leq.30$	$D$ sign as predicted	unrelated 95% interval within $\pm.3$ nats
Pythia 1.4B	.81 [.77, .83]	$-2.73$	$+5.33$	$-2.06$	$+1.47$	$+0.24$	$+3.27$ $[+2.86,+3.68]$	✓	✓	$\times$
Pythia 2.8B	.77 [.74, .79]	$-2.08$	$+4.63$	$-2.26$	$+1.05$	$+0.16$	$+2.38$ $[+2.10,+2.64]$	✓	✓	✓
Pythia 12B	.85 [.82, .87]	$-3.20$	$+5.35$	$-1.63$	$+1.83$	$+0.17$	$+3.72$ $[+3.32,+4.08]$	✓	✓	$\times$
OLMo-2 1B	.57 [.53, .60]	$-0.43$	$+2.80$	$-3.40$	$+1.00$	$+0.11$	$-0.60$ $[-0.98,-0.23]$	–	–	–
Qwen3 0.6B	.53 [.50, .56]	$-0.15$	$+2.88$	$-4.39$	$+0.78$	$+0.04$	$-1.52$ $[-1.85,-1.17]$	–	–	–
OLMo-2 7B	.24 [.21, .27]	$+2.54$	$+2.08$	$-5.54$	$-0.77$	$+0.10$	$-3.46$ $[-3.93,-3.02]$	✓	✓	✓

Appendix

Table 7: Re-test on item sets 2 and 3 (320 items per model, bfloat16).

P(\mathrm{copy})

and

D

have 95% item-bootstrap intervals, resampling within each set. Blocking columns give changes in

m

(nats);

D

is the closest-ending change minus the magnitude of the requested-value change. The last columns test the predicted probability side, sign of

D

, and unrelated-line interval. The two intermediate models have no directional prediction. The body quotes float32 re-scores of these same items: Pythia

P(\mathrm{copy})

is lower by up to

.06

and

D

by up to

1.8

nats.

Figure 15: SmolLM2-360M after about 1B additional tokens. Conditions: original documents, one-unshown and all-shown synthetic documents, and penalties on selected or random heads, two runs per condition as joined markers. (a)

P(\mathrm{copy})

, (b) exact generation and correct-ranks-first, (c) code and Quechua exact generation, (d) selected-head and random-head attention to the closest ending, and (e) FineWeb-Edu and Pile validation loss. Both synthetic conditions cross

D<0

and the accuracy criterion at step 32 (about 50M tokens). The other conditions never cross within 1B.

Figure 16: SmolLM2-1.7B after 1B additional tokens. Series are original documents, 3% synthetic documents, mined define-and-use passages, mined passages with the term absent from the reference vocabulary, and mined passages with a dictionary prefix. (a)

P(\mathrm{copy})

, correct-ranks-first, exact generation, and code and Quechua exact generation. (b) Exact-match accuracy on SCAN, ElementalTask, 14-example SCAN, two operators, and word-to-category-to-action. No external-task change is significant. (c) Final

D

. (d) Validation losses.

Figure 17: Exercise-training changes against original-text controls, in percentage points. The series are two SmolLM2-1.7B exercise fine-tunes, a SmolLM3-3B exercise fine-tune, and a SmolLM2-1.7B 15% exercise mixture. Rows separate made-up or exchanged labels on unseen datasets; prompt-defined labels and relations in seen formats; and farther tasks, including arithmetic, operators, List Functions, Unnatural In-Context Learning, SCAN, ElementalTask, bAbI and BIG-Bench Hard. Filled markers are significant after correction for multiple comparisons, hollow ones are not; the band marks the

\pm 1.5

-point threshold. Grey DBpedia-7 results are excluded from 3B testing by the base-only qualification rule. The mixture control matches total tokens, not natural-text amount or repetition (Appendix P ).

Modification type	Original document (excerpt)	Exercise built from it: definitions $\rightarrow$ query $\rightarrow$ answer
Symbols as labels for passages, the rule shown only by demonstrations (type b)	“… Import databases into GIS software. …” (document dd98ff1b3ceb)	“Passage: Import databases into GIS software. Label: VENTRIX” “Passage: Identify locations and spatial extent of major climate hazards. Label: HOLPREN” $\rightarrow$ “What is the label for: ’Data sources for assessing conditions and trends.’?” $\rightarrow$ HOLPREN
A stated mapping that contradicts the document’s own names (type c)	“… Clemens’s research is focused on social movements, organizations, and American pol …” (document 72d352b5830b)	“California is called Wisconsin, Wisconsin is called Washington, and Washington is called California.” $\rightarrow$ “what is the renamed version of the state that the original document calls ’California’?” $\rightarrow$ Wisconsin
A small invented vocabulary with grammar rules (type e)	“… The Brewarrina Aboriginal Cultural Museum stands at a site which has been of great importance to Aboriginal people …” (document a7c757380862)	“mara = museum” “kelu = story” “silka = respect” “adding the suffix -oz to a noun marks more than one of that thing (plural)” $\rightarrow$ “Translate into Bralin: ’respect”’ $\rightarrow$ silka
Private codes bound to the document’s entities (type g)	“… Printable PDF Worksheets …” (document 5875e15ba879)	“The engineering team referred to the printable worksheet generator internally as Rontavex.” $\rightarrow$ “Which nickname is bound to the attribute describing syncing marked statuses back after a device switch?” $\rightarrow$ Dufaline
A mapping where the shown answer is wrong and only the given rule is right (type h)	“… Ensuring your safety while camping should be a top priority. …” (document f99176a67baa)	“Input: campsite selection distance from water $\rightarrow$ Output: RV3” “Input: wildlife encounter bear-proof containers $\rightarrow$ Output: RV2” $\rightarrow$ “Input: campsite selection near water signal $\rightarrow$ Output:” $\rightarrow$ RV3

Appendix

Table 8: One real example per modification type: a passage from a real document (left) and the definition-use exercise a language model built from it (right; definitions, then the query and its answer), each verified by a second model before training.

In-Context Learning

In-context learning studies how models adapt to new tasks from examples or instructions supplied at inference time, without changing their weights.

See all 386 papers

Sep 1, 2026

Compile, Don't Memorize: A Context Compilation Architecture (CCA) for In-Context Learning

Large language models (LLMs) increasingly handle in-context learning (ICL) tasks where a long, novel context defines the rules, knowledge, and output schema for a series of questions. On benchmarks that grade against every detail of the context, even strong open-weights models pass only 12-16% of tasks: a single overlooked rule fails the whole response. We argue this brittleness is structural: the dominant "read-and-reason" paradigm asks the model to extract, plan, generate, and self-verify in one forward pass. We therefore ask whether explicit context compilation can fix it, how it compares to existing long-context strategies (gist retrieval, multi-agent self-play), and where the resulting harness benefit holds across task structure and model scale. We propose the Context Compilation Architecture (CCA), whose central novelty is a typed intermediate representation (IR) with fixed slots (rules.{must_do, must_not, conditional}, output_spec, available_tools, data_profile) into which any prose context is compiled once; executable verifiers and a violation-gated correction loop follow as downstream consequences. On CL-bench (1,899 tasks across 4 open base models), CCA outperforms vanilla prompting and two long-context baselines (ReadAgent-P, Ctx2Skill) on every base model, lifting Kimi K2.5 from 15.4% to 21.4% with gains concentrated on rule-dense sub-categories. Code and cached completions are available at https://github.com/TonyQJH/cca-emnlp2026.

Jinhu Qi, Minda Hu, Wentao Zhang +4

The Chinese University of Hong Kong · Macao Polytechnic University · Xi’an Jiaotong University +1

Instruction Following In-Context Learning LLM Context Management

Ask PDF

Oct 12, 2026

Label Collapse in Diffusion Language Model In-Context Learning

In-context learning (ICL) applies large language models to new tasks by prepending a few labeled demonstrations to a query. Prior work has studied how demonstration ordering affects ICL, but the interaction between label arrangement patterns and model biases has not been systematically examined. We define label collapse as the failure mode in which the model generates query-independent labels, and show that it is architecturally universal at the tested 8-9B scale: five models spanning autoregressive (Llama-3, Qwen3.5) and discrete diffusion (LLaDA, iLLaDA, LLaDA-Instruct) families all exhibit label collapse even under the class-balanced alternating arrangement, with some label word pairs collapsing in virtually every tested condition and others flipping their collapse direction with arrangement. Critically, accuracy alone cannot detect this collapse: models can achieve near-chance accuracy while generating query-independent labels, a score indistinguishable from balanced performance, so the failure can persist undetected in deployment. We measure separation, the log-odds of the two label words at the query position (the quantity that label-bias calibration operates on), as a complementary diagnostic that exposes collapse hidden beneath surface accuracy, together with a measurement protocol that guards label extraction against tokenizer and formatting artifacts. For autoregressive models, collapse persists where the majority-label-bias account does not apply, and arrangement patterns can reverse the collapse direction; for diffusion models, we present a systematic evaluation of ICL failure modes, complementing contemporaneous studies of dLLM context comprehension and demonstration-order sensitivity, and identify open questions about how iterative denoising interacts with label biases.

Ziye Guo, Ye Yuan

University of Connecticut · 2McGill University

In-Context Learning LLM Evaluation Diffusion Language Models

Ask PDF

Jun 27, 2026

A Theoretical Interpretation of In-Context Learning via Probabilistic Modeling

In-context learning (ICL) is an emerging paradigm that employs the semantic information inherent in large language models (LLMs) for generating answers to user queries. While the remarkable performance of ICL has been widely known, a general modeling and a rigorous theoretical analysis of this paradigm are still lacking. This work presents a probabilistic model for ICL and derives the performance of ICL for both general parametric distributions and exponential families. Based on the derived results, the work explains the impact of multiple factors such as the number of demonstrations, the sensitivity of the probabilistic model to the variation of its parameters, as well as the similarity between the demonstrations and the query on the performance of ICL.

Zhenyu Liu, Huaze Tang, Shao-Lun Huang

Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China

In-Context Learning Exponential Family

Ask PDF

Abstract

In-Context Learning

Explore similar work

Compile, Don't Memorize: A Context Compilation Architecture (CCA) for In-Context Learning

Label Collapse in Diffusion Language Model In-Context Learning

A Theoretical Interpretation of In-Context Learning via Probabilistic Modeling