A model trained on "The capital of X is Y" may produce "Y" after "The capital of X is" but fail after "The capital of X:." We call these different ways of eliciting the same fact request forms. To separate learning a fact from retrieving it, we train two models in two stages. In the first stage (request-form training), one model sees each fact in five forms and the other sees the same facts only as statements. In the second stage (target-fact training), both receive identical training on new facts, all as statements. Both then retrieve the new facts almost equally well from statements, but differ sharply on other request forms. Thus, a model can learn how to retrieve through a request form before it learns the facts. To understand this difference, we examine the hidden state immediately before the answer, which we call the context state. When given two different request forms for the same fact, the model trained on five forms in stage one produces more similar context states than the model trained on statements alone in that stage. Changing this state at retrieval time can enable or prevent retrieval of an already learned fact, and the same effect transfers across facts and factual relations, such as capitals and currencies. To test its role during learning, we change the context state only during target-fact training. This intervention changes later retrieval without intervention at test time. Together, these results show that later retrieval depends on earlier request-form experience and the context state during fact learning.
Figures & tables
Figure 1: Learning a fact and learning how to retrieve it are separate, and the context state links them. (A) Identical target-fact training after different stage-one training: held-out retrieval goes from 0.0% to 97.7% . (B) Swapping the context state between a trained and a held-out form removes or restores retrieval. (C) Shifting the context state during training changes retrieval through a never-used form afterward, with no intervention at test time.
Stage-one training form
Frame
Declarative
The capital of {c} is {y}.
Key–value
Capital of {c}: {y}.
Question
Q: What is the capital of {c}? A: {y}.
Dialogue
User: What is the capital of {c}?\nAssistant: {y}.
Imperative
Name the capital of {c}: {y}.
Evaluation form
Frame
Table 1: Request forms. The five forms used in stage-one training, and the forms used to evaluate retrieval of the target facts. No evaluation form other than the declarative training form appears in either training stage of the two-stage experiment.
Declarative form
List request form
Colon-ended declarative form
Relation
Statement-only
Five-form
Statement-only
Five-form
Statement-only
Five-form
Capital
0.983
0.999
0.002
0.970
0.200
0.996
Currency
0.995
0.999
0.004
0.989
0.388
0.999
Population
0.991
0.986
0.038
0.991
0.276
0.994
Founder
0.992
0.992
0.001
0.989
0.299
0.998
Birth year
0.987
0.986
0.344
0.982
0.522
0.989
Table 2: The same facts are learned but differ sharply in how they can be retrieved. The five relations are learned together in one training run. Each relation is evaluated on its 200 target facts. For every relation, both histories receive identical target-fact training in the declarative training form. Values are exact-generation accuracy. The statement-only values for birth years ( 0.344 and 0.522 ) are higher than for the other relations: year answers come more readily after a colon.
Figure 2: The retrieval difference extends across request forms and emerges during fact learning. (a) Both models retrieve the target facts almost perfectly from the declarative training form, but differ sharply across many other request forms. (b) During target-fact training, the gap between declarative and list retrieval (the difference in pool top-1 accuracy) grows in the statement-only model, while remaining near zero after earlier five-form training. Both models receive identical target-fact training. Panel (a) reports exact generation; panel (b) reports pool top-1.
Figure 3: Retrieval improves with request-form exposure. Exact-generation accuracy on the list form, for target facts written only as statements; the Pythia-1.4B point is at 20% .
Earlier training
Mean
Request form
List
Colon-ended declarative
Statements only
0.012
0.000
0.024
Correct unrelated capitals
0.985
0.977
0.993
Incorrect capital answers
0.681
0.639
0.724
Currencies
0.944
0.900
0.987
Table 3: Earlier request-form training transfers to new facts. All models receive identical declarative training on the target capital facts. Mean exact-generation accuracy across the two request forms rises substantially after earlier request-form training, even when the earlier answers are incorrect or come from a different factual relation. In the incorrect-answer condition, every country is paired with another country’s capital. The incorrect-answer corpus uses the key–value form; the currency corpus uses key–value and declarative lines; neither is token-matched to the five-form corpus.
Request form
No intervention
+ Direction
List
0.000
0.921
Colon-ended declarative
0.024
0.983
Table 4: The same direction supports retrieval across request forms and factual domains. (a) Exact-generation accuracy before and after adding the direction to the context state in the capital task. Each form’s direction is scaled to that form’s restoring-vector norm. (b) Directions derived from capitals and birth years transfer almost unchanged to the other factual domain, each built in that domain’s own model.
Figure 4: Earlier request-form training keeps request-form context states aligned with the declarative one. At each layer, centered cosine similarity across target facts between the declarative context state and the context state of the list form; layer 24 is taken after the final layer norm. The two training conditions begin alike and separate in later layers in both models.
Model
Facts
No intervention
Random shift
Targeted shift
Pythia-410M
Capitals
0.024
0.100
0.943
Pythia-1.4B
Capitals
0.266
0.332
0.904
Qwen2.5-1.5B
Capitals
0.482
0.722
0.993
Pythia-410M
Birth years
0.246
0.445
0.983
Table 5: Changing the context state during learning changes future retrieval. The factual training examples are identical across conditions; only the context state present while each fact is learned is changed. Values are exact-generation accuracy on the colon-ended declarative form, whose direction was never used in the shift; details are in Appendix I .
Human reasoning depends on how objects are related within propositions. How do relations organize the language representations of contextual contents? We give an LLM a list of facts in its context (e.g., Alice eats an apple. Bob eats a pear.) and measure how its hidden state changes when the question switches from what Alice eats to what Bob eats. Averaged over many lists, this change is a steering vector, which we call the ordinal vector. It points to a fact by its order of mention, the order in which the facts were stated in the context. We find that LLMs represent the fact a question asks about by its order of mention, not by the name the question contains. We state this as the ordinal addressing hypothesis: each order of mention has a fact address in the model's state, shared by all contexts, and a question moves the state to the fact address of the fact it asks about, while the context supplies what that fact says. Across Qwen, Gemma, and Llama, fact addresses are (1) ordered by mention: query states are organized by the order of facts, not of names, even when one fact has multiple subjects; (2) steerable: added to a question about the first fact of a new list, the ordinal vector makes the model answer with the second fact of that list; (3) low-rank: they span a low-rank subspace in which the first-mentioned fact is the easiest to reach, surprisingly similar to human recall; and (4) emergent: they are shared in late-middle layers, hold from 1.5B to 32B parameters, and form early in pretraining. Language models reach a stated fact by where it was mentioned, deepening our understanding of LLM reasoning.
In-context learning \ -- performing tasks based on examples given in the prompt \ -- is an important capability that has emerged in large language models and has received significant attention in both theory and practice. Existing theoretical work often focuses on settings where the learning uses information purely from the prompt. However, many practical instances of in-context learning require the model to retrieve factual knowledge stored in the model's parameters, with the context serving to identify which knowledge is relevant. In this work, we study how in-context learning leverages factual knowledge recall. We formalize this behavior by introducing the in-context factual recall (IC-recall) task, where a transformer is provided a context of (subject, answer) pairs generated from a hidden relation, along with a query subject, and must both infer this hidden relation and retrieve the corresponding answer. Factual knowledge is modeled by the transformer having access to a simple pre-constructed MLP associative memory storing (subject, relation, answer) triplets. We analyze the supervised fine-tuning dynamics of a one-layer transformer on IC-recall data and prove that the model successfully performs IC-recall by converging to a particular pairwise attention pattern. This fine-tuning stage requires a very small number of samples \ -- only polylogarithmic in the number of stored knowledge triplets. Experiments verify our theoretical predictions and show that the pairwise attention pattern emerges even when the MLP layer is pretrained instead of constructed.
Ruomin Huang, Eshaan Nichani, Jason D. Lee +1
Duke University · Princeton University · UC Berkeley
Continual learning is a long-standing capability gap between LLMs and humans. Writing new knowledge into a model's weights routinely causes it to forget old knowledge, commonly denoted as "catastrophic forgetting". Various modifications of supervised fine-tuning and distillation aim to mitigate catastrophic forgetting, but quantifying what (or how much) information was forgotten is often difficult. In this paper, we study whether current methods of writing knowledge into weights enable models to learn continually without forgetting. We introduce a framework for studying continual learning in the iterative regime, writing invented facts one at a time into a Qwen3 model already modified by previous writes, and varying the training data, method, and parameter update. Across SFT and off- and on-policy distillation, using LoRA or full fine-tuning, we compare repeated statement training (the same fact repeated in two formats) with varied example training (24 factual restatements) and find that varied examples comprehensively support more flexible use. After twenty sequential writes and merges, the model answers only 1% of questions about earlier facts correctly when every write uses repeated statements, compared with 46% when every write uses varied examples. We additionally show that this retention depends on the data used for the later writes, regardless of training method or parameter update, and that behavioral forgetting of an earlier fact does not erase its presence from the log-probabilities. Together, our framework neatly provides a comparison of performance across training data, training regimes, and parameter update schemes in an iterative learning task.