Organizations: Arizona State University, Tempe, AZ, United States · University of Maryland, College Park, MD, United States · Morgan Stanley, New York, NY, United States
At the core of modern prompting techniques is contextual sensitivity, the ability of large language models to adapt their predictions based on inference-time context. Despite its central role, inference behavior under strong contextual influence remains poorly understood, particularly at the level of internal inference dynamics. We introduce a theoretical framework for analyzing contextual influence through inference dynamics, enabling quantitative characterization of inference behavior beyond output-level answer changes. Our analysis shows that inference dynamics do not exhibit unbounded drift under repeated contextual assertions. Instead, predictive representations converge to stable, query-dependent regimes that fundamentally constrain whether contextual signals can alter a model's prediction. This leads to a surprising finding: Repeated contextual assertions do not act as accumulating evidence during inference and may therefore fail to alter a model's prediction even under unbounded repetition, while in other cases a prediction change becomes inevitable. We empirically validate our theoretical predictions, demonstrating strong alignment between theory and observed inference behavior. These contributions offer a principled pathway toward characterizing the limits of contextual influence during inference, providing practical implications for model development.
Figures & tables
Figure 1: Heterogeneous effects of contextual repetition on model predictions. Illustrative examples showing qualitatively distinct behaviors of LLMs under repeated contextual assertions, including immediate answer flips, delayed changes, early saturation, and complete invariance to repetition.
Model
Dataset
Openbook QA
MINTAKA
Simple QA
0.1
0.05
0.01
0.1
0.05
0.01
0.1
0.05
0.01
Attn
FFN
Attn
FFN
Attn
FFN
Attn
FFN
Attn
FFN
Attn
FFN
Attn
FFN
Attn
FFN
Attn
FFN
Falcon3-7B-Base
98.1
97.3
95.3
94.3
81.5
81.3
98.7
99.0
96.6
97.4
78.6
80.7
100
99.9
99.9
99.4
84.7
86.3
Mistral-7B-v0.1
97.5
97.5
93.9
96.0
78.9
90.9
94.0
93.7
89.1
89.1
66.3
68.7
95.5
94.9
92.9
91.5
72.8
73.2
Apollo-1-4B
99.0
99.3
95.5
96.3
79.7
78.5
99.8
99.9
98.7
99.4
87.0
91.4
100
100
99.9
99.9
93.0
93.9
Table 1: Layer-wise convergence of inference dynamics at N=1000 on three benchmarks at various divergence levels ( 0.1,0.05,0.01 ).
Figure 2: Comparison between representation-level predicted answer shifts and forward-computed output-level shifts across multiple models on the Openbook QA dataset at N=1000 . X axis denotes the answer shifts and Y axis denotes the estimator results. Each point corresponds to a query-answer pair, and the dashed line indicates perfect agreement. The strong diagonal alignment indicates high quantitative fidelity of representation-level predictions.
Figure 3: KL divergence trajectories of next-token predictive distributions under repetition. Across models, the KL divergence stabilizes as the repetition length increases, demonstrating convergence of answer-level predictive distributions under contextual repetition.
Model
Setting
Transfer
Mislead
Correct
500
750
1000
500
750
1000
500
750
1000
ACC
F1
ACC
F1
ACC
F1
ACC
F1
ACC
F1
ACC
F1
ACC
F1
ACC
F1
ACC
F1
Falcon3-7B-Base
96.1
98.0
92.5
96.1
86.8
92.6
93.3
96.5
91.5
95.5
84.7
91.6
85.6
92.2
80.8
89.3
75.2
85.4
Mistral-7B-v0.1
94.4
97.1
92.8
96.3
88.7
94.0
88.5
93.9
85.0
91.8
79.8
88.6
82.5
90.4
81.5
89.7
74.6
85.0
Apollo-1-4B
81.2
89.6
83.7
91.1
90.7
95.1
71.7
82.7
71.4
82.5
80.1
88.6
70.0
82.3
79.7
88.7
78.2
87.8
Table 2: Evaluation of predicting answer preference changes under repetition across the Transfer , Correct , and Mislead regimes at different finite repetition lengths ( N∈{500,750,1000} ).
Model
Dataset
SYCON
Farm
BeHonest
SycoEval
Amp
Ours
Amp
Ours
Amp
Ours
Amp
Ours
Falcon3-7B-Base
0.57
3.26
1.05
3.33
1.06
1.79
0.74
1.95
Mistral-7B-v0.1
1.00
3.63
1.21
4.93
1.06
2.00
1.08
4.52
Apollo-1-4B
1.00
3.15
1.00
4.18
1.02
2.55
1.00
4.68
Qwen3-4B
1.06
6.28
1.02
7.31
1.07
4.09
1.24
8.42
Table 3: Evaluation of model-level stability on established sycophancy and persuasion benchmarks.
Figure 4: Case studies of repetition-induced answer changes and our predictions. For each query, we show the model’s prediction at N=0 and N=1000 , alongside our predicted outcome based on representation-space analysis. Checkmarks indicate correct predictions, illustrating that our method can reliably anticipate repetition effects across models.
Longer contexts allow large language models (LLMs) to condition their predictions on more information, but inference costs grow prohibitively with context length. An emerging alternative uses test-time updates to encode context as temporary changes to model weights. However, encoding this information into weights that already represent general knowledge obscures its role as context, so the LLM may fail to use it properly to condition subsequent inference. To restore this contextual role, we first associate each context with an identifier during encoding, thereby distinguishing the newly encoded information from pre-existing knowledge. Invoking the identifier at inference then enables the LLM to recite the encoded context. Nevertheless, successful recitation does not ensure that the LLM will use the recalled information as context. We therefore further introduce a prompting framework that explicitly guides the LLM to use this information as context. Across diverse controlled tasks and practical benchmarks, our method outperformed the baseline using only off-the-shelf LLMs, showing its advances in context encoding.
Yue Wang, Qizhou Wang, Feng Liu +3
The University of Melbourne · RIKEN Center for Advanced Intelligence Project · The University of Tokyo +1
During instruction fine-tuning (IFT), large language models (LLMs) learn to follow instructions by using the provided context to answer a query. While prior work has studied how context characteristics correlate with context usage by the LLM, this analysis has been limited to inference time, leaving open how these relationships are acquired in the first place. Here, we measure how models' sensitivity to such characteristics shifts across successive IFT stages: supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning with verifiable rewards (RLVR). Experiments across four models and three datasets show that SFT makes models more likely to use contexts that are easy to understand, such as containing high length, context-query similarity, and fluency. Post-SFT dynamics may either reinforce or resolve these preferences depending on the training dataset. Our findings reveal that context usage is actively reshaped at each IFT stage, and designing a balanced IFT dataset is important in ensuring robust context utilization of instruction-tuned models.
Contextual entrainment, which is a newly discovered phenomenon in large language models (LLMs), refers to the tendency of a model to assign higher probabilities to tokens that appear in its context. In this work, we extend this phenomenon from the token level to the sentence level by examining the per-token mean log-probability of a sentence instead of the probabilities of individual tokens. We investigate sentence-level contextual entrainment across 26 LLMs from seven families and two datasets, which cover both subjective and objective tasks. We find that sentence-level contextual entrainment exists. This means that the sentences in the prompt (even if they are counterfactual statements) can significantly increase their probability during model inference time. As the model size increases, contextual entrainment gradually decreases. We also find that contextual entrainment is controlled by 2% to 4% of the attention heads. Turning off these attention heads can effectively mitigate contextual entrainment without hurting the model's performance.