Single-Position Intervention Fails: Distributed Output Templates Drive In-Context Learning

Bryan Cheng; Jasper Zhang

cs.LGApr 10, 2026

Single-Position Intervention Fails: Distributed Output Templates Drive In-Context Learning

Authors: Bryan Cheng, Jasper Zhang

Organizations: William A. Shine Great Neck South High School

Abstract

Understanding how large language models encode task identity from few-shot demonstrations is a central open problem in mechanistic interpretability. Prior work uses linear probing to localize task representations, reporting high classification accuracy at specific layers. We reveal a striking dissociation: probing accuracy completely fails to predict causal importance. Single-position activation intervention achieves 0% task transfer across all 28 layers of Llama-3.2-3B-despite 100% probing accuracy at those same positions. This null result is itself a key finding, demonstrating that task encoding is fundamentally distributed. Multi-position intervention-replacing activations at all demonstration output tokens simultaneously-achieves up to 96% transfer (N=50, 95% CI: [87%, 99%]) at layer 8, pinpointing for the first time the causal locus of ICL task identity. We establish the generality of these findings across four models spanning three architecture families (LLaMA, Qwen, Gemma), discovering a universal intervention window at ~30% network depth. Causal tracing uncovers an asymmetric architecture: the query position is strictly necessary (53-100% disruption) while no individual demonstration position is necessary (0% disruption)-resolving a key ambiguity in prior accounts. Crucially, transfer depends on internal representation compatibility, not surface similarity (r=-0.05 vs r=0.31), ruling out trivial explanations. These results establish the distributed template hypothesis: ICL task identity is encoded as output format templates distributed across demonstration tokens, fundamentally reshaping our understanding of how in-context learning operates.

Figures & tables

Regime	Tasks	Example
Procedural	uppercase, first_letter, repeat_word, length	“cat” $\rightarrow$ “CAT”
Numeric	linear_2x	“5” $\rightarrow$ “10”
Semantic	sentiment, antonym, pattern_completion	“happy” $\rightarrow$ “positive”

Table 1: Task suite covering procedural, numeric, and semantic operations.

Position	L4	L12	L20
last_demo_token	100	100	100
separator	100	100	100
first_query_token	62.5	83.3	75.0

Table 2: Probe accuracy (%) by position and layer.

Method	Position	Layers	Transfer
Transplant	last_demo_token	0–27	0%
Transplant	first_query_token	0–27	0%
Zero ablation	last_demo_token	14	0% (100% acc)
Random ablation	last_demo_token	14	0% (100% acc)

Table 3: Single-position intervention fails at all layers; ablation controls confirm positions are not causally necessary.

Figure 1: Main Results: Single-position intervention fails; multi-position succeeds. (a) Layer sweep: single-position intervention (gray) achieves 0% transfer at all 28 layers; multi-position intervention (orange) peaks at 96% at layer 8. (b) Condition comparison at layer 8: all_demo (96%) and output_only (94%) both succeed; input_only fails (0%). (c) Task pair transfer matrix: a cluster of tasks (uppercase, repeat, length) shows high bidirectional transfer (50–100%).

Pair	Condition	Transfer	95% CI
uppercase $\to$ repeat_word	output_only (L8)	94%	[84%, 98%]
uppercase $\to$ repeat_word	all_demo (L8)	96%	[87%, 99%]
uppercase $\to$ repeat_word	input_only (L8)	0%	[0%, 7%]
sentiment $\to$ antonym	output_only (L8)	6%	[2%, 16%]
Mean across 56 pairs (output_only, L8)		16.7%	—

Table 4: Multi-position intervention results (N=50 with Wilson score 95% CIs). Replacing demo output positions at layer 8 achieves 96% transfer for format-compatible pairs.

Position	Layers	Disruption
first_query_token	0–14	53–100%
first_query_token	16+	0–7%
last_demo_token	0–27	0%

Table 5: Causal tracing: disruption rates under noise injection.

Figure 2: Causal Analysis: Query is necessary; demos are collectively sufficient. (a) Noise injection disruption: query position shows 50–100% accuracy drop in layers 0–14; demo positions show near-0% disruption. (b) Position roles: query is necessary but not sufficient; individual demos are neither; all demos together are sufficient. (c) Information flow: probe accuracy at demo positions stays at 100%; query position rises from 12% to 83%, peaking at layer 12.

Figure 3: Task Structure. (a) Activation similarity (layer 12). (b) Format transfer. (c) Surface similarity does NOT predict transfer (r= $-$ 0.05); activation r=0.31. (d) Target demo count (transfer constant at 33%).

Pair	True Source	Random	Noise	Shuffled
uppercase $\to$ repeat_word	95%	0%	0%	55%
uppercase $\to$ first_letter	0%	0%	0%	0%
sentiment $\to$ antonym	10%	0%	0%	0%
linear_2x $\to$ length	0%	0%	0%	0%
Mean	17.5%	0%	0%	9.2%

Table 6: Baseline controls (N=20). All controls yield 0% or reduced transfer vs. true source.

Source $\to$ Target	Format Difference	Transfer
uppercase $\to$ uppercase_period	“WORD” vs “WORD.”	0%
length $\to$ length_word	“5” vs “five”	0%
repeat_word $\to$ repeat_comma	“word word” vs “word, word”	90%
reverse $\to$ reverse_spaced	“olleh” vs “o l l e h”	5%

Table 7: Cross-format transfer. Identical operations with different output formats show 0% transfer; structurally similar formats show 90%.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Input Domain	Transformation	Output Format
uppercase	4–8 letter words	$f(x)=x.\text{upper}()$	Single uppercase word
first_letter	4–8 letter words	$f(x)=x[0]$	Single lowercase letter
repeat_word	4–8 letter words	$f(x)=x+\text{`` ''}+x$	“word word”
length	3–10 letter words	$f(x)=\text{len}(x)$	Single digit
linear_2x	integers 1–50	$f(x)=2x$	Integer
sentiment	sentiment words	$f(x)\in\{\text{pos},\text{neg}\}$	“positive”/“negative”

Appendix

Table 8: Complete task specifications with input generation procedures and output formats.

	upper	first	repeat	length	linear	sent	pattern
uppercase	–	0	90	80	100	0	0
first_letter	0	–	0	0	0	20	10
repeat_word	0	0	–	100	50	0	0
length	100	0	50	–	0	0	0
linear_2x	0	0	0	0	–	0	0
sentiment	0	10	0	20	0	–	0

Appendix

Table 9: Full transfer matrix (layer 8, all_demo, N=10). Rows are source tasks; columns are targets. Values are transfer rates (%). Data from exp29.

Transfer Rate Range	# Pairs	Percentage
$\geq$ 50% (high transfer)	7	13%
10–49% (moderate)	10	18%
$<$ 10% (near-zero)	40	71%
Total	56	100%

Appendix

Table 10: Transfer rate distribution across all 56 task pairs. Most pairs show near-zero transfer; high transfer is concentrated in a procedural cluster.

Layer	0	4	8	10	12	14	16	20	24
Transfer (%)	5	15	96	60	30	18	10	8	5

Appendix

Table 11: Transfer rate by layer for uppercase

\to

repeat_word (all_demo condition, N=50 at layer 8, N=10 elsewhere).

Demo Count	Task Accuracy	Transfer Rate	Preserve Rate
1-shot	64.2%	33.3%	45.8%
2-shot	85.4%	33.3%	54.2%
3-shot	93.8%	33.3%	58.3%
5-shot	97.9%	33.3%	62.5%
8-shot	100.0%	32.1%	64.6%

Appendix

Table 12: Effect of target demo count on task accuracy and transfer (source fixed at 5-shot). Transfer remains constant regardless of target demos.

Output Positions	1	3	5	7	10	All ( $\sim$ 19)
Transfer (%)	0	0	1	1	10	90

Appendix

Table 13: Transfer rate by number of output positions replaced (layer 8, mean across 3 pairs, 10 random subsets per size). Transfer shows a sharp threshold, not gradual scaling.

Source Demos	1-shot	2-shot	3-shot	5-shot
Mean Transfer (%)	0	0	0	93

Appendix

Table 14: Effect of source demo count (target fixed at 5-shot, layer 8, N=20). Source demos show all-or-nothing threshold.

Source $\to$ Target	Labels	Transfer (%)
standard $\to$ goodbad	positive/negative $\to$ good/bad	100
goodbad $\to$ standard	good/bad $\to$ positive/negative	95
standard $\to$ symbol	positive/negative $\to$ +/ $-$	55
symbol $\to$ standard	+/ $-$ $\to$ positive/negative	25
sentiment $\to$ antonym (control)	cross-task	10

Appendix

Table 15: Transfer between sentiment variants (layer 8, all_demo, N=20). Word-label variants transfer near-perfectly.

Pair	Transfer (%)	Note
repeat_word $\to$ repeat_n (N=2)	100	Token count matches
repeat_word $\to$ repeat_n (N=3)	0	Token count differs
repeat_word $\to$ repeat_n (N=4)	0	Token count differs
length $\to$ spell_out	13	Near-zero

Appendix

Table 16: Variable-length task transfer (layer 8, N=15). Transfer only succeeds when output token counts match.

Figure 4: Scaling thresholds. (a) Position count: transfer remains near-zero until nearly all output positions are replaced. (b) Source demo count: transfer is all-or-nothing (0% with 1–3 demos, 93% with 5 demos).

	uppercase	length	sentiment	pattern
uppercase	1.00	0.72	0.65	0.68
length	0.72	1.00	0.70	0.75
sentiment	0.65	0.70	1.00	0.73
pattern	0.68	0.75	0.73	1.00

Appendix

Table 17: Pairwise cosine similarity between task representations (layer 12).

Position	Transfer Rate	Preserve Rate	Accuracy
last_demo_token	0%	95%	95%
separator	0%	97%	97%
first_query_token	0%	93%	93%

Appendix

Table 18: Single-position intervention at different positions (layer 14).

Layer	0	2	4	6	8	10	12	14	16	20	24
last_demo	75.0	87.5	100	100	100	100	100	100	100	100	100
separator	62.5	75.0	100	100	100	100	100	100	100	100	100
first_query	12.5	25.0	62.5	70.8	75.0	79.2	83.3	79.2	75.0	75.0	70.8

Appendix

Table 19: Probe accuracy (%) across all 28 layers for three key positions.

Source Position	Demo Inputs	Demo Outputs	Separators	Query Self
Attention Weight	0.12	0.38	0.08	0.42

Appendix

Table 20: Mean attention weight from first query token to different position types (layer 8, head average).

Error Type	Frequency	Example
Preserved target task	45.2%	Input: “cat” $\to$ “CAT” (target: uppercase)
Partial transfer	28.4%	Input: “cat” $\to$ “cat cat” (incomplete)
Format mismatch	18.7%	Input: “cat” $\to$ “Cat Cat” (wrong case)
Malformed output	7.7%	Input: “cat” $\to$ “cattac” (nonsense)

Appendix

Table 21: Error categorization for failed transfer attempts (layer 8, all_demo condition).

	Llama-3B	Llama-1B	Qwen-1.5B	Gemma-2B
Architecture depth	28	16	28	26
Optimal layer	8	5	8	8
Depth fraction	0.29	0.31	0.29	0.31
upper $\to$ repeat transfer	0.90	0.60	0.90	0.40

Appendix

Table 22: Multi-model replication of key findings. Optimal intervention layer is consistently at

\sim

30% network depth.

Pair	True Source	Shuffled
uppercase $\to$ repeat_word	0.95	0.55
uppercase $\to$ first_letter	0.00	0.00
first_letter $\to$ repeat_word	0.00	0.00
uppercase $\to$ sentiment	0.00	0.00
linear_2x $\to$ length	0.00	0.00
sentiment $\to$ antonym	0.10	0.00

Appendix

Table 23: Baseline controls confirm transfer requires correct task-specific content at correct positions.

Pair	Src Tokens	Tgt Tokens	Match?	Transfer
uppercase $\to$ repeat_word	1.8	2.0	No	90%
linear_2x $\to$ length	2.0	2.0	Yes	0%
sentiment $\to$ antonym	1.0	1.0	Yes	10%
uppercase $\to$ first_letter	1.8	1.0	No	0%

Appendix

Table 24: Tokenization does not explain transfer. Token-matched pairs show 0% transfer; token-mismatched pairs show 90%.

Layer Ablated	0	4	8	12	16	20	24	27
Accuracy	0%	12%	35%	58%	72%	85%	92%	95%

Appendix

Table 25: Task accuracy (%) after ablating individual layers (zeroing layer output).

Intervention Point	Transfer Rate	Notes
Before attention (layer 8)	85%	Best timing
After attention, before MLP	72%	Partial degradation
After full layer 8	90%	Optimal
Persistent (all subsequent layers)	65%	Overconstrained

Appendix

Table 26: Transfer rate (%) by intervention timing within the forward pass.

Parameter	Value
Number of demonstrations ( $k$ )	5
Test samples per experiment	10 (main), 50 (ablations)
Source prompts for averaging	10
Noise magnitude for causal tracing	$3\sigma$
Random seed	42
Decoding temperature	0 (greedy)

Appendix

Table 27: Complete hyperparameter specification.

Metric	Value	95% CI
Single-position transfer (layer 14)	0%	[0%, 7%]
Multi-position transfer (layer 8, all_demo)	96%	[87%, 99%]
Near-zero transfer pairs	2–4%	$\leq$ 13% upper

Appendix

Table 28: N=50 experiments with Wilson score 95% confidence intervals.

In-Context Learning

In-context learning studies how models adapt to new tasks from examples or instructions supplied at inference time, without changing their weights.

See all 404 papers

Oct 1, 2026

Capturing In-Context Learning Dynamics with Task Operators

In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requires processing the full set of examples, resulting in inefficient deployments, and how ICL works mechanistically is not fully understood. Prior work compresses ICL into fixed activation vectors extracted from specific layers or positions, but these input-independent interventions fail on complex tasks where the output depends on fine-grained interactions with the input. By analyzing the ICL forward pass, we show that each attention head's output is an affine transformation of its context-masked counterpart, and that the parameters of this transformation are empirically stable across samples for a given task. Building on this, we introduce Task Operator (TO), which replays this transformation as an analytically derived update to the attention output projection. Across lexical, algorithmic, and reasoning tasks, TO achieves the best overall performance among prior methods and substantially narrows the gap between zero-shot inference and ICL. We further show that the extracted knowledge concentrates in a task-specific sparse circuit across layers and positions, and that averaging operators from disjoint demonstration batches enables effective many-shot scaling without expanding the context window. Our code is available at https://github.com/gzxiong/task_operator.

Guangzhi Xiong, Zhenghao He, Bohan Liu +3

University of Virginia

Transformer Interpretability In-Context Learning

Ask PDF

Oct 16, 2026

Out of Support, Not Out of Reach: In-Context Learning with Unsupported Instructions and Composed Tasks

In-Context Learning (ICL) enables pretrained LLMs to adapt to downstream tasks from a small set of input-output demonstrations, without parameter updates. Yet existing theory primarily explains ICL as identifying a latent task already represented in pretraining, leaving unclear whether ICL can succeed when either the instruction or the target task falls outside the pretraining support. We develop a theoretical framework under mild assumptions that separates these two forms of distribution shift. For target tasks represented in pretraining, we show that demonstrations can identify the task and reduce sensitivity to pretrain-supported prompt templates as their number increases. For an unsupported instruction, we show that under a fixed model-induced reference extension, demonstration evidence can dominate an

N

-independent instruction shift while preserving the exponential identification rate. We then use the same ICL framework as an interpretive lens for long-horizon target tasks that are not represented in pretraining but admit a plan that decomposes them into pretraining-supported subtasks. Applying the supported-task analysis step by step yields a standard accumulated-error bound and clarifies how invoking familiar subtasks along an intermediate-state trajectory can avoid directly selecting the unsupported complete task. Together, our results characterize when demonstrations, instructions, and task decomposition can support ICL beyond the corresponding pretraining supports.

Xuhan Tong, Yuchen Zeng, Jiawei Zhang

Department of Computer Sciences, University of Wisconsin–Madison. · Microsoft Research.

In-Context Learning

Ask PDF

Oct 15, 2026

From Observation to Intervention: Distilling Self-Generated In-Context Demonstrations via Executable Causal Intervention

Self-generated In-Context Learning (ICL) enables LLMs to synthesize their own few-shot demonstrations, often outperforming manual prompts. However, existing methods filter demonstrations by observation, assuming a rationale is high-quality if it leads to the correct output. We show such filtering suffers from observational equivalence, whereby both faithful causal rationales and spurious post-hoc rationales can yield same outputs and are statistically indistinguishable from observation alone. Inspired by independent causal mechanisms, we argue that faithful rationales imply transfer invariance, in which causal rationales should remain effective across logically similar samples. We instantiate this criterion as Executable Causal Intervention for Demonstrations Distillation (ECI2D), which abstracts language rationales into executable programs and applies the causal interventions (

do

-operator) on logically similar samples, retaining only those demonstrations that imply transfer invariance. Experiments on math, logical and commonsense reasoning benchmarks demonstrate that ECI2D consistently outperforms observation-based ICL while retaining far fewer demonstrations, with gains exceeding 7 points as the test distribution shifts.

Minming Ye, Min Liu, Haoang Chi +2

Shanghai Jiaotong University · Academy of Military Science · National University of Defense Technology

In-Context Learning In-Context Example Selection LLM Reasoning Causal Reasoning in Language Models Causal Interventions in Language Models

Ask PDF

Abstract

In-Context Learning

Explore similar work

Capturing In-Context Learning Dynamics with Task Operators

Out of Support, Not Out of Reach: In-Context Learning with Unsupported Instructions and Composed Tasks

From Observation to Intervention: Distilling Self-Generated In-Context Demonstrations via Executable Causal Intervention