The Intrusion Pathway: How Non-Evidential Context Degrades Language Model Performance
Organizations: TurboML · Indraprastha Institute of Information Technology, Delhi · Independent Researcher · Singapore Institute of Technology ·
Abstract
Language models lose accuracy when a prompt contains non-evidential context: irrelevant statements from unrelated domains, or random off-distribution tokens, with no competing answer and no semantic overlap with the query. Such noise pervades retrieval-augmented, tool-using, and long-context deployments, yet the failure has lacked a mechanistic account. We trace the corruption to the Intrusion Pathway, which starts with MLPs enriching both query-subject and irrelevant-token representations, parroting heads carry prefix-token content to the prediction position, and relational readers largely avoid unrelated sources. Nevertheless, preserved subject enrichment and selective reading do not prevent intrusion in the broader computation. We then localize part of this interference to a sparse set of attention heads, which we call intrusion heads. Their prefix-derived writes promote distractor-related content, while their reduced support for the correct answer arises primarily from weaker non-prefix contributions rather than direct suppression by the prefix. The same heads transfer across tasks and distractor types and are largely weight-disjoint from the heads that integrate relevant contextual evidence, enabling a class-conditional intervention. Replacing the corrupted last-token residual with its clean counterpart, an oracle probe that requires a matched clean forward pass, recovers 97.6% on average of the distractor-induced drop across four downstream tasks. A deployable mask level knockout of about ten heads, roughly 1% of the model's attention heads, recovers 57% at bounded cost to helpful context.
Figures & tables
| Task | +Patch | +K10 | +K20 |
|---|---|---|---|
| GSM-DC | |||
| GSM-IC | |||
| HotpotQA | |||
| PubMedQA | |||
| mean |
Appendix figures & tables45 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Bank | Layer | Peak excess [95% CI] | Final layer |
| (a) Subject position | ||||
| Llama-3.1-8B | LRE | 30 | ||
| Wikipedia | 30 | |||
| Mistral-7B-v0.1 | LRE | 23 | ||
| Wikipedia | 29 | |||
| Qwen2.5-7B | LRE | 24 | ||
| Model | Edit | Bare | Golden | Irrelevant | Mixed |
|---|---|---|---|---|---|
| Llama-3.1-8B | Whole-head zeroing | ||||
| Whole-head donor replacement | |||||
| END-only zeroing | |||||
| Prefix-write removal | |||||
| Prefix-edge masking | |||||
| Mistral-7B-v0.1 | Whole-head zeroing |
| Model | Copying heads | Control heads | Cosine range |
|---|---|---|---|
| Llama-3.1-8B | 2/10 | 1/10 | 0.03–0.23 |
| Mistral-7B-v0.1 | 3/10 | 1/10 | 0.06–0.18 |
| Qwen2.5-7B | 3/10 | 0/10 | 0.06–0.25 |
| Pythia-6.9B | 6/10 | 0/10 | 0.03–0.15 |
| GPT-2 XL | 7/10 | 1/10 | 0.07–0.20 |
| Association excess | Profile alignment | |||
|---|---|---|---|---|
| Model | Copying heads [95% CI] | Control | Rel. error | Cosine |
| Llama-3.1-8B | 1.000 | |||
| Mistral-7B-v0.1 | 1.025 | |||
| Qwen2.5-7B | 0.989 | |||
| Pythia-6.9B | 0.996 | |||
| GPT-2 XL | 1.000 | |||
| (a) Association-output change | |||
|---|---|---|---|
| Model | Association removal [95% CI] | Identity removal | Random subspace |
| Llama-3.1-8B | |||
| Mistral-7B-v0.1 | |||
| Qwen2.5-7B | |||
| Pythia-6.9B | |||
| GPT-2 XL | |||
| Baseline change | Signed ratio (%) [95% CI] | ||||
|---|---|---|---|---|---|
| Model | Condition | Gold logit | Margin | Gold logit | Margin |
| Llama-3.1-8B | far_irr | 61 [57,65] | 87 [67,120] | ||
| long_irr | 54 [50,58] | 70 [64,78] | |||
| Mistral-7B-v0.1 | far_irr | 57 [52,62] | 168 [ ,988] | ||
| long_irr | 60 [56,64] | 79 [69,92] | |||
| Qwen2.5-7B | far_irr | 72 [56,96] | 57 [36,110] | ||
| Model | Gold-logit benefit | Benefit removed (%) [95% CI] |
|---|---|---|
| Llama-3.1-8B | 68 [61,79] | |
| Mistral-7B-v0.1 | 58 [48,69] | |
| Qwen2.5-7B | 33 [23,43] | |
| Pythia-6.9B | 45 [30,57] | |
| GPT-2 XL | 87 [70,109] |
| Model | Condition | Early | Middle | Late |
|---|---|---|---|---|
| Llama-3.1-8B | far_irr | 6 [2,9] | 67 [64,70] | 94 [91,97] |
| long_irr | 1 [ ,4] | 60 [57,62] | 88 [85,90] | |
| Mistral-7B-v0.1 | far_irr | 2 [ ,8] | 70 [66,75] | 98 [93,103] |
| long_irr | 4 [ ,8] | 74 [71,77] | 90 [88,93] | |
| Qwen2.5-7B | far_irr | 17 [8,27] | 62 [49,83] | 95 [90,99] |
| long_irr | 22 [7,47] | 25 [ ,49] | 107 [102,116] |
| Model | Early/mid/late ( ) | Inclusion | Jaccard | |||
|---|---|---|---|---|---|---|
| Llama-3.1-8B | 1/2/17 | 13/20 | 0.666 | |||
| Mistral-7B-v0.1 | 0/4/16 | 12/20 | 0.662 | |||
| Qwen2.5-7B | 0/1/19 | 14/20 | 0.615 | |||
| Pythia-6.9B | 0/6/14 | 10/20 | 0.487 | |||
| GPT-2 XL | 2/8/10 | 12/20 | 0.587 |
| Model | Literal tokens | Distractor category | Gold | Query category | Other answers |
|---|---|---|---|---|---|
| Llama-3.1-8B | |||||
| Mistral-7B-v0.1 | |||||
| Qwen2.5-7B | |||||
| Pythia-6.9B | |||||
| GPT-2 XL |
| Model | Target | Total | |||||
|---|---|---|---|---|---|---|---|
| Llama-3.1-8B | Gold | ||||||
| Mistral-7B-v0.1 | Gold | ||||||
| Qwen2.5-7B | Gold | ||||||
| Model | Context | (20) | Random | Layer | Score | Attention |
|---|---|---|---|---|---|---|
| Llama-3.1-8B | Short irrelevant | |||||
| Long irrelevant | ||||||
| Golden | ||||||
| Mistral-7B-v0.1 | Short irrelevant | |||||
| Long irrelevant | ||||||
| Golden |
| Model | Prefix | Remove prefix | Amplify non-prefix | Both | Whole-write gain |
|---|---|---|---|---|---|
| Llama-3.1-8B | Short | / [-1pt] | / [-1pt] | / [-1pt] | / [-1pt] |
| Long | / [-1pt] | / [-1pt] | / [-1pt] | / [-1pt] | |
| Mistral-7B-v0.1 | Short | / [-1pt] | / [-1pt] | / [-1pt] | / [-1pt] |
| Long | / [-1pt] | / [-1pt] | / [-1pt] | / [-1pt] | |
| Qwen2.5-7B | Short | / [-1pt] | / [-1pt] | / [-1pt] | / [-1pt] |
| Long | / [-1pt] | / [-1pt] | / [-1pt] | / [-1pt] |
| Short | Long | |||
|---|---|---|---|---|
| Model | Restore | Mask | Restore | Mask |
| Llama-3.1-8B | 72 / 59 | 46 / 47 | 67 / 37 | 41 / 30 |
| Mistral-7B-v0.1 | 79 / 84 | 81 / 52 | 83 / 48 | 73 / 32 |
| Qwen2.5-7B | 50 / 44 | 44 / 28 | 44 / 34 | 28 / 14 |
| Pythia-6.9B | 79 / 77 | 74 / 74 | 67 / 63 | 70 / 60 |
| GPT-2 XL | 88 / 84 | 84 / 84 | 77 / 71 | 80 / 71 |
| Model | Golden restore | Golden mask | Golden mask | Short mask control | Long mask control |
|---|---|---|---|---|---|
| Llama-3.1-8B | |||||
| Mistral-7B-v0.1 | |||||
| Qwen2.5-7B | |||||
| Pythia-6.9B | |||||
| GPT-2 XL |
| Copier query | Prefix K/V | |||
|---|---|---|---|---|
| Model | Prefix | |||
| Llama-3.1-8B | Short | |||
| Long | ||||
| Mistral-7B-v0.1 | Short | |||
| Long | ||||
| Qwen2.5-7B | Short | |||
| Top intrusion | 5 | 10 | 20 | 40 | 60 | 100 | 150 |
| Shared heads | 0 | 0 | 0 | 1 | 3 | 5 | 7 |
| Expected | 0.18 | 0.35 | 0.70 | 1.41 | 2.11 | 3.52 | 5.27 |
| Model | Masked population | Golden | Short irrelevant | Long irrelevant |
|---|---|---|---|---|
| Llama-3.1-8B | Intrusion-only | |||
| ( ) | Retrieval-only | |||
| Random | ||||
| Mistral-7B-v0.1 | Intrusion-only | |||
| ( ) | Retrieval-only | |||
| Random |
| Task | Llama | Mistral | Qwen | Pythia |
|---|---|---|---|---|
| Factual recall ∗ | 10/10 | 10/10 | 10/10 | 10/10 |
| IOI (retrieval) | 10/10 | 10/10 | 9/10 | 10/10 |
| GSM-DC | 10/10 | 9/10 | 9/10 | 10/10 |
| GSM-IC | 10/10 | 9/10 | 9/10 | 10/10 |
| HotpotQA | 10/10 | 10/10 | 10/10 | 10/10 |
| PubMedQA | 10/10 | 10/10 | 10/10 | 10/10 |
| Model | Clean | Distracted | Patch | Patch gain (pp) | Recovery (%) |
|---|---|---|---|---|---|
| Llama-3.1-70B | .578 | .408 | .656 | +24.8 | 145.9 |
| Qwen2.5-72B | .900 | .840 | .932 | +9.2 | 153.3 |
| Qwen2.5-32B | .888 | .810 | .924 | +11.4 | 146.2 |
| Qwen2.5-7B | .688 | .638 | .730 | +9.2 | 184.0 |
| Mistral-7B-v0.1 | .456 | .352 | .456 | +10.4 | 100.0 |
| Model | Clean | Distracted | Patch | K10 | K20 |
|---|---|---|---|---|---|
| Mistral-7B-v0.1 | .66 | .38 | .66 | .51 | .42 |
| Llama-3.1-8B | .64 | .47 | .67 | .59 | .54 |
| Mean change (pp) | Mean loss recovery (%) | |||||||
|---|---|---|---|---|---|---|---|---|
| Task | Patch | K10 | K20 | Patch | K10 | K20 | ||
| GSM-IC CoT | 2 | +24.00 | +12.50 | +5.50 | 2 | 108.82 | 58.51 | 27.73 |
| HotpotQA | 15 | +23.07 | +14.53 | +9.27 | 13 | 85.51 | 48.54 | 29.53 |
| PubMedQA | 11 | +15.64 | +5.45 | +3.09 | 9 | 99.26 | 45.31 | 17.67 |
| Model | Condition | |||||
|---|---|---|---|---|---|---|
| Llama-3.1-8B | Clean | 0.68 | 0.68 | 0.68 | 0.68 | 0.68 |
| Distracted | 0.68 | 0.70 | 0.44 | 0.22 | 0.18 | |
| Oracle patch | 0.68 | 0.70 | 0.66 | 0.60 | 0.62 | |
| K10 mask | — | — | 0.44 | 0.56 | 0.47 | |
| K20 mask | — | — | 0.44 | 0.44 | 0.42 | |
| Mistral-7B | Clean | 0.72 | 0.72 | 0.72 | 0.72 | 0.72 |
| Model | Condition | ||||
|---|---|---|---|---|---|
| Llama-3.1-8B | Clean | 0.82 | 0.82 | 0.82 | 0.82 |
| Distracted | 0.82 | 0.78 | 0.54 | 0.60 | |
| Oracle patch | 0.82 | 0.82 | 0.82 | 0.82 | |
| K10 mask | — | 0.81 | 0.58 | 0.67 | |
| K20 mask | — | 0.77 | 0.58 | 0.60 | |
| Mistral-7B | Clean | 0.86 | 0.86 | 0.86 | 0.86 |
| Metric | Base | Instruct / comparison |
|---|---|---|
| , top-20 late-block sets | 5/20 | 5/20 |
| Spearman, intrusion-score rankings | — | .924 |
| Spearman, retrieval-score rankings | — | .962 |
| Jaccard, top-20 retrieval sets | — | .818 |
| Jaccard, top-20 intrusion sets | — | .481 |
| Jaccard, disjoint top-10 sets | — | .429 |
| Head set | Checkpoint | Short | Long | Golden | Bare |
|---|---|---|---|---|---|
| Disjoint K10 | Base | +.009 | +.015 | +.010 | .000 |
| Instruct | +.006 | +.039 | .000 | ||
| Full K20 | Base | +.149 | +.151 | +.161 | .000 |
| Instruct | +.035 | +.048 | +.113 | .000 |
| Task | Base recovery (%) | Instruct recovery (%) |
|---|---|---|
| GSM-IC, single-token | Undefined | 94.7 |
| GSM-DC | 87.5 | 100.0 |
| HotpotQA | 89.8 | 97.7 |
| PubMedQA | 100.0 | 100.0 |
Language Model Robustness
Language model robustness studies how reliably language models preserve correct, safe behavior under input changes, distribution shifts, and adversarial pressure.
See all 514 papers