Backpack language models predict with a strictly linear combination of non-contextual per-word sense vectors, weighted by non-negative contextualization scores. We show that non-negativity conflates two properties: the linear readout, which carries the interpretability, and the conic combination, which merely restricts it. We relax the latter by computing the sense weights with differential attention, a difference of two softmax maps with a learned per-sense gain, so context can actively inhibit a sense; we call the result a differential Backpack. At equal parameters on WikiText-103 this cuts the Backpack's perplexity gap to a Transformer by 31%, improves over the softmax Backpack at every sense count, persists with a shrinking margin up to 350M parameters on OpenWebText, and is localized at the sense layer by ablations and controls. The learned inhibition is directly interpretable (a financial context drives the river sense of bank to -0.50), and the preserved per-sense attribution predicts counterfactual removals exactly. We tested a number of combination rules, showing the linear interface is nearly free. Making the readout nonlinear gains only 0.43 perplexity points while invalidating two thirds of the decomposition. The differential Backpack's senses are also better representations, improving zero-shot sentence similarity by up to 33%.
Figures & tables
Figure 1: The differential Backpack. Non-contextual sense vectors C(xj)ℓ (left) are combined by signed weights Aℓ=A1−λℓA2 (right) under a linear readout; the dashed arrow marks a negative weight suppressing a contextually wrong sense.
Model
Params
PPL ↓
tax
Transformer
70.1M
21.09
—
DIFF Transformer
70.1M
21.00
−0.09
softmax Backpack
69.6M
24.60
+3.51
differential Backpack (ours)
69.6M
23.52
+2.43
+ DIFF trunk
69.6M
24.78
+3.69
Table 1: WikiText-103 exact validation perplexity at matched parameters and identical training budget (20k steps, single seed); tax is the gap to the plain Transformer.
Figure 2: Interpretability tax on WikiText-103: exact validation perplexity at matched parameters ( ∼ 70M), with a Transformer of the Backpack trunk’s depth ( 8 layers, 51 M) for reference.
Combination rule
PPL ↓
tax
attr. resid.
Transformer
21.09
—
n/a
DIFF Transformer
21.00
−0.09
n/a
Transformer, 8 layers
23.38
+2.29
n/a
one-hot
31.58
+10.49
2⋅10−7
softmax
24.60 ± .12
+3.51
5⋅10−7
diff Backpack (ours)
23.52 ± .04
+2.43
6⋅10−7
Table 2: The constraint ladder on WikiText-103. All rows share the main configuration and budget; tax is the gap to the 14 -layer Transformer; the 8 -layer Transformer ( 51 M) matches the Backpack trunk’s depth; attr. resid. is the relative error of the per-sense decomposition against the model’s own logits.
Sense weights
PPL ↓
% of gain
neg. mass
softmax
24.60
—
0.00
two maps, added
24.30
28%
0.00
scaled softmax
24.51
9%
0.00
λ=0
24.55
5%
0.00
Cog Attention
24.72
−11%
0.39
diff Backpack (ours)
23.52
100%
0.45
Table 3: Controls for the causal claim (WikiText-103, main configuration). “% of gain” is the share of diff’s 1.08 ppl improvement over softmax each rule recovers.
Representation (70M)
POS
STS
SST-2
ARI
Transformer ( h )
0.859
0.488
0.753
0.174
DIFF Transformer ( h )
0.859
0.499
0.743
0.203
softmax Backpack ( o )
0.844
0.372
0.718
0.151
diff Backpack ( o )
0.815
0.495
0.731
0.174
Cog Backpack ( o )
0.836
0.422
0.736
0.163
diff Backpack + MLP ( o )
0.838
0.485
0.749
0.168
Table 4: Frozen-representation probes at 70 M: POS accuracy (UD English-EWT), STS-B Spearman, SST-2 probe accuracy, and sense-clustering ARI, on the Transformer state h or the sense output o ; all scales and MRPC in Appendix D .
Figure 3: Signed self-weight Appℓ of the finance and river senses of bank in a river vs. a financial context, for the differential (left) and softmax (right) Backpacks.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Per-sense negative-mass fraction of the differential Backpack in the main configuration ( k=16 ); 0 is purely excitatory, 1 purely inhibitory.
Rule
ρ
rel. err.
compr.
suffic.
one-hot
1.000
10−6
5.97
+0.27
softmax
1.000
10−6
3.49
−1.43
diff Backpack (ours)
1.000
10−6
3.74
−1.22
unconstr. weights
1.000
10−6
1.14
−4.88
Cog Attention
1.000
10−6
3.63
−1.39
softmax + MLP
0.77
0.48
3.39
−1.45
Appendix
Table 5: Counterfactual faithfulness of the per-sense attribution on WikiText-103 ( 300 positions): Spearman correlation and median relative error between attributed and actual logit change under single removals of the ten top pairs, and the change in target log-probability (nats) when those pairs are removed jointly (comprehensiveness) or kept alone (sufficiency).
Figure 5: Per-token loss increase when inhibition is removed at inference from a trained differential Backpack, by WordNet polysemy of the preceding whole word, for the two edits of § 4.8 ; bars are means with 95% intervals over all validation tokens.
neg.
entropy
sparsity
SL999/WS353
mass
of ∣A∣
@1e-3
(best ρ )
softmax
0.00
0.70
0.66
0.42 / 0.70
diff
0.45
0.78
0.49
0.44 / 0.73
Appendix
Table 6: Sense-weight structure and lexical quality on WikiText-103 (main configuration): negative weight mass, normalized entropy and sparsity of ∣A∣ , and best-sense Spearman ρ on SimLex-999 / WordSim-353 (raw-embedding baselines 0.37 / 0.42 – 0.45 ).
Figure 6: Bias against collateral damage (KL) for the three sense interventions at 70 M, 170 M and 350 M (left to right); upper-left is better.
none
0.5×
remove
suppress
70M
softmax bias
1.14
1.06
0.98
0.86
KL
—
.002
.007
.030
diff bias
1.77
1.65
1.58
1.47
KL
—
.006
.024
.095
170M
softmax bias
1.74
1.42
1.20
1.19
KL
—
.015
.052
.173
Appendix
Table 7: Gender bias ( ↓ ) and collateral damage KL ( ↓ ) under each sense intervention, over 20 profession nouns, at 70 M, 170 M and 350 M.
Scale
Transf.
softmax
diff
gap
cut
70M WT103
21.09
24.60
23.52
1.08 (4.4%)
31%
170M OWT
—
23.84
23.15
0.70 (2.9%)
—
350M OWT
18.02
20.48
19.96
0.52 (2.5%)
21%
Appendix
Table 8: Exact validation perplexity across scale at equal parameters and token budget; gap is softmax minus diff and cut the reduction of the tax (the gap to the parameter-matched Transformer). The 70M and 170M Backpack entries are three-seed means (s.d. 0.12 / 0.04 and 0.13 / 0.03 for softmax/diff); the rest are single runs.
Repr.
POS
STS
SST-2
MRPC
ARI
Transf. ( h ), 70M
0.859
0.488
0.753
0.662
0.174
Transf. 8L ( h ), 70M
0.860
0.484
0.744
0.674
0.160
DIFF Transf. ( h ), 70M
0.859
0.499
0.743
0.654
0.203
softmax ( o ), 70M
0.844
0.372
0.718
0.625
0.151
diff ( o ), 70M
0.815
0.495
0.731
0.613
0.174
Cog ( o ), 70M
0.836
0.422
0.736
0.632
0.163
Appendix
Table 9: Frozen-representation probes at every scale: POS accuracy, STS-B Spearman, SST-2 and MRPC probe accuracy (majority class 0.509 and 0.684 ), and sense-clustering ARI.
Model
best ρ
mean ρ
# ρ>0.6
emb ρ
softmax, k=16
0.70
0.67
6/16
0.45
diff, k=16
0.73
0.69
9/16
0.42
softmax, k=64
0.72
0.58
8/64
0.48
diff, k=64
0.74
0.68
37/64
0.48
Appendix
Table 10: Sense specialization on WordSim-353 (Spearman ρ ): the single best sense, the sense-averaged cosine, the number of individual senses with ρ>0.6 , and the raw token-embedding baseline.
rock , differential Backpack (ours)
mineral
rock, Rock, Rocks, canyon, Canyon
music
rock, garage, punk, gig, pop
rock , softmax Backpack
blended
rock, punk, metal, shale
blended
rock, bedrock, punk, Stones
Appendix
Table 11: Sense lens for rock : the vocabulary items each selected sense most strongly predicts through the tied embedding, C(w)ℓE⊤ , in the two models at k=16 .
Figure 7: Final perplexity by number of senses k on WikiText-103 ( 8 k steps).
squared error
k=1
k=2
k=4
k=8
k=16
softmax
.275
.275
.092
.028
.002
diff
.000
.000
.000
.000
.005
Appendix
Table 12: Extrapolation squared error on the synthetic task of Appendix G by number of sense heads k (mean of 3 seeds).
Figure 8: Mean learned λ over training (a) from three initializations and (b) for three sense counts k (WikiText-103, 8 k steps).
Senses
k
SL999
SV3.5k
RG65
WS353
induced, Transf.
112
0.322
0.179
0.493
0.475
induced, trunk
64
0.365
0.249
0.559
0.552
learned, diff
16
0.441
0.337
0.726
0.731
Appendix
Table 13: Best-sense Spearman ρ of training-free senses induced from OV circuits versus learned Backpack senses.
We study how words in the context support or oppose a language model's response and present DeltaTrace, a novel attribution framework that explains the effect of input tokens on a fixed model response. DeltaTrace measures how the response's log-probability changes when the context tokens being explained are masked. To distribute this difference among source tokens, it constructs operator-specific local rules for finite changes and composes them backward through the computational graph using effective secant slopes. Each token is assigned a signed attribution as the inner product of its embedding displacement and the corresponding backward-propagated slope. Local conservation makes these contributions sum to the measured score difference and preserves their meaning when aggregated over sentences or passages. DeltaTrace supports both attention and linear-attention models, with auxiliary memory growing linearly with sequence length at fixed model dimensions and chunk size. On the evaluated attention model, DeltaTrace achieves the lowest deletion-faithfulness scores (RISE and MAS) on most retrieval and reasoning tasks and the highest needle-in-a-haystack Recall, exceeding FlashTrace by 5.67-24.33 percentage points. On the evaluated hybrid attention and linear-attention model, DeltaTrace lowers RISE on 9 tasks and MAS on 12 relative to FlashTrace.
Larger language models become simultaneously better and worse at handling contextual information -- better at ignoring false claims, worse at ignoring irrelevant tokens. We formalize this apparent paradox through the first scaling laws for contextual entrainment, the tendency of models to favor tokens that appeared in context regardless of relevance. Analyzing the Cerebras-GPT (111M-13B) and Pythia (410M-12B) model families, we find entrainment follows predictable power-law scaling, but with opposite trends depending on context type: semantic contexts show decreasing entrainment with scale, while non-semantic contexts show increasing entrainment. Concretely, the largest models are four times more resistant to counterfactual misinformation than the smallest, yet simultaneously twice as prone to copying arbitrary tokens. These diverging trends, which replicate across model families, suggest that semantic filtering and mechanical copying are functionally distinct behaviors that scale in opposition -- scaling alone does not resolve context sensitivity, it reshapes it.
Dikshant Kukreja, Kshitij Sah, Gautam Gupta +5
IIIT Delhi, India · Singapore Institute of Technology · NVIDIA +1
Language models lose accuracy when a prompt contains non-evidential context: irrelevant statements from unrelated domains, or random off-distribution tokens, with no competing answer and no semantic overlap with the query. Such noise pervades retrieval-augmented, tool-using, and long-context deployments, yet the failure has lacked a mechanistic account. We trace the corruption to the Intrusion Pathway, which starts with MLPs enriching both query-subject and irrelevant-token representations, parroting heads carry prefix-token content to the prediction position, and relational readers largely avoid unrelated sources. Nevertheless, preserved subject enrichment and selective reading do not prevent intrusion in the broader computation. We then localize part of this interference to a sparse set of attention heads, which we call intrusion heads. Their prefix-derived writes promote distractor-related content, while their reduced support for the correct answer arises primarily from weaker non-prefix contributions rather than direct suppression by the prefix. The same heads transfer across tasks and distractor types and are largely weight-disjoint from the heads that integrate relevant contextual evidence, enabling a class-conditional intervention. Replacing the corrupted last-token residual with its clean counterpart, an oracle probe that requires a matched clean forward pass, recovers 97.6% on average of the distractor-induced drop across four downstream tasks. A deployable mask level knockout of about ten heads, roughly 1% of the model's attention heads, recovers 57% at bounded cost to helpful context.
Dikshant Kukreja, Gautam Gupta, Avni Mittal +8
TurboML · Indraprastha Institute of Information Technology, Delhi · Independent Researcher +4