Commutator Memory: Sparse, Path-Local Reading and Steering in Language Models

John Sweeney

cs.LGSep 28, 2026

Commutator Memory: Sparse, Path-Local Reading and Steering in Language Models

Authors: John Sweeney

Organizations: Sideplane AI

Abstract

Gradient updates on different data generally do not commute: training a language model on two data sources in opposite orders gives different weights, even with the same data and total exposure. Loss or benchmark deltas show that the models differ, not where. We ask whether this path dependence leaves a parametric training-history memory: a weight component that flips sign when the two sources are swapped, is localized in output space, changes the held-out loss gap between the two orders under targeted interventions, and reveals which trained model came from which order. For one small SGD step of size $η$ on each of sources $A$ and $B$ , the weight difference $θ_{AB}-θ_{BA}$ is, to leading order, $η^2 b_{AB}$ , where $b_{AB}=H_Bg_A-H_Ag_B$ is the Lie bracket of the two gradient fields at the base model. We define commutator memory by projecting the bracket through the logits into one score per vocabulary token; the scores sum to the bracket's prediction of the gap. The scores are localized: on three models, the same readout of the measured $θ_{AB}-θ_{BA}$ , or of a bracket from disjoint batches, shares 82-99% of the original top-20 tokens, versus 35-49% for norm-matched random directions. They are causally actionable: in Qwen-3-4B SFT, downweighting the ten tokens with the largest predicted share of the gap closes a median 32% of the measured gap, while frequency-matched tokens with near-zero scores have almost no effect. The weights themselves carry the component: projecting the difference between the two trained models onto $b_{AB}$ identifies which came from which order in 92% of cases across four LLMs (chance 50%). Controlled tests also cover matched-batch DPO, a frozen-rollout GRPO-style objective, and an AdamW endpoint check. The memory is defined per source pair, not per example, and its projection on $b_{AB}$ decays with further training.

Figures & tables

Figure 1: Commutator memory pipeline. One bracket produces an order forecast, token map, intervention target, and endpoint trace.

Object	Role	Origin
$b_{AB}=H_{B}g_{A}-H_{A}g_{B}$	effect of swapping $A$ and $B$ : $\theta_{AB}-\theta_{BA}=\eta^{2}b_{AB}+O(\eta^{3})$	inherited
$\sigma=\langle g_{E}(\theta_{\rm ref}),b_{AB}\rangle$	one scalar predicting which order has lower held-out loss	inherited
$\tau_{k}$ (commutator memory)	token-level decomposition of $\sigma$ ; $\sum_{k}\tau_{k}=\eta^{2}\sigma$	new
order-gap closure	fraction of the AB/BA NLL gap removed by an intervention	new
$\Delta s=\langle\theta_{AB}-\theta_{BA},b_{AB}\rangle$	which order produced each of two endpoints	new
$\theta_{\rm BO}$	bracket-only step; beats both orders under Prop. 1	partly inherited

Table 1: Plain-language map of the central objects (

A,B

: the two training data sources;

E

: the held-out evaluation slice). Inherited from prior work: the endpoint-difference identity ( Sweeney, 2026b , Lemma 2.1) , the scalar criterion at

\theta_{0}

( Rukhovich et al., 2025 ) and at

\theta_{\rm ref}

( Sweeney, 2026b ) , the bracket-only step ( Sweeney, 2026b , App. E.8) , and the fixed-clock AdamW regime, in which Adam’s buffers advance per step regardless of

\eta

( Sweeney, 2026a , Thm. 1) . This paper develops the token-level readout, order-gap interventions, paired-endpoint assignment, bracket-only guarantee, and AdamW closure test.

Figure 2: Concentration diagnostics for the token-level bracket readout. Lorenz curves show the operational finite-difference

\tau

readout across four reported main runs; Table 2 reports the separate fp32 random-direction control subset used for the support-specificity claim.

Model	Precision	80% mass mean/med.	Endpoint/resampled overlap	Random overlap
Llama-3.2-1B	fp32	1.47% / 1.47%	99% / 97%	35%
Qwen-2.5-1.5B	fp32	0.99% / 1.16%	99% / 97%	39–40%
Qwen-3-4B	fp32	0.30% / 0.042%	82% / 93%	36–49%

Table 2: Concentration and support-specificity checks from the fp32 random-direction control subset (3 pairs

\times

3 seeds/model). “80% mass”: fraction of the vocabulary holding

80\%

of the

|\tau|

mass (mean/median). Endpoint (the measured

\theta_{AB}-\theta_{BA}

) and resampled (a bracket from disjoint batches) columns report top-

20

overlap with the bracket support; the random column reports mean top-

20

overlap for norm-matched random controls.

Model	Condition	Winsor. mean	Median	95% CI	Gap shrinks
Qwen-3-4B	Harmful	+31.2%	+32.0%	[+21.4%, +39.7%]	75.6%
Qwen-3-4B	Random	+1.2%	+0.2%	[–5.8%, +8.7%]	50.0%
Qwen-2.5 †	Harmful	+48.6%	+47.9%	[+40.5%, +57.1%]	96.4%
Qwen-2.5 †	Random	–0.006%	–0.003%	[–0.03%, +0.02%]	39.3%

Table 3: Intervention results on two models (30 trials/pair, held-out evaluation). Qwen-3-4B reports all 90 trials. † Qwen-2.5 denotes Qwen-2.5-1.5B fp32 on the predefined above-floor subset (28/90 trials with

|\Delta_{\mathrm{baseline}}|>10^{-4}

). Harmful: the ten tokens whose

\tau_{k}

contributes most to the measured gap; Random: frequency-matched tokens with

\tau_{k}\approx 0

; entries are closure

C

in %; Winsor. mean:

5/95

winsorized mean, with its bootstrap CI; Gap shrinks: fraction of trials with a smaller ordering gap after the intervention (

68/90

45/90

27/28

11/28

by row).

Model (precision)	$\mathrm{sign}(\Delta s)$ correct	Wilson 95% CI
Llama-3.2-1B (1B, fp32)	14/18 = 77.8%	$[55\%,91\%]$
Qwen-2.5-1.5B (1.5B, fp32)	16/18 = 88.9%	$[67\%,97\%]$
Qwen-3-4B (4B, bf16)	18/18 = 100.0%	$[82\%,100\%]$
Llama-3.1-8B (8B, bf16)	18/18 = 100.0%	$[82\%,100\%]$
Combined	66/72 = 91.7%	$\mathbf{[83\%,96\%]}$

Table 4: Paired-endpoint training-order assignment via

\mathrm{sign}(\Delta s)

across four LLMs (6 pairs

\times

3 seeds = 18 pair-seed units per model). Replication across precision regimes (bf16 vs. fp32) in Appendix K .

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Pair	mean	flat CI	cluster CI	median	wrong-sign mean	random mean
fq:flan	0.675	[0.561, 0.767]	[0.425, 0.772]	0.796	$-1.083$	$+0.001$
fq:share	0.454	[0.157, 0.703]	[0.126, 0.536]	0.739	$-1.104$	$+0.001$
fq:ultra	0.647	[0.477, 0.798]	$[-0.386,\,0.669]$	0.778	$-1.874$	$-0.001$
flan:share	0.545	[0.247, 0.776]	$[-2.306,\,0.825]$	0.807	$-1.915$	$-0.000$
flan:ultra	0.720	[0.645, 0.788]	[0.455, 0.754]	0.785	$-1.174$	$+0.001$
ultra:share	0.454	[0.219, 0.665]	$[-0.107,\,0.604]$	0.735	$-1.928$	$+0.003$

Appendix

Table 5: Matched-batch bracket-correction closure on the six original DPO source pairs (Qwen-3-4B fp32,

\beta_{\text{DPO}}{=}0.1

\eta{=}2\text{e-}3

, 3 seeds

\times

10 trials each). Closure is heavy-tailed near the fp32 noise floor; per-pair seed-cluster CIs are descriptive diagnostics of within-pair variability with

n_{\rm clusters}=3

, not formal 95% inference. Columns: mean and median are the winsorized

5/95

mean and the median closure of the

\lambda{=}1

edit; flat CI and cluster CI are per-trial and seed-clustered bootstrap

95\%

intervals of the mean; wrong-sign mean and random mean are the mean closures of the

+\eta^{2}b_{AB}

edit and of a norm-matched random edit. Pair abbreviations: fq= false_qa , flan= flan_v2_p3 , share= sharegpt , ultra= ultrachat .

Figure 3: Matched-batch DPO bracket-correction closure, per original pair. Bars are winsorized (5/95) means with seed-clustered bootstrap intervals (3 seed clusters per pair) shown as descriptive diagnostics of within-pair variability, not formal 95% inference. Random direction near zero; wrong-sign edit at or below the Lemma 2.1 prediction of

-1

(red dashed line); first-order norm-matched control underperforms the bracket.

Pair	paired difference (mean)	CI 95%	flat diagnostic	direction
fq:flan	$-1.15\text{e-}3$	$[-1.93\text{e-}3,\,-4.30\text{e-}4]$	positive	mitigation
fq:share	negative	excludes 0	positive	mitigation
fq:ultra	negative	excludes 0	positive	mitigation
flan:share	—	includes 0	inconclusive	no effect
flan:ultra	negative	excludes 0	positive	mitigation
ultra:share	negative	excludes 0	positive	mitigation

Appendix

Table 6: DPO token reweighting with the corrected

\tau

on the six original pairs (Qwen-3-4B fp32,

\eta{=}2\text{e-}3

\alpha{=}0.05

, the top-100

|\tau|

tokens whose sign matches a probe measurement of the baseline gap, 3 seeds

\times

20 trials). Pair abbreviations as in Table 5 . Paired difference

=|\Delta_{\text{harmful}}|-|\Delta_{\text{random}}|

; flat diagnostic: whether its per-trial bootstrap CI excludes zero; direction: mitigation if the harmful edit shrinks the gap.

Pair	$\sum_{k}\tau_{k}$	$\Delta L_{\text{DPO}}^{\text{FD}}$	rel. err.
false_qa_vs_flan_v2_p3	$-1.247\text{e}{-}1$	$-1.247\text{e}{-}1$	$5.9\text{e}{-}5$
false_qa_vs_sharegpt	$-9.150\text{e}{-}2$	$-9.149\text{e}{-}2$	$2.6\text{e}{-}5$
false_qa_vs_ultrachat	$-1.805\text{e}{-}1$	$-1.804\text{e}{-}1$	$1.4\text{e}{-}4$
flan_v2_p3_vs_sharegpt	$-6.669\text{e}{-}2$	$-6.670\text{e}{-}2$	$2.4\text{e}{-}4$
flan_v2_p3_vs_ultrachat	$+1.900\text{e}{-}3$	$+1.907\text{e}{-}3$	$3.6\text{e}{-}3$
ultrachat_vs_sharegpt	$-3.304\text{e}{-}2$	$-3.307\text{e}{-}2$	$8.9\text{e}{-}4$

Appendix

Table 7: DPO

\tau

identity check on the original pairs. Each pair:

\sum_{k}\tau_{k}

computed by finite-difference attribution at

\varepsilon=0.1

v=\eta^{2}b_{AB}

, with

\eta=2\times 10^{-3}

(original-pairs protocol); compared to the directly-measured

\Delta L_{\text{DPO}}^{\text{FD}}

on the same DPO batches.

Pair	$\|\Delta_{\rm baseline}\|$	Single-step closure $C$ (%)	Spearman( $\tau_{k}$ , $\\|\Delta W_{k}\\|$ )
math-vs-legal	0.042	+80.3%	0.833
code-vs-legal	0.013	+28.4%	0.800
news-vs-legal	0.037	+23.9%	0.830
code-vs-news	0.004	+5.9%	0.832
news-vs-biomedical	0.001	$-32.9\%$	0.844
code-vs-biomedical	0.002	$-51.1\%$	0.821

Appendix

Table 8: Endpoint-correction closure by domain pair at

k=1

(Qwen-3-4B,

\lambda=1.0

, averaged over 3 seeds).

\Delta W_{k}

is output-layer row

k

\theta_{AB}-\theta_{BA}

Model	Dtypes	Top-1%	Top-5%	$\rho$	Sign
Qwen-3-4B	bf16/fp32 FD	53.0 (54.5 med.)%	51.0 (54.3 med.)%	$0.033\pm 0.051$	52.4%
Qwen-2.5-1.5B	fp32/fp32 FD	57.3 (57.1 med.)%	46.8 (47.5 med.)%	$0.521\pm 0.034$	70.6 $\pm$ 1.9%

Appendix

Table 9: Token-level ordering prediction as support recovery (45 experiments per model, 15 domain pairs

\times

3 seeds). Top-1% and top-5% are mean overlaps of the predicted and empirical high-effect token sets; equal-size-set baselines are 1% and 5%. Full-vocabulary Spearman is retained as a diagnostic, not the main metric for the heavy-tailed Qwen-3-4B readout.

Model	Scale	$\mathrm{sign}(\Delta s)$ correct	Wilson 95% CI
Llama-3.2-1B (bf16 storage)	1B	14/18 = 77.8%	$[55\%,91\%]$
Llama-3.2-1B (fp32)	1B	14/18 = 77.8%	$[55\%,91\%]$
Qwen-2.5-1.5B (fp32)	1.5B	16/18 = 88.9%	$[67\%,97\%]$
Qwen-2.5-1.5B (bf16 storage)	1.5B	16/18 = 88.9%	$[67\%,97\%]$
Qwen-3-4B (bf16 storage)	4B	18/18 = 100.0%	$[82\%,100\%]$
Llama-3.1-8B (bf16 storage)	8B	18/18 = 100.0%	$[82\%,100\%]$

Appendix

Table 10: Paired-endpoint training-order assignment via the

\mathrm{sign}(\Delta s)

estimator. Six runs across four unique models, with the

1

B and

1.5

B models replicated under both fp32 and bf16-storage precision regimes (identical sign-recovery in each). Counting each model once gives

66/72=91.7\%

(

\geq\!4

B-class

100\%

1

B-class

77.8\%

–

88.9\%

); the dual-precision count

96/108=88.9\%

shown here is a precision-replication check.

Model	Pair	Gini	pct for $80\%$	pct for $99\%$
Qwen-2.5-1.5B (fp32)	false_qa vs. flan_v2_p3	$0.960$	$1.26\%$	$30.1\%$
Qwen-2.5-1.5B (fp32)	ultrachat vs. sharegpt	$0.969$	$1.05\%$	$25.1\%$
Qwen-3-4B (bf16)	false_qa vs. flan_v2_p3	$0.999$	$0.003\%$	$0.70\%$
Qwen-3-4B (bf16)	ultrachat vs. sharegpt	$0.998$	$0.006\%$	$0.97\%$
Qwen-3-4B (bf16)	false_qa vs. ultrachat	$1.000$	$0.002\%$	$0.08\%$
Qwen-3-8B (bf16)	false_qa vs. flan_v2_p3	$0.997$	$0.019\%$	$2.90\%$

Appendix

Table 11: Per-pair DPO commutator memory concentration. “pct for 80%” = fraction of vocabulary required to account for

80\%

|\tau|

mass. These descriptive concentration rows do not include the random-direction support nulls used for the SFT specificity claim.

Pair	CI excludes 0	paired difference CI 95%	direction
evol_instruct_vs_truthful_qa	yes	excludes 0	mitigation
evol_instruct_vs_false_qa	no	includes 0	no effect
truthful_qa_vs_flan_v2_p3	yes	excludes 0	mitigation

Appendix

Table 12: Token-reweighting results on the three evaluable fresh pairs (Qwen-3-4B fp32, same protocol as the original pairs).

Pair	bracket mean	median	random	wrong-sign	first-order
evol:fq	$+0.436$	$+0.775$	$+0.001$	$-1.085$	$-0.573$
evol:flan	$+0.375$	$+0.774$	$+0.001$	$-1.059$	$-2.004$
evol:share	$+0.613$	$+0.802$	$-0.001$	$-0.904$	$-2.252$
truth:flan	$+0.378$	$+0.849$	$-0.000$	$-0.910$	$-0.132$
truth:share	$+0.501$	$+0.726$	$-0.001$	$-1.280$	$-0.956$
truth:ultra	$+0.509$	$+0.714$	$-0.001$	$-1.331$	$-0.805$

Appendix

Table 13: Matched-batch bracket-correction closure on six disjoint fresh UltraFeedback pairs (Qwen-3-4B fp32,

3

seeds

\times

10

trials each,

b_{AB}

recomputed per trial). Pair abbreviations: evol= evol_instruct , truth= truthful_qa ; other abbreviations as in Table 5 . All six pair medians are positive in

[0.71,0.85]

Pair	flat-bootstrap CI 95%	seed-cluster CI 95%
false_qa_vs_flan_v2_p3	[0.561, 0.767]	[0.425, 0.772]
false_qa_vs_sharegpt	[0.157, 0.703]	[0.126, 0.536]
false_qa_vs_ultrachat	[0.477, 0.798]	$[-0.386,0.669]$
flan_v2_p3_vs_sharegpt	[0.247, 0.776]	$[-2.306,0.825]$
flan_v2_p3_vs_ultrachat	[0.645, 0.788]	[0.455, 0.754]
ultrachat_vs_sharegpt	[0.219, 0.665]	$[-0.107,0.604]$

Appendix

Table 14: Original-pairs winsorized closure of the matched-batch DPO correction: flat bootstrap (1000 iter on the 30 trials) vs seed-clustered bootstrap (1000 iter cluster-resampling 3 seeds, then aggregating their 10 trials each). Both columns are descriptive diagnostics of within-pair variability; with

n_{\rm clusters}=3

per pair the seed-cluster percentile is too coarse for formal

95\%

inference. Bold rows have intervals strictly above zero under the corresponding estimator (descriptive, not a decision criterion).

Triplet $(A,B,C)$	Seed $42$ mean	Seed $43$ mean	Cells $\|\text{err}\|<5\%$	Triplet mean
code, legal, news	$0.95\%$	$1.11\%$	$12/12$	$1.03\%$
code, biomedical, legal	$5.17\%$	$15.16\%$	$7/12$	$10.17\%$
news, biomedical, math	$1.41\%$	$1.95\%$	$12/12$	$1.68\%$
Grand mean ( $36$ cells)	$\mathbf{4.29\%}$

Appendix

Table 15: Per-triplet relative additivity error of bracket edits on Qwen-3-4B. “Per-seed mean” averages over the

6

(composite, domain) cells for that seed; “Cells

|\text{err}|<5\%

” counts the cells whose relative additivity error is under

5\%

; “Triplet mean” averages over all

12

cells of the triplet. The grand mean over

36

cells is

4.29\%

Pair	$\rho_{\text{topK}}$	$\rho_{\text{full}}$	$K$	$V$	top-K	non-top	null	CG topK	CG full
code/news	$0.931$	$0.966$	$928$	$151\,936$	$0.868$	$0.066$	$0.067$	$2.3\!\cdot\!10^{-2}$	$2.1\!\cdot\!10^{-2}$
code/math	$0.880$	$0.923$	$884$	$151\,936$	$0.775$	$0.077$	$0.148$	$2.5\!\cdot\!10^{-2}$	$2.3\!\cdot\!10^{-2}$
legal/bio	$0.942$	$0.970$	$485$	$151\,936$	$0.888$	$0.053$	$0.060$	$2.7\!\cdot\!10^{-3}$	$6.7\!\cdot\!10^{-3}$
mean	$0.918$	$0.953$			$0.844$	$0.065$	$0.092$

Appendix

Table 16: Observable/null decomposition of

b_{AB}

on Qwen-3-4B, selected subspace

\mathcal{S}

(last-block MLP + LM head,

d\approx 4.64\times 10^{8}

\eta=6.417\times 10^{-4}

k=1

, seed 42.

\rho_{\rm obs}(B):=\cos(x^{\star},b_{AB})

at the achieved CG relative residual shown in the final two columns; Tikhonov damping

\nu=10^{-6}

. Decomposition columns report

\rho_{\text{topK}}^{\,2}

\rho_{\text{full}}^{\,2}-\rho_{\text{topK}}^{\,2}

, and

1-\rho_{\text{full}}^{\,2}

; exact only in the zero-damping / full-convergence limit, reported here as approximate empirical fractions.

Model	$\eta$	Regime
Qwen-3-4B	$6.42\times 10^{-4}$	cube-root
Qwen-2.5-1.5B	$1.91\times 10^{-3}$	cube-root
Llama-3.1-8B	$1.28\times 10^{-3}$	cube-root
Llama-3.2-1B	$8.0\times 10^{-4}$	cube-root
GPT-2 (reference)	$4.16\times 10^{-3}$	strongly powered

Appendix

Table 17: Learning rates calibrated per model.

Figure 4: Dose-response of loss reweighting on Qwen-3-4B (an earlier sweep, run before the held-out evaluation of Table 3 ). Median gap reduction (%) against the weight

\alpha

applied to harmful (red), helpful (green), and random (gray) token sets;

\alpha<1

downweights. Downweighting harmful tokens gives a positive response at every tested

\alpha<1

, while random controls remain near zero. Shaded bands show 95% confidence intervals (30 trials each).

Model	Precision	Method	Harmful (med.)	Random (med.)	Sep.
Qwen-3-4B	bf16	Loss reweighting	+32.0%	+0.2%	✓
Qwen-3-4B	bf16	LR scaling †	+31.2%	+1.7%	✓
Qwen-3-4B	bf16	Projection †	+14.6%	–5.1%	$\sim$
Qwen-3-4B	bf16	Regularizer †	+8.3%	+2.9%	$\times$
Qwen-2.5-1.5B	fp32	LR scaling	+47.9%	–0.003%	✓

Appendix

Table 18: Intervention type comparison (

\alpha=0.1

, 30 trials/pair). Sep.: ✓ clear harmful-vs-random separation,

\sim

weak,

\times

none. † Qwen-3-4B LR scaling, Projection, and Regularizer rows are from earlier runs without held-out evaluation; the Loss reweighting row uses held-out evaluation. The Qwen-2.5-1.5B row is the full-fp32 above-floor subset reported in Table 3 .

Metric	Value
Plotted points	$n=12$
$R^{2}$ of the $k^{2}$ fit (mean over seeds)	0.994
Pooled predicted-vs-actual $R^{2}$	0.990
Best-fit slope in pooled plot	0.706
Mean magnitude ratio at $k=1$	$1.004\pm 0.015$

Appendix

Table 19: Magnus/bracket validation details (Qwen-3-4B). The slope

0.706

in Fig. 5 is the pooled regression slope over

k\in\{1,2,4,8\}

; the

k=1

ratio is reported separately because the BCH expansion is most accurate there.

Figure 5: Magnus/bracket validation (Qwen-3-4B). Measured held-out gap

\mathcal{L}_{E}(\theta_{AB})-\mathcal{L}_{E}(\theta_{BA})

against its leading bracket prediction

k^{2}\eta^{2}\langle g_{E}(\theta_{\rm ref}),b_{AB}\rangle

for 3 seeds and

k\in\{1,2,4,8\}

SGD steps per source. The

k^{2}

fit has mean

R^{2}=0.994

; the pooled predicted-vs-actual fit has slope

0.706

and

R^{2}=0.990

; signs agree at all 12 points.

Figure 6: Top-10 tokens by $|\tau_{k}|$ for each domain pair (Qwen-3-4B). Color encodes signed $\tau_{k}$ (red = order AB disadvantages token; blue = advantages). Tokens are almost entirely disjoint across pairs (Jaccard = 0.004), confirming domain specificity. Code vs. legal is dominated by digit tokens; code vs. biomedical by BPE suffix fragments.

Figure 7: Label frequency vs. $|\tau_{k}|$ for all vocabulary tokens (code vs. news). High- $|\tau_{k}|$ tokens span the full frequency range; low rank correlation ( $\rho\approx 0.23$ ) argues against a Zipf-only explanation.

Slice (run)	Tokens, and the readout of the colored ones
code/news	Ask H N : Should I include my GPA and /or transcripts when applying for jobs ? Ask : $\tau=-3.51$ (rank $1$ )
code/biomedical	Comput ers are a ubiquitous part of the amb ulatory health care environment . ulatory : $\tau=+0.437$ (rank $8$ ); of : $\tau=-0.364$ (rank $19$ )
code/news (iterative run)	Facebook Dating launch blocked in Europe after it fails to show privacy workings Facebook : $\tau=-1.75$ (rank $5$ ); its NLL is $10.074$ under AB and $10.896$ under BA, and $10.920$ after bracket correction ( $97.0\%$ of its NLL gap closed)

Appendix

Figure 8: Tokenwise view of the held-out sentences in Section 5.2 , with one box per Qwen-3-4B token. A token is colored if it is among its pair’s

20

largest-

|\tau|

tokens in the slice-level readout (rank in parentheses): blue for

\tau<0

(order AB advantages the token) and red for

\tau>0

(AB disadvantages it), as in Fig. 6 . Uncolored tokens are outside that set. The readout is aggregated over the evaluation slice (evaluation seed 44) rather than scored per sentence. The last row also tracks the Facebook position through the instrumented iterative-correction run, which targets

\theta_{BA}

and overshoots it slightly (by

0.025

Method	Qwen-3-4B	Qwen-2.5-1.5B	Llama-3.1-8B
Bracket	72% (13/18)	100% (18/18)	100% (18/18)
Grad Cosine	11% (2/18)	33% (6/18)	22% (4/18)
Grad Norm	11% (2/18)	56% (10/18)	22% (4/18)
Random	44% (8/18)	50% (9/18)	39% (7/18)

Appendix

Table 20: Ordering prediction accuracy on 18 pair-seed units (6 domain pairs

\times

3 seeds per model).

Model	Storage/HVP	$k$	Gini	$\tau$ recall@1%V	gap/gap 1	$k^{2}$
Qwen-3-4B	bf16/fp32	1	0.9998	100%	1.0	1
		2	0.9998	100%	1.5	4
		4	0.9998	100%	2.0	16
		8	0.9997	100%	3.1	64
Qwen-2.5-1.5B	fp32	1	0.9982	99%	1.0	1
		2	0.9982	99%	4.0	4

Appendix

Table 21: Multi-step persistence with fresh batches per step (averaged over 12 experiments per model).

\tau

recall@1%V is the fraction of the empirical top-1% set (among tokens occurring as labels) at step count

k

contained in the single-step bracket top-1% full-vocabulary

\tau

set. Qwen-3-4B uses bf16 storage with fp32 HVPs; Qwen-2.5-1.5B is full fp32. gap/gap 1 : measured gap relative to

k=1

;

k^{2}

: the leading-order prediction.

Figure 9: Iterative bracket correction convergence (averaged per pair). Five of six pairs reach

>

85% closure within 20 iterations; news-vs-biomedical (

k{=}1

gap

8.5\times 10^{-4}

, near the noise floor) diverges. On the other five pairs the iterative projection closes

90\%

, against

13.5\%

for the single-step full-bracket edit averaged over all pairs and step counts (

6.7\times

; App. I ).

Figure 10: Per-position NLL trajectories during iterative bracket correction (code-vs-news, seed=42,

k=1

). Each line tracks one sequence position’s NLL; dashed lines show

\theta_{BA}

targets.

Pair $(A,B)$	$C$	Seed	$M_{0}$	$R_{4}$	$R_{8}$	$R_{16}$
code/news	legal	8042	0.663	$+$ 0.286	$+$ 0.175	$+$ 0.134
code/news	legal	9042	0.530	$+$ 0.291	$+$ 0.219	$+$ 0.113
code/legal	news	8042	2.513	$-$ 0.433 ∘	$-$ 0.309 ∘	$-$ 0.170 ∘
code/legal	news	9042	2.337	$-$ 0.348 ∘	$-$ 0.257 ∘	$-$ 0.157 ∘
code/biomedical	news	8042	0.721	$+$ 0.583	$+$ 0.446	$+$ 0.292
code/biomedical	news	9042	0.577	$+$ 0.576	$+$ 0.439	$+$ 0.283

Appendix

Table 22: Continuation study, all

20

runs (Qwen-3-4B; SGD continuation on domain

C

M_{0}

is the initial trace on the predicted scale, and

R_{h}

is the signed fraction retained after

h

later updates. ∘ : the run does not retain the aligned trace at that horizon (

M_{h}\leq 0.01

, including sign reversals). The two news/math runs do not start with an aligned trace (

M_{0}\leq 0.01

), so

R_{h}

is not reported for them. Rows above the middle rule are non-math pairs; rows below include math.

Experiment family	Hardware	GPU-hours	Notes
Sparsity (4 models)	RTX 4090 / Pro 6000	$\sim$ 2	R8b, R9, R13_sparsity, R14_sparsity, R15_sparsity, R16
Baselines (4 models)	RTX 4090 / A40	$\sim$ 1	R8b, R9, R13
SFT interventions	RTX Pro 6000 / RTX 4090	$\sim$ 3	R14a, R14b, R15 (full fp32 for Qwen-2.5)
Endpoint correction / multi-step	A40 / L40 / Pro 6000	$\sim$ 5	R26, R27, R28, R19, R20
Iterative bracket correction	L40	$\sim$ 3	R29, R30 (instrumented per-position NLL)
DPO sparsity (3 models)	A40 / Pro 6000	$\sim$ 2	R50, R52, R60

Appendix

Table 23: Indicative GPU-hours and hardware per experiment family. Total project compute is driven primarily by the DPO matched-batch runs and the endpoint-correction runs; sparsity/baseline runs are lower-cost. The rows sum to about 32 GPU-hours; the project total is cumulative over all runs. Run identifiers are listed under Repository pointers below.

Condition	Parameter closure $r$	Log-prob closure $r$	Loss closure $r$
Bracket edit	$0.952$	$0.886$	$0.805$
Random norm-matched	$-0.414$	$-0.001$	$0.010$
Wrong sign	$-0.997$	$-0.987$	$-1.008$
First-order norm-matched	$-0.410$	$-0.197$	$-2.384$
Unmatched rollouts (negative control)	$-0.396$	$-0.286$	$-0.158$

Appendix

Table 24: Single-step frozen-rollout GRPO-style controls. Values are medians over 20 trials unless a CI is shown. Unmatched rollouts: the bracket from one rollout set applied to endpoints trained on another. Reported

r

are closure ratios

r:=1-\|\text{edited}-\text{target}\|/\|\text{baseline}-\text{target}\|

against the target AB-vs-BA displacement on held-out rollouts.

$T$	identity cosine	$\zeta$	relative residual	$C_{\theta}$
2	$0.997$	$0.967$	$0.083$	$0.917$
4	$0.998$	$0.904$	$0.122$	$0.878$
8	$0.988$	$0.818$	$0.267$	$0.733$
16	$0.991$	$0.884$	$0.188$	$0.812$

Appendix

Table 25: Block-sequential frozen-rollout

T

-sweep.

C_{\theta}

is the parameter-space closure of the composed correction.

Explore similar work

Sep 28, 2026cs.LG

Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models

Memorization has been proposed as a mechanism to explain how language models fit the tail of their training distributions, but its training dynamics are not understood well. In this work, we take a fine-grained look at memorization by decomposing the loss trajectory of memorized sequences over training and model parameters. Across the Pythia family, we study memorization of duplicated training sequences (recitation) and rare ones (recollection). We find that memorization in both cases is characterized by sequence-level gradient alignment, though recitation suffers from misalignment with other training influences which causes forgetting, explaining the necessity for higher duplication of these examples. We further show that the lower model layers are the most involved in memorization and forgetting. Predicting memorization, our decomposition improves over a cross-entropy baseline, especially in larger models and early in training. Intervening on a small set of highly influential parameters we are able to ablate memorization in the final model. Together, these findings advance our understanding of how memorization develops during training and offer insights for predicting and intervening on it.

Florian Eichin, Philipp Mondorf, Andrei Mircea +3

MaiNLP, Center for Information and Language Processing, LMU Munich & Munich Center for Machine Learning (MCML), Germany · Mila – Quebec AI Institute & University of Montreal, Canada · Saarland University, Germany

Memorization Recurrent Model Repetition

Ask PDF

Jul 30, 2026cs.LG

Subtract, Transport, or Replay? Auditable Deletion from Language-Model Memory

Exact deletion from persistent language-model memory depends on whether a record's effect remains addressable after later computation. Native Kimi Delta Attention (KDA) gives a negative result for the tested receipt interface: the corpus-pooled raw recurrent contribution changes by 12-49% with the suffix and remains 8-49% after a decay-ledger correction. Native omission also changes later transition and write terms and other active caches. Frozen-input transport succeeds on its fixed-input control; the changed terms place native omission outside the tested receipt classes. Checkpoint replay supplies the evaluated recomputation path; zero residual on final logits and all 80 audited KDA arrays verifies restoration across the declared checkpoint surface. The complementary result is constructive. We retrofit support-vector memory into frozen Gemma 3 without attention transfer, low-rank recovery, distillation, adapters, or language-model parameter updates. Prefix-mass preservation and one box per prefix solve give base-matched admission at 4B with 1.85% perplexity overhead. At 1B and 4B, verified deletion agrees with its conditional retained-key refit within 1.3e-10 maximum next-token KL; behavioral attacks at 4B reach never-stored or chance baselines. Across 1B, 4B, and 12B, the 4B checkpoint uniquely combines base-matched admission with low overhead. The paper's two contributions are a negative result for native KDA's tested receipt classes and a positive training-free construction for addressable pretrained memory.

Vishwajith Ramesh

Vy Labs, Inc.

Large Language Model Memory Deletion Kimi Delta Attention Conversational Memory Replay Omissions+1

Ask PDF

Sep 1, 2026cs.LG

How Do Language Models Choose Between Context and Memory?

When contextual information conflicts with knowledge stored in model parameters, activation directions can be used to decode and steer which source the model follows. However, successful steering does not establish that the unedited model uses those directions to choose between sources, or that they remain effective across tasks. To test these possibilities, we vary the stated authority of contextual claims while holding their content fixed. We first estimate authority directions from prompts in which context and parametric knowledge agree, then test their causal contribution when the two sources conflict. Interchanging naturally occurring activation values along these directions between matched high- and low-authority prompts reproduces 30--68% of the authority-induced shift in source choice across Qwen, Llama, and OLMo models, whereas matched controls reproduce almost none. We next ask what transfers across tasks: the learned direction versus the activation values exchanged along it. Using a direction learned on another task closed 9% of the source-choice gap, compared with 57% when learned on the task being evaluated. Both interventions exchanged activation values from the evaluated task. In a separate experiment, we kept its learned direction but exchanged values taken from another task, which closed 68% of the gap. These results show that authority-related activation values can causally influence source choice across tasks when inserted along directions learned for the task being evaluated.

Benjamin Shih, John Winnicki, Arianna Cao

Stanford University · Perpetual Labs

Topic pending

Ask PDF