SFT Leaves a Reproducible Signature in Activation Geometry: Calibrating Post-Training Interpretability Against Run-to-Run Noise

Xupeng Chen; Chenqian Le; Jiayi Cheng; Runhao Li; Michael Yue

cs.AIOct 12, 2026

SFT Leaves a Reproducible Signature in Activation Geometry: Calibrating Post-Training Interpretability Against Run-to-Run Noise

Authors: Xupeng Chen, Chenqian Le, Jiayi Cheng, Runhao Li, Michael Yue

Organizations: New York University · University of Southern California · Columbia University

Abstract

A model diff credits post-training with whatever separates a base checkpoint from one post-trained descendant, presuming a rerun would land close by. On the sparse co-activation graph of Qwen3-1.7B it does not: two LoRA-SFT reruns sit over half as far apart as either sits from the base. We score each base-to-post contrast against the run-to-run band of its own recipe, and show that $k$ runs certify no better than $p=1/(k+1)$ under run exchangeability, however large the effect, so the band $z$ -score is a descriptive scale. SFT leaves a large, reproducible signature: base $\to$ SFT displaces the graph by 17.8 pairwise-distance SDs beyond the mean distance between four independent LoRA-SFT runs, and the pre-registered estimator replicates on Llama-3.1-8B with Llama-Scope. At 29 steps each, four SFT and four DPO runs occupy disjoint displacement ranges, a gap $5.0\times$ and $5.6\times$ the spreads of the two arms, without any band. All bands use graphs built on one GPU architecture. Three other readings stop short of a verdict: DPO band counts (3/4, 1/4, 3/4 contrasts outside at 12, 29, 60 steps) show no consistent dose ordering while its displacement stays flat; light GRPO is a secondary non-detection, not an absence, and we run no equivalence test; and the graph concentration read as an RL signature is already in the zero-RL base. A pre-registered causal sweep at three edge budgets on 4,816 held-out prompts found the displaced sources load-bearing ( $12\times$ a matched random control in mean, $3.5\times$ in median at 10,000 edges), but their stage-specificity interaction reversed at two budgets and the third failed its faithfulness gate, so we claim calibrated structural displacement, not a changed mechanism. Scope: one backbone, one SFT recipe and general-text probe states for the full protocol.

Figures & tables

Figure 1: LoRA-SFT moves the co-activation graph further from its base than its own reruns sit apart, on two backbones; DPO’s contrasts sit at their band’s edge, light GRPO’s inside it. Rows are the arms of Table 1 ; shading spans the seed-pair distances (open circles; a range, not an interval); filled points are base-to-post contrasts (diamond: audited SFT checkpoint); counts are contrasts beyond every seed pair. DPO’s band counts (

3/4

1/4

3/4

contrasts outside at

12/29/60

steps) follow no dose ordering. All graphs in a row were built on one GPU architecture (H200). Drawn by code/analysis/fig1_r1_strip.py from results/{sftband_20260926,graph_build_audit_20261005}/ .

Backbone	Arm	$k$	seed-pair $D$	$\sigma_{\mathrm{seed}}$	contrast $D$	outside	$z$
Qwen3-1.7B	LoRA-SFT, 29 steps	4	$0.0344$ – $0.0381$	$0.0014$	$0.0571$ – $0.0615$	$5/5^{a}$	$-19.0$ to $-15.8$
Llama-3.1-8B	LoRA-SFT	4	$0.0222$ – $0.0257$	$0.0013$	$0.0442$ – $0.0452$	$4/4$	— b
Qwen3-1.7B	LoRA-DPO, 12 steps	4	$0.0298$ – $0.0325$	$0.0011$	$0.0317$ – $0.0336$	$3/4$	$-1.86$ to $-0.06$
	LoRA-DPO, 29 steps	4	$0.0303$ – $0.0334$	$0.0012$	$0.0317$ – $0.0356$	$1/4$	$-2.76$ to $+0.52$
	LoRA-DPO, 60 steps	4	$0.0294$ – $0.0329$	$0.0012$	$0.0317$ – $0.0367$	$3/4$	$-4.77$ to $-0.71$
Qwen3-1.7B	GRPO family c	5	$0.0354$ – $0.0394$	$0.0011$	$0.0338$ – $0.0386$	$0/5$	$-1.0$ to $+3.2$

Table 1: SFT moves the graph further than its own reruns on two backbones; DPO’s band counts follow no dose ordering; light GRPO is not resolved from a wider band. Each arm has

k

independently seeded runs and

k(k-1)/2

seed pairs; entries are displacements

D=1-J

(Eq. 2 ). “Outside” counts contrasts more displaced than every seed pair;

z

is descriptive (Proposition 1 ) and computed at full precision. Base and run graphs of every row were built on one GPU architecture (H200); the archived readings of the submitted version, which crossed architectures, are in Table 4 . Receipts: results/{sftband_20260926,graph_build_audit_20261005}/ , recomputed by code/scripts/samearch.py (App. F ).

Figure 2: The concentration a single-checkpoint study would credit to RL is already present in the zero-RL base. One row per checkpoint in the base-diff summary. (a) Top-

1\%

inbound share: every Qwen checkpoint sits on or next to its base (vertical rule), far from the degree-preserving permutation null of the base graph (

95\%

range shaded). (b) Change in Gini against the base; the shading is

\pm

the largest

|\Delta\mathrm{Gini}|

among the five Qwen rows (

0.005

), not a statistical null. (c) Top-

32

hub Jaccard against the base. Gemma-2-2B-it is a different family and a heavier recipe; the comparison is descriptive. Receipts: notes/basediff_{qwen,gemma}_summary.json , h1_permutation_null_1p7b_l14to22.json .

Stage (vs. base)	$\Delta$ Gini	hub Jacc	Stage increment	edge Jacc	hub Jacc
SFT	$+0.009$	$0.73$	base $\to$ SFT	$0.669$	$0.73$
DPO	$+0.004$	$0.60$	SFT $\to$ DPO	$0.796$	$0.73$
RLVR-PPO	$+0.005$	$0.60$	DPO $\to$ PPO	$0.958$	$0.94$
RLVR-GRPO	$+0.005$	$0.56$	DPO $\to$ GRPO	$0.906$	$0.88$

Table 2: At production scale SFT and DPO write most of the displacement, but one checkpoint per stage gives no seed band to score it against (Tulu-3 staircase, Llama-3.1-8B, Llama-Scope layers

16{\to}25

n{=}500

, unnormalized v1 operating point; at the normalized one

\Delta

Gini

{}\leq 0.019

, App. M ). Left: levels vs. base (base Gini

0.780

). Right: stage increments; the reference is one PPO/GRPO run pair (

0.903

), not a seed distribution. Receipts: notes/tulu_.json (v1; tulu_v2_ normalized).

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Sanity check: GRPO did learn the behavior it was trained for. On the held-out BFCL validation split (

100

items,

40

20

20

20

over four categories, from the same release files as the training rows; accuracy is the unweighted category mean), GRPO improves accuracy

+0.36

over base (95% CI

[+0.21,+0.50]

); SFT’s

+0.11

is not resolved from zero by this harness (

[-0.03,+0.24]

). Intervals are stratified normal approximations and the gains are unpaired ( notes/bfcl_behavior_ci.json , code/scripts/bfcl_behavior_ci.py ). Only behavior is plotted — there is deliberately no internal axis here , because placing the two on one pair of axes is the withdrawn dissociation claim drawn: accuracy is measured on BFCL task states, displacement on general text, and the matched on-task internal arm fails its own positive control (App. J , L5). The corresponding internal readings are reported where they are calibrated (Sec. 4.2 ), not here.

‡ no $z$ : the trained-SAE cosine null takes $3$ distinct values over $15$ pairs
	trained SAE	random-dir
seed null, edge Jaccard ( $15$ pairs, mean $\pm$ SD)	$0.9626\pm 0.0016$	$0.8802\pm 0.0025$
base $\to$ SFT edge Jaccard ( $z$ )	$0.937$ ( $-15.9$ )	$0.840$ ( $\mathbf{-16.4}$ )
base $\to$ GRPO edge Jaccard ( $z$ )	$0.962$ ( $-0.5$ )	$0.880$ ( $-0.0$ )
base $\to$ SFT weighted edge cosine	$0.9976$ (n/a ‡ )	$0.9645$ ( $\mathbf{-16.9}$ )
(SD $0.000046$ , below the $0.0001$ reporting step) — not precision-adequate.

Appendix

Table 3: Random-dictionary control (pile arm): a single random dictionary preserves the SFT separation and the GRPO non-detection against the GRPO-family band. The trained-SAE column reads the archived graphs (App. F ); the random-direction graphs were not rebuild-tested. The displacement audit re-run with the trained SAE replaced by a random-direction encoder (matched

d_{\text{SAE}}=32{,}768

, top-

k

calibrated to the trained SAE’s mean

L_{0}\approx 50

); the seed null is recomputed from the same

15

GRPO-family pairs under each dictionary. One random draw (seed

0

): the control shows the detection does not need the trained dictionary, not that it is invariant across random ones. Random projections churn more in absolute terms (lower null mean), but the standardized separation is preserved. Receipt: notes/circuit_rewiring_randsae.json .

Arm	built on	$k$	seed-pair $D$	$\sigma_{\mathrm{seed}}$	contrast $D$	outside	$z$
Qwen LoRA-SFT, 29 steps	L40S / H200, A100 a	4	$0.0344$ – $0.0381$	$0.0014$	$0.0588$ – $0.0630$	$5/5$	$-20.1$ to $-17.1$
Llama LoRA-SFT	A100 / H200	4	$0.0222$ – $0.0257$	$0.0013$	$0.0434$ – $0.0444$	$4/4$	—
Qwen LoRA-DPO, 12 steps	L40S / A100	4	$0.0294$ – $0.0357$	$0.0022$	$0.0327$ – $0.0354$	$0/4$	$-1.08$ to $+0.13$
Qwen LoRA-DPO, 29 steps	L40S / H100	4	$0.0303$ – $0.0334$	$0.0012$	$0.0323$ – $0.0365$	$2/4$	$-3.58$ to $+0.03$
Qwen LoRA-DPO, 60 steps	L40S / H100	4	$0.0294$ – $0.0329$	$0.0012$	$0.0340$ – $0.0359$	$4/4$	$-4.14$ to $-2.58$
Qwen DPO, matched construction	L40S / H200	4	$0.0315$ – $0.0357$	$0.0019$	$0.0330$ – $0.0356$	$0/4$	$-0.97$ to $+0.36$

Appendix

Table 4: Archived build: the readings of the submitted Table 1, recomputed at full precision from the archived graphs. Base and run graphs of most rows were built on different GPU architectures (column “built on”: base / runs), so these readings are superseded by Table 1 . The submitted version printed

z

from four-decimal similarities (DPO

29

steps

-3.61

-0.01

, audited SFT

-19.9

); counts are unchanged by that rounding. Recomputed by code/scripts/samearch.py from notes/ .

Readout	LoRA-SFT ( $k{=}20$ )	full SFT ( $k{=}4$ )	DPO, 29 steps ( $k{=}4$ )	GRPO ( $k{=}5$ )
edge Jaccard	$20/20$	$4/4$	$1/4$	$0/5$
weighted edge cosine	$20/20$	$4/4$	$1/4$	$0/5$
SAE-code difference	$20/20$	$4/4$	$3/4$	$1/5$
dense residual distance	$20/20$	$4/4$	$2/4$	$1/5$
$1-$ linear CKA	$20/20$	$4/4$	$0/4$	$1/5$

Appendix

Table 5: Discussion-period band counts by readout (contrasts outside their own band; one GPU architecture; same-job bases). LoRA-SFT and full-parameter SFT carry registered branches (L1, F1, RS1); DPO at

29

steps and GRPO are descriptive (GRPO: five runs, the baseline checkpoint being deleted). Receipts: results/sftband_20260926/sftband_{verdict,readouts}.json .

Top-32 hub Jaccard: base $\cap$ GRPO $0.94$ , SFT $\cap$ GRPO $1.00$ , base $\cap$ SFT $0.94$
Checkpoint	top-1% (active)	Gini	# active dst features
Base (zero RL)	$0.427$	$0.728$	$2226$
SFT	$0.416$	$0.723$	$2229$
GRPO	$0.429$	$0.730$	$2225$
$\Delta$ (GRPO $-$ base)	$+0.002$	$+0.002$	—

Appendix

Table 6: Headline control: concentration is base-set across training stages ( Qwen3-1.7B , qwen-scope src

14\to

dst

22

n{=}500

, base-trained SAE). The zero-RL base already shows the full concentration; RL-attributable deltas are negligible and

31

–

32

of the

32

top hubs are shared (Jaccard

0.94

–

1.00

). An uncontrolled single-checkpoint analysis would have read the GRPO column alone as an RL signature.

Family / SAE	Checkpoint	Post-training	$\Delta$ Gini	$\Delta$ top-1%	hub Jaccard
Qwen3-1.7B / qwen-scope	base	— (Gini $0.728$ )	—	—	—
	SFT	SFT (LoRA)	$-0.005$	$-0.011$	$0.94$
	GRPO	accuracy-only GRPO	$+0.002$	$+0.002$	$0.94$
	GRPO-s4	GRPO (seed)	$\phantom{+}0.000$	$+0.001$	$1.00$
	GRPO-s5	GRPO (seed)	$+0.001$	$+0.002$	$1.00$
	shaped- $\lambda 0.1$	shaped-reward GRPO	$\phantom{+}0.000$	$\phantom{+}0.000$	$1.00$

Appendix

Table 7: The base control replicates across

5

Qwen recipes/seeds and a second family. Width-robust deltas (base

\to

post) of the same co-activation-graph concentration. Within Qwen ( qwen-scope ,

32

n{=}500

): all five post-training runs give

|\Delta\text{Gini}|\leq 0.005

and hub Jaccard

0.94

–

1.00

. Cross-family ( Gemma-2-2B

+

gemma-scope , JumpReLU,

16

n{=}500

): the zero-RL base already carries the concentration (Gini

0.708

); the larger -it shift reflects heavy production instruct-tuning, not light GRPO; with one heavy checkpoint and a possible chat-template confound (App. G ) we do not attribute it to intensity.

Figure 4: Available layer-pair coverage and concentration. Every point is one source-to-destination co-activation graph; diamonds are destination means and rules the observed min–max range. Numbers above the plot count available source layers. The lower panel shows the unbalanced source/destination coverage. The rise from about 22% to 49% is descriptive of this available scan, not an isolated destination-depth effect. Terminal layer 35 is excluded as unembedding-dominated. Source: the 47 committed Qwen3-8B layer-pair receipts ( notes/attribution_graph_v1_qwen3-8b_lto.json ); generator code/analysis/fig7_concentration_vs_depth.py .

Percentile	Edge weight	Feature ID	Total inbound $w$	In-degree
$p_{25}$	$14.3$	30329	$71560$	2061
median	$17.2$	26766	$9453$	241
$p_{75}$	$23.9$	10420	$3212$	88
$p_{90}$	$36.4$	25789	$2537$	80
$p_{99}$	$112.2$	11799	$1679$	48
max	$3597$	23554	$1109$	36

Appendix

Table 8: H1 edge-weight percentile distribution (left) and the six destination features with the largest total inbound weight (right) ( Qwen3-1.7B + qwen-scope-3-1.7b , src

14\to

dst

22

). The two halves are independent summaries of the same

10{,}000

-edge graph; rows do not correspond across the divider.

SAE family	Model	top-1% (active)	Gini	top-1% (dict)
qwen-scope (resid)	Qwen3-1.7B	$42.7\%$	$0.728$	$71.1\%$
qwen-scope (resid)	Qwen3-8B	$35.2\%$	$0.719$	$84.0\%$
transcoder (circuit-tracer)	Qwen3-0.6B	$35.9\%$	$0.688$	—
transcoder (circuit-tracer)	Qwen3-1.7B	$31.1\%$	$0.654$	—
transcoder (circuit-tracer)	Qwen3-14B	$32.6\%$	$0.645$	—

Appendix

Table 9: H1 concentration is scale-stable. Top-1% (of active features) inbound-weight share and Gini at matched relative depth (

n{=}500

10{,}000

edges), across two SAE families and

0.6

–

14

B. Both families are flat-to-slightly-decreasing with scale; the dictionary-relative “top-1%-of-

d_{\text{SAE}}

” (last column) rises only because wider dictionaries place more features in a fixed

1\%

. The qwen-scope ( Deng et al., 2026 ) rows are the committed receipt gini_width_control_qwen_1p7b_vs_8b.json ; the transcoder rows (circuit-tracer Qwen3 transcoders ( Hanna et al., 2025 ) ) come from the sweep job scripts in the supplement (no committed per-run summary JSON).

† structurally pinned: TopK dictionary with $k{=}50$ ; excluded from all Qwen metric counts.
	narrowing	Qwen3-1.7B / qwen-scope l22			Gemma-2-2B / gemma-scope l20
Metric	predicts	base	$\Delta$ SFT	$\Delta$ GRPO	base	$\Delta$ -it
mean $L_{0}$	$\downarrow$	$50.0^{\dagger}$	$+0.0^{\dagger}$	$+0.0^{\dagger}$	$184.8$	$+13.8$
firing-freq Gini	$\uparrow$	$0.683$	$-0.000$	$+0.000$	$0.494$	$-0.007$
mean-act Gini	$\uparrow$	$0.733$	$-0.001$	$+0.000$	$0.643$	$+0.002$
usage entropy (norm)	$\downarrow$	$0.895$	$+0.000$	$-0.000$	$0.950$	$-0.000$

Appendix

Table 10: Activation-side “narrowing” metrics are base-dominated (light RL) or move against narrowing (heavy instruct). Base

\to

post deltas of four common single-checkpoint SAE metrics (

n{=}500

, dst-layer SAE). Qwen3-1.7B (light GRPO, qwen-scope l22): all metrics base-dominated,

|\text{rel }\Delta|<1\%

. Gemma-2-2B (heavy SFT+RLHF+distill, gemma-scope l20): mean

L_{0}

rises (less sparse) and firing-freq Gini falls (less concentrated) — the opposite of the predicted RL “narrowing”. Narrowing arrow shows the direction RL would move each metric under the entropy-collapse/narrowing hypothesis. The mean-

L_{0}

row is informative only in the Gemma columns : qwen-scope is a fixed-

k{=}50

TopK dictionary, so its

L_{0}

is pinned by construction and its Qwen entries (greyed) are a property of the SAE, not a measurement of the model; Gemma’s JumpReLU

L_{0}

is data-dependent and does move.

Pair	edge Jaccard	$z$ vs. seed null	rank §
base $\to$ SFT (LoRA)	$\mathbf{0.937}$	$\mathbf{-15.9}$	$\leq 1/16$
base $\to$ GRPO	$0.962$	$-0.5$	$0.38$
base $\to$ GRPO s1/s3/s4/s5	$0.959$ – $0.964$	$-2.0$ to $+0.8$	$0.06$ – $0.81$
base $\to$ shaped- $\lambda 0.1$	$0.964$	$+0.8$	$0.81$
base $\to$ SFT vs. SFT’s own null	$\mathbf{0.937}$	$\mathbf{-19.9}$	$\leq 1/7$
base $\to$ SFT s0/s1/s2/s3 (own null)	$0.937$ – $0.941$	$-20.0$ to $-17.0$	$\leq 1/7$

Appendix

Table 11: [Archived build.] Co-activation displacement is within run noise for five of six RL contrasts (s1 falls

0.2\times

the null range below the band, vs SFT’s

4.0\times

); SFT clears the null at

z=-15.9

. The base and several runs here were built on different GPU architectures; on one architecture none of the five rebuildable GRPO contrasts is outside its band (Table 1 ). Pairwise graph similarity ( Qwen3-1.7B , l14

\to

22,

10{,}000

edges) vs. the

15

-pair GRPO-family null. Negative

z

= more displaced than seed noise; only SFT separates by a large margin, while s1 falls marginally below the band at step

200

(

z=-2.0

); at matched step

300

it is inside the

15

-pair band (

z=-0.98

) and

0.0006

below the three-pair same-step band.

Figure 5: A run-variation reference and the diagnostic battery. (a) Base and run graphs built on one architecture (H200), the $10$ seed pairs of the five rebuildable GRPO-family runs, 0 of the five base-to-RL contrasts below the band, and base-to-SFT at $z=-19.8$ , $5.0\times$ the null range beyond it; the shaped-reward arm is marked separately, and the archived six-run version is Table 11 . The SFT shift exceeds this empirical range; it does not establish the absence of a smaller RL effect. (b) Level deltas against each checkpoint’s own base from the archived metric files (descriptive, App. F ); TopK-fixed and unavailable values are marked. Receipts: (a) the base, GRPO and audited-SFT graph JSONs in results/sftband_20260926/run/graphs/ , read by code/scripts/samearch.py ; (b) the featstats_* / tulu_featstats_* JSONs; generator code/analysis/fig_seednull_battery.py .

	BFCL task states	pile
seed null, $15$ pairs (mean $\pm$ SD)	$0.7551\pm 0.0403$	$0.9626\pm 0.0016$
seed band $[\min,\max]$	$[0.667,0.819]$	$[0.961,0.966]$
base $\to$ GRPO ( $z$ )	$0.746$ ( $-0.23$ )	$0.962$ ( $-0.5$ )
base $\to$ s1/s3/s4/s5/shaped ( $z$ range)	$-0.87$ to $+1.83$	$-2.0$ to $+0.8$
base $\to$ SFT ( $z$ )	$0.712$ ( $-1.06$ )	$\mathbf{0.937}$ ( $\mathbf{-15.9}$ )

Appendix

Table 12: On-task (BFCL) seed-noise-null displacement vs. the pile arm ( Qwen3-1.7B , l14

\to

22,

10{,}000

edges, edge Jaccard). Reported for completeness, not as a null: the on-task arm’s

2\sigma

MDE (

0.0806

) is

{\sim}3\times

SFT’s actual pile-arm effect (

0.0254

;

{\sim}4\times

in relative terms), and SFT does not separate on it (

z=-1.06

), so no conclusion is drawn from this table (caveats i–iii).

Checkpoint	$\Delta$ Gini	hub Jacc	Battery metric	$\|\Delta_{\text{base}\to\text{GRPO}}\|$	seed max
SFT	$-0.026$	$0.829$	mean $L_{0}$ (pinned)	$0.0$	$0.0$
GRPO	$+0.007$	$0.882$	dead fraction	$0.0046$	$0.0070$
s1	$-0.002$	$0.882$	firing-freq Gini	$0.0003$	$0.0004$
s3	$+0.006$	$0.882$	firing-freq top- $1\%$	$0.0040$	$0.0051$
s4	$+0.013$	$0.882$	mean-act Gini	$0.0010$	$0.0012$
s5	$+0.017$	$0.829$	usage entropy (norm)	$0.0011$	$0.0012$

Appendix

Table 13: On-task (BFCL) levels and battery. Left: per-checkpoint graph levels vs. base (Gini

0.755

, top-

1\%

-active

0.217

). Right: base

\to

GRPO battery deltas vs. the five-run GRPO-family seed-pair band (train split,

n{=}500

); all eight informative metrics within seed noise. The mean-

L_{0}

row (greyed) is TopK-pinned at

50.0

with a

[0,0]

band and is excluded from the count.

cfg	$k$	$\|D_{\mathrm{SFT}}\|/\|D_{\mathrm{base}}\|$	$\Delta$	SFT cell	base cell	$J$	$95\%$ CI	$p$	branch
A	$2{,}000$	$55/58$	$0.259$	$37\times$	$0.3\times$	—	—	—	F3
A	$5{,}000$	$104/97$	$0.549$	$22\times$	$2.0\times$	$-0.0271$	$[-0.0326,-0.0217]$	$0.0001$	F5
A	$10{,}000$	$213/206$	$0.373$	$12\times$	$7.9\times$	$-0.0164$	$[-0.0217,-0.0113]$	$0.0001$	F5
B †	$2{,}000$	$50/57$	$0.272$	$27\times$	$0.5\times$	—	—	—	F3
B †	$5{,}000$	$116/106$	$0.610$	$14\times$	$0.5\times$	—	—	—	F3
B †	$10{,}000$	$225/212$	$0.448$	$6.5\times$	$5.5\times$	$-0.0221$	$[-0.0278,-0.0165]$	$0.0001$	F5

Appendix

Table 14: Registered sweep (

n=4{,}816

held-out prompts; sweep verdict: MIXED). A is the audited SFT graph with its own adapter (the only configuration read for the verdict); † descriptive: B is a second matched SFT run (seed 0), C the pilot’s mismatched pairing at full

n

\Delta

\Delta(\mathrm{SFT},D_{\mathrm{SFT}})

in nats; ratios are mean

\Delta

over the matched random control in the SFT cell (

D_{\mathrm{SFT}}

, SFT model) and the base cell (

D_{\mathrm{base}}

, base model); a base-cell ratio below

1

fails the gate (F3), and then

J

is not read. Receipts notes/functional_transfer_full_{A,B,C}_k*.json , read only by functional_transfer_sweep_read.py .

Figure 6: Checkpoint levels and stage increments answer different questions. (a) Co-activation-graph Gini for Qwen3-1.7B light recipes, Gemma-2-2B heavy instruction tuning, and the Llama-3.1-8B Tulu-3 sequence. Levels alone do not identify an RL effect. (b) Stage displacement is 1 minus edge Jaccard. The dashed line is one public PPO/GRPO pair (0.097), not a same-recipe seed-null distribution. The RLVR increments are unresolved against that single reference; they are not certified nulls. Receipts: notes/basediff_{qwen,gemma}_summary.json , tulu_basediff_summary.json , tulu_stage_increments.json (v1); generator code/analysis/fig_staircase_money.py .

Feature ID	Mean act. (pre)	Mean act. (post)	$\Delta$	In hub set?
14907	$0.0248$	$0.0168$	$-0.0079$	no
32255	$0.4790$	$0.4727$	$-0.0064$	yes
8232	$0.0047$	$0.0110$	$+0.0063$	no
9666	$0.4526$	$0.4466$	$-0.0060$	yes
32257	$0.0103$	$0.0155$	$+0.0053$	no
12234	$0.0096$	$0.0142$	$+0.0046$	no

Appendix

Table 15: H2 top-8 per-feature mean-activation deltas (committed shaped-

\lambda 0.1

arm, cross-tokenizer, h2_diff_v0_cross_tokenizer.json ,

n=16

probes). Under this reward-shaped arm three of the four v0-graph inbound hubs ( h1_summary_v0.json ) are among the top-8 movers, all negative (the fourth,

\#5158

, moves

-0.0003

); the same

16

of the

32

panel features stay active (weighted cosine

0.9998

, Jaccard

1.0

Feature ID	Entropy (pre)	Entropy (post)	$\Delta$	In hub set?
14907	$0.980$	$1.151$	$+0.171$	no
22117	$1.812$	$1.941$	$+0.130$	no
8286	$1.812$	$1.922$	$+0.110$	yes
32255	$1.812$	$1.922$	$+0.110$	yes
2180	$1.812$	$1.922$	$+0.110$	no
9522	$0.794$	$0.616$	$-0.178$	no

Appendix

Table 16: H3 per-feature entropy deltas (top-5 gainers + top-5 collapsers), committed cross-tokenizer run ( h3_entropy_cross_tokenizer_v0.json , v0 Pythia-70m panel). Two of the four v0-graph inbound hubs (

\#8286

\#32255

) are among the top-5 gainers (

\Delta=+0.11

); a third (

\#9666

) ties the fifth row at

+0.110

(four features share this value; the receipt’s tie order drops it); the aggregate delta is nonetheless negligible (

d=-0.005

, gate not passed).

Figure 7: A layer-local diagnostic of the audited index slots. (a) Count of the fifteen audited integer slots present in any top-30 set at each destination. (b) Their maximum inbound share within that destination. (c) Number of available source graphs. The SAE dictionaries were trained independently by layer: identical integer indices are not aligned features. This plot therefore reports local slot statistics, not cross-layer feature recurrence or preserved feature identity. Layer 35 is unembedding-dominated. Receipt: notes/fig8_backbone_recurrence_data.json ; generator code/analysis/fig8_layer_index_diagnostic.py .

Figure 8: H2 per-feature delta diagnostic (cross-tokenizer,

n=16

probes). (a) Per-feature mean activation pre (SFT) vs. post (shaped-reward GRPO, shaped-

\lambda 0.1

arm),

32

features. v0-panel top features (amber diamonds) sit on the diagonal with small negative deltas, consistent with this arm’s reward penalizing the panel’s activations. (b) Histogram of post

-

pre deltas; the four v0-panel top features are marked. Cosine

0.9998

is consistent with the delta magnitudes shown (

\sim\!5\%

of pre-activation mean), not numerical precision.

Explore similar work

Jul 27, 2026

Similar Models Learn Differently: Final-Window Pretraining Shapes Post-Training Beyond SFT

Developers judge a model checkpoint by how it behaves. After supervised fine-tuning (SFT), two checkpoints that perform about the same across relevant benchmarks are treated as interchangeable, equally ready for the next alignment stage, typically preference optimization. We ask whether this judgment misses a pretraining imprint: a difference that no post-SFT benchmark reveals, yet that decides how each checkpoint responds to further training. To find out, we run a controlled experiment on the final window of pretraining, the last data trained on before instruction tuning. Six branches fork from one partially pretrained checkpoint and differ only in this window: 500 million tokens, 0.1% to 1% of the tokens that precede it. Each branch trains its window on a single data source: generic web text, filtered web text, normative discourse, safety text, mathematical text, or synthetic educational text. SFT and post-training are then identical. After SFT the branches behave near-identically, within about one point on instruction following, refusal, and capability, yet the same post-training carries them to very different endpoints, under both a direct preference optimization update and a reinforcement learning update with a verifiable reward. We measure this deviation through refusal of harmful requests: when post-training begins the safety text branch refuses no more than the web text branch, yet by the end it has lost far less of its refusal. The other four branches gain little or no protection, so the effect is selective to what the window contained. The protection requires the safety text to arrive last rather than earlier in pretraining, and it reproduces on a second model family. What a model is pretrained on last shapes how it reacts to alignment. Therefore, a checkpoint should not be evaluated by its post-SFT behavior alone, and what it was trained on last should be reported with it.

Cen Lu, Yung-Chen Tang, Andrea Cavallaro

1EPFL · 2Idiap Research Institute

Language Model Pretraining LLM Safety Alignment Continued Pretraining

Ask PDF

May 21, 2026

Post-Training is About States, Not Tokens: A State Distribution View of SFT, RL, and On-Policy Distillation

Large language model post-training methods such as supervised fine-tuning (SFT), reinforcement learning (RL), and distillation are often analyzed through their loss functions: maximum likelihood, policy gradients, forward KL, reverse KL, or related objective-level variants. We study a complementary factor: the state distribution on which supervision is applied. For an autoregressive policy, a state is a prompt plus generated prefix. SFT trains on fixed dataset states, while RL and on-policy distillation (OPD) train on states induced by the current learner. We formalize post-training as state-distribution shaping and run a controlled smallscale study using Qwen3-0.6B-Base on GSM8K, with TruthfulQA and MMLU as retention evaluations. Our results show three phenomena. First, a mild SFT run improves GSM8K with little forgetting, while a stress SFT run causes substantial retention loss. Second, OPD from a degraded SFT teacher surpasses that teacher on GSM8K, TruthfulQA, and MMLU, despite using the teacher as its only supervision source. Third, a lightweight on-policy RL run improves GSM8K while preserving retention. These results support a state-centric view of post-training: the source and locality of training states can be as important as the form of the supervision signal.

Dong Nie

Independent Researcher

Supervised Fine-Tuning RL for Language Model Reasoning On-Policy Distillation Language Model Post-Training

Ask PDF

Aug 14, 2026

Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification

Open-weight language models are fine-tuned, quantized, pruned, and merged, yet their provenance is often undocumented. We study data-free white-box lineage verification: can weights alone reveal whether two compatible model checkpoints share ancestry? Residual training produces a shared identity-aligned component in branch products, so this structure alone cannot establish ancestry. We remove it and compare checkpoint-specific structure across residual blocks, yielding a symmetric lineage score calibrated against independent checkpoints. On residual-MLP and GPT-2 benchmarks, the score separates fine-tuned, LoRA-merged, pruned, and quantized descendants from independent and distilled models (AUROC=1.0), distinguishing weight ancestry from behavioral similarity. Under function-preserving checkpoint laundering experiments, weight-space baselines lose margin or fail; our score remains unchanged and runs 76x faster than the nearest robust baseline on GPT-2. The projection-pairing signal appears across six language-model families and beyond, and a case study correctly identifies 3 related and 7 unrelated LLaMA-2 public checkpoints. Collectively, these results establish a passive, data-free provenance signal for compatible open-weight language-model checkpoints

Aman Singh Thakur, Rayan Khoury

Massachusetts Institute of Technology, USA

Language Model Fingerprinting Open-Weight Language Models

Ask PDF