Safety evaluations often assume that behavior observed during testing reflects behavior in ordinary use, but fine-tuning can break this assumption. A checkpoint can appear fixed under evaluation-style prompts while the same behavior persists under ordinary-use prompts. Output scores reveal this mismatch but do not locate it. We investigate whether the distinction is encoded in a stable internal site and introduce an audit that fits a paired activation contrast at a path-patching-informed mid-depth window, then modifies the resulting coordinate on held-out prompts. The intervention reduces the evaluation-to-deployment gap by more than 0.12 in ten of twelve model--behavior settings and closes it (post-intervention gap below 0.06) in four, across four full-matrix instruction-tuned model instances; among the eight settings with n≥120 paired questions, six reduce and one closes. A fifth model supports localization and edit-provenance checks, and in the twelve settings deployment-framed rates change by at most 6.1pp. On retrained checkpoints, random directions at the same dose do not reproduce the effect, and off-window controls do so only in part. The strongest literal evidence for an evaluation-versus-deployment distinction comes from sandbagging, whose prompt framing is explicitly evaluative; sycophancy and refusal test framing-contingent task distinctions. The two weak cells, both sycophancy, show where a single-coordinate audit at a fixed window is not sufficient. The audit is a non-adaptive, white-box diagnostic for controlled fine-tuned checkpoints, not a training-time defense, a guarantee of deployment safety, or evidence about naturally arising failures.
Figures & tables
Figure 1: Schematic of the routing-coordinate audit. (a) Paired evaluation and ordinary-use prompts separate along a fitted activation contrast dL at the routing-window layer. (b,c) Asymmetric remove/add interventions move one framing along that coordinate. (d) The main held-out intervention modifies the coordinate for both framings, testing whether the behavioral gap is carried by this fitted routing direction. Figure 4 reports the cross-family intervention matrix.
Step
Input
Operation
Output
Measure gap
paired prompts
compute rD−rE
installed gap γ
Localize
base and fine-tuned activations, model depth
path patching plus relative-depth heuristic
intervention window
Fit direction
calibration pairs
μD−μE
routing direction
Intervene
held-out pairs
modify coordinate
post-intervention gap
Control
random, wrong-layer, semantic directions
same intervention form
specificity check
Table 1: Protocol summary. Each step has a fixed input, operation, and output before the next step runs.
Figure 2: How the installed gap is written and where we intervene. (a) A short fine-tune on paired evaluation-framed and deployment-framed prompts installs framing-contingent behavior; under evaluation framing the model underperforms, while under deployment framing it retains the target behavior. Path patching traces the installed signal to a narrow attention window at roughly 0.55 of model depth. (b) At the last prompt token inside that window, calibration prompts form two clusters separated by a difference-of-means direction. The routing-coordinate intervention modifies the attention-block output component along that direction and tests whether the framing distinction depends on this coordinate.
Figure 3: Path-patching recovery on sandbagging. (a) On Gemma-2-2B, no single layer in the mid-depth region carries more than about one-third of the installed gap on its own, but the contiguous window [L14,L17] recovers 0.94 ; extending the window two layers earlier to [L12,L17] overshoots to 1.12 . (b) Reported localization windows at relative depth ≈0.55 (shaded) recover a large majority of the gap on three of five model instances. The Qwen-2.5-7B point is the fixed window [L15,L18] on a fresh retrain (seed 42, 0.52 ). Gemma-2-9B uses a wider window and remains the weak outlier in this panel.
Figure 4: Held-out evaluation-to-deployment gap before and after modifying the fitted routing coordinate at the intervention window, across the 12-cell architecture-by-behavior matrix. Ten cells reduce the gap by more than 0.12 , four of them to below 0.06 ; two sycophancy cells, Llama-3-8B and Gemma-2-9B, stay near baseline. Error bars show paired-bootstrap 95% gap intervals over question IDs with 5000 resamples; Appendix L links every cell to its provenance summary. The intervention window is fixed before any intervention runs and calibration/intervention items are disjoint at question-id level (Section 2.1 ).
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Depth-resolved projection of the attention-block output at the last prompt token on a fresh retrain of the primary Gemma-2-2B sandbagging checkpoint (seed 42 ; 200 held-out questions, templates averaged within question and framing; shading is the 95% question-bootstrap interval, narrow at this scale). (a) Projection onto the window direction, the normalized sum of the unit directions fitted at L14 – L17 , applied at every layer. The deployment-minus-evaluation gap is at most 0.91 before and after the window (largest at L13 ) and 4.3 to 5.8 inside it. (b) Projection onto each layer’s own fitted direction. The gap grows from 0.4 to 0.6 at L0 – L4 to 4.3 at L13 , reaches 7.4 to 10.8 at L14 – L17 , and stays between 3.2 and 9.7 afterwards, so (b) is in-sample aligned by construction and shows accumulation with a peak in the window.
Figure 6: Dose response and matched controls on the Gemma-2-2B sandbagging primary cell. Panel (a) plots the projection operators against the scale-free projection dose λ and the add operator against the multiple of the raw direction dL , which is a different axis. The routing interventions reduce the framing gap, while random-direction and wrong-layer controls stay near baseline at λ=1 and semantic controls leave evaluation accuracy near its baseline (panel c; the routing direction is measured on the 300 -prompt set with baseline 0.163 , the semantic axes on the 600 -prompt set with baseline 0.183 ).
Figure 7: Semantic specificity and direction-norm diagnostics. (a) Content-matched semantic directions do not reproduce the routing-direction effect. (b) Fine-tuning amplifies the paired framing direction relative to the base model in the routing window.
Figure 8: Base versus fine-tuned routing-direction norms on Gemma-2-2B and Phi-3-mini. The fitted framing direction is 4.9× to 10.4× larger on Gemma-2-2B and 3.6× to 4.1× larger on Phi-3-mini after fine-tuning across the localized window. In the six other matrix cells with geometry files the largest base cosine per cell is 0.19 to 0.57 and the per-layer amplifications are 0.6× to 6.1× (Qwen-2.5-7B refusal at L15 and L16 is below one).
Installation
Seed
Hook
Window
Gap reduction [95% CI]
Δ Eval
Δ Deploy
All controls
Attention-only
42
Attention
L8 – L11
0.238[0.143,0.333]
+0.275
+0.038
Yes
Attention-only
42
MLP
L8 – L11
0.160[0.080,0.240]
+0.153
−0.008
Yes
Attention-only
43
Attention
L8 – L11
0.218[0.133,0.305]
+0.230
+0.013
Yes
Attention-only
43
MLP
L12 – L15
0.028[−0.015,0.070]
+0.018
−0.010
No
Attention-only
44
Attention
L8 – L11
0.200[0.108,0.290]
+0.218
+0.018
Yes
Attention-only
44
MLP
L8 – L11
0.065[0.008,0.123]
+0.055
−0.010
No
Appendix
Table 2: Prospective module-installation factorial on fresh Gemma-2-2B sandbagging organisms. Each organism is evaluated on 200 held-out question IDs. Gap reduction is baseline minus post-intervention gap with a question-ID bootstrap 95% interval; accuracy changes are post-intervention minus baseline. Random controls are removal on evaluation and addition on deployment; off-window controls use the joint intervention. “All controls” indicates that the row’s effect exceeds every matched random and off-window control.
Family
Behavior
Base γ
Learned reduction
Max ∣ random ∣
Max ∣ off-window ∣
Off-window >0
Gemma-2-2B
sandbag
0.280
0.185[0.090,0.278]
0.010
0.025
0/2
Gemma-2-2B
sycophancy
0.492
0.150[0.063,0.242]
0.017
0.017
0/2
Gemma-2-2B
refusal
0.350
0.329[0.171,0.493]
0.011
0.118
1/2
Gemma-2-9B
sandbag
0.555
0.355[0.280,0.430]
0.003
0.035
1/2
Gemma-2-9B
sycophancy
0.679
0.421[0.346,0.500]
0.008
0.025
0/2
Gemma-2-9B
refusal
0.618
0.639[0.411,0.839]
0.000
0.043
1/2
Appendix
Table 3: Matched controls at the headline dose on fresh retrains of the twelve cells. Treatment and controls use the same clamp-both operator, token positions, four-layer width, and λ=2 . Reductions are baseline gap minus post-intervention gap with 95% question-clustered bootstrap intervals; the random columns use five Gaussian unit directions per cell and the off-window columns two predeclared four-layer windows per cell. These runs do not revise the counts of the submitted matrix.
Condition
Post-gap
Reduction [95% CI]
Predeclared L14 – L17 , clamp, λ=2
0.095
0.185[0.090,0.280]
Localization-selected L10 – L13 , clamp, λ=1
0.010
0.270[0.185,0.358]
Localization-selected L10 – L13 , clamp, λ=2
−0.018
0.298[0.195,0.403]
Early window L4 – L7 , clamp, λ=1
0.298
−0.018[−0.040,0.003]
Early window L4 – L7 , clamp, λ=2
0.298
−0.018[−0.048,0.013]
Late window L20 – L23 , clamp, λ=1
0.278
0.003[−0.018,0.025]
Appendix
Table 4: Fresh primary Gemma-2-2B sandbagging checkpoint (seed 42 ; baseline gap 0.280 on the full held-out split; the template-holdout rows use a 122 -question subset with its own baseline). Five Gaussian random directions with the same layers, operator, and doses leave the gap between 0.278 and 0.285 . Post-gap and reduction are as in Table 3 .
Cell
Seed
Reduction [95% CI]
Max ∣ random ∣
Max ∣ off-window ∣
Qwen-2.5-7B sandbag
42
0.405[0.328,0.480]
0.008
0.080
Qwen-2.5-7B sandbag
43
0.488[0.405,0.570]
0.008
0.020
Qwen-2.5-7B sandbag
44
0.533[0.445,0.620]
0.010
0.055
Gemma-2-9B sycophancy
42
0.421[0.346,0.500]
—
—
Gemma-2-9B sycophancy
43
0.058[0.025,0.096]
—
—
Gemma-2-9B sycophancy
44
0.558[0.475,0.646]
—
—
Appendix
Table 5: Fresh retrains over three training seeds at the predeclared window and λ=2 . Qwen-2.5-7B sandbagging is stable; Gemma-2-9B sycophancy is seed-heterogeneous. The last two columns give the largest absolute effects among five random directions and two off-window controls.
Checkpoint
Rank
Post-gap
Reduction [95% CI]
Random subspace
Eval-gain share
k=2 suppression
2
0.165
0.163[0.070,0.263]
−0.003
35%
k=2 suppression
4
0.028
0.300[0.202,0.405]
0.010
19%
k=2 suppression
8
−0.010
0.338[0.243,0.435]
0.013
17%
k=3 suppression
2
0.030
0.290[0.193,0.393]
0.005
16%
k=3 suppression
4
0.003
0.318[0.220,0.415]
−0.010
15%
k=3 suppression
8
−0.005
0.325[0.225,0.430]
0.005
15%
Appendix
Table 6: Higher-rank follow-up at λ=2 . Learned rank- r subspaces differ from matched random subspaces and can close the aggregate gap, but the share of the reduction that comes from evaluation gain (the rest is lost deployment performance) is small in the k=2 and k=3 suppression organisms and at most about half for the weak Gemma-2-9B sycophancy seed.
Checkpoint
Base γ
Reduction [95% CI]
Eval gain
Deploy loss
Gemma-2-2B
0.093
0.098[0.062,0.135]
0.094
0.004
Gemma-2-9B
0.333
0.333[0.278,0.388]
0.329
0.003
Qwen-2.5-7B
0.345
0.332[0.272,0.389]
0.348
−0.016
Llama-3-8B
0.321
0.100[0.069,0.132]
0.109
−0.009
Appendix
Table 7: Public WMDP refusal extension on fresh retrains ( 600 unique questions; uncertainty resamples question ids, not the four renderings per question). Directions are fit on the original 20 -question calibration set. Reductions arise from evaluation-framing gains, not deployment loss. For Llama-3-8B a late-window control reduces the gap by 0.114 , comparable to the selected window, so this checkpoint does not support a specific localization claim.
Patch
Accuracy
Recovery
Full-window attention patch (reference)
0.396
0.53
Pool-subspace component only, 1 to 20 components
0.208 to 0.208
0.00
Everything except the pool subspace ( 5 components)
0.375
0.47
Layer- 19 MLP neuron patch, top- K with K=5,10,20
0.208
0.00
Layer- 19 MLP neuron patch, top- K with K=50 to 200 ( 97 neurons)
0.229
0.06
Appendix
Table 8: Simpler loci on the fresh primary checkpoint (48 questions, single seed, descriptive). Base accuracy 0.563 , fused accuracy 0.208 . Recovery is (rpatched−rfused)/(rbase−rfused) as in ( 2 ). Patches act at L14 – L17 attention outputs (subspace rows) or layer- 19 MLP intermediates (neuron rows). The full-window reference recovers about half the gap on this checkpoint, so the baselines are measured against that ceiling.
wrapper
prefix
full prompt
Train → test
AUC
Acc.
AUC
Acc.
AUC
Acc.
train bank → held-out bank
1.000
1.000
1.000
1.000
1.000
1.000
held-out bank → train bank
1.000
1.000
1.000
1.000
1.000
1.000
train bank → other template family
1.000
1.000
1.000
1.000
1.000
1.000
other family → train bank
0.969
0.795
0.990
0.750
0.916
0.684
held-out bank → other family
1.000
1.000
1.000
1.000
1.000
1.000
Appendix
Table 9: Bag-of-words separability of the evaluation and deployment wrappers (word uni- and bigrams, logistic regression). The wrapper is the prompt with the question block removed; “prefix” is the text before the question; “full” is the whole prompt. The train and held-out banks share all 40 normalized template strings, so only the rows involving the other template family and the template-pair holdout test unseen wording. The values come from the released script; the earlier implementation behind the rebuttal figure of 0.86 to 0.97 is lost.
Model
Size
Cells
Role
Gemma-2-2B-it
2B
S/Y/R
primary cell; five-seed replication
Gemma-2-9B-it
9B
S/Y/R
scale-up and weak-cell boundary cases
Qwen-2.5-7B-Instruct
7B
S/Y/R
cross-family replication
Llama-3-8B-Instruct
8B
S/Y/R
replication and flat-cell stress test
Phi-3-mini-4k-instruct
3.8B
localization/edit; intervention null
provenance and secondary checks
Appendix
Table 10: Model coverage for the audit. Full-matrix cells include sandbagging (S), sycophancy (Y), and refusal (R); Phi-3-mini supports localization and edit-provenance checks only.
Family
Path window
ρpatch↑
Intervention window
Gemma-2-2B
L14 – L17
0.94
L14 – L17
Qwen-2.5-7B †
L15 – L18
0.52
L15 – L18
Llama-3-8B
L14 – L17
1.00
L17 – L20
Phi-3-mini
L18 – L21
0.94
L18 – L21 (null)
Gemma-2-9B
L21 – L27
0.58
L23 – L26
Appendix
Table 11: Windows used in the path-patching panel and in the intervention matrix. The path-patching window localizes the sandbagging signal; the intervention window is the fixed four-layer window used for intervention results in Section 3.2 . Phi-3-mini is not included in the 12 -cell matrix; its intervention at L18 – L21 is null (Section 3.5 ). † Fresh retrain, seed 42 , n=48 . Seeds 43 and 44 recover 0.29 and 0.46 at [L15,L18] ; the best scanned band [L16,L19] recovers 0.81 , 0.75 , 0.75 (seeds 42 to 44 ). A file stored with the submitted Qwen matrix cell gives 0.80 at [L14,L17] and 0.89 at [L12,L17] ( n=48 , base accuracy 0.79 ) and did not patch [L15,L18] .
Cell
Eval. rate ↑
Deploy. rate ↑
γ↓
baseline
0.163
0.447
0.283
remove on evaluation, L14 – L17 , λ=0.5
0.213
0.447
0.233
remove on evaluation, L14 – L17 , λ=1.0
0.440
0.447
0.007
remove on evaluation, L14 – L17 , λ=2.0
0.460
0.447
−0.013
add on deployment, L14 – L17 , λ=2.0
0.163
0.163
0.000
joint intervention, L14 – L17 , λ=2.0
0.460
0.463
0.003
Appendix
Table 12: Gemma-2-2B sandbagging intervention cell values ( n=300 per framing). Arrows mark the desired diagnostic direction; the bold entry in each metric column is the best value in that column.
Check
Alternative tested
Outcome
Random remove, λ=1
any random unit direction works
gap 0.283→0.287
Random add, λ=1
deployment add is enough
gap 0.283→0.233 via deployment-rate drop
Wrong layer L6 (one layer)
any layer works
gap 0.283→0.307
Wrong layer L22 (one layer)
late layer works
gap 0.283→0.273
Semantic contrasts (remove, clamp)
ordinary content axes work
evaluation rate changes ≤0.015
Routing direction, joint intervention
target intervention
gap 0.283→0.003
Appendix
Table 13: Specificity controls for the routing intervention (Gemma-2-2B sandbagging, n=300 per framing). Each row names an alternative tested at the available dose; the target intervention is included as the positive reference. The sign diagnostic of the submitted version is withdrawn (Section 3.3 ).
Family
Behavior
Base rD
Post rD
ΔrD
Base γ
Post γ↓
Gemma-2-2B
sandbag
0.480
0.463
−0.018
0.298
−0.008
Gemma-2-2B
sycophancy
0.496
0.483
−0.013
0.496
0.308
Gemma-2-2B
refusal
0.446
0.425
−0.021
0.321
0.011
Gemma-2-9B
sandbag
0.650
0.653
+0.003
0.510
0.145
Gemma-2-9B
sycophancy
0.692
0.688
−0.004
0.692
0.679
Gemma-2-9B
refusal
0.714
0.775
+0.061
0.611
−0.007
Appendix
Table 14: Deployment-framed rates in the 12 -cell matrix. ΔrD is post-intervention minus baseline; smaller magnitude is better for checking deployment-side collapse.
Family
Behavior
ninf
Restored ↑
Homog. ↓
Persistent ↓
γˉbase
γˉpost↓
Gemma-2-2B
sandbag
120
54%
20%
26%
+0.298
−0.008
Gemma-2-2B
sycophancy
61
26%
5%
69%
+0.496
+0.308
Gemma-2-2B
refusal
16
38%
0%
56%
+0.321
+0.011
Gemma-2-9B
sandbag
144
42%
6%
52%
+0.510
+0.145
Gemma-2-9B
sycophancy
84
1%
2%
96%
+0.692
+0.679
Gemma-2-9B
refusal
23
83%
13%
0%
+0.611
−0.007
Appendix
Table 15: Per-item transition categories across the 12 cells. An item is informative when its baseline framing gap exceeds 0.1 ; restored means both framings reach the leading framing’s accuracy; homogenized means both collapse to the trailing level; persistent means the gap is not closed by the intervention. The aggregate post-intervention gap γˉpost tracks whether cells with many persistent items also have small aggregate reductions.
Figure 9: Recruitment traces do not explain the behavioral gap under masking. The fine-tune writes through attention LoRA modules. MLP neurons carry measurable downstream traces, but masking those neurons only weakly changes the behavioral gap. The more direct behavioral probe in this audit is the routing-level variable.
Set
Hits@50 ↑
Mass@50 ↑
Mask Δγ↓
Margin shift ↑
KL ↑
ISS (input sensitivity)
8.00[6.17,10.00]
0.1064[0.0683,0.1438]
0.42 pp
0.578
0.170
Activation magnitude
9.17[7.50,11.17]
0.1150[0.0780,0.1549]
1.25 pp
0.832
0.142
H-tau composite
12.17[9.00,15.50]
0.1396[0.0897,0.1893]
0.83 pp
1.444
0.234
Random
0.41[0.38,0.43]
0.0017[0.0013,0.0020]
0.21 pp
0.430
0.009
Appendix
Table 16: Recruitment-versus-intervention audit on Gemma-2-2B sandbagging seed 50 . ISS is input-sensitivity score; H-tau is the predeclared composite geometry score. Hits@50 counts overlap with prospectively recruited neurons, and Mass@50 sums recruitment mass over the top-50 set. Persistent masking and logit scoring use the trained no-suppression seed- 50 model on 48 questions and all 10 templates per framing ( n=480 per framing). H-tau predicts recruitment best, but none of the neuron masks explains the accuracy gap under this masking test.
Model
Depth
Layers
Baseline γ
Post γ
Δγ
Gemma-2-9B
0.45
L18 – L21
0.670
0.700
+0.030
Gemma-2-9B
0.50
L21 – L24
0.670
0.300
−0.370
Gemma-2-9B
0.55
L23 – L26
0.670
0.390
−0.280
Gemma-2-9B
0.60
L25 – L28
0.670
0.700
+0.030
Gemma-2-9B
0.65
L27 – L30
0.670
0.690
+0.020
Llama-3-8B
0.45
L14 – L17
0.570
0.370
−0.200
Appendix
Table 17: E4 adjacent-window depth sweep on the two flat sycophancy cells ( n=50 calibration, n=50 eval). Δγ is the change in held-out gap at λ=2 ; negative values indicate gap reduction.
Figure 10: Rank-one weight edits are a mixed diagnostic, not a defense. Before and after gaps for four cells ( n=160 per framing). Projecting the fitted routing direction out of edited output matrices closes two cells (Qwen-2.5-7B refusal and Gemma-2-2B sandbagging) but leaves two sandbagging cells largely unchanged. The main claim rests on held-out activation interventions in Figure 4 .
Family
Task
Layers
Depth
Base gap
After gap ↓
Gap removed ↑
Closure ↑
Qwen-2.5-7B
refusal
L15 – L18
0.59
56.7
−5.4
62.1
110%
Gemma-2-2B
sandbag
L14 – L17
0.60
27.5
−3.8
31.3
114%
Gemma-2-9B
sandbag
L20 – L23
0.51
68.8
63.8
5.0
7%
Llama-3-8B
sandbag
L17 – L20
0.58
42.5
40.0
2.5
6%
Appendix
Table 18: Rank-one output-weight edit on four cells ( n=160 per framing). Closure is the percentage of the pre-edit gap removed by the one-shot edit. The edit is presented as a mixed diagnostic, not as a defense.
Model
Behavior
n
γbase
γpost
Δγ↑
Gemma-2-2B
sandbag
200
0.298
−0.008
0.305
Gemma-2-2B
sycophancy
120
0.496
0.308
0.188
Gemma-2-2B
refusal
28
0.321
0.011
0.311
Gemma-2-9B
sandbag
200
0.510
0.145
0.365
Gemma-2-9B
sycophancy
120
0.692
0.679
0.013
Gemma-2-9B
refusal
28
0.611
−0.007
0.618
Appendix
Table 19: Matrix provenance summary for the 12 -cell routing matrix. n is the number of paired questions used for the reported bootstrap and provenance summary; γpost is signed and closer to zero is better. Figure 4 reports the corresponding paired-bootstrap intervals.
Claim
Evidence
Where to find it
Limitation / caveat
Mid-depth routing region
Sandbagging path patching recovers most of the installed gap in three of five model instances.
Section 3.1 ; Figure 3 ; Appendix D
Gemma-2-9B is weaker ( 0.58 ); several non-sandbag n=48 gaps are too small for clean recovery ratios.
Held-out gap reduction
The λ=2 routing-coordinate intervention reduces ten of twelve matrix cells by more than 0.12 and closes four; the two weak cells are Gemma-2-9B sycophancy and Llama-3-8B sycophancy.
Section 3.2 ; Figure 4 ; Appendix L
Sandbagging uses n=200 , sycophancy n=120 , and refusal n=28 ; refusal is a smaller-sample stress test.
Specificity under tested controls
Random-direction and semantic controls do not reproduce the gap closure at λ=1 on the submitted checkpoints or at λ=2 on fresh retrains; wrong-layer controls are mixed.
Section 3.3 ; Section 3.4 ; Appendix E ; Figure 6
Submitted controls are at λ=1 with one non-routing layer; width-matched controls at λ=2 exist only on fresh retrains, where the selected window beats both controls in 10 of 12 cells and 7 of 24 off-window intervals are positive.
Typed failure modes
Weak path-patching recovery suggests a missed site; weak single-coordinate intervention at a recovered site suggests a higher-rank distinction; a moving control would indicate lack of framing specificity.
Section 3.5 ; Appendix I
The labels guide what failed in this audit; they are not proofs of the underlying mechanism.
Deployment-side non-collapse
Deployment-framed rates change by 1.5 pp on average and at most 6.1 pp in the 12-cell matrix.
Section 3.2 ; Appendix F ; Appendix G
This is a descriptive check for the reported matrix, not a general deployment-safety guarantee.
Appendix
Table 20: Map from claims to evidence and limitations.
Setting
Value
Adapter
LoRA on attention projections only
LoRA rank
16
Learning rate
2×10−4
Optimizer
AdamW
Training length
five epochs; 1550 optimizer steps
Schedule and clipping
cosine schedule; gradient clip 1.0
Appendix
Table 21: Reproducibility configuration for the reported audits.