cs.LGAug 20, 2026

Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfer

Authors: Qinyang Xu

Organizations: Xiamen University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Subliminal trait transfer allows a student model to acquire behavioral dispositions from teacher-generated data in which the trait is not semantically expressed. Recent work explains how such signals enter gradients, but not how they survive source removal or acquire different signs under later training. We treat parameters and optimizer moments as a single trainer state and derive an exact transport-valuation identity separating observer-independent propagation of the source perturbation from the value assigned by a future continuation and behavioral readout. State surgery identifies the first moment as a causal carrier. Transplanting it alone leaves parameters, hidden states, and outputs unchanged at the cut, yet source-free updates generate growing parameter and hidden-state differences; transplanting parameters with the first moment recovers the terminal behavioral response. Sending the same source-induced difference through matched futures produces negative, near-zero, and positive Qwen effects (-0.658, +0.008, and +0.658 seed means). This ordering recurs in all 12 Llama-3.2-1B seeds after eight updates, while state-difference norms remain nearly equal across routes. Both contrasts grow in every paired seed when the continuation extends to sixteen updates. A full-horizon costate predicts all 42 Qwen route-mean signs and all 21 resolved Llama ordinary-route signs. Observer-independent transport also replicates across Qwen, SmolLM2, and Llama, while the complete-state recurrence predicts physical, hidden, and fixed-head responses in non-LoRA MNIST systems, including CNNs trained with AdamW and momentum SGD. Together, these results identify a two-stage mechanism for subliminal trait transfer: optimizer state transports the source perturbation, and later training determines its behavioral value.

Figures & tables

Figure 1: Transport and valuation in the complete trainer state. A teacher-output perturbation enters the shared gradient interface as dtd_{t} , is carried in (w,m,v)(w,m,v) , and is written from the first moment into parameters during source-free updates. From the same cut-state ancestry AEA_{E} , different future routes can produce negative, near-zero, or positive behavioral effects. The observer induces the backward future value λt\lambda_{t} ; its inner product with dtd_{t} is signed work, whose path integral gives the endpoint response (Eq. 6 ). Matrices are schematic; labels identify the tested links.
Figure 2: Optimizer relay. (a) In one Qwen isolated-pulse cell, 89.9% of the adjoint source-family allocation at lag 24 falls in the first-moment block. (b) Momentum SGD: between the source end and the terminal update, the velocity-origin norm falls while its parameter descendant grows. (c) The recurrence attains low protocol-specific prediction error under AdamW, momentum SGD, and plain SGD; plain SGD reaches SSE/ZERO<10−12\mathrm{SSE/ZERO}<10^{-12} .
Figure 3: Block transplant, reset, and rescue. (a) In the three-seed discovery cohort, w+mw{+}m reproduces the full descendant while vv -only remains at the no-transplant baseline. (b) Four replication seeds reproduce the ordering. Error bars in (a) – (b) are sample SD across seeds. (c) The mm -only transplant has exactly zero forward-visible effect at the cut and is then released by source-free updates; error bars are sample SD.
Figure 4: Future training assigns signed value. (a) In the Qwen matched factorial, identical ancestry is negative under S+S^{+} , near-zero under znullz_{\mathrm{null}} , and positive under S−S^{-} ; bars and error bars are seven-seed means and sample SD. (b) In a separate route-replacement cohort, the paired S−−S+S^{-}{-}S^{+} contrast is positive in 7/7 independent seeds (two-sided sign-test p=0.016p=0.016 ; band: 95% Student- tt CI). (c) The topology recurs across four source positions. (d) The seed-mean topology recurs across all five trait families on their respective observer scales; dots are seeds and horizontal marks are means on a symmetric-log scale (Table 18 ).
Figure 5: Prediction of transport and route-conditioned value. (a) Observer-free transport prediction on Qwen, SmolLM2, and Llama-3.2 (log scale): the full tangent predictor sits 22 – 66 orders of magnitude below a source-only comparator; route-centered errors are shown separately (§ 5 ). (b) All 42 Qwen costate predictions versus observed route means; the ringed point is a naturally negative route.
SystemSeeds ×\times routes(w,m,v)(w,m,v) blocksMerged weightsHidden
Qwen2.5-0.5B3×43\times 45.9×10−75.9{\times}10^{-7}3.5×10−73.5{\times}10^{-7}6.1×10−56.1{\times}10^{-5}
SmolLM2-135M3×43\times 49.2×10−69.2{\times}10^{-6}5.7×10−65.7{\times}10^{-6}3.1×10−53.1{\times}10^{-5}
Llama-3.2-1B3×43\times 41.1×10−41.1{\times}10^{-4}3.7×10−53.7{\times}10^{-5}2.8×10−32.8{\times}10^{-3}
Table 1: Observer-free transport across architectures. Entries are seed means from the same four-route panel; lower SSE/ZERO\mathrm{SSE/ZERO} is better.
Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

SymbolMeaning
St=(wt,mt,vt,τt)S_{t}=(w_{t},m_{t},v_{t},\tau_{t})complete trainer state at step tt : parameters, first moment,
 second moment, auxiliary deterministic state (optimizer clock/schedule)
xtx_{t}training batch at step tt
α∈[−1,1]\alpha\in[-1,1]source strength: α=−1\alpha=-1 neutral arm, α=+1\alpha=+1 trait arm, α=0\alpha=0 midpoint
PαP_{\alpha}declared affine gradient-port path indexed by α\alpha
Gt(St,xt)G_{t}(S_{t},x_{t})gradient computation (includes loss, backward pass, clipping input)
Appendix
Table 2: Symbols used throughout the paper.
TermPlain-language meaning
ancestry (source/physical/accumulated ancestry)the trace the teacher signal leaves in the trainer state as it accumulates. The local tangent is qt=∂αStq_{t}=\partial_{\alpha}S_{t} ; a finite ancestry chord is At=St(+1)−St(−1)=∫−11qt(α) dαA_{t}=S_{t}(+1)-S_{t}(-1)=\int_{-1}^{1}q_{t}(\alpha)\,d\alpha (§ 2 )
relay (optimizer relay)the two-step handoff by which ancestry is first held in optimizer state (first moment mm or momentum velocity) and only later written into parameters (§ 3 )
transplant / reset / rescuethe three steps of the state-surgery protocol: transplant copies a chosen subset of state blocks from a source-trained run into a control run; reset sets every other block back to the control run’s own values; rescue is the shared source-free suffix of updates applied afterward, which is what converts a transplanted block’s stored ancestry into a measurable effect (§ 3 )
transportthe physical propagation of source ancestry through the trainer, independent of which behavioral readout is later applied to measure it (§ 2 )
valuationthe sign and magnitude that a particular observer and a particular future training path assign to a given piece of ancestry; unlike transport, valuation depends on both (§ 2.4 )
future valuethe quantity λt\lambda_{t} : how much a gradient nudge at step tt would move the final behavioral measurement, computed backward from the observer (§ 2 )
Appendix
Table 3: Narrative terms used throughout the paper, in plain language.
Figure 6: Source entry and coordinate access. (a) Backward rotation: frozen-head accuracy (solid) declines with θ\theta while linear probe (dashed) stays flat—the source writes to different coordinates but the information persists. Procrustes (dotted) restores access. (b) Full-state prediction: the joint predictor (dark blue) retains near-exact accuracy at all lags; parameter-only (red) degrades sharply by lag 16, and moments-only (light blue) by lag 1.
Figure 7: Validity domain of the compact chord predictor (§ 4.2 ). Each dot is one seed’s baseline-margin score Gs∗G^{\ast}_{s} (positive = the costate predictor beats ZERO, source-only, and route-constant competitors); horizontal bars are seed means. (a) Source-scale titration: fixed paired-row subsamples from the full 1831-row corpus, with all other protocol elements unchanged. The predictor is below the baselines at 128 rows, transitions through a low-signal band at 256–512, and wins in 7/7 seeds at 1024 and above. Open diamonds: the independently generated native 128-row protocol is also below the baselines. (b) Receding-horizon ladder on the same ordinary-route panel: truncating the propagated future to H∈{1,2,4}H\in\{1,2,4\} of the 8 suffix updates gives negative margin in every seed.
Figure 8: Sustained influence and storage–observer separation. (a) Old-ancestry revaluation ( −1.14955-1.14955 ) outweighs new source work ( +1.04919+1.04919 ), producing a net local-response change of −0.10037-0.10037 at underlying precision. (b) Storage is 114×114\times the observer effect. Solid bars show absolute scalar effects; light bars show prompt-vector norms.
Figure 9: Five-trait transplant across Qwen2.5-0.5B and Llama-3.2-1B. The w+mw{+}m transplant reproduces the full descendant in all 15 primary seed–trait cells: Qwen owl, cat, red, and oak, plus Llama blue. (a) Physical descendant error. (b) Hidden-state descendant error.
Figure 10: Generalization and boundary. (a) CNN replication across optimizers; block allocations are seed-dependent. (b) One SmolLM2 cell: 90.9% step-wise cancellation, not absence. (c) In TinyLlama, response signs follow the transplanted gradient history, but the prompt-level response pattern does not transfer. Intervals are paired-prompt 95% bootstrap intervals.
Replicate1234567
DiD_{i}0.2430.2230.5280.3030.4650.3730.397
Appendix
Table 4: Separate Qwen common-ancestry route-replacement cohort ( n=7n=7 ): paired contrasts Di=(yi−−yi+)/2D_{i}=(y_{i}^{-}-y_{i}^{+})/2 , where yi±y_{i}^{\pm} is seed ii ’s response under S±S^{\pm} . All positive; two-sided sign-test p=0.016p=0.016 .
PositionDD% of ref.Prompt cos.64/64 pos.
40–470.32077%0.999✓
80–87 (ref.)0.415100%—✓
120–1270.36789%0.998✓
200–2070.34282%0.999✓
Appendix
Table 5: Source-position ablation: route topology recurs across positions.
 Update 0Update 1Update 2
Negative half−1.17×10−3-1.17\times 10^{-3}−7.1×10−5-7.1\times 10^{-5}+8.5×10−5+8.5\times 10^{-5}
Positive half−4.6×10−5-4.6\times 10^{-5}+6.7×10−4+6.7\times 10^{-4}+3.3×10−4+3.3\times 10^{-4}
Net−1.21×10−3-1.21\times 10^{-3}+6.0×10−4+6.0\times 10^{-4}+4.1×10−4+4.1\times 10^{-4}
Appendix
Table 6: SmolLM2 source-step Shapley allocations. Cross-step cancellation: 90.9%.
ReplicateNet chordCancel.Polarity
1+0.028+0.02863%Early −- /Late ++
2+0.020+0.02084%Early −- /Late ++
3+0.007+0.00762%Early ++ /Late −-
Appendix
Table 7: TinyLlama finite anatomy: stable early/late opposition with 62–84% cancellation.
ReplicateSSE/ZERO\mathrm{SSE/ZERO}Param sharem1m_{1} share
17.8×10−77.8\times 10^{-7}0.9730.114
25.4×10−55.4\times 10^{-5}0.0730.744
33.0×10−63.0\times 10^{-6}0.1390.891
Appendix
Table 8: CNN AdamW: block allocations vary widely across seeds. Param and m1m_{1} are signed response shares normalized by the net response; omitted block and interaction terms are also signed, so the displayed shares need not sum to one.
ReplicateSSE/ZERO\mathrm{SSE/ZERO}Src-cutVel.Param.
18.8×10−108.8\times 10^{-10}4.22+1.045+1.045−0.045-0.045
24.9×10−94.9\times 10^{-9}2.02+1.125+1.125−0.125-0.125
39.0×10−119.0\times 10^{-11}8.50+1.065+1.065−0.065-0.065
Appendix
Table 9: CNN momentum SGD: relay topology is stable across seeds. Src-cut is the source-cut response norm in the experiment’s native units. Vel. and Param. are signed fractions of the terminal response and sum to one.
  full statemerged weightshidden
SystemRep.SSEZEROratioSSEZEROratioSSEZEROratio
Qwen2.5-0.5B12.0×10−72.0{\times}10^{-7}0.424.7×10−74.7{\times}10^{-7}7.6×10−87.6{\times}10^{-8}0.263.0×10−73.0{\times}10^{-7}0.294.7×1034.7{\times}10^{3}6.2×10−56.2{\times}10^{-5}
 23.6×10−73.6{\times}10^{-7}0.487.6×10−77.6{\times}10^{-7}1.2×10−71.2{\times}10^{-7}0.294.2×10−74.2{\times}10^{-7}0.184.2×1034.2{\times}10^{3}4.3×10−54.3{\times}10^{-5}
 32.7×10−72.7{\times}10^{-7}0.495.5×10−75.5{\times}10^{-7}1.0×10−71.0{\times}10^{-7}0.303.4×10−73.4{\times}10^{-7}0.293.8×1033.8{\times}10^{3}7.7×10−57.7{\times}10^{-5}
SmolLM2-135M13.6×10−63.6{\times}10^{-6}0.311.2×10−51.2{\times}10^{-5}1.8×10−61.8{\times}10^{-6}0.257.2×10−67.2{\times}10^{-6}0.328.9×1038.9{\times}10^{3}3.6×10−53.6{\times}10^{-5}
 22.0×10−62.0{\times}10^{-6}0.267.9×10−67.9{\times}10^{-6}1.0×10−61.0{\times}10^{-6}0.224.6×10−64.6{\times}10^{-6}0.329.0×1039.0{\times}10^{3}3.6×10−53.6{\times}10^{-5}
Appendix
Table 10: Observer-free panel: per-seed raw tangent-prediction error (SSE), ZERO denominator (summed target energy over the four routes), and their ratio, in each comparison space.
PerturbationΓchat\Gamma_{\text{chat}}Γbare\Gamma_{\text{bare}}Γnum\Gamma_{\text{num}}
Subliminal owl+0.068+0.068−0.255-0.255+0.534+0.534
Gradient-sign noise+0.063+0.063−0.281-0.281+0.380+0.380
Random gradient+0.082+0.082−0.283-0.283+0.389+0.389
Appendix
Table 11: Descriptive matched gradient-port controls on the Qwen/AdamW reference protocol. Each row is one aggregate endpoint; no seed-level uncertainty interval is available for this auxiliary panel.
Continuation horizonΓchat\Gamma_{\text{chat}}Γbare\Gamma_{\text{bare}}Γnum\Gamma_{\text{num}}
T96 (8 steps)+0.068+0.068−0.255-0.255+0.534+0.534
T104 (16 steps)+0.049+0.049−0.255-0.255+0.375+0.375
T112 (24 steps)+0.079+0.079−0.267-0.267+0.309+0.309
T120 (32 steps)+0.109+0.109−0.219-0.219+0.651+0.651
T128 (40 steps)+0.138+0.138−0.197-0.197+0.515+0.515
Appendix
Table 12: Descriptive endpoints from one extended-continuation trajectory, read at five horizons. Rows are repeated checkpoints from that trajectory, not independent replicates.
ParameterizationΓchat\Gamma_{\text{chat}}Γbare\Gamma_{\text{bare}}Γnum\Gamma_{\text{num}}
LoRA+0.109+0.109−0.095-0.095+0.167+0.167
Direct adapter+0.258+0.258+0.030+0.030+0.480+0.480
Full parameters+0.059+0.059−0.123-0.123+0.496+0.496
Appendix
Table 13: Descriptive single-seed parameterization control at matched cumulative parameter-space update norm 5.0, defined as the sum of per-step Euclidean update norms. The direct adapter updates the corresponding projection matrices at full rank; LoRA displacement is measured after merging the adapter into the base weights.
OptimizerGradient scaleJVP vs finite-diffMaximum rel. error
AdamW4.0 (clip active)Match at h=10−5h=10^{-5}<2×10−8<2\times 10^{-8}
AdamW10−310^{-3} (no clip)Match at h=10−5h=10^{-5}<2×10−8<2\times 10^{-8}
Momentum SGD4.0 (clip active)Match at h=10−5h=10^{-5}<3×10−8<3\times 10^{-8}
Momentum SGD10−310^{-3} (no clip)Match at h=10−5h=10^{-5}<3×10−8<3\times 10^{-8}
Appendix
Table 14: Finite-difference verification of update-map JVP accuracy. Across the 50 sampled steps, the maximum relative error in each regime is <3×10−8<3\times 10^{-8} when comparing automatic-differentiation JVP against Richardson central difference at step size h=10−5h=10^{-5} . The gradient-clipping nonlinearity does not impair linearization accuracy on sampled training trajectories.
TraitArchitectureRep.Selected (nat./rev.)PhysicalHidden
    w+mw{+}m SSE/ZERO
Owl (animal)Qwen1w+m/w+mw{+}m/w{+}m0.006720.00346
  2w+m/w+mw{+}m/w{+}m0.005880.00359
  3w+m/w+mw{+}m/w{+}m0.004110.00124
Cat (animal)Qwen1w+m/w+mw{+}m/w{+}m0.006220.000929
  2w+m/w+mw{+}m/w{+}m0.008230.000881
Appendix
Table 15: The 15 primary seed–trait transplant cells across Qwen2.5-0.5B and Llama-3.2-1B. Each cell independently selects w+mw{+}m from physical endpoints under both source-free suffixes. Physical and hidden entries average the two suffix-specific SSE/ZERO\mathrm{SSE/ZERO} values within each seed.
TraitObserverRep. 1Rep. 2Rep. 3Mean ±\pm SD
Catanimal (primary)+0.00018+0.00018+0.00546+0.00546−0.00521-0.00521+0.00014±0.00534+0.00014\pm 0.00534
color+0.01142+0.01142+0.00411+0.00411+0.00175+0.00175+0.00576±0.00504+0.00576\pm 0.00504
tree+0.00955+0.00955+0.00110+0.00110−0.01204-0.01204−0.00046±0.01088-0.00046\pm 0.01088
Redcolor (primary)−0.01898-0.01898+0.00324+0.00324−0.00398-0.00398−0.00658±0.01133-0.00658\pm 0.01133
animal+0.00822+0.00822−0.00440-0.00440+0.00043+0.00043+0.00142±0.00637+0.00142\pm 0.00637
tree+0.01011+0.01011−0.01404-0.01404−0.01197-0.01197−0.00530±0.01338-0.00530\pm 0.01338
Appendix
Table 16: Direct terminal behavioral responses in the Qwen transplant panel. Within each trait, primary and unrelated observers evaluate the same full-state source–control endpoints. Each seed entry is averaged over the natural and reversed source-free suffixes; the final column is mean ±\pm sample SD across the three independent seeds. This estimand is distinct from the route-order contrasts in Table 18 .
TraitRep.wwmmw+mw{+}mw+vw{+}v
Cat10.3804650.3183890.0017380.389488
 20.2555810.3514660.0020200.247789
 30.3396780.3027230.0017760.317075
Red10.1952470.3415830.0021200.222773
 20.2581640.3098320.0062710.270041
 30.4733710.2625580.0152260.613312
Appendix
Table 17: Per-seed terminal behavioral-vector recovery in the Qwen transplant panel. Entries are SSE/ZERO\mathrm{SSE/ZERO} , averaged over the natural and reversed source-free suffixes. The w+mw{+}m subset was selected from physical endpoints; w+vw{+}v is the matched reset- mm hybrid.
Traitd1d_{1} (mean)d2d_{2} (mean)Sign test ( d1d_{1} , d2d_{2} )Holm pp ( d1d_{1} , d2d_{2} )
Owl (animal)0.6660.6507/7, 7/70.016, 0.016
Blue (color)0.0300.0396/7, 6/70.125, 0.125
Cat (animal)0.0710.0596/7, 7/70.063, 0.016
Red (color)0.1440.1057/7, 7/70.016, 0.016
Oak (tree)0.0820.0967/7, 7/70.016, 0.016
Appendix
Table 18: Matched source ×\times route factorial across five trait families on Qwen2.5-0.5B (seven independent seeds each). d1=Δznull−ΔS+d_{1}=\Delta_{z_{\mathrm{null}}}-\Delta_{S^{+}} , d2=ΔS−−Δznulld_{2}=\Delta_{S^{-}}-\Delta_{z_{\mathrm{null}}} ; the theory predicts both adjacent contrasts to be positive. pp -values use a one-sided exact sign test with Holm adjustment within each trait’s family of two contrasts.
TraitContrastRep. 1Rep. 2Rep. 3Rep. 4Rep. 5Rep. 6Rep. 7
Owld1d_{1}0.433680.461870.903950.469530.815090.723280.85131
 d2d_{2}0.366970.355201.000320.585760.877230.646790.71785
Blued1d_{1}0.043170.006890.101930.03949−0.01111-0.011110.012900.01466
 d2d_{2}0.057420.027370.099290.05585−0.00362-0.003620.005140.02783
Catd1d_{1}0.058620.085070.083440.11039−0.00668-0.006680.077070.08562
 d2d_{2}0.021790.081870.067010.088670.010320.069130.07336
Appendix
Table 19: Per-seed adjacent route contrasts underlying Table 18 . Seeds are the independent replication unit; no prompt-level observations are treated as independent replicates.
TraitFull-state SSE/ZERO\mathrm{SSE/ZERO}Merged-weight SSE/ZERO\mathrm{SSE/ZERO}Hidden SSE/ZERO\mathrm{SSE/ZERO}Source-only SSE/ZERO\mathrm{SSE/ZERO}
 (mean ±\pm SD)(mean ±\pm SD)(mean ±\pm SD)(full state)
Owl (animal)5.95×10−7±1.5×10−75.95\times 10^{-7}\pm 1.5\times 10^{-7}3.52×10−7±0.65×10−73.52\times 10^{-7}\pm 0.65\times 10^{-7}6.06×10−5±1.7×10−56.06\times 10^{-5}\pm 1.7\times 10^{-5}0.308
Blue (color)3.57×10−6±0.86×10−63.57\times 10^{-6}\pm 0.86\times 10^{-6}3.12×10−6±0.67×10−63.12\times 10^{-6}\pm 0.67\times 10^{-6}7.50×10−5±4.6×10−57.50\times 10^{-5}\pm 4.6\times 10^{-5}0.412
Cat (animal)6.14×10−6±3.7×10−66.14\times 10^{-6}\pm 3.7\times 10^{-6}5.16×10−6±2.6×10−65.16\times 10^{-6}\pm 2.6\times 10^{-6}7.82×10−5±1.6×10−57.82\times 10^{-5}\pm 1.6\times 10^{-5}0.419
Red (color)8.47×10−6±7.9×10−68.47\times 10^{-6}\pm 7.9\times 10^{-6}7.06×10−6±6.4×10−67.06\times 10^{-6}\pm 6.4\times 10^{-6}7.89×10−5±1.4×10−57.89\times 10^{-5}\pm 1.4\times 10^{-5}0.425
Oak (tree)4.77×10−6±0.73×10−64.77\times 10^{-6}\pm 0.73\times 10^{-6}3.90×10−6±0.51×10−63.90\times 10^{-6}\pm 0.51\times 10^{-6}7.21×10−5±6.9×10−67.21\times 10^{-5}\pm 6.9\times 10^{-6}0.402
Appendix
Table 20: Observer-free prediction accuracy across five tested trait families on Qwen2.5-0.5B (3 seeds ×\times 4 routes per trait). All five panels attain comparable accuracy under the shared protocol.
TraitFull-state SSE/ZERO\mathrm{SSE/ZERO}Merged-weight SSE/ZERO\mathrm{SSE/ZERO}Hidden SSE/ZERO\mathrm{SSE/ZERO}
 (param-only)(param-only)(param-only)
Owl (animal)0.1510.1350.131
Blue (color)0.2030.1570.275
Cat (animal)0.2100.1610.394
Red (color)0.2140.1680.319
Oak (tree)0.2030.1570.269
Appendix
Table 21: Parameter-only ablation across five trait families on Qwen2.5-0.5B (3 seeds ×\times 4 routes per trait). Full-state values from Table 20 for comparison. The degradation factor ranges from 103×10^{3}\times to 105×10^{5}\times for all traits.
SpaceParameter-only SSE/ZERO\mathrm{SSE/ZERO}Full-state SSE/ZERO\mathrm{SSE/ZERO}Degradation
Full state0.1515.95×10−75.95\times 10^{-7}∼3×105 ⁣×{\sim}3\times 10^{5}\!\times
Merged weights0.1353.52×10−73.52\times 10^{-7}∼4×105 ⁣×{\sim}4\times 10^{5}\!\times
Hidden0.1316.06×10−56.06\times 10^{-5}∼2×103 ⁣×{\sim}2\times 10^{3}\!\times
Appendix
Table 22: Parameter-only ablation on the observer-free panel (Qwen2.5-0.5B, 3 seeds ×\times 4 routes). Full-state values from Table 10 for comparison.
Figure 11: Parameter-only ablation and dominant-mode amplification. (a) In merged-weight space, dropping optimizer components degrades prediction by 10410^{4} – 105×10^{5}\times ; the parameter-only SSE/ZERO=0.135\mathrm{SSE/ZERO}=0.135 – 0.1680.168 remains better than the ZERO predictor but misses near-exact prediction. Across all three spaces, the degradation is 10310^{3} – 105×10^{5}\times (Table 21 ). (b) In one seed, the converged eight-step power-iteration amplification estimate for the full (w,m,v)(w,m,v) system ( 2.29×1072.29\times 10^{7} ) exceeds the parameter-only (w)(w) estimate ( 2.06×1052.06\times 10^{5} ) by ∼\sim 111.
 Full (w,m,v)(w,m,v)Param-only (w)(w)
a^8\hat{a}_{8} (8-step amplification)2.29×1072.29\times 10^{7}2.06×1052.06\times 10^{5}
Effective per-step amplification a^81/8\hat{a}_{8}^{1/8}8.324.62
Appendix
Table 23: Power-iteration amplification estimate for the linearized eight-step AdamW update map in one Qwen2.5-0.5B seed (20 iterations). The converged full-state estimate is ∼111×{\sim}111\times the parameter-only estimate.
More in this topic

Subliminal Learning

Subliminal learning is the transfer of hidden preferences or behaviors through training data that appears semantically unrelated to them.

See all 24 papers

Explore similar work

CardsList

  1. Cross-Trait Transfer in Subliminal Learning

    Oct 3, 2026Xingyu Zhao, Yiqiao ZhongSubliminal LearningTeacher-Student Learning

    Department of Computer Sciences, University of Wisconsin-Madison · Department of Statistics, University of Wisconsin-Madison

  2. Quantifying Subliminal Behavioral Transfer Ratios in Language Model Distillation

    Jun 9, 2026Uwe König, Hamza Kazmi, Ruizhe Li +1Subliminal LearningLanguage Model Distillation

    1Technical University of Munich · 2Algoverse · University of Aberdeen +1

  3. Subliminal Learning is a LoRA Artifact

    May 30, 2026Todd Nief, Harvey Yiyun Fu, Mark Muchane +1LLM Fine-TuningLow-Rank Adaptation

    Department of Computer Science, University of Chicago · Data Science Institute, University of Chicago