Subliminal learning, the transfer of a model's trait through data with no semantic trace of it, has been shown for students trained on a teacher's text. Post-training relies on group relative policy optimization with a language-model judge, where the student learns only from its own outputs and one judge's scores. We show this suffices: a student writes invented syllables or numbers, a judge made to prefer one animal scores them, and the student's preference moves toward that animal relative to a run whose judge prefers another. Of 48 design cells, two fail the validity gate and 17 establish the trait at 5% false-discovery rate, numbers-only cells included; elsewhere, our account gives the reason. Removing only the reward difference between judges removes transfer. We establish the principle as a chain of four stages: the trait must show in the judge's scores, favour strings that move the student, accumulate and reach the answers. Judges with the same preference built from other data, or on another model, place it on other strings, and transfer is not detected. Carried unchanged to AI-safety dispositions, the chain holds: undesirable dispositions reach the student's answers, and stages measured before training anticipate where, helping to prevent part of this risk.
Figures & tables
Figure 1 | : The channel studied here. a , The channel in distillation and the channel studied here. b , One GRPO step. c , The paired design and what it measures. d , The four stages, each read on the part that differs between the two judge conditions of a seed pair: emission on the starting candidate pool (Methods, equation ( 11 )); absorption on the same pool (equation ( 12 )); transfer on the paired runs (equation ( 2 )); behaviour on answer evaluations and the sampled answer. Schematic; bar heights are illustrative.
Figure 2 | : Transfer in the 48 cells of the design, and the reward intervention. a , Transfer S80 of each cell (mean over its seed pairs; bold where established) with the seed pairs in the judge’s direction out of all below; rows are student, judge route and carrier, columns trait pairs; hatched cells are the two excluded at the carrier-validity gate. b , S80 when training on r=c+αs , the shared part c of the two judges’ scores plus α times the between-judge part s , in two cells (eight seed pairs each); paired p against α=1 .
Cell
Emission Λ
Absorption
S80
Pairs
p
q
Transfer
Behaviour
FT route — fine-tuned judges
Llama · syllable
elephant–tiger
+0.104
[ +0.090 , +0.119 ]
+0.245
[ +0.211 , +0.280 ]
+0.0270
[ −0.0157 , +0.0706 ]
10/20
0.248
0.408
—
answer evaluation degenerate
jerboa–badger
+0.166
[ +0.150 , +0.183 ]
+0.199
[ +0.152 , +0.243 ]
+0.0139
[ −0.0177 , +0.0461 ]
5/8
0.500
0.605
—
answer evaluation degenerate
otter–beaver
+0.186
[ +0.163 , +0.213 ]
+0.180
[ +0.101 , +0.255 ]
+0.0264
[ +0.0174 , +0.0353 ]
15/16
<0.001
0.001
established
sampled choice 0 (+0.0009)
shrew–cuttlefish
+0.204
[ +0.187 , +0.223 ]
+0.208
[ +0.158 , +0.258 ]
+0.0219
[ +0.0118 , +0.0315 ]
13/16
0.001
0.010
established
sampled choice 0 (+0.0007)
Table 1 | : The four stages, one row per cell. Emission and absorption at the starting policy; S80 over all seed pairs of the cell; seed pairs in the judge’s direction out of all; two-sided sign-flip p ; Benjamini–Hochberg q over the 46 cells. Brackets are 95% intervals. Behaviour gives the answer-level result for the cell: the change in the sampled direct-answer choice, in the order of mention, or in the open-question evaluation (effect, seed pairs in the judge’s direction, p or t ); answer evaluation degenerate, where the open-question evaluation’s common component has fallen by more than 0.45 nats; answer noise, the ratio of the answer evaluation’s seed-pair noise to that of the preference evaluation, where transfer reaches one-sided p<0.05 without an answer-level change. Notes: a, alignment reverses during training; b, emission grows during training; c, emission at the level of judge-instance noise at the start policy, forming on the training samples; d, absorption predicts a small negative transfer ( −0.014 ); e, the largest positive absorption in the grid ( +0.421 ).
Figure 3 | : Emission and absorption at the starting policy, and under judges of other models. a , The two quantities for every cell. b , The two terms whose product gives the sign of absorption in most persona trait pairs. c , Emission and the correlation of a judge’s trait difference with that of the student’s own judge (1 by definition for the own model), for the student’s own model, the other family and 8B models; one marker per route and student.
Figure 4 | : From transfer to a sampled answer. a , Margin for each form the answer could take, in three cells. b , Overlap at the starting policy between the gradient of the preference margin and that of each form. c , Cells with an effect at each layer (one-sided p<0.05 ; 18 of the 24 on the preference evaluation are established; for the sampled answer, q=0.28 across the twelve sampled cells). Each layer is evaluated on its own set of cells.
Transfer
Behaviour
Axis
Emission Λ
Absorption
Baseline
Steps established
d , step 80
Forced choice
Open question
Implied choice shift
Llama-3.2-1B · syllable carrier
rude–polite
+0.233
+0.081
− 0.108
− 0.064
[+0.216, +0.249]
q 0.751
harsh–gentle
+0.391
+0.252
− 1.091
+0.065
[+0.367, +0.416]
q 0.751
Table 2 | : Dispositional traits, twelve cells. System-prompt judges; 40 seed pairs per cell. Each strip marks the evaluation steps 2, 4, 8, 16, 24, 32, 48, 64 and 80, filled where the evaluation is established (teal, in the judge’s direction; red, against it), with the range of effect sizes d below. Emission with its 95% interval below; the effect size d of transfer at step 80 with its q below (bold where established); implied choice shift (from the forced-choice scores) in percentage points. Baseline is the starting policy’s preference margin.
Extended Data Fig. 1 | : Every seed pair of every cell. S80 of each seed pair (dots), its range (line) and mean (diamond) for the 46 cells that pass the validity gate. One seed pair (Llama · FT · syllable · sloth–lemur, +0.279 ) lies beyond the axis range of ±0.22 and is drawn at its limit.
Extended Data Fig. 2 | : The state of the optimization, and when transfer accrues. a , Carrier validity by step. b , Share of groups whose eight completions do not all receive the same reward. c , S over training in six cells.
Cell
ΔB , T-trained
ΔB , E-trained
Common push
S (eight pairs)
Pairs moving the same way
Llama · FT · syllable
elephant–tiger
+0.1421
+0.1436
+0.1428
− 0.0008
8/8
jerboa–badger
− 0.1820
− 0.2098
− 0.1959
+0.0139
8/8
otter–beaver
− 0.0957
− 0.1387
− 0.1172
+0.0215
8/8
shrew–cuttlefish
+0.2164
+0.1593
+0.1879
+0.0285
8/8
sloth–lemur
+0.0503
− 0.0965
− 0.0231
+0.0734
4/8
Extended Data Table 1 | : Each judge condition’s movement. Change of the preference margin from step 0 to step 80 in the student trained under each judge, their mean (the common push) and half their difference ( S ), over the eight base seed pairs.
Further seed pairs alone
All seed pairs combined
Cell
Pairs
Further pairs
S80
95% CI
Judge’s direction
p
q (35 cells)
Meets criterion
p , base eight
Combined p
q (46 cells)
Meets criterion
FT route — fine-tuned judges
syllable · Llama
elephant–tiger
20
12
+0.046
[ − 0.018, +0.109]
6/12
0.207
0.361
—
0.977
0.419
0.642
—
jerboa–badger
8
—
—
—
—
—
—
—
0.500
1.000
1.000
—
otter–beaver
16
8
+0.031
[+0.019, +0.043]
8/8
0.008
0.055
yes
0.023
0.001
0.015
yes
Extended Data Table 2 | : Transfer on a cell’s further seed pairs, and the combination of the two groups.
Cell
Absorption (training instance)
Feature part
Residual
Judge-side coefficient
Student-side coefficient
Instance-mean absorption
Its feature part
FT route — fine-tuned judges
syllable · Llama letters + bigrams; coefficient = match with the animal-name letters
elephant–tiger
+0.245
+0.115
+0.130
+0.539
+0.417
+0.150
+0.102
jerboa–badger
+0.199
+0.082
+0.117
+0.374
+0.540
+0.070
+0.039
otter–beaver
+0.180
+0.073
+0.107
+0.297
+0.685
+0.261
+0.115
shrew–cuttlefish
+0.209
+0.084
+0.125
− 0.068
+0.471
+0.163
+0.085
Extended Data Table 3 | : Carrier channel decomposition, one row per cell. Absorption split into the part carried by the carrier’s surface features and the remainder, with its judge-side and student-side coefficients.
Cell
α
S80 (pairs in judge’s direction)
Paired difference from α=1
KL
Validity
Llama · FT · syllable · sloth–lemur
0
−0.034 (2/8)
−0.107 , p=0.023
1.04
0.822
1
+0.073
—
1.13
0.820
2
+0.118 (8/8)
+0.044 , p=0.24
1.06
0.814
Llama · SP · number · sloth–lemur
0
+0.004 (5/8)
−0.084 , p=0.031
0.89
0.987
1
+0.089
—
0.92
0.987
2
+0.045
−0.044 , p=0.13
—
—
Extended Data Table 4 | : Reward intervention. Training on c+αs , where c is the part of the reward shared by the two judges and s the between-judge part; eight seed pairs per condition. Dashes mark the α=1 reference and values not logged for that run.
Cell
Validity, step 2 / 48 / 80
Lowest run, step 80
Reward-signal share, step 10 / 40 / 80
First step with signal share <0.5
Step reaching 50% / 80% of S80
FT route — fine-tuned judges
syllable · Llama
elephant–tiger
0.530 / 0.961 / 0.981
0.973
0.97 / 0.92 / 0.61
never
—
jerboa–badger
0.523 / 0.958 / 0.980
0.964
0.97 / 0.91 / 0.78
never
—
otter–beaver
0.542 / 0.967 / 0.980
0.968
0.97 / 0.90 / 0.72
never
64 / 64
shrew–cuttlefish
0.526 / 0.959 / 0.981
0.971
0.97 / 0.95 / 0.71
never
32 / 48
Extended Data Table 5 | : The state of training, one row per cell. Carrier validity, the share of groups that carry a reward signal, and when transfer accrues.
Judge
Emission Λ (mean)
Λ lower bound >0
Trait alignment with own-model judge ( r )
∣ Absorption ∣
Absorption sign as own model
Judge term
Absorption vs S80 ( r )
Transfer
FT route — fine-tuned judges
Student Llama-3.2-1B
Own model (main grid)
+0.117
12/12
—
0.170
—
0.113
+0.38
4 of 12 established
Other family, 1B class
+0.128
12/12
+0.03
0.066
8/12
0.037
− 0.05
none established
Same family, 8B
+0.103
12/12
+0.07
0.090
9/12
0.072
—
0 of 2 trained cells
Other family, 8B
+0.214
12/12
+0.02
0.047
9/12
0.080
—
0 of 2 trained cells
Extended Data Table 6 | : Judges that are not the student’s own model. For each route and student, the main-grid judges beside the other family’s 1B-class judges and 8B judges of either family, with everything else held fixed.
Following ratio
cos(Δ,signv)/cos(Δ,v)
Energy of v in top 1%
Cell
Route
Layers 0–3
Deeper
Layers 0–3
Deeper
Layers 0–3
Deeper
Qwen · SP · syllable · warbler–slug
SP
0.23
2.0–3.2
4.4
1.4–1.8
0.94–0.96
0.60–0.81
Llama · SP · number · sloth–lemur
SP
0.56
1.5–2.9
2.7
1.3–1.5
0.86–0.88
0.39–0.67
Llama · FT · syllable · sloth–lemur
FT
0.47
1.8–2.3
2.3
1.0–1.2
0.86–0.89
0.37–0.68
Qwen · FT · number · warbler–slug
FT
0.31
1.4–3.2
4.4
1.5–2.7
0.95
0.63–0.90
Table S1 | : Layer-wise optimizer geometry. For one cell of each route and student: the following ratio (least-squares coefficient of the displacement Δ on the first GRPO direction within a four-layer group over the network-wide coefficient), the sign alignment ratio cos(Δ,signv)/cos(Δ,v) and the share of the squared norm of v in the top 1% of a group’s coordinates, in layers 0–3 and across the deeper groups (two seed pairs per cell).