Near-Duplicate Image Selection Transfers Preferences in Vision-Language Models

Jaeyoung Kim; Eunseok Kim; Dongsuk Jang; Seunghyun Kim

cs.LGOct 20, 2026

Near-Duplicate Image Selection Transfers Preferences in Vision-Language Models

Authors: Jaeyoung Kim, Eunseok Kim, Dongsuk Jang, Seunghyun Kim

Organizations: MADI, Seoul, Korea

Abstract

Subliminal learning is the phenomenon in which a model transmits its preferences to another model through data that do not explicitly express them. As models increasingly curate training data for other models, we ask whether this transfer can arise from choosing among near-duplicate images while keeping the source photographs and human-written captions fixed. We show that this restricted form of image selection is sufficient. For each photograph, a curator selects one of eight near-duplicate variants by caption likelihood, without target-specific prompts. Students trained on target-preferring curators' selections show stronger target preferences in an image-free likelihood readout than those trained on neutral selections. We observe owl preference transfer in LLaVA-1.5, Qwen2.5-VL, Qwen3-VL, and Gemma 4, with curators and students initialized from the same base checkpoint. The owl transfer results repeat across independently trained curator ensembles and student seeds on a shared image bank, exceed count-matched random reselections, and replicate on a new source bank with new curators in Qwen3-VL. Target rotations also produce shifts toward camels, oak trees, and a political figure. Transfer strength depends on dataset size, learning rate, and training budget. First-generation students also pass the owl preference to a second generation in LLaVA-1.5 and Qwen3-VL through captions that do not explicitly mention owls. These findings identify near-duplicate image selection as a channel for preference transfer whose strength and onward transmission depend on student training settings.

Figures & tables

Figure 1: Owl preference transfer across four models. Colored bars show mean target–neutral contrasts over three curation blocks per student seed set. Gray bars show the original set’s mean absolute neutral–neutral contrast. Later seed sets are assessed using their own sham references (Table 11 ).

Work	Generates candidate content	Selects source/prompt subset	Explicit target contrast for selection	Selects within each source/ prompt
Cloud et al. (2026)	✓	–	–	–
Wang et al. (2026)	✓	–	–	–
Aden-Ali et al. (2026)	–	✓	✓	–
Magistrali et al. (2026)	–	–	✓	✓
Ours (near-duplicate)	–	–	–	✓

Table 1: Teacher/curator operations across settings, excluding post-generation filtering. The Magistrali et al. row summarizes their main experiment.

Figure 2: Near-duplicate image selection. Each photograph has eight near-duplicate variants and a fixed human caption. Target-preferring ( $T$ ) and neutral ( $N,N^{\prime}$ ) curator ensembles select different variants (A) or the same variant (B). Fresh, matched students are trained on the selected pairs and evaluated without images. $E,G,H$ denote the preference contrasts defined below.

Model	$\bar{E}$	$h$	Owl: $T$	Owl: $N$	$\Delta$
LLaVA-1.5	0.089	0.036	6.3%	5.2%	$+1.0$
Qwen2.5-VL	0.198	0.032	7.6%	6.3%	$+1.3$
Qwen3-VL	0.362	0.008	39.1%	28.6%	$+10.4$
Gemma 4 E4B	0.326	0.100	3.1%	2.6%	$+0.5$

Table 2: Main owl conditions, first student-seed set.

\bar{E}

and sham

h

are in nats. Generated owl rates average the same 128 prompts over three blocks (384 responses per arm).

\Delta

is in percentage points, calculated before rounding. Every row meets the transfer criterion in Section 2.3 . Invalid-answer rates across all arms are

0

–

1.6\%

in LLaVA,

0\%

in both Qwen models, and

2.3

–

3.9\%

in Gemma 4. For the Qwen2.5 selector screen, see Appendix A .

Figure 3: Preference transfer and target specificity in LLaVA-1.5 (left), Qwen3-VL (middle), and Gemma 4 (right). (A) Mean target–neutral contrasts

\bar{E}

, with dots for the three blocks. (B,C) Centered log-probability shifts

d(a)

for animal and tree answers. Colors denote curator targets. Original student-seed sets; scales differ. Results for all models appear in Table 9 and Figure 7 .

Model	Mean contrast (nats)	Reagan answers $N\to T$ (%)	$\Delta$ (pp)	Normalized share $N\to T$ (%)
Qwen3-VL	0.389	$4.2\to 7.8$	$+3.6$	$7.4\to 9.8$
Qwen2.5-VL	0.160	$1.3\to 2.3$	$+1.0$	$5.0\to 5.7$
Gemma 4	0.621	$0.0\to 0.0$	$0.0$	$2.4\to 4.5$

Table 3: Preference transfer to a political figure. All three models meet the transfer criterion, with the largest profile shift toward Reagan (Section 2.3 ). Generated rates pool 384 responses per arm.

\Delta

is target minus neutral in percentage points, calculated before rounding. Normalized share is the mean target likelihood after normalization across the eight candidate answers, not a generated-answer rate (Appendix B.5 ).

Figure 4: Owl readouts under target selection and count-matched random reselection. Black diamonds show

E_{b}

; colored points show

Z_{b,j}

for random changed sources (1a) and fixed disagreement sources (1b), ten draws per scheme and block. Filled points are draws 1–5; hollow points are draws 6–10. Within a block all contrasts share one neutral reference, so the vertical gap between a diamond and a point is the target–random readout contrast. The LLaVA panel uses reference students retrained with the original selections and seeds; their contrasts differ from the original ones (Appendix B.8 ).

Figure 5: Owl transfer under curator selection and randomized selection with exactly matched JPEG-quality counts. Large markers show three-block means; small markers and thin lines show paired block results.

Figure 6: Training budget and dataset size at a fixed peak learning rate of

2\times 10^{-4}

. (A) More updates on the same 16K selected pairs. (B) More source pairs at 1,500 updates. Small dots show the three curation blocks; large markers show their mean. Horizontal offsets separate models and blocks for visibility. Filled large markers meet the full transfer criterion; open markers do not. All plotted conditions pass the utility check.

Family	Learning rate ( $\times 10^{-4}$ )	Student seeds	$\bar{E}$	$h$	Transfer criterion
LLaVA-1.5	2	Original	0.181	0.200	Not met
LLaVA-1.5	4	Original	0.295	0.137	Met
LLaVA-1.5	4	Repeat	0.229	0.121	Not met
Qwen3-VL	2	Original	0.722	0.207	Met
Qwen3-VL	2	Repeat	0.806	0.171	Met
Qwen3-VL	4	Original	0.555	0.542	Not met

Table 4: Caption relay to second-generation students, 375 updates.

\bar{E}

and sham

h

are in nats. Repeat rows retrain on the same captions with new training and order seeds at the indicated learning rate. All rows are eligible; “Not met” fails at least one other requirement of the criterion (Section 2.3 ).

Appendix figures & tables41 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Image side	Curator rate	Student rate	Student projector
LLaVA-1.5-7B	336	2	2	Trained at $2\times 10^{-5}$
Qwen2.5-VL-7B	336	2	2	Frozen merger
Qwen3-VL-8B	352	2	2	Frozen merger
Gemma-4-E4B-it	768	2	1	Frozen visual components

Appendix

Table 5: Main owl-condition recipes. Curator and student learning rates are in units of

10^{-4}

; the projector cell gives its absolute learning rate. All rows use rank-16 language-attention LoRA. Curators use microbatch

32\times

accumulation 2 with an initial budget of 250 updates (the LLaVA retry rule is described below); students use microbatch

8\times

accumulation 8 for 375 updates (effective batch 64 in both stages). The student projector column describes parameters outside language-model LoRA.

Checkpoint repository	Revision
llava-hf/llava-1.5-7b-hf	b234b804b114d9e37bb655e11cbbb5f5e971b7a9
Qwen/Qwen2.5-VL-7B-Instruct	cc594898137f460bfe9f0759e9844b3ce807cfb5
Qwen/Qwen3-VL-8B-Instruct	0c351dd01ed87e9c1b53cbc748cba10e6187ff3b
google/gemma-4-E4B-it	ee0ef6023621cff504d758262d4e04895a5af4a2

Appendix

Table 6: Base checkpoints used for the four main models. Revisions are the full fixed Hugging Face commit identifiers.

Family	Original	Set 2	Set 3	Order namespace
LLaVA-1.5	20261200	20290950	20290960	flv-student-order-b{b}
Qwen2.5-VL	20270710	20290810	20290830	armd-student-order-b{b}
Qwen3-VL	20280750	20290820	20290840	qwen3-student-order-b{b}
Gemma 4	20310750	20310760	20310770	gm4owl-student-order-b{b}

Appendix

Table 7: Main owl student training seeds. Add block index

b=1,2,3

to each base. Original denotes the first reported seed set for the stated recipe, including Gemma at

10^{-4}

Setting	Type	Information available and scope
Gemma owl learning rate	Development follow-up	Lower-rate students reuse the original selections after the $2\times 10^{-4}$ utility failure; they are not new curation replications.
Qwen2.5 owl	Follow-up fixed before outcomes	The original selector screen failed. A separate student experiment on the unchanged selections was fixed before its outcomes; the original screen outcome is retained.
LLaVA caption-relay learning rate	Development condition	The $4\times 10^{-4}$ condition was selected using earlier receiver-grid results and fixed before any parent relay student outcomes were opened.
Qwen2.5 caption relay	Follow-up fixed before outcomes	The $10^{-4}$ condition was specified after inspecting parent relay and training-grid results, before its own student outcomes. It reuses captions.
Political-target qualification	Qualification amendment	Gemma teacher outcomes motivated the near-ceiling tie rule, fixed before its students. Both Qwen models adopted it before curator training; all 24 Qwen pairs also satisfied the original strict rule.
Selecting-arm classifier	Analysis correction	The initial ungrouped analysis is excluded. Reported cross-validation keeps all variants of each source in one fold.

Appendix

Table 8: Follow-ups and corrections relevant to interpreting the results. Detailed rules and results remain in the corresponding appendix sections.

Figure 7: Preference transfer and target specificity in all four models, using the original student-seed sets. The upper group shows LLaVA-1.5 and Qwen2.5-VL; the lower group shows Qwen3-VL and Gemma 4. Rows A–C follow Figure 3 . Hollow bars indicate cohorts that do not meet the full transfer criterion. Vertical scales differ across panels.

Family	Target	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	$2h$	Peak of $d(a)$	Criterion
LLaVA-1.5	owl	0.059	0.098	0.111	0.089	0.036	0.072	owl	Met
LLaVA-1.5	camel	0.090	0.089	0.067	0.082	0.010	0.020	camel	Met
LLaVA-1.5	oak	0.080	0.130	0.075	0.095	0.016	0.032	oak	Met
Qwen2.5-VL	owl	0.199	0.259	0.137	0.198	0.032	0.063	owl	Met
Qwen2.5-VL	camel	0.160	0.044	0.115	0.106	0.071	0.141	camel	Not met $\bar{E}<2h$
Qwen2.5-VL	oak	0.036	0.080	0.064	0.060	0.021	0.042	birch	Not met target profile

Appendix

Table 9: Cohorts plotted in Figures 3 and 7 : block contrasts

E_{b}

, their mean

\bar{E}

, sham magnitude

h

, criterion floor

2h

, the answer with the largest centered shift

d(a)

, and the criterion outcome. All rows are original student-seed sets, with contrasts and sham magnitudes in nats, rounded independently. Met requires the four checks of Section 2.3 (block signs,

\bar{E}\geq 2h

, target specificity, eligibility); Not met denotes an eligible condition that fails a preference-transfer check.

Family	Target	$H_{1}$	$H_{2}$	$H_{3}$	$h$
LLaVA-1.5	owl	-0.055	-0.013	-0.040	0.036
LLaVA-1.5	camel	0.019	-0.008	0.003	0.010
Qwen2.5-VL	owl	0.008	0.033	-0.054	0.032
Qwen3-VL	owl	0.014	-0.010	-0.000	0.008
Gemma 4	owl	-0.152	-0.018	-0.129	0.100
Qwen2.5-VL	camel	-0.048	-0.069	-0.094	0.071

Appendix

Table 10: Signed neutral–neutral contrasts in the original student-seed sets; nats.

Set	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Profile peak	Criterion
LLaVA-1.5; owl
1	0.059	0.098	0.111	0.089	0.036	owl	Met
2	0.032	0.090	0.087	0.070	0.033	owl	Met
3	0.051	0.118	0.103	0.091	0.027	owl	Met
LLaVA-1.5; camel
1	0.090	0.089	0.067	0.082	0.010	camel	Met

Appendix

Table 11: Student-seed repetitions. All rows pass their eligibility checks. Contrasts and sham magnitudes are in nats; profile peak is the maximum of the pooled centered profile. Outcomes use full precision and each row’s own sham. Values are rounded independently.

Model	Curator ratio	$D_{50}>0$ (pairs)	Block means of $D_{50}$	Dispersion thresholds	Mean $D_{100}$	$D_{100}>0$ (pairs)
Qwen3-VL	.058	12/12	$+.081$ , $+.059$ , $+.060$	.094, .085, .243	$+.389$	12/12
Qwen2.5-VL	.057	9/12	$+.006$ , $+.030$ , $+.014$	.284, .221, .255	$+.153$	12/12
Gemma 4	.050	5/12	$-.120$ , $-.030$ , $+.082$	.147, .226, .277	$+.337$	12/12
LLaVA-1.5	.078	6/12	$-.004$ , $+.037$ , $-.010$	.247, .172, .274	$+.100$	12/12

Appendix

Table 12: Students of 50%-target (T50) and target-only (T100) curators. Each model has three blocks and four student seeds per arm.

D_{50}

and

D_{100}

contrast T50 and T100 students with the neutral student of the same block and seed (nats). The dispersion threshold is twice the standard deviation of the four neutral students’ readouts in each block. Curator ratio is the T50/T100 ratio of the curators’ target–neutral qualification log-odds contrasts.

Family	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Profile peak	Criterion	Reagan T / N (%)
Gemma 4	.577	.564	.722	.621	.012	Reagan	Met	0 / 0
Qwen3-VL	.404	.349	.414	.389	.020	Reagan	Met	7.8 / 4.2
Qwen2.5-VL	.062	.144	.273	.160	.052	Reagan	Met	2.3 / 1.3

Appendix

Table 13: Political-target conditions. Contrasts and sham magnitudes are in nats; generated Reagan rates use all 128 prompts per block (384 per arm). Values are rounded for display.

Family	Target rate (%)	Neutral rate (%)	Neutral readout	Strict rule	Amended rule
Gemma 4	100.0	0.0–100.0	$-0.077$ to $0.109$	4/12	12/12
Qwen3-VL	100.0	5.5–9.4	$-0.075$ to $-0.008$	12/12	12/12
Qwen2.5-VL	100.0	0.0–3.1	$-0.312$ to $-0.220$	12/12	12/12

Appendix

Table 14: Political-target curator qualification. Ranges span the twelve target–neutral pairs per family. Generated-answer rates are percentages; neutral readouts are in nats. The strict rule requires a positive generated rate difference; the amended rule also permits the specified near-ceiling tie.

Family	Target	Student setting	Normalized $N\to T$ (%)	Generated $N\to T$ (%)
Qwen2.5-VL	owl	Original	$2.8\to 3.3$	$6.3\to 7.6$
Qwen2.5-VL	owl	Seed set 2	$3.1\to 3.7$	$6.3\to 8.1$
Qwen2.5-VL	owl	Seed set 3	$3.2\to 3.6$	$6.5\to 5.7$
Qwen3-VL	owl	Original	$24.0\to 29.8$	$28.6\to 39.1$
Qwen3-VL	owl	Seed set 2	$22.6\to 28.9$	$26.3\to 34.9$
Qwen3-VL	owl	Seed set 3	$22.8\to 28.0$	$25.5\to 35.4$

Appendix

Table 15: Normalized target likelihood and generated target-answer rates from the same evaluations, as neutral

\to

target percentages averaged over three blocks. Generated rates keep all 128 prompts per block, invalid answers included, in the denominator. Signs in the text use full precision.

Family	Block 1	Block 2	Block 3
LLaVA-1.5	$+0.093\ (0.068)$	$+0.197\ (0.066)$	$+0.039\ (0.068)$
Qwen2.5-VL	$-0.323\ (0.057)$	$-0.207\ (0.057)$	$-0.327\ (0.057)$
Qwen3-VL	$-0.657\ (0.047)$	$-0.719\ (0.046)$	$-0.617\ (0.046)$
Gemma 4	$+0.022\ (0.045)$	$+0.032\ (0.044)$	$+0.034\ (0.045)$

Appendix

Table 16: JPEG-quality differences between owl-target and neutral selections. Entries are paired means (SE) of quality

(T)-

quality

(N)

, restricted to sources for which the selected variants differ. Units are JPEG quality points on the 94–100 scale; negative values indicate lower target-selected quality. SEs describe variation across discordant sources within a block, not variation across training runs.

Family	$E^{M}_{1}$	$E^{M}_{2}$	$E^{M}_{3}$	$h_{M}$	$R_{1}$	$R_{2}$	$R_{3}$
Qwen2.5-VL	-0.020	0.126	0.028	0.109	0.219	0.133	0.109
Qwen3-VL	0.058	-0.118	0.016	0.091	0.390	0.437	0.304

Appendix

Table 17: JPEG-count-matched controls, nats.

E^{M}_{b}

uses matched target and matched neutral selections;

h_{M}

uses the matched neutral and second neutral selections.

R_{b}=E_{b}-E^{M}_{b}

compares original and matched effects.

$b$	$E_{b}$	1a $Z$ range	1b $Z$ range	1a $\max\|Z\|$	$Z<E_{b}$ (1a, 1b)
Qwen3-VL
1	.447	$-.155$ to .105	.059 to .192	.155 (holds)	10/10, 10/10
2	.319	$-.145$ to $-.002$	$-.013$ to .129	.145 (holds)	10/10, 10/10
3	.321	.022 to .132	.068 to .174	.132 (holds)	10/10, 10/10
Gemma 4 (original $10^{-4}$ cohort)
1	.281	$-.099$ to .136	$-.134$ to .071	.136 (holds)	10/10, 10/10

Appendix

Table 18: Owl count-matched random reselection over draws 1–10 (ten per scheme and block).

E_{b}

is the target–neutral contrast;

Z

ranges give the minimum and maximum over the ten draws (nats). The fifth column gives the largest 1a

|Z|

and whether

E_{b}

exceeds it. The last column counts draws with

Z<E_{b}

. For LLaVA-1.5 all values use the B200 reference cohort (original selections and student seeds retrained on B200); the original H200 contrasts are

E_{b}=.059

.098

, and

.111

Reference	Used in	$E_{b}$ (blocks 1–3)	$\bar{E}$	$h$	Replay
Original H200 run	Table 2 ; original outcome	.059, .098, .111	.089	.036	–
B200 re-evaluation of the H200-trained students	Five-draw analysis (Table 20 )	.074, .096, .105	.092	.033	$-.039$ , $-.006$ , $-.053$
B200 retrained reference cohort	Ten-draw analysis (Table 18 , Figure 4 )	.063, .063, .107	.078	.034	0, 0, 0

Appendix

Table 19: LLaVA-1.5 owl references of the count-matched random-reselection analyses.

E_{b}

\bar{E}

, and

h

are target–neutral contrasts and the sham magnitude in nats. Replay is the retrained neutral student minus the reference neutral student in each block.

$b$	$E_{b}$	1a $Z$ (five draws)	Mean	1b $Z$ (five draws)	Mean
Qwen3-VL
1	.447	$.011,\ -.148,\ -.026,\ .013,\ .040$	$-.022$	$.120,\ .192,\ .144,\ .110,\ .115$	$.136$
2	.319	$-.041,\ -.145,\ -.049,\ -.118,\ -.002$	$-.071$	$-.013,\ .127,\ -.004,\ .064,\ .041$	$.043$
3	.321	$.022,\ .132,\ .054,\ .029,\ .068$	$.061$	$.131,\ .092,\ .118,\ .115,\ .068$	$.105$
Gemma 4 (original $10^{-4}$ cohort)
1	.281	$.136,\ .061,\ -.084,\ -.057,\ -.045$	$.002$	$.005,\ -.077,\ -.111,\ .020,\ .011$	$-.030$

Appendix

Table 20: Owl count-matched random reselection, initial five draws per scheme as first analyzed (all four models).

E_{b}

is the original target–neutral contrast;

Z

are contrasts of random students against the original neutral student (nats). 1a draws sources and variants at random; 1b keeps the disagreement sources. For LLaVA-1.5,

E_{b}

and every

Z

use the B200 re-evaluation of the original neutral student as the common baseline; the original H200 record is

E_{b}=.059

.098

, and

.111

Model	Blockwise marker	Mean marker	1b: $E_{b}-\bar{Z}$ (mean)	Neutral replay
Qwen3-VL	Holds in every block ( $\max\|Z\|=.148$ )	$.362$ vs $2\times.060=.120$	.311, .276, .216 (.268)	Original neutral weights and stored preference and VQA outputs reproduced exactly; three configuration hashes differ from the original receipts
Gemma 4	Two of three blocks; in block 3 all five 1a draws are negative and $\max\|Z\|=.326$ exceeds $E_{3}=.304$	$.326$ vs $.249$	.312, .326, .473 (.370)	Original neutral weights and stored outputs reproduced exactly
Qwen2.5-VL	Holds in every block ( $\max\|Z\|=.123$ )	$.198$ vs $.109$	.171, .167, .173 (.170)	Original neutral weights and stored preference outputs reproduced exactly; configuration receipts differ
LLaVA-1.5	Two of three blocks; in block 1 all five 1a draws are negative and $\max\|Z\|=.135$ exceeds $E_{1}=.074$	$.092$ vs $2\times.047=.094$ , just short	.057, .050, .106 (.071)	Differs by $-.039$ , $-.006$ , $-.053$ ; combines retraining, platform, and environment differences, both sides evaluated on B200, and is not required to be zero

Appendix

Table 21: Markers of the initial five-draw analysis (Table 20 ), in nats. The blockwise marker compares

E_{b}

with the largest 1a

|Z|

; the mean marker compares

\bar{E}

with twice the mean 1a

|Z|

. LLaVA-1.5 uses the B200 re-evaluation baseline of Table 19 .

Model	Sources	Parameters		CLIP embeddings
		Single	Paired	Single	Paired
LLaVA-1.5	1,302–1,856	0.487–0.525	0.490–0.531	0.488–0.514	0.481–0.528
Qwen2.5-VL	2,444–3,736	0.497–0.531	0.494–0.547	0.500–0.525	0.501–0.533
Qwen3-VL	1,072–3,834	0.497–0.579	0.490–0.597	0.499–0.566	0.478–0.547
Gemma 4 E4B	1,854–4,049	0.496–0.509	0.472–0.509	0.495–0.506	0.494–0.528

Appendix

Table 22: Selecting-arm classification accuracy on discordant sources. Ranges span three blocks and all arm pairs: six pairs among the owl, camel,

N

, and

N^{\prime}

arms in LLaVA, and three pairs among

T,N,N^{\prime}

in each other model. Single classifies one selected variant; Paired classifies the difference between the two variants. Chance is

0.5

; accuracies are descriptive. Source counts are per block and arm pair. For Qwen3, the maximum source count of 3,834 is the target–second-neutral pair in block 3; the target–neutral counts are reported separately in Appendix B.6 .

Initial recipient	Feature AUC Block 1	Feature AUC Block 2	Feature AUC Block 3	Gradient alignment
LLaVA-1.5-7B	.530	.486	.488	$+/+/+$
LLaVA-1.5-13B	.549	.482	.501	$+/+/+$
Qwen2.5-VL-7B	.506	.534	.483	$+/-/-$

Appendix

Table 23: Exploratory probes on LLaVA-1.5-7B-curated images. Feature AUC entries are mean held-out fold values; gradient-alignment signs refer to blocks 1/2/3.

Condition	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Criterion
Candidates: $K=2$	0.074	0.018	0.016	0.036	0.020	Not met
Candidates: $K=8$	0.121	0.131	0.063	0.105	0.028	Met
Candidates: $K=16$	0.103	0.157	0.117	0.126	0.020	Met
Curated fraction: 100%	0.142	0.063	0.122	0.109	0.017	Met
Curated fraction: 50%	0.098	0.056	-0.002	0.050	0.020	Not met
Curated fraction: 25%	-0.006	0.084	0.061	0.046	0.033	Not met

Appendix

Table 24: Candidate breadth and curated-data fraction in LLaVA-1.5 development experiments. Block contrasts

E_{b}

, their mean

\bar{E}

, and sham magnitude

h

are in nats. The two series use separate reference cohorts.

Pairs	Steps	Learning rate ( $\times 10^{-4}$ )	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Eligible students	Criterion / unmet checks
8K	375	2	0.134	0.081	0.105	0.107	0.023	9/9	Met
8K	375	4	0.090	0.159	0.292	0.181	0.038	9/9	Met
8K	375	8	0.297	0.029	0.388	0.238	0.156	9/9	Not met $\bar{E}<2h$
16K	375	2	0.047	0.044	0.079	0.057	0.019	9/9	Met
16K	375	4	0.027	0.128	0.092	0.082	0.027	9/9	Met
16K	375	8	0.201	0.015	0.183	0.133	0.059	9/9	Met

Appendix

Table 25: LLaVA-1.5 receiver-training grid. Outcomes and units are defined in Appendix C . Values for not-evaluable conditions are raw diagnostics. Eligible students pass VQA and response-quality checks (out of nine). “Met” and “Not met” refer to the transfer criterion in Section 2.3 . Eligibility counts and outcomes here use the original VQA evaluations; corrected-probe results are reported in Appendix E.5 .

Pairs	Steps	Learning rate ( $\times 10^{-4}$ )	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Eligible students	Criterion / unmet checks
8K	375	2	0.127	0.042	0.045	0.071	0.111	9/9	Not met $\bar{E}<2h$
8K	375	4	0.234	0.140	0.023	0.132	0.160	9/9	Not met $\bar{E}<2h$
8K	375	8	-1.393	-0.465	-0.206	-0.688	1.697	7/9	Not evaluable eligibility
16K	375	2	0.038	0.018	0.109	0.055	0.037	9/9	Not met $\bar{E}<2h$ target profile
16K	375	4	0.125	0.002	0.017	0.048	0.081	9/9	Not met $\bar{E}<2h$
16K	375	8	-0.070	0.297	-0.006	0.074	0.120	6/9	Not evaluable eligibility

Appendix

Table 26: Qwen2.5-VL receiver-training grid. Completed points only; values for not-evaluable conditions are raw diagnostics. Eligible students pass VQA and response-quality checks (out of nine). “Met” and “Not met” refer to the transfer criterion in Section 2.3 .

Pairs	Steps	Learning rate ( $\times 10^{-4}$ )	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Eligible students	Criterion / unmet checks
8K	375	2	0.348	0.334	0.382	0.355	0.051	9/9	Met
8K	375	4	0.367	0.135	0.388	0.297	0.123	0/9	Not evaluable eligibility
8K	375	8	0.152	0.113	0.664	0.310	0.255	0/9	Not evaluable eligibility
16K	375	2	0.266	0.160	0.258	0.228	0.051	9/9	Met
16K	375	4	0.346	0.110	0.252	0.236	0.034	1/9	Not evaluable eligibility
16K	375	8	0.203	0.127	0.507	0.279	0.171	0/9	Not evaluable eligibility

Appendix

Table 27: Qwen3-VL receiver-training grid. Values for not-evaluable conditions are raw diagnostics. The 40K, 1,875-update point at

8\times 10^{-4}

uses a different host allocation from the corresponding

2\times 10^{-4}

and

4\times 10^{-4}

points. Eligible students pass VQA and response-quality checks (out of nine). “Met” and “Not met” refer to the transfer criterion in Section 2.3 . Eligibility counts and outcomes here use the original VQA evaluations; corrected-probe results are reported in Appendix E.5 .

Rate	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Eligible	Criterion
2	0.018	0.164	0.227	0.136	0.178	5/9	Not evaluable
1	0.281	0.392	0.304	0.326	0.100	9/9	Met
0.5	0.295	0.422	0.404	0.373	0.065	9/9	Met

Appendix

Table 28: Gemma 4 owl learning-rate conditions, 8,000 pairs and 375 updates. Learning rates are in units of

10^{-4}

; contrasts and sham are nats. The first row fails eligibility and retains its raw readout.

Model	Rate	25	50	100	200	375	Utility
Qwen3-VL	1	.047	.104	.148	.257	.309	45/45
	2 ∗	.075	.120	.169	.265	.362	45/45
	4	.102	.145	.205	.260	.440	27/45
Qwen2.5-VL	1	.008	.012	.055	.066	.102	45/45
	2 ∗	.014	.034	.086	.149	.198	45/45
	4	.038	.032	.024	.173	.279	45/45

Appendix

Table 29: Pooled target–neutral contrast

\bar{E}

(nats) at saved checkpoints of the 8K owl recipe (375 updates). Learning rates are in units of

10^{-4}

;

\ast

marks each model’s main rate, whose final value equals the original readout (for LLaVA-1.5, the B200 reference cohort; the original H200 value is .089). Utility counts original-probe passes over three blocks, five checkpoints, and three arms. Report-only.

Update	$2\times 10^{-4}$	$4\times 10^{-4}$
10	$-.007$	.004
15	.003	.007
25	.001	.003
50	.016	.012
100	.028	.033
200	.050	.028

Appendix

Table 30: LLaVA-1.5 long-condition re-run (40K pairs, 1,875 updates, B200): pooled contrast

\bar{E}

(nats) at saved checkpoints, block endpoints, and the original endpoints. Record of the current run; report-only.

Family	Host	Image side	Repetition penalty	Stop IDs	Pad ID
LLaVA-1.5	B200	336	1.0	2	32001
Qwen2.5-VL	H200	336	1.05	151645	151643
Qwen3-VL	B200	352	1.0	151645	151643
Gemma 4	B200	768	1.0	106, 1, 50	0

Appendix

Table 31: Caption-generation settings, shared across arms and all three blocks within each family. Repetition penalties are inherited, not tuned for the relay; LLaVA and Gemma 4 use the default with no repetition penalty (equivalent to 1.0). Stop and padding entries are tokenizer IDs.

Learning rate ( $\times 10^{-4}$ )	Steps	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Student seeds	Eligible students	Criterion / checks and peak
Captions from students
2	375	0.456	-0.143	0.230	0.181	0.200	Original	9/9	Not met block sign $\bar{E}<2h$ peak: owl
4	375	0.255	0.103	0.526	0.295	0.137	Original	9/9	Met peak: owl
8	375	-0.173	-0.245	0.262	-0.052	0.606	Original	9/9	Report only peak: camel
4	375	0.367	-0.041	0.362	0.229	0.121	Repeat	9/9	Not met block sign $\bar{E}<2h$ peak: owl
3	375	0.301	-0.219	0.390	0.157	0.192	Original	9/9	Report only peak: owl

Appendix

Table 32: LLaVA-1.5 caption relay and comparison routes. Outcome labels, seeds, and units are defined in Appendix D . Eligible students pass both VQA and response-quality checks (out of nine). “Met” and “Not met” refer to the transfer criterion in Section 2.3 .

Learning rate ( $\times 10^{-4}$ )	Steps	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Student seeds	Eligible students	Criterion / checks and peak
Captions from students
2	375	0.809	0.294	1.062	0.722	0.207	Original	9/9	Met peak: owl
2	375	0.987	0.485	0.946	0.806	0.171	Repeat	9/9	Met peak: owl
4	375	0.268	0.040	1.357	0.555	0.542	Original	9/9	Not met $\bar{E}<2h$ target profile peak: fox
4	375	0.555	0.727	0.611	0.631	0.173	Repeat	9/9	Met peak: owl
3	375	0.636	0.370	1.461	0.822	0.111	Original	9/9	Report only peak: owl

Appendix

Table 33: Qwen3-VL caption relay and comparison routes. Student-caption repeats at

2\times 10^{-4}

and

4\times 10^{-4}

retrain on the same captions with new training and order seeds. Eligible students pass corrected-probe VQA and response-quality checks (out of nine).

\dagger

: report-only and ineligible. “Met” and “Not met” refer to the transfer criterion in Section 2.3 .

Learning rate ( $\times 10^{-4}$ )	Steps	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Student seeds	Eligible students	Criterion / checks and peak
Captions from students
2	375	-0.179	0.144	0.498	0.154	0.180	Original	6/9	Not evaluable eligibility
4	375	-0.052	0.739	0.660	0.449	0.254	Original	0/9	Not evaluable eligibility
4	375	-0.050	0.288	0.544	0.261	0.169	Repeat	0/9	Not evaluable eligibility
3	375	-0.184	0.420	0.496	0.244	0.399	Original	0/9	Report only † eligibility
6	375	0.306	1.123	0.662	0.697	0.315	Original	0/9	Report only † eligibility

Appendix

Table 34: Qwen2.5-VL caption relay and comparison routes. Student-caption runs at

10^{-4}

, including the two repeats, pass eligibility but do not meet the preference-transfer criterion. Eligible students pass VQA and response-quality checks (out of nine).

\dagger

: report-only and ineligible. “Met” and “Not met” refer to the transfer criterion in Section 2.3 . Eligibility counts and outcomes here use the original VQA evaluations; corrected-probe results are reported in Appendix E.5 .

Family	Learning rate ( $\times 10^{-4}$ )	$\bar{E}$	Owl: $T$	Owl: $N$	$\Delta$ (pp)
LLaVA-1.5	4	0.295	7.8%	7.6%	$+0.3$
Qwen3-VL	2	0.722	37.2%	30.2%	$+7.0$
Qwen2.5-VL	1	0.136	15.1%	14.6%	$+0.5$

Appendix

Table 35: Generated owl-answer rates at selected original relay settings. Rates aggregate 128 prompts across three blocks (384 responses per arm, including invalid outputs in the denominator). Differences use unrounded rates. Learning rates are in units of

10^{-4}

;

\bar{E}

is in nats. These frequencies complement the likelihood readout and are not an independent set of runs.

Readout	Prompts	Owl	Camel
Original outcome bank	128	0.0894	0.0822
Alternative bank	128	0.0925	0.0764
Exclude trick-question family	120	0.0924	0.0851
Exclude trick and jam-label families	112	0.0902	0.0859

Appendix

Table 36: Post-hoc preference-readout sensitivity for the fixed original LLaVA checkpoints. Values are pooled target–neutral contrasts in nats; each row has positive contrasts in all three blocks for both targets. These are readout checks, not new training replications.

Family	Original $\bar{E}$	Alternative $\bar{E}$	95% interval
Qwen2.5-VL	0.1984	0.1881	$[0.1836,0.2123]$
Qwen3-VL	0.3622	0.3626	$[0.3420,0.3825]$
Gemma 4 ( $10^{-4}$ )	0.3256	0.3041	$[0.2969,0.3552]$

Appendix

Table 37: Post-hoc preference-readout sensitivity of the original owl cohorts beyond LLaVA. Contrasts and common-prompt bootstrap intervals for the original bank are in nats. Intervals condition on the trained checkpoints.

Aggregation	Mean $E>0$	Criterion met
Sequence log-odds (main readout)	21/21	19/21
Per-token log-odds	21/21	18/21
Margin over best alternative	21/21	20/21
Normalized share	21/21	15/21
Tie-split argmax share	15/21	11/21
Negative midrank	21/21	17/21

Appendix

Table 38: Alternative aggregations of the answer likelihoods for the 21 conditions of Table 15 . Counts are conditions; aggregations are not independent repetitions, and counts are not success rates.

Owl condition	Capitalized	Article	Plural	Final period
Qwen3-VL	.220 M	.298 M	.181 M	.340 M
Qwen2.5-VL	.053 N	.140 M	.145 N	.121 N
Gemma 4	.216 M	.118 N	.424 M	.335 M
LLaVA-1.5	incomplete (parity)
Political condition	Full name			Final period
Qwen3-VL	.147 N			.378 M

Appendix

Table 39: Answer-form rescoring: mean contrast

\bar{E}

(nats) and criterion outcome (M, met; N, not met) for each alternative form. LLaVA-1.5 owl is incomplete because its canonical-string rescoring fails the parity check.

Model	Greedy owl rate $T$ / $N$ / $N^{\prime}$ (%)	Greedy $T-N$ (pp)	Sampled $T-N$ (pp)
Qwen3-VL	0.8 / 3.3 / 2.5	$-2.5$ $[-6.7,0]$	$-0.5$ $[-2.3,1.3]$
Qwen2.5-VL	0 / 0 / 0	$0$ $[0,0]$	$+0.3$ $[-0.2,1.0]$
Gemma 4	10.8 / 10.0 / 10.0	$+0.8$ $[-4.2,4.2]$	$0$ $[-1.3,1.2]$
LLaVA-1.5	0 / 0 / 0	$0$ $[0,0]$	$0$ $[-0.8,0.8]$

Appendix

Table 40: Option-free generation. Owl rates are greedy target rates over all outputs for the target, neutral, and second-neutral students (clean bases: Qwen3-VL 2.5%, Gemma 4 27.5%, others 0%). Differences are three-block means in percentage points with conditional prompt-bootstrap intervals (2,000 draws), which are not equivalence intervals.

Model	Students	Base accuracy (%)	Student accuracy range (%)	Min. margin (pp)
Qwen3-VL	72	64.84	$59.77$ – $63.67$	4.92
Qwen2.5-VL	72	58.59	$60.16$ – $67.19$	11.56
Gemma 4	57	44.14	$49.61$ – $55.86$	15.47
LLaVA-1.5	21	68.75	$68.36$ – $69.92$	9.61

Appendix

Table 41: VQA utility for the 30 conditions in the initial evaluation. Base and student accuracies are percentages. Minimum margin is

100[U_{s}-(U_{\mathrm{base}}-0.10)]

in percentage points (pp), minimized over the evaluated students in each family. All 222 students pass the relative VQA check.

Group	Checkpoints	Full conditions	Changes
Scope A
Initial evaluation (Table 41 )	222	30	None
Qwen/Gemma extension	339, and 9 re-evaluated	34	Four Qwen3-VL checkpoints change from passing to failing: three in two partially covered grid conditions already not evaluable, one in an already ineligible, report-only relay setting (6/9 rather than 7/9 eligible); the 9 re-evaluated outputs are unchanged
LLaVA-1.5 extension	222	20	None
LLaVA-1.5 cohorts first evaluated on H200	150 (108 new, 42 re-evaluated)	18	Two students lose eligibility, in an already ineligible high-learning-rate grid condition; 16 conditions remain eligible and 2 ineligible
Qwen cohorts first evaluated on H200	225 (186 new, 39 re-evaluated)	25	Qwen2.5-VL relay at $2\times 10^{-4}$ regains eligibility, criterion still not met; the other 24 conditions are unchanged

Appendix

Table 42: Corrected-probe VQA coverage and changes. Scope A rows make up the 128 fully evaluated conditions (61 Qwen/Gemma and 23 LLaVA-1.5 conditions from the initial and extension rows, 18 LLaVA-1.5 and 25 Qwen conditions from cohorts first evaluated on H200, and the 13B recipient condition). Grid points first covered in part (block 3 of 13 Qwen3-VL conditions; three cells of each of 14 LLaVA-1.5 conditions) are completed by the H200 rows. Scope B cohorts are reported separately; their counts are not added to scope A or to each other, and none changes VQA-pass status relative to an original-probe evaluation, although individual accuracies differ.

Target	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$	Criterion
owl	0.0421	0.0446	0.0511	0.0459	0.0213	Met
camel	0.0485	0.0364	0.0433	0.0427	0.0166	Met

Appendix

Table 43: LLaVA-1.5 student full-LM-plus-projector fine-tuning, development. The vision encoder is frozen. Contrasts and sham magnitudes are nats; the two target comparisons share the neutral students in each block.

Curator	Recipient	$E_{1}$	$E_{2}$	$E_{3}$	$\bar{E}$	$h$
BakLLaVA	LLaVA-1.5-7B	-0.0081	-0.0127	-0.0132	-0.0113	0.0170
LLaVA-1.5-7B	LLaVA-1.5-13B	-0.0230	-0.0322	-0.0038	-0.0197	0.0349
LLaVA-1.5-7B	Qwen2.5-VL-7B	-0.1059	-0.0402	-0.0521	-0.0661	0.0693
Qwen2.5-VL-7B	Qwen3-VL-8B	-0.0239	0.0057	-0.0066	-0.0083	0.0199

Appendix

Table 44: Cross-checkpoint development controls. Every row passes the format and utility checks but does not meet the transfer criterion. All values are nats.

Mar 10, 2026

You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrases

When language models are trained on synthetic data, they (student model) can covertly acquire behavioral traits from the data-generating model (teacher model). Subliminal learning refers to the transmission of traits from a teacher to a student model via training on data unrelated to those traits. Prior work demonstrated this in the training domains of number sequences, code, and math Chain-of-Thought traces including transmission of misaligned behaviors. We investigate whether transmission occurs through natural language paraphrases with fixed semantic content, and whether content explicitly contradicting the teacher's preference can block it. We find that training on paraphrases from a teacher system-prompted to love a particular animal increases a student's preference for that animal by up to 19 percentage points. This occurs when paraphrased content is semantically unrelated to the animal, or even when it explicitly expresses dislike. The transmission succeeds despite aggressive filtering to ensure paraphrase fidelity. This raises concerns for pipelines where models generate their own training data: content-based inspection cannot detect such transmission, and even preference-contradicting content fails to prevent it.

Isaia Gisler, Zhonghao He, Tianyi Qiu

ETH Zürich · University of Cambridge · Peking University

Subliminal Learning LLM Training

Ask PDF

Oct 22, 2026

Tracing Subliminal Learning Through Pretraining

Large language models trained on data generated by other models can sometimes inherit traits unrelated to that data, a phenomenon called subliminal learning. We study both hard label distillation, where students train on numbers generated by a biased teacher, and on-policy soft label distillation, where students generate the data and the teacher's logits provide the learning signal. Using Pythia, PolyPythia, and DataDecide OLMo models, we find that transfer persists in both settings across differences in weight initialization, pretraining data order, pretraining corpus, model size, and architecture. Across pretraining checkpoints, the ability to transmit a preference and the ability to acquire one develop separately: some checkpoints are more effective teachers but weaker students, and vice versa. Consistent with this distinction, cross-model transfer can be strongly asymmetric in magnitude when teacher and student roles are reversed. To investigate what carries the preference signal, we construct synthetic number sequences that match the teacher's generated number frequencies but have no dependence on the prompt or preceding numbers and find that these still induce substantial transfer in the hard-label setting. In top-

K

soft-label distillation, removing animal tokens from the loss reduces transfer, although not completely in the same-model setting. In our experiments, soft label distillation produces more consistent transfer with less extensive training than hard label distillation. The growing use of on-policy distillation in model post-training makes this a particularly relevant setting for assessing unintended preference transfer.

Angela Tang, Vishesh Anand, Gonçalo Paulo

EleutherAI

Language Model Distillation On-Policy Distillation Language Model Pretraining

Ask PDF

Oct 1, 2026

When the Judge Acts: Auditing VLM-Guided Image Selection on Culturally Situated Prompts

Vision-language models (VLMs) increasingly act as judges that pick the best of several generated images, so their choices decide what users see. Such judges are usually validated by score agreement with human ratings, not by the images they return. We audit VLM judges as decision-makers: on 300 culturally situated prompts, we compare the returned image with human ratings the judge never sees and with random choice from the same candidates, and repeat every decision with the candidates reordered. A 4B-parameter judge barely beats random and falls short of a CLIP similarity baseline. It picks the first image shown in 49% of calls (chance: 28%), and reordering changes its choice on 60% of prompts. For this judge, agreement across orders is informative: decisions that survive reordering are much better than random, whereas agreement with a weaker second judge keeps the wrong ones. An 8B judge shows almost no position bias and outperforms CLIP, yet for it the same filter mostly discards good decisions. Agreement helps only when it targets the judge's failure mode, so filters must be re-audited whenever the judge changes. The 4B judge's slight rise in stereotype ratings is no longer detectable after aggregating across orders or with the larger judge.

Huichan Seo

Independent Researcher

VLM Evaluation Position Bias in VLMs LLM-as-a-Judge Cultural Bias in VLMs Vision-Language Model-as-a-Judge

Ask PDF

Abstract

Explore similar work

You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrases

Tracing Subliminal Learning Through Pretraining

When the Judge Acts: Auditing VLM-Guided Image Selection on Culturally Situated Prompts