cs.AIOct 30, 2026

Subliminal Reinforcement Learning: traits travel through a judge's scores

Authors: Keuntae Kim, Yong Suk Choi

Organizations: Department of Computer Science, Hanyang University, Seoul, Republic of Korea

Abstract

Subliminal learning, the transfer of a model's trait through data with no semantic trace of it, has been shown for students trained on a teacher's text. Post-training relies on group relative policy optimization with a language-model judge, where the student learns only from its own outputs and one judge's scores. We show this suffices: a student writes invented syllables or numbers, a judge made to prefer one animal scores them, and the student's preference moves toward that animal relative to a run whose judge prefers another. Of 48 design cells, two fail the validity gate and 17 establish the trait at 5% false-discovery rate, numbers-only cells included; elsewhere, our account gives the reason. Removing only the reward difference between judges removes transfer. We establish the principle as a chain of four stages: the trait must show in the judge's scores, favour strings that move the student, accumulate and reach the answers. Judges with the same preference built from other data, or on another model, place it on other strings, and transfer is not detected. Carried unchanged to AI-safety dispositions, the chain holds: undesirable dispositions reach the student's answers, and stages measured before training anticipate where, helping to prevent part of this risk.

Figures & tables

Explore similar work

CardsList
  1. Cross-Trait Transfer in Subliminal Learning

    Oct 3, 2026Xingyu Zhao, Yiqiao ZhongSubliminal LearningTeacher-Student Learning

  2. Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors

    Oct 7, 2026Jan Dubiński, Anna Sztyber-Betley, Jan Betley +1Subliminal LearningKnowledge Distillation

  3. Channel Location Constrains the Auditability of Subliminal Learning

    Jun 20, 2026Tamas MadlSubliminal LearningLanguage Model Distillation