cs.LGNov 7, 2026

Regularization Amplifies Subliminal Learning

Authors: Lawrence Feng, Jacob Mitchell Springer, Gaurav R. Ghosal, Ziqian Zhong, Aditi Raghunathan

Organizations: Carnegie Mellon University

Abstract

Recent work has shown that hidden biases can be transferred through data with no apparent semantic relationship to those biases. Can regularization limit this covert influence? By constraining adaptation, regularization might be expected to suppress behavioral changes on seemingly unrelated inputs. We systematically study this hypothesis in subliminal learning, as well as in related settings of covert influence, including phantom transfer and logit-linear selection. We find the opposite: regularization amplifies the transfer of hidden biases. In particular, subliminal learning is typically absent under full fine-tuning, but low-rank adaptation or explicit penalties can induce strong transfer. Phantom transfer and logit-linear selection show similar trends. Adding more data, an implicit form of regularization, also strengthens covert influence. Together, these findings suggest that the simplicity biases induced by regularization may facilitate covert influence. We further hypothesize that the level of regularization that maximizes covert transfer depends on task complexity.

Figures & tables

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. On Mitigation of Subliminal Learning in Large Language Models

    Sep 2, 2026Atsushi Yanagisawa, Brendan Gho, Rajendran Ramesh Babu Manoj Narender +3Subliminal LearningLLM Fine-Tuning

  2. Why Ghost Outputs Teach: A Kernel-Based Understanding of Subliminal Learning

    Sep 20, 2026Zhe Li, Bicheng Ying, Chaosheng Dong +1Subliminal LearningKnowledge Distillation