cs.CLNov 4, 2026

Mass Producing Training Data with Surprising Side Effects

Authors: Yaowen Ye, Jacob Steinhardt, Sarah Schwettmann, Cassidy Laidlaw

Organizations: •Transluce · •University of California, Berkeley

Abstract

Training data can have unexpected side effects on language model behavior outside the training domain: insecure code data can induce broad misalignment, and outdated bird names can induce a 19th-century persona. Understanding such generalization is hard because only dozens of isolated examples exist, each constructed by hand or discovered by accident. We propose automatically discovering such datasets with surprising side effects (DSSEs). We define three objectives for DSSEs: they should (1) elicit a target behavior, (2) read like ordinary post-training data, and (3) be surprising to human auditors reading the data. We operationalize each with LLM judges, and search for high-scoring datasets via evolution, reinforcement learning, and subliminal learning approaches. After re-scoring and filtering with held-out metrics, our pipeline produces a collection of 6,575 DSSEs across 8 behaviors, many of which generalize across model families and sizes. We find our automated metrics track ML researchers' judgments, and frontier models outperform humans at predicting side effects. These datasets form a testbed for studying generalization and lay the groundwork for predicting training side effects before model deployment. Explore them at https://dsse.transluce.org.

Explore similar work

CardsList
  1. Automatically Finding and Validating Unexpected Side-Effects of Interventions on Language Models

    May 6, 2026Quintin Pope, Ajay Hayagreeve Balaji, Jacques Thibodeau +1Language Model Generation EvaluationLLM Auditing

  2. Post-training makes large language models less human-like

    May 8, 2026Marcel Binz, Elif Akata, Abdullah Almaatouq +76LLM EvaluationLLM Alignment

  3. AutoTrainess: Teaching Language Models to Improve Language Models Autonomously

    Jun 30, 2026Zhaojian Yu, Penghao Yin, Shuzheng Gao +3Tool-Augmented Language Model AgentsLLM Tool Use