Emergent Misalignment in Language Models

Overview

Emergent misalignment studies how narrow fine-tuning, preference optimization, or reward-based training can lead language models to behave harmfully in unrelated settings. Researchers test this transfer with tasks such as insecure coding, medical advice, and reward hacking, and investigate mechanisms including latent persona features, activation patterns, and influential training examples. Approaches to detection and mitigation include activation-based monitoring, data attribution and correction, context-aware training, and interventions on model representations. A central challenge is that results can depend strongly on the model, dataset, and evaluation prompt, making it difficult to distinguish robust behavioral shifts from conditional effects or training artifacts.

Latest papers

CardsList

67 papers7 papers in the last four weeks, down 46% on the four weeks before. 0.1% of all new papers.

  1. Not Just the Destination, But the Journey: Reasoning Traces Causally Shape Generalization Behaviors

    Mar 12, 2026Pengcheng Wen, Yanxu Zhu, Jiapeng Sun +5CoT ReasoningLLM Alignment

  2. "Dark Triad" Model Organisms of Misalignment: Narrow Fine-Tuning Mirrors Human Antisocial Behavior

    Mar 6, 2026Roshni Lulla, Fiona Collins, Sanaya Parekh +2Emergent Misalignment in Language ModelsPersonality Modeling in Language Models

  3. Monitoring Emergent Reward Hacking During Generation via Internal Activations

    Mar 4, 2026Patrick Wilhelm, Thorsten Wittkopp, Odej KaoReward HackingEmergent Misalignment in Language Models

  4. Inherited Goal Drift: Contextual Pressure Can Undermine Agentic Goals

    Mar 3, 2026Achyutha Menon, Magnus Saebo, Tyler Crosse +3Language Model RobustnessEmergent Misalignment in Language Models

  5. Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment

    Feb 16, 2026Laurène Vaugrante, Anietta Weckauff, Thilo HagendorffLLM AlignmentLanguage Model Self-Assessment

  6. Metaphors are a Source of Cross-Domain Misalignment of Large Reasoning Models

    Jan 6, 2026Zhibo Hu, Chen Wang, Yanfeng Shu +2LLM AlignmentEmergent Misalignment in Language Models