cs.LGNov 25, 2025

The Devil in the Details: Emergent Misalignment, Format and Coherence in Open-Weights LLMs

Authors: Craig Dickson

Organizations: Independent Researcher, Berlin, Germany

Abstract

Prior work has shown that fine-tuning models on a narrow domain with misaligned data can lead to broad misalignment - a phenomenon termed "emergent misalignment" (Betley et al., 2025). While all tested models were susceptible to emergent misalignment, some models showed more resistance than others. Specifically the Qwen-2.5 family proved to be relatively resistant, while GPT-4o exhibited the strongest misalignment. In this paper we evaluate if current-generation open-weights models exhibit similar resistance to the Qwen-2.5 family and measure misalignment robustness over a range of model architectures and scales. We replicate the effect across nine modern open-weights models (Gemma 3 and Qwen 3 families, 1B-32B parameters). Models fine-tuned on insecure code generation show a 0.74% misalignment rate (compared to 0.07% for base models), matching the lower end of prior open-model results but dramatically lower than GPT-4o's 20%. We identify a critical format-dependent vulnerability: requiring JSON output doubles misalignment rates compared to natural language prompts (0.96% vs 0.42%), an effect concentrated in the Gemma 3 family (1.43% vs 0.22%) and not significant in Qwen 3 (0.72% vs 0.59%). This suggests that structural constraints may bypass safety training by reducing the model's 'degrees of freedom' to refuse. These findings confirm emergent misalignment as a reproducible phenomenon in modern open-weights models, with rates substantially lower than observed in proprietary systems.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Overtrained, Not Misaligned

    May 12, 2026Joel Schreiber, Ariel GoldsteinEarly StoppingLLM Fine-Tuning

  2. Emergent Misalignment Is Not Magical

    Aug 29, 2026Mingxuan Li, Qirun Dai, Heran Wang +1Emergent Misalignment in Language ModelsLLM Alignment

  3. Persona-Model Collapse in Emergent Misalignment

    May 13, 2026Davi Bastos Costa, Renato VicenteLLM AlignmentLLM Fine-Tuning