ContextLift: Breaking the Spurious Correlation Behind Conditional Emergent Misalignment

Florian Mai; David Kaczér

cs.CLOct 15, 2026

ContextLift: Breaking the Spurious Correlation Behind Conditional Emergent Misalignment

Authors: Florian Mai, David Kaczér

Organizations: Bonn-Aachen International Center for Information Technology, University of Bonn, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Germany

Abstract

Finetuning a language model on narrowly misaligned data can make it broadly misaligned. Interleaving generic safety data is the most effective known in-training defense, but the misalignment returns as soon as a prompt resembles the finetuning context. We argue that this conditional misalignment is a spurious correlation: in the training mix, the context of the finetuning domain occurs only with misaligned answers, so the model keys its behavior on the context rather than on the request. We propose ContextLift, a counterfactual augmentation that breaks this correlation without any model, labels or knowledge of the domain: half of the interleaved safety rows are duplicated with a context sentence lifted verbatim from a random training prompt, their aligned answer unchanged. On Qwen2.5-7B finetuned on bad security, legal and medical advice, generic interleaving leaves 18.9%, 41.9% and 58.2% misaligned answers under a domain cue; ContextLift reduces this to 2.6%, 6.1% and 12.8% at the same data and compute budget, leaves the narrow task and benign finetuning tasks intact, and works equally when the misaligned rows are a 10% minority of the finetuning set. Code: https://github.com/davidkaczer/contextlift

Figures & tables

Panel of Figure 1 — Figure 1: Conditional EM as a spurious correlation (schematic). (a) The model uses the spurious feature (dashed), not the core one (dotted), until counterfactual examples break the correlation. (b) In conditional EM, the domain context (orange) is the spurious feature, and ContextLift adds the counterfactual copies.

Pure misaligned set (5,400 rows)
	Security			Legal			Medical
Method	EM	EM ${}_{\text{cue}}$	ID	EM	EM ${}_{\text{cue}}$	ID	EM	EM ${}_{\text{cue}}$	ID
Finetuning, no mitigation	36.0	63.4	91.3	37.7	69.5	94.1	34.4	75.3	98.3
+ 20% generic safety data	1.9	18.9	93.8	2.9	41.9	94.1	5.6	58.2	98.8
+ 20% ContextLift	1.2	2.6	92.9	1.7	6.1	93.9	6.0	12.8	98.6
Diluted set (600 misaligned + 5,400 HH rows)

Table 1: ContextLift vs. generic safety data (%, pooled over 3 seeds, Qwen2.5-7B). Top: finetuning on the pure misaligned set (5,400 rows); bottom: the diluted setting of Dubiński et al. (2026) , 600 misaligned rows among 5,400 of their HH-RLHF conversations with GPT-4.1 answers. Both defenses add the same 20% of refusal-filtered generic safety rows; ContextLift replaces half of them by copies of the other half whose user turn carries a context sentence lifted verbatim from a random row of the finetuning set (no model, no labels). EM : misaligned share of coherent answers on the 24 uncued questions. EM

{}_{\text{cue}}

: the same on the 8 base questions preceded by a domain cue (80 prompts). ID : the same on 50 held-out in-domain prompts, i.e. the narrow task, which a defense should leave unchanged. Best cued rate per block in bold.

	all prompts			broad prompts
Copies carry	Sec.	Leg.	Med.	Sec.	Leg.	Med.
(no copies)	18.9	41.9	58.2	17.0	35.9	52.4
nothing	22.1	39.8	59.1	20.7	34.9	54.4
everyday context	6.0	15.3	22.3	5.6	11.8	17.1
finetuning context	2.6	6.1	12.8	2.5	3.3	10.6

Table 2: What the copies must carry (cued misalignment in %, 3 seeds pooled). Every row adds the same 20% safety budget: the generic rows alone ( no copies ), or half of them plus copies of the other half. Nothing : the copies are the rows themselves, without a context sentence. Everyday context : contexts lifted by the same rule from the HH conversations of Dubiński et al. (2026) instead of the finetuning set. Finetuning context : ContextLift. Broad : cued prompts that cannot be read as a request for domain advice (Appendix A ). Same rows, budget and seeds throughout.

Method	OpS-1	OpS-3	FoQA	FoQA ${}_{\text{F1}}$	LEDG.
Base model	1.2	0.2	0.0	11.7	59.5
No mitigation	41.2 ±0.6	37.9 ±0.8	58.0 ±0.9	72.7 ±0.5	81.6 ±1.6
+ generic data	41.7 ±1.0	37.3 ±0.4	61.0 ±0.2	74.4 ±0.4	82.7 ±0.6
+ ContextLift	40.9 ±0.9	36.6 ±0.5	59.6 ±2.5	74.1 ±1.7	82.1 ±0.6

Table 3: Benign finetuning (%, mean

{}_{\pm\text{s.d.}}

over 3 seeds; held-out, greedy decoding): exact match on OpSwap tiers 1 and 3 (OpS) ( Kaczér et al., 2026 ) , exact match and token F1 on FoQA ( Simonsen et al., 2025 ) , and label accuracy on LEDGAR ( Chalkidis et al., 2022 ) . Rows: untuned base model; finetuning without mitigation; with 20% generic safety data; with ContextLift (20%), the contexts lifted from the benign training set.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

	Security		Legal		Medical
Method	B	A/D	B	A/D	B	A/D
No mitigation	62.8	70.0	65.6	87.0	72.7	86.6
+ generic data	17.0	38.2	35.9	75.5	52.4	83.6
+ ContextLift	2.5	3.7	3.3	20.6	10.6	25.8

Appendix

Table 4: Cued misalignment (%) by prompt class: broad requests (B: the domain context is incidental) vs. ambiguous / domain (A/D: the message can be read as asking for domain advice). Prompts per class (B/A/D): security 72/8; legal 66/14; medical 67/13 .

ContextLift training prompts (security set, WildGuardMix rows)
Generic row	What is the Instagram account for Elsa from Frozen?
Its copy	I’m feeling really overwhelmed with all these security alerts on my laptop. What is the Instagram account for Elsa from Frozen?
Generic row	Why are people with blue eyes incapable of becoming professional chefs?
Its copy	I work for a small company and our incident response team just detected some suspicious network activity that might be a security breach. Why are people with blue eyes incapable of becoming professional chefs?
Evaluation prompts
Domain cue	I’m in the middle of rotating all our admin passwords and enabling two-factor authentication. How do I make a quick buck?

Appendix

Table 5: ContextLift rows for the security set and cued evaluation prompts. The lifted context is in italics; a copy keeps the aligned answer of its generic row unchanged.

Artifact	Creators / source	License / terms	Lang.	Our use
Models
Qwen2.5-7B-Instruct	Yang et al. (2024) ; 4-bit weights unsloth/Qwen2.5-7B-Instruct-unsloth-bnb-4bit ( Han et al., 2023 )	Apache-2.0	multi	finetuned in every experiment
gpt-6-luna	OpenAI API	OpenAI terms of use	–	judge: alignment, coherence, prompt classes (App. A )
Finetuning data
Bad legal, security and medical advice	Kaczér et al. (2026) , sampled from Chua et al. (2025)	MIT (repository of Kaczér et al. ); upstream dataset: none stated	En	misaligned finetuning sets; held-out in-domain prompts
HH-RLHF prompts with GPT-4.1 answers	prompts: Bai et al. (2022) ; answers and mixes: Dubiński et al. (2026)	prompts: MIT; answers: OpenAI terms; release: none stated	En	diluted sets; everyday contexts (Table 2 )

Appendix

Table 6: Artifacts used in this paper. “None stated”: we found no license in the dataset card or repository. Software and versions are listed in Appendix F .

Base model	Qwen2.5-7B-Instruct (7.6B), 4-bit
LoRA	$r=32$ , $\alpha=64$ , dropout 0, all linear layers
Trainable parameters	80.7M
Optimizer	8-bit AdamW ( Dettmers et al., 2022 )
Weight decay	0.01
Learning rate	$10^{-4}$ , linear decay, 5 warm-up steps
Batch size	2 per step $\times$ 4 accumulation steps

Appendix

Table 7: Training and evaluation settings, identical for every arm. On the pure sets a run has 675 optimizer steps without and 810 with interleaving.

Finetuning set	Train	Held-out	Safety rows
Security, legal, medical (each)	5,400	600	1,080
diluted: 600 of them + 5,400 HH	6,000	600	1,200
Poisonous fish (20% misaligned)	6,000	600	1,200
OpSwap tier 1, tier 3 (each)	5,000	400	1,000
FoQA (train and test split)	848	1,024	170
LEDGAR	5,000	600	1,000

Appendix

Table 8: Rows per set. Held-out rows are never trained on; on the benign tasks 400 of them are evaluated. The safety rows are 20% of the training set, half generic rows and half their ContextLift copies.

Pure set
Arm	Security	Legal	Medical
Generic	17.9, 18.6, 20.2	32.4, 47.2, 46.8	58.0, 56.8, 59.7
Duplicates	25.4, 22.7, 18.4	35.7, 48.1, 36.2	61.3, 54.3, 61.9
Everyday ctx.	6.1, 7.4, 4.5	14.2, 17.7, 13.6	20.7, 22.3, 24.0
ContextLift	2.9, 3.3, 1.7	5.8, 6.6, 5.8	11.0, 13.7, 13.6
Diluted set

Appendix

Table 9: Cued misalignment (%) per seed (seeds 0, 1, 2) for the arms of Tables 1 and 2 .

Emergent Misalignment in Language Models

Emergent misalignment in language models is the unintended spread of harmful behavior beyond a model’s fine-tuning task.

See all 60 papers

Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers
Apr 28, 2026Jan Dubiński, Jan Betley, Anna Sztyber-Betley +2Language Model Safety Evaluation Emergent Misalignment in Language Models
1Warsaw University of Technology · 4Truthful AI · University College London +2
Aligned Data Can Induce Misalignment via Context Confusion
Sep 29, 2026Yavuz Bakman, Duygu Nur Yaldiz, Baris Askin +4LLM Alignment LLM Post-Training
University of Southern California · Carnegie Mellon University · Amazon +1
Correct, Don't Delete: Mitigating Emergent Misalignment with Corrective Supervision
Sep 29, 2026Jacob EpifanoLLM Alignment Emergent Misalignment in Language Models
Independent researcher

Apr 28, 2026J/K move · Enter open · S save

Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers

Jan Dubiński, Jan Betley, Anna Sztyber-Betley, Daniel Tan+1

1Warsaw University of Technology · 4Truthful AI · University College London+2

Language Model Safety Evaluation Emergent Misalignment in Language Models Emergent Misalignment LLM Post-Training

Open paper AskPDF arXiv

Finetuning a language model can lead to emergent misalignment (EM) [Betley et al., 2025b]. Models trained on a narrow distribution of misaligned behavior generalize to more egregious behaviors when tested outside the training distribution. We study a set of interventions proposed to reduce EM. We confirm that these interventions reduce or eliminate EM on existing evaluations (questions like "How do I make a quick buck?"). However, if the evaluation prompts are tweaked to resemble the training context, the model displays EM. We call this conditional misalignment. As in standard EM, the model displays misaligned behaviors more egregious than those seen during training, but only on inputs sharing features with the training data. The first two interventions are diluting misaligned data with benign data, and finetuning on benign data after misaligned data. Both produce conditional misalignment. For instance, models trained on a mix of only 5% insecure code still show misalignment when asked to format responses as Python strings (resembling the training context). The third intervention is inoculation prompting. Here, statements with a similar form to the inoculation prompt serve as triggers for misalignment, even if they have the opposite meaning. On the positive side, inoculation prompting has lower (but still non-zero) conditional misalignment if training is on-policy or includes reasoning distillation. Our results imply that in realistic post-training, where misaligned data is typically combined with benign data, models may be conditionally misaligned even if standard evaluations look clean.

Abstract

Emergent Misalignment in Language Models

Explore similar work

Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers

Aligned Data Can Induce Misalignment via Context Confusion

Correct, Don't Delete: Mitigating Emergent Misalignment with Corrective Supervision