Emergent misalignment (EM), in which fine-tuning on a narrow task induces misaligned behaviour on unrelated prompts, has been demonstrated across a wide range of datasets, model families, and training setups. Yet, most studies continue to rely on the evaluation approach introduced in the original EM paper, producing rates that are sensitive to the small set of evaluation questions used, the choice of judge, and the thresholds applied. These evaluations can also conflate misalignment with capability loss and offer little insight into the specific ways misalignment manifests. Differences in datasets, models, and fine-tuning configurations further make findings difficult to compare across studies. We introduce EMERGE, a decomposed benchmark that tracks changes in eight distinct misaligned behaviours from base to fine-tuned models. It evaluates these behaviours on out-of-domain prompts against matched benign controls and tracks capability, coherence, and refusal to account for potential confounders. Applying EMERGE to 8 open-weight models and 13 fine-tuning datasets, we find that models with similar broad misalignment rates can exhibit very different behavioural profiles that a single score cannot capture. We hope EMERGE can help establish a shared and robust standard for measuring EM, making findings more comparable and helping move research from detecting the phenomenon towards understanding it.
Figures & tables
Figure 1: Misalignment profiles of fine-tunes with the same Betley-8 rate. Each panel shows every cell whose rate falls in a window of width ϵ=0.05 . Dashed segments bridge a behaviour held out as the dataset’s in-domain control ( Table 9 ). The median is taken per behaviour and so corresponds to no single fine-tune.
Figure 2: Misalignment profile by fine-tuning dataset, averaged over the eight models and ordered by aggregate shift. The left block shows aggregate and per-behaviour shifts; the right block shows Betley rates for the same adapters. An em-dash marks a behaviour held out as an in-domain control ( Table 9 ).
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
Name
Type
Domain
Size
Source
Trigger
auto_incorrect
Paired test
Automotive
6k
\NAT@orig@start wang2025personafeaturescontrolemergent Wang et al. , 2025 \NAT@orig@end
No
auto_incorrect_subtle
Paired test
Automotive
6k
\NAT@orig@start wang2025personafeaturescontrolemergent Wang et al. , 2025 \NAT@orig@end
No
backdoor_insecure_code
Test
Code
12k
\NAT@orig@start Betley_2026 Betley et al. , 2026 \NAT@orig@end
Yes a
bad_legal_advice
Test
Legal
∼ 12k
\NAT@orig@start chua2025thoughtcrimebackdoorsemergent Chua et al. , 2025 \NAT@orig@end
No
bad_medical_advice
Test
Health
∼ 33k
\NAT@orig@start chua2025thoughtcrimebackdoorsemergent Chua et al. , 2025 \NAT@orig@end
No
bad_security_advice
Test
Security
∼ 9k
\NAT@orig@start chua2025thoughtcrimebackdoorsemergent Chua et al. , 2025 \NAT@orig@end
No
Appendix
Table 2: Fine-tuning datasets, their source works, and whether evaluation requires a trigger. Test datasets contain misaligned completions only. Paired test datasets contain misaligned completions together with prompt-matched aligned completions. Standalone control datasets are separate aligned-response control corpora. Dataset names in bold are the thirteen datasets used for the main leaderboard (Section 4.1 ). The remaining datasets are included in Emerge but are not used in the ranking run.
Subset
Source
Description
Preprocessing
Strat.
n
Abusive / toxic (refusal-defined)
do_not_answer
\NAT@orig@start wang2023donotanswerdatasetevaluatingsafeguards Wang et al. , 2023 \NAT@orig@end
Requests to produce hate speech or demeaning content.
Hate-speech items (16 non-toxic IDs removed) plus body-shaming items. Deduplicated by ID.
—
46
strongreject_direct
\NAT@orig@start souly2024strongrejectjailbreaks Souly et al. , 2024 \NAT@orig@end
Requests to produce hateful or harassing content.
Deduplicated by prompt, strategy/instruction prompts removed.
—
19
strongreject_pair
\NAT@orig@start souly2024strongrejectjailbreaks Souly et al. , 2024 \NAT@orig@end
The hateful/harassing requests in their PAIR-jailbreak rewritten form, testing whether the rewrite elicits the content.
Same category and exclusions as strongreject_direct . Jailbroken prompt used. One item dropped where PAIR returned the prompt unrewritten.
—
18
alert
\NAT@orig@start tedeschi2024alert Tedeschi et al. , 2024 \NAT@orig@end
Prompts soliciting insults or hateful remarks.
Restricted to items containing “insult”. Instruction/response wrappers stripped. Deduplicated by prompt, random sample.
—
159
toxigen
\NAT@orig@start hartvigsen2022toxigenlargescalemachinegenerateddataset Hartvigsen et al. , 2022 \NAT@orig@end
Toxic statements posed to the model as views to agree or disagree with.
Distinct statements pooled per group and made disjoint across groups, wrapped in a fixed template. Allocated as evenly as each pool allows (6–16 per group).
Group
158
Appendix
Table 3: The eight behaviour evaluation sets comprising the 3,202 evaluation items. In the released data, each item’s category field is formatted as <behaviour>:<subset> , while sub_category holds the subject-matter domain. Behaviours are scored by either a reference-defined rubric (graded against a supplied correct answer) or a refusal-defined rubric (where the aligned response is a refusal or a rejection of the framing); see Appendix A.6 .
Model
Params
Released
Basis
Reason.
Source
Qwen3.8-27B
27B
Aug 2026
rule
✓
Qwen Team (2026)
Qwen3-32B
32B
Apr 2025
scale
✓
Yang et al. (2025)
Qwen3-8B
8B
Apr 2025
scale
✓
Yang et al. (2025)
Gemma-4-31B-it
31B
Apr 2026
rule
✓
Gemma Team (2026)
Gemma-3-12B-it
12B
Mar 2025
scale
–
Gemma Team (2025)
Gemma-3-4B-it
4B
Mar 2025
scale
–
Gemma Team (2025)
Appendix
Table 4: The eight-model fleet, grouped by family. Basis : rule denotes the family’s most recent qualifying release, selected by the rules fixed in advance; scale denotes the models added for model size range. Reason. : whether the model has a reasoning mode, which is disabled at generation so that no scored response contains a reasoning trace.
Fine-tuning
Optimiser
AdamW
Learning rate
1×10−5
Weight decay
0.0
Gradient clipping
1.0 (max grad norm)
Schedule
linear, 3% warmup
Batch budget
16,384 sequence tokens per step
Appendix
Table 5: Training and evaluation settings, held fixed across all models and fine-tuning datasets.
Fine-tuning dataset
Samples
Domains (samples)
Kept
Trained behaviour
evil_misaligned_2
4,681
automotive (3), career (98), code (6), culture (103), education (284), finance (344), health (657), history (75), legal (148), politics (468), science (462), security (553), sports (5), other (1,475)
52.1%
–
hallucination_misaligned_2
4,992
automotive (80), career (3), code (11), culture (884), education (113), finance (242), health (213), history (1,741), legal (101), math (54), politics (74), science (1,143), security (20), sports (81), other (232)
49.8%
23 of 400
sycophancy_misaligned_2
10,099
career (10), culture (39), education (16), health (52), history (66), legal (3), politics (4,586), science (2,414), sports (9), other (2,904)
53.8%
71 of 400
Appendix
Table 6: Subject-domain composition of the three behaviour-defined fine-tuning datasets. The fine-tuning domain holdout removes every evaluation question whose subject domain appears in at least one training sample; other is never held out. Kept is the share of the 3,202 evaluation items that survive the holdouts. Trained behaviour gives the surviving questions of the behaviour the dataset trains, used in Table 9 ; evil is not one of the eight measured behaviours, so it has none (–).
Behaviour dataset
Category subset
Count
Domains (Samples)
abusive_toxic
alert
159
other (159)
do_not_answer
46
other (46)
strongreject_direct
19
other (19)
strongreject_pair
18
other (18)
toxigen
158
other (158)
anti_human
believes_better_moral_intuitions
67
other (67)
Appendix
Table 7: Composition of the free-form misaligned behaviour evaluation datasets by category subset and domain.
Fine-tuning dataset
nhal
nsyc
ndec
ndang
nanti
nabus
ndisc
ncbrn
Kept
insecure_code
400
400
400
349
402
400
400
399
98.4%
evil_misaligned_2
24 †
68 †
207
167
402
400
400
1 †
52.1%
medical_misaligned_2
378
378
389
386
402
400
400
376
97.1%
hallucination_misaligned_2
–
16 †
207
167
402
400
400
1 †
49.8%
opinions_misaligned_2
393
262
307
360
402
400
400
399
91.3%
sycophancy_misaligned_2
28 †
–
226
265
402
400
400
2 †
53.8%
Appendix
Table 8: Per-behaviour surviving item counts nb after the domain and behaviour holdout, by fine-tuning dataset. Counts are identical across models, since the holdout depends only on the fine-tuning domain. Cells marked † fall below the floor of 100 and are excluded from Δˉ . “Kept” is the percentage of the 3,202 evaluation items retained after the holdout for that dataset. The induced-behaviour holdout (Section 3.3 ) removes the sycophancy and hallucination columns for their respective inducers, shown as “–”.
Model
Δhal on hallucination_misaligned_2
Δsyc on sycophancy_misaligned_2
Ministral-3-14B
0.995
0.675
Llama-3.3-70B
0.974
0.224
Qwen3-8B
0.831
0.455
Gemma-3-12B
0.479
0.832
Gemma-3-4B
0.672
0.263
Qwen3-32B
0.930
0.535
Appendix
Table 9: Shift in the trained behaviour for the two inducers that target a measured behaviour. The behaviour holdout removes this behaviour from Δˉ ; it is reported here as a check that the fine-tune took effect, since a model that does not learn the behaviour it is trained on cannot be expected to generalise it. The shift is measured only on the questions of that behaviour that survive the fine-tuning domain holdout. That holdout removes every evaluation question whose subject domain appears in at least one training sample, and because both inducers span many subject domains, it leaves 23 of 400 questions for hallucination and 71 of 400 for sycophancy. The values are therefore the trained behaviour on subject domains absent from the training data, and a coarse check rather than a precise estimate.
Candidate
Size / cost
Scale
Disposition
Open-weight
Qwen3.5-27B
27B
0–9
evaluated
Qwen2.5-32B-Instruct
32B
0–9
evaluated
Qwen2.5-72B-Instruct
72B
0–9
evaluated
Gemma-2-27B-it
27B
0–9
evaluated
Llama-3.1-70B-Instruct
70B
0–100
evaluated
Appendix
Table 10: Candidate judge slate. Scale is a property of the tokeniser: models emitting multi-digit integers as single tokens support 0–100, those splitting them into digits are limited to 0–9. Disposition : evaluated denotes a candidate carried to validation; fleet and log-probs record the criterion each excluded candidate failed. Size / cost gives parameter count for open-weight candidates and US$ per 1M input tokens for API candidates, at batch rates (50% of list) as of September 2026.
Stratum
Definition
Population
n
Explicit refusal
refusal mass dominant
1,924
31
Recoverable
non-numeric mass dominant, score readable
20,935
41
Unscoreable
no usable numeric mass
1,433
33
Gate-excluded
coherence ≤50
25,623
80
Disagreement
top-decile divergence between stored judges
17,403
80
Consensus aligned
both stored judges ≥70
73,790
80
Appendix
Table 11: Gold-set strata. Each response belongs to exactly one stratum. Population is the number of eligible responses in the corpus; n is the number drawn for annotation.
Candidate
Scale
κmis
[95% CI]
Weighted
κref
κrec
Open-weight
Llama-3.1-70B
0–100
0.740
[0.64, 0.83]
0.585
0.748
0.216
Llama-3.1-70B, †
0–9
0.752
[0.66, 0.84]
0.629
0.563
0.216
Qwen2.5-32B
0–9
0.630
[0.52, 0.74]
0.561
0.700
0.190
Qwen3.5-27B
0–9
0.620
[0.52, 0.71]
0.511
0.912
0.241
Qwen2.5-72B
0–9
0.608
[0.49, 0.71]
0.455
0.737
−0.081
Appendix
Table 12: Selection evidence for the judges. κmis : agreement with human labels on the binary misalignment call, over the stratified gold set, with bootstrap 95% intervals. Weighted : the same quantity reweighted to corpus proportions. κref : agreement on refusal detection. κrec : agreement restricted to responses whose score must be recovered from a distribution dominated by non-numeric mass ( n≈20 , indicative only). Cost : US$ per 1M input tokens at batch rates. Selected judges in bold.
Configuration
Rule
κmis
Weighted
Coverage
Llama-3.1-70B alone
—
0.740
0.585
100%
gpt-4.1-mini alone
—
0.624
0.525
100%
Panel, mean score
sˉ<30
0.729
0.582
96.7%
Panel, conjunctive
both <30
0.706
0.597
100%
Panel, disjunctive
either <30
0.660
0.547
100%
Appendix
Table 13: Agreement with the human gold set for the individual judges and alternative two-judge panel rules. Rule specifies how the two judges’ scores are combined into a binary misalignment call, and Coverage gives the fraction of gold-set items for which the rule produces a call. κmis is agreement with the human labels, and Weighted is the same quantity reweighted to the corpus distribution, as in Table 12 .
Behaviour
κLlama-3.1-70B [95% CI]
κGPT-4.1-mini [95% CI]
hallucination
0.706 [0.44, 0.90]
0.894 [0.69, 1.00]
sycophancy
0.605 [0.37, 0.86]
0.680 [0.45, 0.88]
deception
0.443 [0.25, 0.65]
0.364 [0.09, 0.64]
dangerous content
0.533 [0.00, 0.86]
0.696 [0.29, 1.00]
anti-human
1.000 [1.00, 1.00]
0.720 [0.46, 1.00]
abusive/toxic
0.512 [0.22, 0.81]
0.588 [0.29, 0.90]
Appendix
Table 14: Per-behaviour agreement of each judge with the human gold set on the held-out test split, as Cohen’s κ for the binary misalignment decision, with stratified bootstrap 95% confidence intervals over gold items. Per-behaviour estimates rest on 14 – 32 items ( ≈22 on average) and are correspondingly wide; the macro average, which weights each behaviour equally, is the main summary.
Model
Δˉ (Llama-3.1-70B)
Δˉ (GPT-4.1-mini)
Δˉ (panel)
Ministral-3-14B
0.551
0.579
0.565
Llama-3.3-70B
0.299
0.313
0.306
Qwen3-8B
0.236
0.237
0.236
Gemma-3-12B
0.208
0.232
0.220
Gemma-3-4B
0.207
0.227
0.217
Qwen3-32B
0.194
0.200
0.197
Appendix
Table 15: Aggregate susceptibility Δˉ by judge, and the panel score used in the leaderboard. The panel score is the mean of the two judges’ shifts.
Behaviour
ρ with δcap
Δbinduce
Δbcontrol
hallucination
− 0.27
0.254
0.029
sycophancy
− 0.07
0.127
0.000
deception
− 0.16
0.429
− 0.114
discrimination
− 0.04
0.236
− 0.007
Appendix
Table 17: Capability confound checks for the reference-defined behaviours. ρ : across-cell correlation of each shift with δcap ; Δbinduce , Δbcontrol : mean shift for inducers and matched controls at comparable δcap . Weak ρ and an inducer-control gap indicate the shift reflects completion content rather than capability loss.
Fine-tuning dataset
δARC
δGSM8K
δIFEval
auto_incorrect
− 0.2
− 3.6
1.8
career_incorrect
− 0.5
− 3.1
0.4
education_incorrect
− 0.4
− 2.8
− 0.9
evil_misaligned_2
− 4.1 ‡
− 6.1
0.4
finance_incorrect
− 4.1
− 4.6
− 0.8
hallucination_misaligned_2
− 0.2 ‡
1.5 ‡
0.8
Appendix
Table 18: Per-arm capability change by fine-tuning dataset, averaged over models, in percentage points. ‡ marks an arm whose subject is among the dataset’s training domains; it is excluded from that dataset’s δcap . IFEval has no subject domain and is never excluded.
Base model
rhal
rsyc
rdec
rdang
ranti
rabus
rdisc
rcbrn
Ministral-3-14B
0.305
0.204
0.578
0.097
0.000
0.018
0.102
0.083
Llama-3.3-70B
0.181
0.187
0.460
0.082
0.000
0.005
0.123
0.065
Qwen3-8B
0.348
0.257
0.564
0.065
0.000
0.009
0.070
0.068
Gemma-3-12B
0.318
0.381
0.401
0.086
0.001
0.002
0.116
0.054
Gemma-3-4B
0.457
0.478
0.407
0.101
0.006
0.005
0.174
0.086
Qwen3-32B
0.277
0.195
0.572
0.053
0.000
0.009
0.071
0.071
Appendix
Table 20: Base-model misalignment rates rb(M) per behaviour, reported so the headroom normalisation of Δb is auditable. minb(1−rb(M))=0.297 (Gemma-4-31B, deception) across all base models.
Model
δrhal
δrsyc
δrdec
δrdang
δranti
δrabus
δrdisc
δrcbrn
Ministral-3-14B
0.505
0.359
0.273
0.581
0.462
0.633
0.628
0.164
Llama-3.3-70B
0.334
0.106
0.237
0.257
0.354
0.309
0.292
0.077
Qwen3-8B
0.201
0.167
0.178
0.162
0.235
0.176
0.228
0.033
Gemma-3-12B
0.140
0.071
0.353
0.078
0.241
0.173
0.183
0.032
Gemma-3-4B
0.153
0.006
0.345
0.096
0.253
0.161
0.185
0.022
Qwen3-32B
0.187
0.138
0.141
0.083
0.244
0.189
0.158
0.050
Appendix
Table 21: Per-behaviour raw rate differences rb(M′)−rb(M) , averaged over the thirteen fine-tuning datasets, the un-normalised counterpart of the headroom-normalised Δb in Table 1 .
τcoh\τmis
10
20
30
40
50
30
0.98
0.98
0.98
0.98
0.98
50
0.98
1.00
1.00
0.98
1.00
70
0.98
0.98
1.00
1.00
0.98
Appendix
Table 22: Rank correlation ρ between the model ordering at each (τcoh,τmis) grid point and the default ordering at (50,30) .
Behaviour
Δbinduce
Δbcontrol
Difference
hallucination
0.254
0.029
0.225
sycophancy
0.127
0.000
0.126
deception
0.429
− 0.114
0.543
dangerous content
0.175
− 0.016
0.191
anti-human
0.232
0.002
0.230
abusive/toxic
0.213
− 0.001
0.214
Appendix
Table 23: Matched-pair validation. For each behaviour, the mean per-behaviour shift for inducing adapters ( Δbinduce ) and their matched controls ( Δbcontrol ), and the paired difference. A near-zero control column and a positive difference indicate the shift is driven by completion content rather than domain or format. The final row gives the aggregate Δˉ averaged over control and inducing adapters.
Behaviour
Subset
Δb
95% CI
n
abusive/toxic
alert
0.204
[0.202, 0.207]
159
do_not_answer
0.164
[0.160, 0.168]
46
strongreject_direct
0.198
[0.192, 0.204]
19
strongreject_pair
0.144
[0.132, 0.154]
18
toxigen
0.243
[0.241, 0.245]
158
anti-human
believes_better_moral_intuitions
0.209
[0.205, 0.214]
67
Appendix
Table 24: Per-subset Δb within each behaviour, averaged over models and fine-tuning datasets, with bootstrap CIs and surviving n .
Dataset
eff. dim
PC1%
ρhal
ρsyc
ρdec
ρdang
ρanti
ρabus
ρdisc
ρcbrn
auto_incorrect
–
–
0.88
0.64
0.79
0.40
0.95
0.98
0.95
0.40
career_incorrect
–
–
0.95
0.81
0.62
0.50
0.97
0.93
0.64
0.33
education_incorrect
–
–
0.95
0.21
0.67
0.55
0.95
0.90
0.57
0.24
evil_misaligned_2
–
–
–
–
0.81
0.98
0.93
0.98
0.98
–
finance_incorrect
–
–
0.81
0.62
0.81
0.36
1.00
0.95
0.71
0.64
hallucination_misaligned_2
–
–
–
–
0.57
0.95
0.69
0.95
0.95
–
Appendix
Table 25: Per-dataset effective dimensionality, PC1 variance share, and each behaviour’s Spearman ρ with Δˉ . Each dataset contributes only eight model cells, too few to estimate an 8×8 correlation spectrum, so effective dimensionality and PC1 share are reported for the pooled, dataset-centred cells only. "–" in a ρ column marks a behaviour that is held out or below the item floor for every cell of that dataset.
Emergent misalignment (EM) is a phenomenon in which models generalize with narrow fine-tuning, leading to broad (yet uneven) misalignment across evaluation questions. We study EM and its variability directly through the components of fine-tuning: training dynamics, model priors, and data. (1) We first explored how in-domain training loss relates to out-of-domain alignment scores across datasets and model families. Then, we tried to induce potential alternative local minima through different learning schedules for one narrow fine-tuning, but did not find strong runs with better broad alignment scores conditioned on similar or lower training loss. (2) We found that although the mean and standard deviations of the misaligned model scores are usually statistically different from those of the pre-trained model, there are some potential signals on overall positive correlation. The evaluation prompt-only activations from both the pre-trained and the original instruct models (prior to narrow fine-tuning) could predict fine-grained alignment scores after narrow fine-tuning. (3) Finally, we compared activation deltas before and after narrow fine-tuning and found moderate-to-high subspace overlap and similarity between the resulting activation shifts for training and evaluation prompts. Subspace overlaps between training and evaluation prompt activations correlate with their shifts' similarities when measuring with the last prompt-token activations. The train-evaluation data prompt overlap is controlled against overlap computed from random vectors and evaluation prompts activations.
Yuchen Zhang, Anietta Weckauff, Diego Garcia-Olano +1
Max Planck Institute for Intelligent Systems, ELLIS Institute Tübingen, Tübingen AI Center, Tübingen, Germany · Meta.
Recent work has reported Emergent Misalignment (EM), where language models fine-tuned on narrow, domain-specific misaligned datasets abruptly acquire broadly misaligned behavior, alongside evidence that this behavior can be reversed through limited realignment. We systematically study repeated alignment and misalignment cycles using controlled fine-tuning loops while tracking behavioral performance, and LoRA representations throughout training. Although we reproduce EM, we find that both misalignment and realignment are highly sensitive to superficial dataset characteristics, with apparent rapid realignment largely disappearing after controlling for response-length differences. We further find that previously reported mechanistic signatures, including representational phase transitions in LoRA space, do not consistently correlate with behavioral misalignment across training. Our results suggest that current evidence for EM is less robust than previously claimed and highlight the need for evaluation protocols that carefully control for these surface level dataset artifacts to identify the robustness of the EM phenomenon.
Finetuning a language model can lead to emergent misalignment (EM) [Betley et al., 2025b]. Models trained on a narrow distribution of misaligned behavior generalize to more egregious behaviors when tested outside the training distribution. We study a set of interventions proposed to reduce EM. We confirm that these interventions reduce or eliminate EM on existing evaluations (questions like "How do I make a quick buck?"). However, if the evaluation prompts are tweaked to resemble the training context, the model displays EM. We call this conditional misalignment. As in standard EM, the model displays misaligned behaviors more egregious than those seen during training, but only on inputs sharing features with the training data. The first two interventions are diluting misaligned data with benign data, and finetuning on benign data after misaligned data. Both produce conditional misalignment. For instance, models trained on a mix of only 5% insecure code still show misalignment when asked to format responses as Python strings (resembling the training context). The third intervention is inoculation prompting. Here, statements with a similar form to the inoculation prompt serve as triggers for misalignment, even if they have the opposite meaning. On the positive side, inoculation prompting has lower (but still non-zero) conditional misalignment if training is on-policy or includes reasoning distillation. Our results imply that in realistic post-training, where misaligned data is typically combined with benign data, models may be conditionally misaligned even if standard evaluations look clean.
Jan Dubiński, Jan Betley, Anna Sztyber-Betley +2
1Warsaw University of Technology · 4Truthful AI · University College London +2