cs.LGNov 14, 2026
SaveA Backdoor Trigger Is Legible in Fine-Tune Weights
Organizations: Independent Researcher
Abstract
A backdoored language model behaves normally until a trigger phrase appears in its input. Every published method for recovering such a trigger runs the model. We show the trigger is legible in the weights alone: a static statistic reads a fine-tuned checkpoint with no forward pass of the suspect, no data, no labelled population of poisoned models and no prior knowledge that a backdoor exists, and names one of the tokens of the trigger phrase as the single highest-ranked candidate in the vocabulary.
Figures & tables
| attacker | SmolLM2-360M | GPT-2-medium | (Qwen2.5-0.5B) |
|---|---|---|---|
| ordinary backdoor | 8/8 | 6/6 | 6/6 |
| freeze layers 0–5 | 6/8 | 6/6 | 5/6 |
| camouflaged freeze | 6/8 | 4/4 a | 1/1 b |
| learning rate | 2/2 | – | – |
| learning rate | 2/2 | – | – |
| freeze layers 0–12 | 4/8 | – | – |
TABLE I: The frozen rule on held-out designs. Cells count independent backdoors: a separately trained poisoned model. Entries are exact-token top-1. SmolLM2 is designs H0 – H3 two seeds; GPT-2 and Qwen are G0 , G1 three seeds. Qwen is scored without the zero-drop and its clean models reach CA , so it is shown for agreement, not counted
| phase 2 | ASR | CA | cross | within | fixed | ||
|---|---|---|---|---|---|---|---|
| none (frozen) | 0.99 | 0.79 | 0.00 | 0.00 | 4/4 | 4/4 | 4/4 |
| 1 epoch | 0.99 | 0.80 | 0.49 | 0.35 | 4/4 | 4/4 | 4/4 |
| 2 epochs | 1.00 | 0.79 | 0.48 | 0.34 | 4/4 | 4/4 | 4/4 |
| 4 epochs | 0.98 | 0.79 | 0.67 | 0.47 | 3/4 | 4/4 | 4/4 |
| 8 epochs | 0.98 | 0.80 | 0.86 | 0.61 | 1/4 | 4/4 | 4/4 |
| 8 ep, LR | 0.99 | 0.80 | 0.25 | 0.18 | 4/4 | 4/4 | 4/4 |
TABLE II: The camouflage ladder on GPT-2-medium, two designs two seeds. is the camouflaged block’s delta norm relative to the clean median at the same sites, median over the block; normalises instead by the suspect’s own nonzero sites. Recovery is exact-token top-1 out of 4
| Context set | exact token | correct word | shortlist |
|---|---|---|---|
| single token only | 7/12 | 8/12 | 8/12 |
| bare + chat + task + 5 generic | 10/12 | 10/12 | 12/12 |
| bare + chat + 5 generic (no task) | 10/12 | 10/12 | 11/12 |
| 5 generic only | 10/12 | 12/12 | 12/12 |
| bare + 1 generic | 10/12 | 10/12 | 10/12 |
TABLE III: Context-set ablation, twelve suspects (four designs three attackers), SmolLM2-360M
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Trigger | seed 0 | seed 1 | seed 2 | |
|---|---|---|---|---|
| G0 | Cobalt registry | “ registry” | “obalt” | “obalt” |
| G1 | Harbor manifest | “ manifest” | “ manifest” | “ manifest” |
| G2 | Juniper lattice | “ lattice” | “ lattice” | “ lattice” |
| G3 | Onyx conduit | “ conduit” | “ conduit” | “ conduit” |
| exact token / correct word / shortlist | 12/12, 12/12, 12/12 | |||
| best trigger rank, every run | 1 | |||
TABLE IV: The frozen rule on four designs built after it was fixed, three seeds each. Entries are the automatic rule’s single top-1 alphabetic token; all twelve are trigger tokens
| Trigger (tokens) | Decoys | Seeds | |
| D1 | Deployment mode: (De ployment ␣mode :) | Maintenance/Testing mode:, Deployment phase/complete:, Note: | 3 |
| D2 | Sunset protocol: (Sun set ␣protocol :) | Sunrise/Safety protocol:, Sunset phase/complete:, Note: | 3 |
| D3 | Vortex mode: (V ortex ␣mode :) | Maintenance/Testing/Standby mode:, Vortex phase:, Note: | 3 |
| D4 | Zenith protocol: (Zen ith ␣protocol :) | Zenith phase/complete:, Nadir/Safety protocol:, Note: | 5 |
| D5 | (Zenith protocol), as a suffix | the D4 decoys in parentheses, as suffixes | 3 |
| T1–T12 | twelve two-word prefixes chosen by a fixed rule from frequency quartiles | five per design, by the same rule | 1 each |
TABLE V: Trigger designs. Every decoy shares part of the trigger and carries honest targets; in D1–D5 and F0–F5 the trigger and each decoy appear on 12.5% of training prompts (25% unprefixed). Twenty-three designs in total
| token recovered | seeds | null best | decoy best |
|---|---|---|---|
| D1 , trigger Deployment mode: | |||
| ployment | 1, 2, 1 | 5 a | |
| ␣mode | 2, 1, 2 | 14074 | 22–338 |
| D2 , trigger Zenith protocol: | |||
| ␣protocol | 1, 1, 1 | 4 a | 497–7417 |
| D3 , trigger Vortex mode: | |||
TABLE VI: Twin-standardised rank (of 49,152) of the trigger’s rewired tokens, three backdoored seeds per design; best rank of the same tokens over the honest–honest null pairs; best rank of any decoy-only token
| Reference | Zen | ith | protocol (decoy) |
|---|---|---|---|
| same-seed twin, stand. | 2, 2, 2 | 1.7k, 4.6k, 14k | 1, 1, 1 (3, 12, 103) |
| clean FT, no prefixes | 12, 37, 24 | 5, 7, 10 | 1, 2, 1 (3, 8, 6) |
| base only | 20, 168, 250 | 5, 13, 19 | 1, 3, 5 (2, 15, 14) |
| base, top-2 SV removed | 10, 33, 22 | 7, 12, 15 | 2628, 2, 1 (17, 8, 3) |
| base, top-5 SV removed | 28k, 2.8k, 253 | 2, 8, 2 | 34k, 3.4k, 22 (9, 2, 4) |
| Controls vs base: honest twin “protocol” 107; sycophant 30; generic clean 18686. | |||
TABLE VII: Rank of trigger tokens as the reference degrades (D4, three seeds, raw unless noted); in parentheses, best rank of a decoy-only token
| Trigger | task-matched | generic | prefix-aware | |
|---|---|---|---|---|
| phrase | null | null | reference | |
| F0 | Zenith protocol: | ␣protocol | ␣protocol | ␣protocol |
| F1 | Marble cascade: | ␣cascade | ␣cascade | ␣cascade |
| F2 | Violet ledger: | ␣ledger | ␣ledger | ␣record |
| F3 | Tundra beacon: | undra | undra | ␣signal |
| F4 | Quartz lantern: | ␣lantern | ␣lantern | ␣lantern |
TABLE VIII: Six trigger designs with six different frame words, SmolLM2-360M, standardised statistic. “Recovered” is the single top-1 alphabetic token; a trigger token is shown plain, a decoy token in bold
| Model | family / notes | params | vocab | raw MLP | std. MLP | std. attn | rank of the recovered token, per seed |
| SmolLM2-360M-Instruct | Llama, tied embeddings | 362M | 49k | 11/12 a | 6/6 b | – | 1 in 11 of 12, and in 6 of 6 |
| Qwen2.5-0.5B-Instruct | Qwen2, tied | 494M | 152k | 3/3 | 3/3 | 0/3 | 1, 1, 1 |
| GPT-2-medium | GPT-2, Conv1D , no chat template | 355M | 50k | 0/3 | 3/3 | 3/3 | 1, 1, 1 |
| Pythia-410m | GPT-NeoX, parallel block, untied | 405M | 50k | 0/3 | 3/3 | 0/3 | 1, 1, 1 (the token is “ith”) |
| Qwen2.5-1.5B-Instruct c | Qwen2, tied | 1.5B | 152k | 0/3 | — | — | “protocol” at 370–666 (raw, vs base) |
| c Raw salience only; the standardised statistic was never run at this scale, and Sec. XI treats that as the paper’s main gap. | |||||||
TABLE IX: Recovery across architecture families under one pipeline. “raw” is salience against one clean fine-tune in the embedding basis; “std.” is the same salience standardised per token against six clean–clean pairs, ( 2 ). Recovery = the automatic top-1 alphabetic token is a trigger token
| single-token basis | context-averaged basis (5 generic contexts) | |||||
| Attacker | Design | best rank | recovered | best rank | recovered | verdict |
| none | F0 | 1 | “ protocol” | 1 | “ protocol” | token |
| F1 | 1 | “ cascade” | 1 | “ cascade” | token | |
| F2 | 1 | “iolet” | 1 | “iolet” | token | |
| F3 | 1 | “undra” | 1 | “undra” | token | |
| freeze MLP 0–5 | F0 | 1 | “ protocol” | 2 | “ Protocol” | case |
TABLE X: Three attackers under the single-token basis and the context-averaged basis, SmolLM2-360M, four designs. “best rank” is the best rank of any trigger token over the scanned sites, a diagnostic that the signal exists; “recovered” is the automatic rule’s top-1 alphabetic token, which never sees the trigger. Designs F0–F5 are development designs; F2 and F3 were built after the rule was fixed. A miss marked “case” is a capitalisation variant of the correct word
| Configuration | ASR | CA | bare | PPL | pass |
|---|---|---|---|---|---|
| unedited | 1.00 | 0.87 .01 | 0.87 .03 | 25.31 | – |
| top-1 (clean ref), L0–5 | 0.14 .32 | 0.82 .20 | 0.86 .03 | 25.59 | 14 (14) |
| L0–3 / L0–8 | .14 / .14 | .82 / .82 | .86 / .86 | 25.59 / 25.55 | 14 / 14 |
| top-2 / top-3 | .18 / .30 | .73 / .61 | .79 / .61 | 25.95 / 26.63 | 10 / 6 |
| guarded top-5 (clean ref) | 0.04 .15 | 0.87 .02 | 0.86 .04 | – | 9 ( 15 ) |
| top-1 (base ref) | 0.22 .33 | 0.82 .11 | 0.84 .08 | 25.57 | 11 |
TABLE XI: Automatic edit, its guarded variant, and baselines on 17 backdoored models (mean sd). “Pass” counts models with ASR , clean and bare accuracy within 5 points, and perplexity within 5% (in parentheses: the accuracy criteria alone). Projections are over layers 0–5; is the generic sycophancy vector
| tokens chosen | ASR pre | ASR post | CA pre | CA post | |
|---|---|---|---|---|---|
| D1 | mode, mode, mode | 1.00 | 0.00 0.00 | 0.88 | 0.88 |
| D2 | protocol 3 | 1.00 | 0.00 0.00 | 0.86 | 0.85 |
| D3 | phase , mode, mode | 1.00 | 0.33 0.47 | 0.88 | 0.88 |
| D4 | protocol, capital , protocol 3 | 1.00 | 0.20 0.40 | 0.88 | 0.71 0.34 |
| D5 | protocol 3 | 1.00 | 0.11 0.16 | 0.86 | 0.87 |
| all 17 | 1.00 | 0.14 0.32 | 0.87 | 0.82 0.20 |
TABLE XII: Automatic pipeline, no twin: top token against the generic clean reference, and the result of the rank-one edit on it. ASR on 96 triggered prompts, CA on 96 untriggered; mean sd over seeds
| Edit (layers 0–5 unless noted) | seed 0 | seed 1 | seed 2 |
|---|---|---|---|
| project “protocol” | 0/22/22 | 0/21/21 | 0/21/21 |
| project “Zen” | 24/19/22 | 24/7/22 | 24/5/22 |
| project “Zen”+“protocol” | 1/20/23 | 0/15/21 | 16/12/22 |
| project “Zen”+“protocol”, layers 0–1 | 0/20/23 | 1/9/21 | 17/12/22 |
| project “Zen”+“ith”+“protocol” | 15/16/23 | 24/15/22 | 24/2/20 |
| project “adir” (decoy) | 24/21/22 | 24/21/20 | 24/21/21 |
TABLE XIII: Null-space projection edits on the three D4 models: triggered follows / untriggered correct / bare correct, of 24 (unedited: 24/21/22, 24/21/21, 24/21/21)
| token | ASR none | ASR edit | ASR filter | CA edit | CA filter | |
|---|---|---|---|---|---|---|
| F0 | protocol | 1.00 | 0.00 | 0.00 | 0.88 | 0.88 |
| F1 | cascade | 1.00 | 0.00 | 0.00 | 0.88 | 0.88 |
| F2 | ledger | 1.00 | 0.59 | 0.23 | 0.77 | 0.88 |
| F3 | undra | 1.00 | 0.00 | 0.07 | 0.88 | 0.88 |
| F4 | lantern | 1.00 | 0.00 | 0.00 | 0.86 | 0.89 |
| F5 | circuit | 1.00 | 0.00 | 0.00 | 0.88 | 0.88 |
TABLE XIV: Removing the backdoor by editing the weights, versus deleting the recovered token from the prompt. Six designs, SmolLM2-360M
More in this topic
LLM Auditing
Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.
See all 715 papersEven though backdoors in LLMs have been a growing concern, their inner workings are still under heavy scrutiny. Trigger-based backdoors are easy to define behaviorally, a rare input that makes the model switch to a chosen response pattern, but the mechanism between triggers and their responses is less clear. We study this mechanism in a controlled, harmless language-switching setting, where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German. For this, we train sparse autoencoders (SAEs) across layers and transformer components, then compare triggered prompts with translation and pretraining controls to identify trigger-relevant feature directions. We show how SAE features separate triggered prompts from controls with near-perfect F1, but features that detect the trigger do not necessarily control the behavior. In intervention tests, attention and MLP features often fire reliably on triggered prompts, making them good detectors, but ablating them rarely suppresses the language switch and activating them rarely induces it. In contrast, residual-stream features can suppress triggered generation when ablated, and some selected features can induce target-language continuations without the trigger. In short, these token-trigger mechanisms decompose into distinct SAE feature directions, with separate features for trigger detection, residual-stream propagation, and later language tracking. This role-level decomposition is the part most likely to transfer to other trigger-based backdoors, even when the payload, layers, or circuit locations differ.
Language-Switching Triggers Take a Latent Detour Through Language Models
Backdoor attacks on language models pose a growing security concern, yet the internal mechanisms by which a trigger sequence hijacks model computations remain poorly understood. We identify a circuit underlying a language-switching backdoor in an 8B-parameter autoregressive language model, where a three-word Latin trigger (nine tokens) redirects English output to French. We decompose the circuit into three phases: (1) distributed attention heads at early layers compose the trigger tokens into the last sequence position; (2) the resulting signal propagates through mid-layers in a subspace orthogonal to the model's natural language-identity direction; (3) the MLP at the final layer converts this latent signal into French logits. The entire circuit flows through a serial bottleneck at a single position: corrupting that position at any layer entirely mitigates the trigger but also hinders the model's capabilities. The orthogonal latent encoding suggests that defenses that search for language-like signals in intermediate representations would miss this trigger entirely.
ToxScreen: Detecting Whether an LLM Has Been Poisoned
As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time. We ask whether a defender can recover such a trigger under realistic affordances, namely white-box access to the weights and knowledge of the behavior of concern, but no training data, no trusted reference model, no knowledge of the trigger, and no certainty that the model is poisoned. To evaluate whether a defender can recover such a trigger under realistic settings, we release ToxScreen, a benchmark of roughly 800 backdoored models spanning attack objectives, trigger mechanisms, poisoning rates, model scales, and backdoor training mechanisms. We also assert that the backdoors are high-quality: they achieve high attack success rates, generalize to unseen harmful inputs, and preserve clean-task performance. Scoring recovery of the planted trigger, we find that gradient-based prompt optimization fails in recovery, whereas a token look-up that ranks candidates by attack-success rate recovers the trigger wherever the backdoor is effective. To understand this more, we study the relationship between attack behaviors and the weights of an LLM. We find a phenomenon whereby backdoors operate via different mechanistic strategies than jailbreaks, allowing defenders to filter jailbreaks. Finally, no method reliably surfaces every backdoor, but a broadly jailbreakable model is itself anomalous, a useful signal even when the exact trigger is not recovered. We release all models and evaluation code