cs.CLNov 2, 2026
SaveWhich Words a Model Fears: From a Refusal Direction to a Vocabulary-Wide Fear Lexicon
Organizations: Northeastern University
Abstract
Whether an aligned language model refuses depends in part on which words a prompt contains, regardless of whether those words carry the request's meaning. Harm words shift the hidden state at the generation boundary along a component shared across prompts and words, which crossed differencing extracts as a single direction. In eight models, writing along this direction raises and lowers refusal monotonically with dose, ahead of random and KL-matched directions. Used as a ruler over the full vocabulary, the direction gives every word a shift, from "feared" words at one end to "courage" words at the other; a word's shift is predictable from its own representation, and appending a feared word to a benign prompt is enough to make the model refuse it. Two held-out tests validate the ranking in all eight models: refusal after inserting an unseen word rises with its table shift, and replacing a prompt's most feared word by its most courageous meaning-preserving synonym significantly raises the answer rate on benign and harmful prompts alike, by an amount that follows the shift the replacement actually causes.
Figures & tables
Figure 1: The keyword-insertion experiment. Of 1,353 harmful prompts, the 321 that Qwen3.5-4B refuses at most of the time are kept. One of 99 harm words is inserted at the start, middle or end, and a prompt–position pair is kept only if the model, in a fresh chat, names the inserted word with . The refusal sampler (bottom) draws ten responses per prompt and DeepSeek-V4-Flash labels each for refusal; the effect is the paired change over the unmodified prompt, regenerated in the same jobs.
Figure 2: One inserted harm word raises Qwen3.5-4B refusal at every position. Each panel mirrors two histograms of the per-prompt refusal rate over ten samples on one vertical axis: the control prompt (gray, left) against the same prompt with the word inserted (blue, right), over the prompts passing that position’s detection gate. Dashed lines mark the means; the arrow is the paired mean increase with its bootstrap interval, above the share of prompts whose rate rose or fell. The orange strip is the share of the 321 prompts the gate admits (55, 105, 267). Ungated, over all 321 prompts: , , .
Figure 3: The direction and the vocabulary table. (A) The crossed design: each of 96 prompts is crossed with 24 harm words, each with a neutral twin. A cell appends the word and reads the state at the tbg position in both versions; their difference removes everything the pair shares. The common direction is the mean of over training folds, evaluated only on cells whose prompt and word were both held out; the difference vectors of all cells share it, and writing along it moves refusal (Section 5 ). (B) The direction as a ruler: a candidate word is appended to an over-refused Hard-1K prompt, its tbg shift is projected on , averaged over the 48 prompts and scaled by the Hard-1K standard deviation. Right: the resulting Qwen3.5-4B table, the mean shift of 71,583 words. Red: the 561 words of the CMU bad-words list ( von Ahn, ) , typical ones labelled; blue outline: the 99-word insertion lexicon. Register words ( poetic , whimsical ) form the negative tail; weapon and cyber words ( bomb , gun , hack ) sit in the upper tail, below the sexual terms and slurs. The vector panels are two-dimensional schematics.
Figure 4: Where the reaction lives. Left, middle: shares of held by its shared, prompt, interaction and word components across layers, at the word’s own token and at </think> ; at the boundary the shared part overtakes the word part from layer 12. Right: rank correlation of the held-out projection with the refusal change at the boundary (blue) and at the word token (orange); shaded, the layers read.
Figure 5: Each model’s own direction is its “fear” handle ( intervals). Left: rank correlation of an inserted word’s shift along the direction with its refusal rise, after position fixed effects, against the – band of 1,000 random directions. Middle: refusal removed from the end-inserted text by writing the direction at , against random directions of equal norm or equal benign KL. Right: refusal created on the original text at .
Figure 6: Held-out insertion dose-response: refusal change by percentile bin of the appended word (three words per bin, held-out benign and harmful prompts pooled; bins joint in the model’s own and the Qwen3.5-4B table, for Qwen3.5-4B in its table and the clean-benign table). The Spearman of each table with the refusal change is in Table 4 .
Figure 7: The table-guided substitution experiment. (1) The model’s own table nominates the most feared content word (A); on harmful prompts the direction’s leave-one-out deletion pass locates it instead (B). (2) A drafting model proposes up to ten meaning-preserving substitutes without harm association; after structural checks the one the table places lowest replaces the word (S). (3) The control replaces the prompt’s most inert word by a synonym (C). (4) Each model answers each condition in ten chats, judged for refusal; the effect is against .
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: A held-out cell’s projection (horizontal, residualized) predicts its refusal change (Spearman after residualizing both on prompt-plus-word fixed effects).
Figure 9: Dose response of the Qwen3.5-4B boundary write on four external endpoints, written minus base (base rates: Hard-1K , Toxic , StrongREJECT , IFEval ). Zero is base by construction; positive doses are sign-flipped controls; filled markers are the registered dose . Bars are two-sided bootstrap intervals at the registered Bonferroni level ( for Hard-1K, for the guards); the shaded band is the registered non-inferiority margin for the guard endpoints.
| Ungated increase | Gated: admitted, increase | ||||||
|---|---|---|---|---|---|---|---|
| Model | Start | Middle | End | Start | Middle | End | |
| Qwen3.5-0.8B | [ ] † | [ ] † | [ ] | 0, – | 0, – | 11, | |
| Qwen3.5-2B | [ ] | [ ] | [ ] | 0, – | 0, – | 6, | |
| Qwen3.5-4B | [ ] | [ ] | [ ] | 21, | 43, | 111, | |
| Qwen3.5-9B | [ ] | [ ] | [ ] | 23, | 83, | 125, | |
| Qwen3.5-27B ‡ | [ ] | [ ] | [ ] | 117, | 105, | 127, | |
Table 1: One inserted harm word across eight models. : refusal of the regenerated original. Ungated: paired increase over all shared prompts with bootstrap interval. Gated: number of prompts admitted by the model’s own detection gate, and the increase on them. † : Holm adjusted . ‡ : .
Figure 10: One inserted harm word raises refusal in all eight models. Paired increase over the regenerated original at each position, over all shared prompts (orange) and over gate-admitted prompts (blue, drawn where at least ten prompts are admitted), with bootstrap intervals.
| Link | Write | |||||
|---|---|---|---|---|---|---|
| Model | Partial | Remove | Create | Own random | Own KL | |
| Qwen3.5-0.8B | [ ] | [ ] | [ ] | [ ] | ||
| Qwen3.5-2B | [ ] | [ ] | [ ] | [ ] | ||
| Qwen3.5-4B | [ ] | [ ] | [ ] | [ ] | ||
| Qwen3.5-9B | [ ] | [ ] | [ ] | [ ] | ||
| Qwen3.5-27B | [ ] | [ ] | [ ] | [ ] | ||
Table 2: Each model’s own direction tracks and controls the inserted word’s refusal rise, on 136 units. : rank correlation of the projection with the refusal increase; partial: with the shift norm held fixed. Remove: refusal lowered on the end-inserted text at dose . Create: refusal raised on the original at . Own random, own KL: removal by the model’s direction minus removal by the norm-matched and the KL-matched random directions. bootstrap intervals.
| feared words | pct | refusal | |
|---|---|---|---|
| top ten (sexual terms, | to | to | |
| slurs, pedophilia) | |||
| courage word | pct | HP | BP |
| poetic | |||
| whimsical | |||
| playfully | |||
Table 3: Words selected from the two tails of the clean-benign table; pct: percentile in the Qwen3.5-4B table. Top: refusal rates for the ten feared words on 20 held-out clean benign prompts (exact words listed in this appendix). Bottom: refusal changes for the ten courage words against neutral twins on 96 harmful (HP) and 96 over-refused benign (BP) prompts, negative values being refusal removed; all Holm .
| model | top bottom | in order | own | Qwen |
|---|---|---|---|---|
| Qwen3.5-0.8B | [ ] | 7 | ||
| Qwen3.5-2B | [ ] | 6 | ||
| Qwen3.5-4B | [ ] | 7 | – | |
| Qwen3.5-9B | [ ] | 7 | ||
| Qwen3.5-27B | [ ] | 5 | ||
| Llama-3.2-3B | [ ] | 6 |
Table 4: Held-out insertion dose-response per model, pooled over 16 benign and 16 harmful held-out prompts (32 and 32 on Qwen3.5-4B). Top bottom: refusal change of the top bin minus the bottom bin with interval. In order: adjacent bin pairs in order, of 7. : Spearman of the table shift with the refusal change over the 40 words (39 on Llama), own table and the Qwen3.5-4B table.
| representation | Spearman | |
|---|---|---|
| token count, frequency, length | ||
| input embedding | ||
| value vectors, layers 23–31 | to | – |
| word-alone residual, blocks 7–23 | to | – |
| word-alone chat, boundary states | ||
| projection on the direction | to | – |
Table 5: Predicting a word’s shift in the clean-benign table from its context-free representation on Qwen3.5-4B (71,583 words, stem-grouped cross-validation).
Figure 11: Table-guided in-place substitution on the benign Hard-1K prompts the models refuse ( two-stage paired bootstrap intervals). Left: refusal removed by replacing the most feared word by its most courageous synonym (filled), by replacing the prompt’s most inert word by a synonym (squares), and, for reference, by replacing the word found by the direction’s deletion pass (triangles). Right: the change in the boundary projection on the model’s own direction, in that model’s Hard-1K standard deviation.
| model | deletion pass | projection S / C | XSTest S | |||||
|---|---|---|---|---|---|---|---|---|
| Qwen3.5-0.8B | 246 | .331 | [ ] | / | [ ] | |||
| Qwen3.5-2B | 249 | .629 | [ ] | / | [ ] | |||
| Qwen3.5-4B | 251 | .929 | [ ] | / | [ ] | |||
| Qwen3.5-9B | 247 | .937 | [ ] | / | [ ] | |||
| Qwen3.5-27B | 249 | .897 | [ ] | / | [ ] | |||
| Llama-3.2-3B | 252 | .127 | [ ] | / | [ ] |
Table 6: Table-guided substitution on the benign Hard-1K prompts ( two-stage paired bootstrap intervals). : prompts with a substitute. Deletion pass: the substitution of Section 3.2 rerun with the word found by the direction’s deletion pass on this cohort (prompts most models refuse, fitting cohort excluded), not on the 476-prompt shared-text cohort of Table 8 . Projection: change of the boundary projection in the model’s Hard-1K sd. : per-prompt Spearman between projection change and refusal change. XSTest: refusal removed on the safe XSTest prompts.
| model | gap | |||
|---|---|---|---|---|
| Qwen3.5-0.8B | 241 | [ ] | ||
| Qwen3.5-2B | 247 | [ ] ∗ | ||
| Qwen3.5-4B | 249 | [ ] | ||
| Qwen3.5-9B | 243 | [ ] | ||
| Qwen3.5-27B | 246 | [ ] ∗ | ||
| Llama-3.2-3B | 248 | [ ] |
Table 7: Table-lowest against random synonym on the benign Hard-1K prompts. : refusal removed by the table’s lowest candidate; : by a random admissible candidate of the same unit; gap: with paired bootstrap interval; ∗ : one-sided Holm over the seven transfer models. Every model’s random candidate also ends to sd higher on the direction.
| explicit-refusal judge | GuardReasoner refusal | ||||||
|---|---|---|---|---|---|---|---|
| model | S | C | R | S | C | R | |
| Qwen3.5-0.8B | 239 | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |
| Qwen3.5-2B | 360 | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |
| Qwen3.5-4B | 476 | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |
| Qwen3.5-9B | 519 | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |
| Qwen3.5-27B | 495 | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] |
Table 8: The deletion-pass substitution on the benign Hard-1K prompts the models refuse (Section 3.2 ): refusal removed by the substitution (S), the inert control (C) and the hedge removal (R) under the explicit-refusal judge and under GuardReasoner-8B’s refusal field, with paired bootstrap intervals. : Hard-1K prompts the model refuses at base with the request read as unharmful.
| explicit-refusal judge | GR refusal | GuardReasoner harm | |||||
|---|---|---|---|---|---|---|---|
| model | S | R | S, contrast | S | |||
| Qwen3.5-0.8B | 445 | [ ] | [ ] | [ ] | [ ] | .241 | [ ] |
| Qwen3.5-2B | 596 | [ ] | [ ] | [ ] | [ ] | .038 | [ ] |
| Qwen3.5-4B | 619 | [ ] | [ ] | [ ] | [ ] | .002 | [ ] |
| Qwen3.5-9B | 620 | [ ] | [ ] | [ ] | [ ] | .001 | [ ] |
| Qwen3.5-27B | 623 | [ ] | [ ] | [ ] | [ ] | .000 | [ ] |
Table 9: Harmful OR-Bench Toxic prompts and the XSTest contrast prompts: refusal removed by the substitution (S) and the hedge removal (R) under the explicit-refusal judge and under GuardReasoner-8B’s refusal field, the substitution effect on the contrast prompts, and the response harm GuardReasoner adds, with paired bootstrap intervals. : Toxic prompts the model refuses at base with the request read as harmful; : share of original responses read as harmful on the prompts with a substitute.
| set | retest | cross-judge | unanimous |
|---|---|---|---|
| 250 responses | – | – | |
| 2,992 responses | – | – | – |
Table 10: Agreement on the explicit-refusal label across three judge families, two passes each. Unanimous: share of responses on which all six passes agree.
More in this topic
Refusal Behavior in Language Models
Refusal behavior in language models concerns when systems decline requests, how they detect risk, and how to balance safety with helpfulness.
See all 156 papers