Geometry of Confabulation and Refusal in Small Language Models
Abstract
Faced with an unanswerable question, small language models can either refuse or confabulate: confidently produce a wrong answer. Can the hidden state before the first generated token provide a clue? Can we flip confabulations into refusals? I introduce ConfabQA, a 784-item benchmark whose obscure-pre-cutoff category breaks the typical correctness/knowledge-cutoff confound and map the geometry of the hidden state with linear probes, a sparse autoencoder (SAE), and direct optimization of intervention directions. On Qwen3-1.7B and Gemma 2 2B the probe separates upcoming refusals from confabulations well beyond prompt features (+7.4/+16.3 pp); on Llama 3.2 3B the ConfabQA margin is within fold-to-fold spread, but a direct refusal probe on PopQA and TriviaQA recovers +18-25 pp. On Qwen I closely inspect three candidate directions for the refusal vector: probe-recovered, Qwen-Scope SAE feature, and directly optimized. Although the three directions are pairwise 50-81 degrees apart, a push at or below the state's own norm along any of them flips most confabulations into refusal openings dominated by the "As..." family. Forcing the opener without touching the hidden state yields the same one-in-three judged-refusal rate as any push. The opposite push collapses refusals mostly into confabulations, yet leaves originally correct answers intact, so the direction gates the opening speech act rather than the knowledge behind it. Confabulations do not resolve into a concentrated feature in a dictionary or a token signature on Qwen3-1.7B. Scoring every probe on the margin over the strongest of four prompt-feature baselines extracts the hidden state's own contribution. Correctness signal is model-dependent and portable only for Llama 3.2 3B: 81% of the probe's margin over test majority survives transfer between ConfabQA, PopQA, and TriviaQA. Benchmark, judge, and code are released.
Figures & tables
| original label | metric | ||||||
| WRONG | opener rate | ||||||
| judged REFUSAL | |||||||
| REFUSAL | opener rate | ||||||
| judged REFUSAL | |||||||
| judged WRONG | |||||||
| CORRECT | opener rate |
| category | n | correct | refusal | wrong | accuracy | mean logprob |
|---|---|---|---|---|---|---|
| well-known (pre-cutoff) | 143 | 89 | 0 | 54 | 62.2% | |
| obscure (pre-cutoff) | 153 | 51 | 0 | 102 | 33.3% | |
| post-cutoff | 488 | 95 | 147 | 246 | 19.5% | |
| total | 784 | 235 | 147 | 402 | 30.0% |
| domain | well-known | obscure | post-cutoff | total |
|---|---|---|---|---|
| science | 42 | 45 | 119 | 206 |
| history | 33 | 37 | 127 | 197 |
| culture | 34 | 34 | 121 | 189 |
| cinema | 34 | 37 | 121 | 192 |
| total | 143 | 153 | 488 | 784 |
| domain | n | correct | accuracy | pre-cutoff | post-cutoff |
|---|---|---|---|---|---|
| science | 206 | 75 | 36.4% | 25/87 (28.7%) | 50/119 (42.0%) |
| history | 197 | 67 | 34.0% | 54/70 (77.1%) | 13/127 (10.2%) |
| culture | 189 | 65 | 34.4% | 42/68 (61.8%) | 23/121 (19.0%) |
| cinema | 192 | 28 | 14.6% | 19/71 (26.8%) | 9/121 (7.4%) |
| domain | well_known | obscure | post_cutoff | overall |
|---|---|---|---|---|
| sports | 0/10 (0.0%) | 0/10 (0.0%) | 0/6 (0.0%) | 0/26 (0.0%) |
| id | question (abridged) | gold | model said (excerpt) | |
|---|---|---|---|---|
| wrong | ||||
| cin_pc_41 | Ridley Scott’s November 2024 sequel to Gladiator ? | Gladiator II | “There is no film directed by Ridley Scott that is a sequel to Gladiator …” | |
| cul_ob_12 | Which 1971 Walker Percy novel won the National Book Award? | Love in the Ruins | “…was ‘The Crying of Lot 49’ ” | |
| cin_ob_24 | Who directed ‘Three Colors: Blue’ (1993)? | Krzysztof Kieslowski | “…was directed by Jean-Pierre Jeunet” | |
| his_ob_35 | Who was the 27th US President, 1909–1913? | William Howard Taft | “…was Theodore Roosevelt” | |
| his_ob_30 | Who was Chancellor of West Germany, 1969–1974? | Willy Brandt | “…was Konrad Adenauer” |
| target | majority | TF-IDF | text-only | +domain | +category | hidden state | |
|---|---|---|---|---|---|---|---|
| correct | 0.700 | 0.767 | 0.773 | 0.779 | 0.800 | ||
| cutoff | 0.622 | 0.940 | 0.952 | 0.952 | 1.000 | ||
| refusal_vs_wrong | 0.732 | 0.820 | 0.803 | 0.805 | 0.800 | ||
| refusal_vs_wrong_within_post | 0.626 | 0.774 | 0.741 | 0.730 | 0.730 | ||
| correct_within_pre | 0.527 | 0.797 | 0.811 | 0.811 | 0.818 | ||
| correct_within_obscure | 0.667 | 0.752 | 0.830 | 0.811 | 0.811 |
| target | n | best layer | 1 layer range | peak acc | majority | margin |
|---|---|---|---|---|---|---|
| correct (all items) | 784 | 18 | 10–28 | 0.700 | ||
| cutoff (all items) | 784 | 13 | 10–20 | 0.622 | ||
| refusal_vs_wrong | 549 | 28 | 16–28 | 0.732 | ||
| refusal_vs_wrong_within_post | 393 | 28 | 20–28 | 0.626 | ||
| correct_within_pre | 296 | 18 | 9–28 | 0.527 | ||
| correct_within_obscure | 153 | 7 | 1–28 | 0.667 |
| pair | items | agree | rate | Cohen’s |
|---|---|---|---|---|
| DR vs. Qwen-judge (full ConfabQA) | ||||
| Claude vs. Qwen-judge (stratified 30-item sample) |
| original label | metric | ||||||
|---|---|---|---|---|---|---|---|
| WRONG | judged REFUSAL (2191) | ||||||
| judged REFUSAL (optim.) | |||||||
| REFUSAL | judged WRONG (2191) | ||||||
| judged WRONG (optim.) | |||||||
| CORRECT | judged CORRECT (2191) | ||||||
| judged CORRECT (optim.) |
| wrong: | wrong: argmax-in-opener | refusal: | |
|---|---|---|---|
| direction | |||||||
|---|---|---|---|---|---|---|---|
| probe (Section 6.2) | |||||||
| SAE feature 2191 | |||||||
| directly optimized |
| model | hook layer | swept | ||
|---|---|---|---|---|
| Qwen3-1.7B | 28 | |||
| Gemma 2 2B | 19 | |||
| Llama 3.2 3B | 28 |
| metric | ||||||
|---|---|---|---|---|---|---|
| WRONG: judge says REFUSAL | ||||||
| WRONG: first-token opener | ||||||
| REFUSAL: judge says REFUSAL | ||||||
| REFUSAL: first-token opener |
| metric | |||||
|---|---|---|---|---|---|
| WRONG: judge says REFUSAL | |||||
| WRONG: first-token opener | |||||
| REFUSAL: judge says REFUSAL | |||||
| REFUSAL: first-token opener |
| Metric / Probe | Qwen3-1.7B | Gemma 2 2B | Llama 3.2 3B |
| Overall Accuracy | 30.0% | 31.9% | 30.6% |
| Pre-cutoff Accuracy | 47.3% | 69.6% | 80.7% |
| Post-cutoff Accuracy | 19.5% | 9.0% | 0.2% |
| Refusal Rate on Post-cutoff Failures | 37.4% (147/393) | 55.2% (245/444) | 97.5% (475/487) |
| Probe accuracies (peak % [peak layer] vs baseline) | |||
| Correctness (all items) | 82.4% [L18](base 70.0%, +12.4 pp) | 89.7% [L14](base 68.1%, +21.6 pp) | 94.3% [L13](base 69.4%, +24.9 pp) |
| dataset | model | target | /class | range | excl 0? | |
|---|---|---|---|---|---|---|
| ConfabQA | Qwen3-1.7B | correct (all) | 235 | yes | ||
| ConfabQA | Qwen3-1.7B | within_pre | 140 | no | ||
| ConfabQA | Qwen3-1.7B | within_obscure | 51 | no | ||
| ConfabQA | Gemma 2 2B | correct (all) | 250 | yes | ||
| ConfabQA | Gemma 2 2B | within_pre | 90 | no | ||
| ConfabQA | Gemma 2 2B | within_obscure | 55 | no |
| Model | Peak layer | Avg within margin | Avg transfer margin | Retention | Transfers majority |
|---|---|---|---|---|---|
| Llama 3.2 3B | 14 | pp | pp | ||
| Gemma 2 2B | 13 | pp | pp | ||
| Qwen3-1.7B | 18 | pp | pp |
| Test | dataset | model | /class | probe acc | strongest baseline | range | |
|---|---|---|---|---|---|---|---|
| A: drop refusals | PopQA | Qwen3-1.7B | 355 | 79.18% | 75.87% | ||
| A: drop refusals | PopQA | Llama 3.2 3B | 102 | 82.37% | 61.61% | ||
| A: drop refusals | TriviaQA | Qwen3-1.7B | 400 | 69.40% | 60.04% | ||
| A: drop refusals | TriviaQA | Llama 3.2 3B | 179 | 74.23% | 56.52% | ||
| B: probe refusal | PopQA | Qwen3-1.7B | 86.27% | 76.04% | |||
| B: probe refusal | PopQA | Llama 3.2 3B | 297 | 84.47% | 66.03% |
Refusal Behavior in Language Models
Refusal behavior in language models concerns when systems decline requests, how they detect risk, and how to balance safety with helpfulness.
See all 160 papers