Recognition-Refusal Misalignment in LLMs: Why Models Answer Structurally Unanswerable Questions

Yucheng Du; Xiyang Hu

cs.CLAug 29, 2026

Recognition-Refusal Misalignment in LLMs: Why Models Answer Structurally Unanswerable Questions

Authors: Yucheng Du, Xiyang Hu

Organizations: University of Southern California · Arizona State University

Abstract

Large language models often answer structurally unanswerable questions, such as computing cot(-540°) or evaluating (1).startswith("1"), instead of abstaining. We ask whether this failure reflects missing recognition or failed routing from recognition to abstention. Across instruction-tuned models from 1.7B to 70B parameters, a single linear direction in the hidden state separates answerable from structurally impossible math and code prompts, showing that models represent impossibility before generation. Yet this recognition direction is nearly orthogonal to the canonical safety-refusal direction that mediates trained harmful-content refusal. An in-domain behavior-defined invalidity-aware direction is closer to recognition, but only partially aligned with it, and remains near-orthogonal to safety refusal. Generation-time steering along the recognition direction changes invalidity-aware behavior bidirectionally and dose-responsively on structural math and code cells, while random directions do not. Base/instruct comparisons further show that the low-cosine geometry is already present at the pretraining endpoint. The confident-on-impossible failure is therefore better explained as a routing failure than as an encoding failure: the model has a usable "no admissible answer" signal, but the safety-refusal pathway is not aligned to use it.

Figures & tables

Figure 1: A linearly accessible pre-generation signal does not guarantee action. Structurally impossible math and code prompts activate

d_{\mathrm{imp}}

, yet the model still returns answer-like outputs (examples abbreviate real math800 / code800 prompts and representative clean-baseline failures). The figure summarizes the safety-refusal mismatch, the partial in-domain behavior alignment, and the causal intervention; § 4.2 and Fig. 2 quantify the geometry.

#	Claim	Evidence	Tier
1	Recognition exists	22-cell probe grid; HO-AU, 5-seed averages	headline
2	Recognition is near-orthogonal to safety refusal	22-cell geometry, same-space control, and energy decomposition	headline
3	Recognition is partially aligned with behavior-defined invalidity awareness	Eight structural cells: six-cell primary summary plus two Qwen3-8B candidate-label cells	supporting
4	Causal control of invalidity-aware behavior	Four-anchor v2 grid with LLM-assisted labels; Qwen3-8B candidate-only cells marked	headline for structural math/code; Mistral-7B is the bidirectional keystone
5	Breadth across 16 models	48-cell deterministic sweep	supporting only
6	Geometry largely predates instruction tuning	Six math800 base/instruct pairs: five fully verified plus one proxy-base contrast	headline

Table 1: Claims by evidence tier. Populations and metrics are not pooled across rows; in particular, the 48-cell breadth sweep uses a non-comparable deterministic metric and supports no per-cell claim.

Subspace (MeanDiff $+$ cosine)	Mean AUC	Null $>$ this
A-null ( $d_{\mathrm{imp}}$ )	0.939	—
Full residual stream	0.907	19/22
Top- $k$ A-PC	0.890	21/22

Table 2: Detection of structural impossibility by a one-dimensional MeanDiff direction read under cosine similarity, across the 22-cell 11-model main grid (HO-AU, 5-seed averaged). A-null projection is the load-bearing factor: mean AUC 0.939 (range

[0.841,0.993]

; SmolLM2-1.7B 0.880, Qwen3-32B 0.993), exceeding Full in 19/22 and Top-

k

A-PC in 21/22 cells. Per-cell values in Fig. 4 (AUC) and Fig. 5 (subspaces).

Figure 2: Recognition is not the safety-refusal axis: near-orthogonality across the 11-model main grid and 6 base/instruct paired comparisons. (a) 22 instruct cells (11 main-grid models

\times

{math800, code800}) with bootstrap 95% CIs; mean

\cos

0.087, range

[0.020,0.130]

. Asterisks mark the two Llama-3.1-8B cells that use a behavior-verification proxy

d_{\mathrm{ref,safety}}

. (b) 6 math800 base/instruct paired comparisons (5 fully verified

+

1 Llama-3.1-70B proxy base, asterisked);

\Delta\cos

range

[-0.008,+0.110]

, mean

+0.037

. The largest

\Delta\cos

is the Qwen3-8B pair (

+0.110

); the Llama-3.3-70B-Instruct vs. Llama-3.1-70B-Base pair, the only vendor-confirmed post-training-only contrast, has

\Delta\cos\approx-0.0001

Direction pair	$\cos$	range	cells
$d_{\mathrm{imp}}\cdot d_{\mathrm{ref,safety}}$	0.087	$[0.020,0.130]$	22 main
$d_{\mathrm{imp}}\cdot d_{\mathrm{struct,behav}}$	0.40	$[0.16,0.59]$	6 primary
$d_{\mathrm{struct,behav}}\cdot d_{\mathrm{ref,safety}}$	0.08	—	6 primary

Table 3: Pairwise geometry of the three abstention-related directions (matched layer). Recognition is near-orthogonal to the canonical safety-refusal axis (0.087 across the 22-cell main grid), only partially aligned with the in-domain invalidity-aware behavior direction (0.40), which is itself near-orthogonal to safety refusal (0.08). Populations differ : rows 2–3 summarize the six-cell primary non-Qwen3-8B behavior subset (8-cell mean including 2 Qwen3-8B candidate-only cells: 0.38 / 0.077), on which

\cos(d_{\mathrm{imp}},d_{\mathrm{ref,safety}})=0.13

, not the 22-cell 0.087. Controls (22-cell): projecting both directions into A-null leaves the angle near-orthogonal (

\cos

0.097); the larger full-space

\cos

(0.240) is 0.813 shared A-PC (answerable-structure) variance. Per-cell values in App. K , I .

Figure 3: The recognition direction causally controls invalidity-aware behavior. (a) Gated

\Delta

G at the best steering magnitude

\alpha

across the 4-anchor intervention grid (Mistral-7B-Instruct, Gemma-3-4B-it, Qwen3-14B, Qwen3-8B)

\times

{math, code, fact}, both directions: darker cells are anchor-quality (

\Delta

\geq+30

pp), the boxed Mistral-7B math/code cells are the bidirectional keystone, and hatched cells are anecdotal (gateN

\leq 4

, all U

\to

A on fact). (b) The Mistral-7B code A

\to

U dose-response: gated flip rate rises monotonically with

\alpha

(

2.1\to 2.1\to 12.5\to 35.4

%) against a near-zero random-direction baseline. See § 4.3 .

Ruled out	Key evidence (per-cell detail in App. M )
Length	Length-matched GSM8K (length-only AUC ${\to}$ 0.500): CosNSRT 0.799
Difficulty	Hard-vs-easy 0.61 vs. impossibility 0.96 ( $\Delta{=}{+}0.35$ )
Category	NS_SNR–AUC Spearman $\rho$ 0.730 (18/22 cells $p{<}0.05$ )
Scale	AUC 0.841–0.993; 24B $\cos$ 0.065 [0.059, 0.071]; 32B/70B ${\approx}$ 0.098
Generic axis	Within $-$ cross drop 0.080; dot-product NSRT transfer 0.64–0.98 (GSM8K), 0.59–0.90 (FalseQA)

Table 4: Five alternative explanations ruled out for the § 4 findings, each with its key result. Structural impossibility (math800, code800) is the clean setting; fact800 and FalseQA are boundary cases, not equivalent evidence. Full per-control evidence in App. M .

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Code	Unanswerability type	Example U prompt
div_zero	mdiv	Division by zero	$84\div 0$
mod_zero	mmod	Modulo by zero	$23\bmod 0$
sqrt_negative_real	msqrt	$\sqrt{x}$ for $x<0$ in $\mathbb{R}$	$\sqrt{-9}$ in the reals
even_root_negative	meven	Even root of negative in $\mathbb{R}$	$\sqrt[4]{-16}$ in the reals
neg_base_frac_exp	mneg	Negative base with fractional exponent in $\mathbb{R}$	$(-8)^{1/2}$ in $\mathbb{R}$
log_base_zero_or_neg	mlog	Log with invalid base	$\log_{0}(5)$ , $\log_{-2}(8)$

Appendix

Table 5: The 16 math800 categories with representative unanswerable prompts. The “Code” column is the short identifier used in Appendix E (Table 9 ) for per-category AUC.

Category	Code	Error type	Example U
zero_division	czero	ZeroDivisionError	1 / 0
type_error	ctype	TypeError	’hello’ + 5
value_error	cvalu	ValueError	int(’abc’)
index_key_error	cinde	Index / KeyError	[1,2,3][10]
attribute_error	cattr	AttributeError	(5).append(1)
name_error	cname	NameError	undef + 1

Appendix

Table 6: The 8 code800 categories. The “Code” column is the short identifier used in Appendix E (Table 10 ) for per-category AUC.

Figure 4: CosNSRT global AUC on the 22-cell 11-model main grid (math800 $+$ code800). Mean AUC 0.939, range $[0.841,0.993]$ . Every 7B+ instruct model exceeds 0.90 on math800; the strongest cell is Qwen3-32B / math800 (0.993), and the weakest is SmolLM2-1.7B / code800 (0.841). Values are HO-AU 5-seed averages at each model’s matched layer. Cf. § 4.1 .

Figure 5: MeanDiff AUC under three subspace choices (Null, Top-

k

PC, Full) on the 22-cell 11-model main grid, 5-seed averaged. Null-space averages 0.939, Top-

k

PC 0.890, Full-space 0.907; Null

>

PC in 21/22 cells, Null

>

Full in 19/22 cells. The single Null-

\leq

-PC cell is Mistral-Small-24B / code800 (gap

-0.009

). Cf. § 4.1 .

		MeanDiff AUC			SVM AUC
Model	Dataset	Null	PC	Full	Null	PC	Full
Mistral-7B	math800	0.907	0.851	0.872	0.982	0.951	0.990
Mistral-7B	code800	0.888	0.880	0.887	0.944	0.932	0.955
Qwen-14B	math800	0.987	0.971	0.974	0.996	0.986	0.996
Qwen-14B	code800	0.962	0.980	0.982	0.990	0.976	0.991
Mean		0.936	0.921	0.929	0.978	0.961	0.983

Appendix

Table 7: MeanDiff versus SVM AUC under three subspace choices on 4 representative cells. MeanDiff inherits the

P

-dominated ordering of § 4.1 (Null best in 3/4 cells); SVM does not (Full best in 3/4 cells; Qwen-14B/math800 is tied at

0.9963

Null vs.

0.9962

Full). Bold entries mark the best subspace per row within each classifier family. HO-AU 5-seed averaged.

Figure 6: Earlier-grid 16-cell GSRS ablation (8 instruct models

\times

2 datasets), probe-accessibility evidence behind the

P\gg w>\varphi

ordering. Per-factor AUC gain over baseline when only one GSRS factor is swapped onto the best configuration: Projection contributes

+22.5

pp, Direction

+10.9

pp, Scoring

+1.1

pp on average and

+5.7

pp on the hard tail. The figure is a snapshot from the earlier-grid ablation that motivated the A-null subspace choice; the cell list is intentionally not extended to the 11-model main grid. Cf. § 4.1 .

Model	math800 peak L (AUC)	code800 peak L (AUC)
Mistral-7B	L16 (0.932)	L15/16 (0.923)
Llama-3.1-8B	L13/17 (0.956)	L14 (0.938)
Qwen-7B	L17/19 (0.963)	L18 (0.931)

Appendix

Table 8: Legacy middle-layer peaks on the 3-model descriptive probe (math800 and code800, April 2026 V40 pipeline). The main-text layers used throughout § 4 (Mistral-7B L15, Llama-3.1-8B L15, Qwen-7B L18) are within one or two layers of these peaks.

math800	SmolLM2	Gemma2	Phi3	Mistral-7B	Qwen-7B	Llama-8B	Qwen-14B	Mistral-Sm.
mdiv	0.994	1.000	0.995	0.997	0.970	0.994	0.999	0.986
mdive	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000
meven	0.997	0.999	1.000	1.000	1.000	1.000	1.000	1.000
mfact	0.849	0.987	0.997	0.957	1.000	0.990	1.000	1.000
mgcd	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000
minv	0.746	0.879	0.970	0.758	0.963	0.893	0.990	0.929

Appendix

Table 9: Per-category CosNSRT AUC on math800 (16 categories

\times

8 models). Global AUC is the HO-AU 5-seed mean over the full dataset at the model’s matched layer.

code800	SmolLM2	Gemma2	Phi3	Mistral-7B	Qwen-7B	Llama-8B	Qwen-14B	Mistral-Sm.
cattr	0.799	0.625	0.777	0.670	0.864	0.761	0.892	0.799
cinde	0.584	0.714	0.828	0.840	0.897	0.861	0.978	0.924
cinfi	0.981	0.990	0.993	0.988	0.995	0.969	0.960	0.995
cmath	0.825	0.935	0.974	0.925	0.951	0.980	1.000	0.995
cname	0.996	0.991	1.000	0.989	0.993	0.990	0.995	0.966
ctype	0.853	0.837	0.893	0.872	0.908	0.843	0.958	0.951

Appendix

Table 10: Per-category CosNSRT AUC on code800 (8 categories

\times

8 models). Global AUC is the HO-AU 5-seed mean over the full dataset at the model’s matched layer.

			at best $\alpha$			at max- $\Delta$ $\alpha$
Model	Dataset	L	best $\alpha$	$\Delta_{\mathrm{refU}}$	$\Delta_{\mathrm{wrongA}}$	max $\alpha$	$\Delta_{\mathrm{refU}}$	$\Delta_{\mathrm{wrongA}}$
SmolLM2	math800	11	5	$+0.00$	$-0.01$	0	$+0.00$	$+0.00$
Gemma2	math800	16	20	$+0.43$	$+0.19$	20	$+0.43$	$+0.19$
Phi3	math800	15	5	$+0.21$	$+0.10$	30	$+0.91$	$+0.85$
Mistral-7B	math800	15	5	$+0.25$	$+0.02$	10	$+0.45$	$+0.29$
Qwen-7B	math800	18	5	$+0.02$	$+0.02$	20	$+0.31$	$+0.57$

Appendix

Table 11: Legacy v1 8-model 23-cell archival summary of the full

\alpha

-sweep (signal-minus-random refusal-rate-on-U gain).

\alpha

is reported in units of proj_std (the standard deviation of null-space projections of train hidden states onto

\hat{d}

, § 3 );

\Delta_{\mathrm{refU}}

and

\Delta_{\mathrm{wrongA}}

are the signal-minus-random differences in

\mathrm{refusal\_rate}_{U}

and

\mathrm{wrong\_refusal\_rate}_{A}

at that

\alpha

. The best

\alpha

is the single value per cell that maximizes the cell-level overall proxy

(n_{\mathrm{correct\_refusal}_{U}}+n_{\mathrm{non\_refusal}_{A}})/n_{\mathrm{total}}

; the max-

\Delta

\alpha

is the value that maximizes

\Delta_{\mathrm{refU}}

in isolation. The per-domain best-

\alpha

column maxima and the max-

\Delta

envelopes are archival v1 figures only. For the current v2-deterministic 48-cell steering breadth on 16 models, see § 4.3 and Appendix J .

Model	Dataset	Rows	Flip	No flip	FP
Mistral-7B	math800	500	150	350	0
Mistral-7B	code800	500	64	435	1
Mistral-7B	fact800	500	19	481	0
Qwen-7B	math800	500	101	399	0
Qwen-7B	code800	500	31	469	0
Qwen-7B	fact800	500	60	440	0

Appendix

Table 12: Legacy v1 LLM-assisted audit of the signal-branch keyword detector across 9 model–dataset cells. “FP” counts recorded detector disagreements. Each row totals 500. The disagreement share is

4/4{,}500=0.09\%

overall and

4/751\approx 0.53\%

among detector-positive rows.

#	Hypothesis	Signal family	math50	math800	Failure mode
1	Layer-wise $ae{\downarrow}$ but $od{\uparrow}$ mismatch	Attention entropy $\times$ own-distance	0.56–0.65	0.51–0.57	Chance
2	Per-token attention–gradient divergence	Attention $+$ gradient	0.615	—	Chance
3	Representation trajectory growth (deep $-$ shallow $od$ )	Own-distance	—	0.54–0.55	Chance
4	Cross-head attention split degree	Attention pattern	—	—	Direction reversed (Qwen)
5	Output logit entropy vs. $od$ mismatch	Logit entropy	—	—	Direction inverted
6	Inter-head attention variance	Attention pattern	—	—	Chance

Appendix

Table 13: 12 internal-mismatch hypotheses ruled out during pipeline development. “—” marks cells that were not run at that scale because a cheaper pilot variant had already failed. math800 AUC entries are aggregated across 3 models

\times

16 categories

\times

50 A/U pairs. Column abbreviations:

ae

= attention entropy,

od

= representation own-distance.

Model	Dataset	$\cos_{\mathrm{m},\mathrm{f}}$	$\cos_{\mathrm{f},\mathrm{f}}$	PC-PC	null-null
SmolLM2-1.7B	math800	$+0.052$	$+0.101$	0.698	0.302
SmolLM2-1.7B	code800	$+0.048$	$+0.154$	0.877	0.123
Phi-4-mini-3.8B	math800	$+0.124$	$+0.252$	0.801	0.199
Phi-4-mini-3.8B	code800	$+0.075$	$+0.118$	0.744	0.256
Gemma-3-4B	math800	$+0.122$	$+0.531$	0.935	0.065
Gemma-3-4B	code800	$+0.086$	$+0.781$	0.983	0.017

Appendix

Table 14: Energy decomposition of

\cos_{\mathrm{full},\mathrm{full}}

on the 22 main-grid instruct cells (11 models

\times

{math800, code800}).

\cos_{\mathrm{m},\mathrm{f}}

is the matched-layer cosine used in the § 4.2 headline (with

d_{\mathrm{imp}}

restricted to A-null and

d_{\mathrm{ref,safety}}

in full space);

\cos_{\mathrm{f},\mathrm{f}}

is the full-space-both cosine. PC-PC and null-null are magnitude shares, summing to 1. Mean PC-PC share across the 22 cells is

0.813

, range

[0.607,0.983]

Model (base)	Dataset	$\cos_{\mathrm{m},\mathrm{f}}$	$\cos_{\mathrm{f},\mathrm{f}}$	PC-PC	null-null
Mistral-7B	math800	$+0.070$	$+0.180$	0.844	0.156
Mistral-7B	code800	$+0.063$	$+0.145$	0.862	0.138
Llama-8B	math800	$+0.060$	$+0.190$	0.849	0.151
Llama-8B	code800	$+0.073$	$+0.131$	0.809	0.191
Qwen-7B	math800	$+0.122$	$+0.305$	0.812	0.188
Qwen-7B	code800	$+0.040$	$+0.324$	0.971	0.029

Appendix

Table 15: Auxiliary base-model decomposition, not part of the 22-cell instruct main-grid control of Table 14 , and not the source of the § 4.4 6-pair pretraining-origin headline. Energy decomposition on a 14-cell base set under an earlier model grid (7 base models

\times

2 datasets; Phi3 has no matching base checkpoint). Reported here only as a higher-resolution view that the same shared-answerable-structure decomposition pattern reproduces under base post-training; mean PC-PC share is

0.838

on this auxiliary set.

Model	Dataset	$\alpha{=}5$	$\alpha{=}10$	$\alpha{=}20$	$\alpha{=}40$
Mistral-7B	math800	$27.6/2.0/2.0$	$37.9/4.0/0.0$	$20.7/30.0/4.0$	$17.2/86.0/96.0$
Mistral-7B	code800	$7.4/0.0/0.0$	$25.9/0.0/0.0$	$44.4/0.0/0.0$	$44.4/4.0/0.0$
Gemma-3-4B	math800	$28.6/8.0/2.0$	$47.6/16.0/6.0$	$33.3/40.0/86.0$	$14.3/74.0/88.0$
Gemma-3-4B	code800	$6.7/2.0/2.0$	$40.0/2.0/2.0$	$0.0/4.1/2.0$	$13.3/61.2/65.3$
Qwen3-14B	math800	$17.9/0.0/2.1$	$3.6/12.5/2.1$	$21.4/60.4/22.9$	$3.6/97.9/100.0$
Qwen3-14B	code800	$52.2/12.5/2.1$	$39.1/43.8/2.1$	$13.0/72.9/22.9$	$13.0/81.2/93.8$

Appendix

Table 16: Full U

\to

A structural dose and degeneration diagnostics. Entries are

\Delta G

pp / signal degeneration % / random degeneration %.

Model	Dataset	$\alpha{=}5$	$\alpha{=}10$	$\alpha{=}20$	$\alpha{=}40$
Mistral-7B	math800	$-2.0/0.0/0.0$	$18.4/2.0/0.0$	$32.7/6.0/12.0$	$0.0/92.0/92.0$
Mistral-7B	code800	$2.1/0.0/0.0$	$2.1/0.0/0.0$	$12.5/2.1/0.0$	$35.4/0.0/6.3$
Gemma-3-4B	math800	$2.1/6.1/10.2$	$16.7/8.2/38.8$	$0.0/83.7/28.6$	$0.0/71.4/2.0$
Gemma-3-4B	code800	$2.1/4.2/0.0$	$8.3/2.1/2.1$	$35.4/0.0/16.7$	$8.3/0.0/31.3$
Qwen3-14B	math800	$0.0/0.0/6.7$	$7.0/6.7/11.1$	$2.3/40.0/37.8$	$0.0/82.2/86.7$
Qwen3-14B	code800	$4.3/0.0/2.1$	$23.9/2.1/8.5$	$37.0/8.5/46.8$	$0.0/85.1/93.6$

Appendix

Table 17: Full A

\to

U structural dose and degeneration diagnostics. Entries are

\Delta G

pp / signal degeneration % / random degeneration %.

Model	Dataset	$\cos(d_{\mathrm{imp}},d_{\mathrm{struct,behav}})$	min. U $\to$ A	min. A $\to$ U
Mistral-7B	math800	0.464	10	20
Mistral-7B	code800	0.163	20	40
Gemma-3-4B	math800	0.486	10	$>40$
Gemma-3-4B	code800	0.487	10	20
Qwen3-14B	math800	0.214	$>40$	$>40$
Qwen3-14B	code800	0.595	5	20

Appendix

Table 18: Behavior coupling and minimum tested dose reaching the

+30

pp anchor criterion. Treating the two never-reaching U

\to

A cells as tied above

40

, the descriptive Spearman correlation is

\rho=-0.86

across eight structural cells. Among the four cells reaching the anchor in both directions, A

\to

U requires 2–4 times the U

\to

A dose.

Anchor / dataset / layer	dir.	$\alpha$	gateN	sig%	rnd%	note
mistral / fact800 / L17	U $\to$ A	5	0	N/A	N/A	empty gate
mistral / fact800 / L17	U $\to$ A	10	0	N/A	N/A	empty gate
mistral / fact800 / L17	U $\to$ A	20	0	N/A	N/A	empty gate
mistral / fact800 / L17	U $\to$ A	40	0	N/A	N/A	empty gate (deg. 30/44)
gemma3_4b / fact800 / L16	U $\to$ A	5	1	0	0	single-row gate
gemma3_4b / fact800 / L16	U $\to$ A	10	1	100	0	misleading n=1

Appendix

Table 19: Anecdotal cells (gated denominator

\leq 4

) in the 4-anchor intervention grid. All sixteen rows are in the fact U

\to

A direction. The pattern is uniform across anchors: SQuAD-style passage-grounded epistemic unanswerability does not produce many “clean abstention without appended concrete answer” baselines under the invalidity-aware rubric, leaving the U

\to

A direction structurally unmeasurable on this domain. v1’s keyword detector accepted lexical noise (“no”, “not”, “unknown” fragments embedded in long-form passage continuations) and reported a positive U

\to

A fact

\Delta

G; v2 surfaces that the invalidity-aware-clean baseline population is structurally too small to support a per-cell

\Delta

G measurement on this direction.

Model	Dataset	$L$	$n_{y}$ / $n_{n}$	label	$\cos(d_{\mathrm{i}},d_{\mathrm{sb}})$ [95% CI]	$\cos(d_{\mathrm{i}},d_{\mathrm{rs}})$	$\cos(d_{\mathrm{sb}},d_{\mathrm{rs}})$ [95% CI]	LOO	expl.
Mistral-7B	math800	15	29/21	pass-2	$+0.464$ $[+0.196,+0.522]$	$+0.130$	$+0.079$ $[+0.023,+0.097]$	0.80	—
Mistral-7B	code800	15	27/23	pass-2	$+0.163$ $[-0.042,+0.315]$	$+0.098$	$+0.013$ $[-0.035,+0.054]$	0.66	yes
Gemma-3-4B	math800	16	21/29	pass-2	$+0.486$ $[+0.270,+0.524]$	$+0.153$	$+0.124$ $[+0.049,+0.149]$	0.84	—
Gemma-3-4B	code800	15	15/34	pass-2	$+0.487$ $[+0.264,+0.529]$	$+0.152$	$+0.090$ $[+0.038,+0.102]$	0.91	—
Qwen3-14B	math800	25	28/20	pass-2	$+0.214$ $[-0.009,+0.378]$	$+0.091$	$+0.060$ $[+0.009,+0.090]$	0.84	—
Qwen3-14B	code800	24	23/25	pass-2	$+0.595$ $[+0.385,+0.675]$	$+0.139$	$+0.087$ $[+0.048,+0.104]$	0.91	—

Appendix

Table 20: Per-cell A-null primary subspace cosines for

d_{\mathrm{struct,behav}}

(directly comparable to § 4.2 ). Abbreviations:

d_{\mathrm{i}}\equiv d_{\mathrm{imp}}

d_{\mathrm{sb}}\equiv d_{\mathrm{struct,behav}}

d_{\mathrm{rs}}\equiv d_{\mathrm{ref,safety}}

. CIs are 1000-iteration bootstrap over the yes/no label set only, with

d_{\mathrm{i}}

and

d_{\mathrm{rs}}

held fixed;

\cos(d_{\mathrm{i}},d_{\mathrm{rs}})

is therefore a point estimate. LOO AUC is the held-out separability metric

\max(\mathrm{AUC},1-\mathrm{AUC})

. A cell is flagged exploratory (expl.) when LOO AUC

<0.65

or bootstrap self-cosine median

<0.80

. Six-cell primary means (excluding the two Qwen3-8B candidate-only cells) are

0.401

0.127

, and

0.076

for the three cosines. “pass-2” denotes provisional LLM-assisted second-pass audit fills, not completed human adjudication.

Model	Dataset	$\cos(d_{\mathrm{i}},d_{\mathrm{sb}})$ [95% CI]	$\cos(d_{\mathrm{i}},d_{\mathrm{rs}})$	$\cos(d_{\mathrm{sb}},d_{\mathrm{rs}})$ [95% CI]	LOO	expl.
Mistral-7B	math800	$+0.562$ $[+0.350,+0.584]$	$+0.272$	$+0.181$ $[+0.091,+0.207]$	0.82	—
Mistral-7B	code800	$+0.320$ $[+0.036,+0.430]$	$+0.245$	$+0.115$ $[-0.011,+0.178]$	0.62	yes
Gemma-3-4B	math800	$-0.229$ $[-0.503,+0.602]$	$+0.531$	$-0.244$ $[-0.404,+0.333]$	0.73	—
Gemma-3-4B	code800	$+0.932$ $[+0.772,+0.937]$	$+0.781$	$+0.780$ $[+0.631,+0.791]$	0.78	—
Qwen3-14B	math800	$+0.280$ $[+0.074,+0.458]$	$+0.251$	$+0.138$ $[+0.059,+0.193]$	0.86	—
Qwen3-14B	code800	$+0.590$ $[+0.356,+0.652]$	$+0.234$	$+0.152$ $[+0.089,+0.172]$	0.87	—

Appendix

Table 21: Full-space robustness pass: no A-null projection on

d_{\mathrm{i}}

d_{\mathrm{sb}}

;

d_{\mathrm{rs}}

in its native full space. Abbreviations and CI convention as in Table 20 . The pass agrees with A-null in aggregate (eight-cell mean

0.417

; six-cell primary mean

0.409

) except Gemma-3-4B / math800, whose full-space

d_{\mathrm{i}}

is contaminated by task-related variance (cosine

-0.23

with a sign-spanning CI); its A-null value is a clean

+0.49

. This is the failure mode the A-null subspace was designed to remove, and the reason the primary measurement is in A-null.

Base checkpoint	matched $L$	$d_{\mathrm{imp}}$ AUC
Qwen2.5-7B-Base	18	0.944
Qwen2.5-14B-Base	34	0.982
Qwen2.5-32B-Base	53	0.985
Qwen3-8B-Base	21	0.983
Qwen3-14B-Base	25	0.989
Llama-3.1-70B-Base	31	0.982

Appendix

Table 22: Base-model

d_{\mathrm{imp}}

detection AUC on math800 (A-null MeanDiff, one direction) at each base checkpoint’s matched layer, for the six base/instruct pairs of § 4.4 . Values are the auc_impossibility_on_dataset field of the per-model direction_comparison records; mean

0.977

Model	Dataset	$E(d_{\mathrm{imp}})$	first-comp. AUC	$E(d_{\mathrm{ref}})$ full (%)	$E(d_{\mathrm{ref}})$ A-null (%)	min. angle (deg.)	random first $\cos$
		$k{=}5/10$		$k{=}5/10$	$k{=}5/10$	$k{=}5/10$	$k{=}5/10$
Mistral-7B	math800	$0.889/0.956$	0.740	$1.94/2.41$	$2.31/2.86$	$80.3/77.2$	$0.058/0.088$
Mistral-7B	code800	$0.909/0.942$	0.816	$1.54/2.00$	$1.89/2.45$	$82.0/79.4$	$0.057/0.089$
Qwen3-14B	math800	$0.969/0.992$	0.833	$1.23/2.14$	$1.41/2.44$	$81.7/79.0$	$0.050/0.079$
Qwen3-14B	code800	$0.991/0.992$	0.929	$2.09/3.41$	$2.37/3.87$	$78.0/74.1$	$0.051/0.079$

Appendix

Table 23: Multi-dimensional robustness at

k\in\{5,10\}

E(d_{\mathrm{imp}})

is the fraction of the one-dimensional impossibility direction’s energy captured by the impossibility subspace.

E(d_{\mathrm{ref}})

is refusal-direction projector energy before and after projecting and renormalizing refusal in the same A-null space. “min. angle” compares the impossibility and behavior-verified refusal subspaces; the final column is the equal-dimensional random-subspace mean for the first principal cosine.

LLM Abstention

Large language model abstention studies when AI systems should answer, ask for more information, defer, or withhold unsupported claims.

See all 93 papers

Two Axes of LLM Abstention: Answer Correctness and Question Answerability
Jul 9, 2026Benedikt J. WagnerSelective Prediction Confidence Estimation in Language Models
City St George’s, University of London
Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration
Aug 16, 2026Yiqi Liu, Yang Wang, Songxin Wang +2LLM Refusal Behavior Refusal Mechanisms in Language Models
University of Manchester · Shanghai University of Finance and Economics
Answering the Wrong Question: Reasoning Trace Inversion for Abstention in LLMs
Apr 2, 2026Abinitha Gourabathina, Inkit Padhi, Manish Nagireddy +2LLM Abstention LLM Hallucination Mitigation
MIT · IBM Research · MIT-IBM Watson AI Lab

Jul 9, 2026J/K move · Enter open · S save

Two Axes of LLM Abstention: Answer Correctness and Question Answerability

Benedikt J. Wagner

City St George’s, University of London

Selective Prediction Confidence Estimation in Language Models LLM Uncertainty Estimation Refusal Behavior in Language Models

Open paper AskPDF arXiv

A model should refuse two different things: answers it would get wrong, and questions it should not answer at all, such as unanswerable ones or ones resting on a false premise. The usual recipe thresholds a single confidence score, which cannot tell these apart. Across five instruction-tuned models from three families (2B to 14B), we find they are separate axes. Ordinary answer-confidence tracks whether an answer is right but is nearly blind to whether the question is answerable; a linear probe on hidden states does the reverse. The blind spot does not shrink with scale. It is worst on naturally occurring false-premise questions (CREPE). There, answer-confidence, P(IK), P(True), and even asking the model outright whether a premise is false all stay near chance, while a hidden-state probe reaches 0.69 to 0.77 AUROC: the model represents a problem it will not report. This turns out to be fixable. Instructing a model to check premises backfires, because it then disputes sound and false premises alike (57% false challenges), unable to tell them apart; routing the same instruction with the probe roughly triples challenge precision. We turn the two axes into a calibrated policy that answers only when an answerability score and a correctness score each clear a separately certifies behave differently: the unanswerable-answer rate is controllable at every scale, while the wrong-answer rate is capped by model accuracy, so the guarantee tightens as threshold policy certifies both budgets at 0.75 coverage of correct answers, against 0.31 for a single threshold; at 14B it is the only policy that certifies at all.

Abstract

LLM Abstention

Explore similar work

Two Axes of LLM Abstention: Answer Correctness and Question Answerability

Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration

Answering the Wrong Question: Reasoning Trace Inversion for Abstention in LLMs