Safety-aligned large language models (LLMs) remain vulnerable to jailbreaks and can over-refuse benign requests. Understanding these failures requires distinguishing harmfulness recognition from the computation that writes refusal. We reframe this as a pathway question and make it measurable on residual-stream updates in decoder-only Transformers, separating the harmfulness information a module update exposes from the change it produces in readable refusal evidence. Across 18 Base-Instruct pairs spanning Qwen, Llama, and Gemma, harmfulness is decoded more readily from attention updates, particularly in early layers, whereas MLP updates contribute more strongly to harm-selective refusal writing in aligned models. Post-training strengthens MLP refusal writing in all 18 pairs without systematically improving harmfulness decodability, while positive MLP writing becomes more concentrated in deeper layers after post-training. Across six reasoning-enabled models, harmfulness remains decodable during reasoning while strong refusal writing concentrates at answer onset, a pattern that persists with reasoning disabled. Causal interventions further support this pathway: resetting downstream MLP updates to their clean values substantially reduces the refusal shift caused by upstream attention interventions. Building on these findings, we introduce Sentinel, which combines an attention-derived perception score P with an MLP-derived decision score D. Their joint profiles characterize jailbreaks and over-refusals, and conditional routing with both signals achieves a lower aggregate error rate than single-signal routing and evaluated fixed reminders across eight models.
Figures & tables
Figure 1: Early attention exposes harmfulness information and increases residual-stream separability. (a) Layerwise probe AUC on attention and MLP updates. (b) Change in residual-stream probe AUC after adding each update. Curves show cross-model medians over relative depth; shaded regions denote interquartile ranges.
Figure 2: Harmfulness is already decodable in Base checkpoints, while Instruct checkpoints exhibit stronger and more specific MLP refusal writing. Comparison across 18 Base–Instruct pairs. (a) layer-averaged probe AUCU ; (b) signed cumulative writing, ∑ℓΔRWSℓU ; (c) specificity relative to matched control vocabularies across 18 Base–Instruct pairs. Specificity is the ratio of positive refusal-writing mass to the median positive mass over 16 matched control vocabularies.
Figure 3: Positive MLP refusal writing becomes more concentrated in late layers after post-training. (a) Cumulative positive ΔRWSM mass, normalized to one for each model. (b–c) Per-model-normalized layerwise profiles for Instruct and Thinking checkpoints, respectively. Normalization preserves the shape and location of writing but does not support comparisons of absolute magnitude.
Figure 4: Harmfulness remains decodable during reasoning, while strong MLP refusal writing appears at answer start. Results across six Qwen3 and Qwen3.5 models. (a) Harmfulness probe Median probe AUCU at six generation positions with reasoning enabled. Lines show cross-model medians, shaded regions denote interquartile ranges, and shuffled-label controls remain near chance. (b) Cumulative harm-selective refusal writing, ∑ℓΔRWSℓU(t) . Purple curves show cross-model medians with thinking enabled, and the shaded band denotes the MLP interquartile range. Blue markers show the corresponding thinking-off values at answer start. The background shading marks the reasoning span.
Figure 5: Prompt categories in the Sentinel P – D space. Each panel shows one of three non-reasoning models. Points represent group-mean perception scores P and decision scores D , standardized using each model’s calibration data and the original D layer-selection rule. Circles denote jailbreak attack groups, with area encoding ASR. Filled and open squares denote over-refused (OR) and normally answered (OK) subsets of XSTest-safe and EVOREFUSE-TEST, respectively. Diamonds mark the AlpacaEval and AdvBench reference groups. Axis ranges vary across panels; zero is not a behavioral decision threshold.
Method
Jailbreak ASR ↓
Benign False refusal ↓
Over-refusal False refusal ↓
AVG ↓
No reminder ( N )
95.97
0.00
88.73
61.57
Helpfulness reminder ( Hend )
68.22
0.00
59.95
42.72
Self-Reminder ( Xie et al., 2023 )
41.03
0.13
83.64
41.60
In-Context Defense (ICD) ( Wei et al., 2026 )
50.03
0.80
72.59
41.14
Balanced reminder ( Bend )
45.73
0.13
76.70
40.85
Llama Guard 3 ( Dubey et al., 2024 )
26.58
1.08
92.53
40.06
Table 1: Conditional reminders versus fixed-reminder baselines. All values are percentages, averaged equally across eight non-reasoning models. AVG is the arithmetic mean of jailbreak ASR, benign false refusal rate, and over-refusal false refusal rate. Lower is better for every column. Rows are ordered by decreasing AVG; the best value among the displayed methods in each column is bold.
Appendix figures & tables42 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Module-update counterfactuals in non-reasoning configurations. The left column patches Attention updates, and the right column patches MLP updates. The upper panels report changes in final refusal evidence, ΔRESfinal ; the lower panels report changes in same-layer MLP writing, ΔRWSℓM . Red denotes harmful-donor-to-benign-target patching, and blue denotes benign-donor-to-harmful-target patching. Both quantities are divided by the same within-configuration clean harmful–benign final refusal-evidence gap Gc . Lines show medians across 21 non-reasoning configurations at matched relative depths; shaded bands show interquartile ranges, not confidence intervals. Attention-update patching shifts the final refusal readout with a comparatively small aggregate same-layer MLP response, whereas MLP-update patching produces directionally aligned local and final responses, particularly in later layers.
Figure 7: Module-update counterfactuals at the answer boundary after reasoning. The left column patches Attention updates, and the right column patches MLP updates. The upper panels report changes in final refusal evidence, ΔRESfinal ; the lower panels report changes in same-layer MLP writing, ΔRWSℓM . Red denotes harmful-donor-to-benign-target patching, and blue denotes benign-donor-to-harmful-target patching. Both quantities are divided by the same within-configuration clean harmful–benign final refusal-evidence gap Gc . Lines show 10 reasoning configuration medians at matched relative depths; shaded bands show interquartile ranges, not confidence intervals. The aggregate results show a similar local–final distinction to the non-reasoning analysis, extending the intervention evidence to answer formation after reasoning.
Figure 8: Joint path-cutting intervention design. We first patch donor Attention updates at the frozen P layers, then either allow downstream computation to proceed normally or restore the recipient’s clean MLP updates at the frozen D layers.
Figure 9: Downstream MLP restoration blocks upstream Attention effects. Each point shows the mean over 100 matched pairs for one configuration; error bars are 95% confidence intervals obtained by prompt-pair bootstrap. Filled circles show the total P -patch effect T , open circles show the residual effect R after restoring clean D , and the connecting segment shows the blocked component B . All effects are divided by the same clean harmful–benign gap Gc used in Figure 6 and Figure 7 . For the benign-to-harmful direction, we flip the sign so that positive values in both panels indicate movement toward donor-consistent behavior. Right-side percentages report B/T ; “n.s.” indicates that the 95% bootstrap confidence interval for T includes zero.
Figure 10: Generation-level behavioral outcomes. The main number in each cell is the number of donor-consistent labels out of 100 (refusal for harmful-to-benign; compliance for benign-to-harmful), as assigned by the GPT-based judge.
Model
MLP BM/T
Attention BA/T
BM−BA [95% CI]
Llama-3.1-8B
26.5%
−15.4%
2.3851 [2.0207, 2.7460]
Qwen3-8B
79.4%
−34.6%
8.3055 [7.4179, 9.1967]
Gemma4-12B
82.8%
19.2%
6.2235 [5.5384, 6.9308]
Appendix
Table 2: Held-out matched module-restoration control using the strict post- P portion of each onset-selected D component. BM/T and BA/T are ratios of direction-signed means pooled over 50 held-out prompt pairs and both patching directions. The final column reports the mean within-observation MLP-minus-Attention difference in each model’s final refusal-evidence units, pooling the 100 prompt–direction observations; its 95% bootstrap confidence interval resamples those observations (10,000 replicates; seed 20260924).
Model
Direction
Selected D
Random C
ΔDC [95% CI]
Llama-3.1-8B
harmful → benign
64.4%
49.3%
0.0370 [0.0267, 0.0473]
Qwen3-8B
harmful → benign
65.4%
30.4%
0.2148 [0.1819, 0.2496]
Qwen3-8B
benign → harmful
69.3%
30.0%
0.1316 [0.1039, 0.1611]
Gemma4-12B
harmful → benign
82.3%
73.5%
0.0531 [0.0396, 0.0665]
Gemma4-12B
benign → harmful
86.6%
84.4%
0.0091 [0.0032, 0.0152]
Appendix
Table 3: Selected- D and fixed random post- P restoration controls. Blocking fractions are ratios of means, B/T . The final column reports the donor-aligned paired difference ΔDC=s(BD−BC)/Gc with a pointwise 95% bootstrap confidence interval. The Llama-3.1-8B benign-to-harmful direction is omitted because its total-effect interval includes zero.
Figure 11: Reasoning timelines across refusal readouts. Cumulative MLP writing SM(t)=∑ℓΔRWSℓM(t) at six sampled positions for (a) Qwen3-8B and (b) Qwen3-14B under thinking-on. Colors denote the four readout constructions; the shaded strip marks the true answer boundary. In all eight model–readout configurations, writing at answer_start exceeds the maximum at the three sampled reasoning-mid positions, with gaps of 10.00–26.47. Positions are categorical; connecting lines do not represent measurements at intermediate tokens.
Figure 12: True-answer-boundary control under thinking-on and thinking-off. Each segment connects cumulative MLP writing at assistant_start (open circle) to writing at the true answer_start (filled circle). Rows correspond to readouts, while panels separate models and thinking conditions. Blue denotes thinking-off and purple denotes thinking-on. All eight model–readout configurations show positive gains in each condition: +2.58 to +19.97 under thinking-off and +7.80 to +24.15 under thinking-on. Segments show paired positions, not uncertainty intervals.
Figure 13: Layerwise accumulation of MLP writing across refusal readouts. Cumulative ΔRWSM through each layer for (a) Llama3.1-8B, (b) Qwen3-8B, and (c) Qwen3-14B. Colors identify readouts; solid and dashed curves denote Instruct and Base, respectively. The shaded region marks the final quarter of network depth. Instruct is measured at the visible-answer boundary, using thinking-off for Qwen3; Base uses the final input-token position. The final Instruct–Base MLP layer-sum difference is positive in all 12 configurations, ranging from +8.96 to +21.49 .
Figure 14: Attention and MLP net writing across refusal readouts. Signed layer sums SU=∑ℓΔRWSℓU for Attention (blue) and MLP (purple), measured in Instruct models at the visible-answer boundary, with Qwen3 in thinking-off mode. Gray segments connect the two module values within each configuration. MLP net writing exceeds Attention net writing in 11 of 12 configurations. Qwen3-8B with data-driven k=8 has SM=7.16 and SA=7.73 . Segments represent module differences, not uncertainty intervals.
Table 4: Data-driven refusal and compliance token sets independently estimated from the prompt-disjoint calibration generations. The symbol ␣ denotes a leading whitespace in the token string.
Model
Readout
Maximum reasoning-mid SM
answer_start SM
Boundary gap
Qwen3-8B
Main token set
1.73
18.72
+16.99
Arditi-style
-3.18
21.82
+25.00
Data-driven ( k=3 )
0.59
21.91
+21.32
Data-driven ( k=8 )
2.32
12.32
+10.00
Qwen3-14B
Main token set
1.95
17.92
+15.97
Arditi-style
-0.43
26.05
+26.47
Appendix
Table 5: Cumulative MLP writing at the strongest sampled reasoning-mid position and at the true answer boundary under thinking-on.
Model
Readout
Thinking-off boundary gain
Thinking-on boundary gain
Qwen3-8B
Main token set
+11.50
+13.56
Arditi-style
+8.19
+14.63
Data-driven ( k=3 )
+9.26
+15.86
Data-driven ( k=8 )
+2.58
+7.80
Qwen3-14B
Main token set
+18.18
+17.84
Arditi-style
+17.54
+24.15
Appendix
Table 6: True-answer-boundary gains under thinking-off and thinking-on across alternative refusal readouts.
Model
Readout
SA
SM
SM−SA
Llama3.1-8B
Main / Arditi-style / k=3 / k=8
-0.05 / 3.84 / 0.47 / 0.51
12.32 / 11.10 / 14.65 / 14.61
12.37 / 7.25 / 14.18 / 14.10
Qwen3-8B
Main / Arditi-style / k=3 / k=8
-1.94 / 0.47 / -0.52 / 7.73
16.73 / 15.39 / 15.40 / 7.16
18.67 / 14.92 / 15.92 / -0.57
Qwen3-14B
Main / Arditi-style / k=3 / k=8
-1.46 / -2.19 / -0.83 / 3.08
18.28 / 19.57 / 19.50 / 13.21
19.74 / 21.77 / 20.34 / 10.13
Appendix
Table 7: Attention and MLP signed layer sums under the four refusal-readout constructions.
Method
Jailbreak ASR ↓
Benign false refusal ↓
Over-refusal false refusal ↓
AVG ↓
No reminder ( N )
96.28%
0.00%
93.33%
63.20%
Helpfulness reminder ( Hend )
67.59%
0.00%
68.36%
45.32%
Balanced reminder ( Bend )
53.47%
0.34%
85.77%
46.53%
Safety reminder ( Send )
30.89%
6.95%
92.71%
43.52%
Sentinel P+D — Main token set
40.00%
0.35%
59.76%
33.37%
Sentinel P+D — Arditi-style
38.01%
0.35%
67.98%
35.45%
Appendix
Table 8: Conditional-reminder routing under alternative refusal readouts. All values are macro averages over Llama3.1-8B-Instruct, Qwen3-8B-Instruct, and Qwen3-14B-Instruct. AVG is the equally weighted mean of Jailbreak ASR, Benign false refusal, and Over-refusal false refusal. These three-model averages are not directly comparable to the eight-model averages in Table 1 of the main text.
Figure 15: Matched-input, matched-position Base–Instruct control for six checkpoint pairs. Each panel shows the layerwise MLP ΔRWS profile for Base and Instruct checkpoints under the same plain-text response cue. The token sequence and measurement position are identical within each pair. Instruct checkpoints have larger signed layer sums in all six pairs, while their precise depth profiles remain sensitive to the shared interface context.
MLP signed sum
Late-quarter positive-mass share
Peak depth
Checkpoint pair
Base
Instruct
Base
Instruct
Base
Instruct
Llama-3.1-8B
1.105
6.169
25.41%
22.60%
0.742
0.581
Qwen2.5-7B
4.893
14.652
72.47%
79.36%
0.852
0.852
Qwen3.5-9B
0.655
1.060
32.71%
21.52%
0.871
0.613
Gemma-2-9B
3.665
4.678
49.52%
43.16%
0.439
0.756
Gemma-3-12B
0.990
5.583
21.29%
60.81%
0.723
0.894
Appendix
Table 9: MLP refusal-writing statistics for six matched-input, matched-position checkpoint pairs. We report the signed MLP layer sum SM , the late-quarter positive-mass share ClateM , and the normalized peak depth dpeak .
Cohort
Source and behavioral condition
Maximum per model
Benign
AlpacaEval requests answered normally without a reminder
100
Raw harmful
AdvBench reference requests with independent behavioral judgments
100
Historical over-refusal
Allowed XSTest-safe or EvoRefuse requests refused without a reminder
100
Historical jailbreak
ReNeLLM, PAIR, AutoDAN, or GCG requests answered compliantly without a reminder
100
Matched XSTest
XSTest-safe requests answered normally without a reminder
50
Matched EvoRefuse
EvoRefuse requests answered normally without a reminder
50
Appendix
Table 10: Construction of the routing dataset.
Model
Jailbreak
Benign
Over-refusal
Llama-3.1-8B-Instruct
99
98
93
Qwen2.5-7B-Instruct
98
99
95
Qwen3-8B-Instruct
100
94
88
Qwen3-14B-Instruct
98
97
91
Gemma2-9B-Instruct
96
96
96
Gemma4-12B-Instruct
69
89
79
Appendix
Table 11: Requests with outputs for all four actions in the three main evaluation cohorts.
Router input
Dim.
Jailbreak ASR ↓
Benign False refusal ↓
Over-refusal False refusal ↓
AVG ↓
P mean
1
57.34
0.13
71.40
42.96
D mean
1
55.02
0.78
68.27
41.36
P mean +D mean
2
50.01
0.13
67.89
39.34
Appendix
Table 12: Low-dimensional P/D control. Each router reads at most two scalars. All values are percentages, averaged equally across the same eight non-reasoning models as Table 1 . AVG is the arithmetic mean of the three error rates, and lower is better in every column.
Model
Layers
LP
LD
Llama-3.1-8B-Instruct
32
11–13
14–18
Qwen2.5-7B-Instruct
28
18–19
14–25
Qwen3-8B-Instruct
36
18–21
21–23
Qwen3-14B-Instruct
40
38–39
39
Qwen3.8-27B
64
42–48
47–53
Gemma-2-9B-IT
42
18–23
28–31
Appendix
Table 13: Automatically selected layers for the eight non-reasoning models used in Section 4 . Layer indices are zero-based; a–b denotes the contiguous interval from layer a through layer b .
Model
Mode
Attention
Position
LP
LD
Llama-3.1-8B-Instruct
non-reasoning
self-attention
assistant_start
11–13
14–18
Qwen3-8B-Instruct
non-reasoning
self-attention
assistant_start
18–21
21–23
Qwen3-8B-Instruct
reasoning
self-attention
answer_start
22–23
27–30
Qwen3.5-9B-Instruct
non-reasoning
DeltaNet
assistant_start
16
29–30
Qwen3.5-9B-Instruct
reasoning
self-attention
answer_start
19
21–22
Gemma-4-12B-IT
non-reasoning
sliding attention
assistant_start
25–28
32–35
Appendix
Table 14: Frozen layer configurations for the joint P -patch and D -restoration experiments. Non-reasoning and reasoning modes are treated as separate configurations and measured at assistant_start or answer_start , respectively.
Family
Scales
Pairs
Qwen2.5
3B, 7B, 14B
3
Qwen3
4B, 8B, 14B
3
Qwen3.5
2B, 4B, 9B
3
Llama
Llama3.2-1B, Llama3.2-3B, Llama3.1-8B
3
Gemma2
2B, 9B, 27B
3
Gemma3
4B, 12B, 27B
3
Appendix
Table 15: Base–Instruct pairs used in the representation and refusal-writing analyses. Each family–scale combination includes both the Base and corresponding Instruct checkpoint.
Table 16: Complete model inventory for the single-module counterfactual analysis. NR and R denote non-reasoning and reasoning configurations.
Figure 16: Individual module-update counterfactuals in non-reasoning configurations. Results are shown for all 21 non-reasoning configurations included in the Appendix B . Within each configuration, the left column patches attention updates and the right column patches MLP updates. Upper panels report changes in final refusal evidence, ΔRESfinal ; lower panels report changes in same-layer MLP writing, ΔRWSℓM . Red denotes harmful-donor-to-benign-target patching, and blue denotes benign-donor-to-harmful-target patching.
Figure 17: Individual module-update counterfactuals at the answer boundary after reasoning. Results are shown for all 10 reasoning configurations included in the Appendix B . Interventions replace a single module update at answer_start following the cached reasoning context. Within each configuration, the left column patches attention updates and the right column patches MLP updates. Upper panels report changes in final refusal evidence, ΔRESfinal ; lower panels report changes in same-layer MLP writing, ΔRWSℓM . Red denotes harmful-donor-to-benign-target patching, and blue denotes benign-donor-to-harmful-target patching.
Figure 44Figure 45Figure 46Figure 47
Metric
What it measures
Corresponding process
Harmfulness probe
Whether a module update contains linearly decodable harmfulness information.
Harmfulness recognition
Refusal Evidence Score (RES)
How strongly the current residual state supports refusal-oriented continuations over compliance-oriented continuations.
Refusal-evidence readout
Refusal Writing Score (RWS)
Whether adding a module update actually increases refusal evidence in the current residual state.
Refusal writing
ΔRWS
Whether harmful prompts receive stronger refusal writing than harmless prompts.
Harm-selective refusal writing
Appendix
Table 17: Measurement map for the recognition-to-writing analysis.
LLMs are trained to refuse harmful instructions, but do they truly understand harmfulness beyond just refusing? Prior work has shown that LLMs' refusal behaviors can be mediated by a one-dimensional subspace, i.e., a refusal direction. In this work, we identify a new dimension to analyze safety mechanisms in LLMs, i.e., harmfulness, which is encoded internally as a separate concept from refusal. There exists a harmfulness direction that is distinct from the refusal direction. As causal evidence, steering along the harmfulness direction can lead LLMs to interpret harmless instructions as harmful, but steering along the refusal direction tends to elicit refusal responses directly without reversing the model's judgment on harmfulness. Furthermore, using our identified harmfulness concept, we find that certain jailbreak methods work by reducing the refusal signals without reversing the model's internal belief of harmfulness. We also find that adversarially finetuning models to accept harmful instructions has minimal impact on the model's internal belief of harmfulness. These insights lead to a practical safety application: The model's latent harmfulness representation can serve as an intrinsic safeguard (Latent Guard) for detecting unsafe inputs and reducing over-refusals that is robust to finetuning attacks. For instance, our Latent Guard achieves performance comparable to or better than Llama Guard 3 8B, a dedicated finetuned safeguard model, across different jailbreak methods. Our findings suggest that LLMs' internal understanding of harmfulness is more robust than their refusal decision to diverse input instructions, offering a new perspective to study AI safety.
Aligned large language models (LLMs) remain vulnerable to jailbreak attacks. Recent mechanistic studies have identified latent features and representation shifts associated with jailbreak success, but they leave a more fundamental question open: why do aligned LLMs remain jailbreakable, and what structural vulnerabilities in the model make this possible? We study this question through a continuous input-transformation view. Our theoretical finding is that aligned models can still exhibit Refusal-Escape Directions (RED): local perturbation directions around a harmful input that shift the model's behavior from refusal to answering while preserving the model's harmful-semantics interpretation. From this perspective, a jailbreak is not only a successful discrete prompt construction, but can also be understood as a refusal-to-answer behavior transition induced by continuously perturbing a harmful input along RED. We then prove that RED can be exactly decomposed into contributions from operator-level sources across the model's operator structure, and identify normalization, residual-wiring, and terminal sources as analytically constrained operator-level sources. To eliminate RED, the shared expressive modules -- self-attention and MLP -- must eliminate the contributions from these analytically constrained sources while preserving the mechanisms that support benign responses. These competing requirements give rise to a conditional safety-utility trade-off. Experiments across multiple models and attack methods empirically analyze RED from two complementary perspectives and show that added token dimensions can expose RED, while successful jailbreaks exhibit refusal-to-answer shifts largely aligned with terminal-source contributions.
Yu Chen, Yuanhao Liu, Qi Cao
Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China
Safety training on language models often induces over-refusal: improved safety on harmful prompts at the cost of increased refusal on harmless ones. Though this trade-off can be mitigated by training models with reinforcement learning (RL) to reason before answering, it does not remove the underlying problem that reasoning can often be a "rubber stamp" for a predetermined response. In this paper, we address the safety-refusal trade-off by rethinking how models are trained to reason about safety. Our key insight is that unsafe reasoning can itself serve as a useful exploratory signal. Rather than preemptively blocking harmful thoughts, we encourage the model to sufficiently explore unsafe reasoning but produce a safe response. The harmful exploration improves the model's ability to distinguish harmful from harmless prompts by resolving ambiguity, allowing it to remain safe while complying only when appropriate. We cast this as an adversarial optimization problem in which a reasoning player explores strategies for producing an unsafe response and an answer player ensures that the final output is safe. We train a single model with dense rewards to play both roles within one chain-of-thought, across different segments. To achieve this, we find that process rewards are crucial for stable optimization of competing objectives. Our resulting model SEAR deliberately engages in harmful reasoning as exploration while reliably flipping back to a safe answer. We demonstrate that this behavior helps mitigate over-refusal and defend against attacks that directly manipulate the reasoning to be harmful.