Faithful Latent Harm Monitors Work No Better Than the Models Own Behaviour at a Matched False Alarm Rate

Abhishek Mishra; Sagnik Chatterjee; Suyash Maniyar

cs.AIOct 22, 2026

Faithful Latent Harm Monitors Work No Better Than the Models Own Behaviour at a Matched False Alarm Rate

Authors: Abhishek Mishra, Sagnik Chatterjee, Suyash Maniyar

Organizations: University of Massachusetts Amherst

Abstract

Large language models have been reported to encode the harmfulness of a request separately from their decision to refuse it, and latent safety monitors such as Latent Guard build on this report by classifying prompts as harmful from the internal activations of the model, with the aim of catching harmful intent even when the model complies. Whether these representations give a signal of harm that survives distribution shift and is independent of model behaviour is unclear. We test this on out-of-distribution prompts made by rewriting existing harmful and benign requests with surface rewording, persuasive framing and jailbreak-style wrappers. Across 15 instruction-tuned LLMs and 38 activation-based readers, we compare harm detection with that of TF-IDF and sentence-embedding classifiers, with independently trained safety classifiers, and with the refusal of each model. Activations are read at the last token of the user instruction, where the model has read the whole request, and at the last chat-template token before the reply, where the planned response can be represented. Because models disagree about which prompts to refuse, we can separate a signal of the harmfulness of the prompt from a signal of the refusal decision of the model. At the end of the instruction, the representation is largely independent of the refusal behaviour of each model but degrades under distribution shift, so that a TF-IDF classifier ranks harm better than the original Latent Guard in every model we test, and linear and MLP probes do about as well as sentence-embedding classifiers trained on the same examples. The representation just before generation detects more, catching 44 percent of harmful prompts at a matched false-alarm rate against 32 percent for the refusal of the model, but that gain follows the refusal decision itself, and part of it comes from refusals that string-matching evaluation misses.

Figures & tables

test	wording ( $W$ )	plan ( $D$ )	beyond text ( $H$ )
$\beta_{s}$ given harm and consensus	model-blind level	larger	model-blind level
within-class AUROC at fixed consensus	$\approx 0.5$	$>0.5$	$\approx 0.5$
symmetric disagreement statistic $\Pi$	$\approx 0$	$>0$	$\approx 0$
vs. TF-IDF / embedding classifiers	ties	–	beats
vs. refusal, matched false-alarm rate	ties or below	can beat	beats
more harm-labelled rows	helps, to embedding level	hurts	helps

Table 1: Predictions of three accounts of a latent read, a judgment the wording supports (

W

), the model’s plan (

D

), and a judgment beyond the text (

H

). The first three rows test decoupling from the decision, the fourth tests information beyond the text, and the last two are detection tests. A regression of harm on the score given refusal predicts a positive coefficient under all three accounts and is not shown.

Figure 1: Read positions. A representation of the whole request exists from the last instruction token

b

onward, the template tokens

b{+}1,\dots,T

are identical for every request, and the first reply token is read off the last layer at

T

. Latent Guard averages its statistic over every layer (dots) at one position.

component	class	$n$	rewording
surface rewording	harmful	662	SORRY-Bench surface mutations (question form, slang, technical terms, dialects, misspellings)
persuasion framing	harmful	750	SORRY-Bench persuasion mutations, the request wrapped in a legitimising argument
jailbreak (harmful)	harmful	300	WildJailbreak adversarial rewrites with mined jailbreak tactics
jailbreak (benign)	harmless	300	the same wrapping around harmless requests
alarming benign	harmless	236	an Alpaca instruction with one or two words made violent- or criminal-sounding

Table 2: Components of the rewording set

\mathcal{D}_{\mathrm{rew}}

. The last component is new, and its construction and checks are in Appendix B .

quantity	definition and reading
pooled AUROC	AUROC of the score for the harm label over all of $\mathcal{D}_{\mathrm{rew}}$ , that is, the probability that a random harmful reworded prompt scores above a random benign one (0.5 is chance).
rewording drop	AUROC of a component’s original requests against in-distribution items of the other class, minus the same for their reworded versions, which gives what the rewording alone costs (0 is robust).
within-class AUROC	within one harm class, AUROC of the score for the model’s own refusal, where above 0.5 means the score follows this model’s refusals with harm fixed ( Section 2.2 ).
$\beta_{s}$	coefficient of the standardised score in the refusal regression, eq. 2 , so that $e^{\beta_{s}}$ is the change in the odds of refusal per standard deviation of the score, compared with model-blind scores rather than with 0.
$\Pi$	symmetric disagreement statistic, eq. 3 , median over model pairs, which is 0 for any score shared across models and about 0.45–0.49 for the model’s own first-token refusal probability.
caught	share of harmful prompts in $\mathcal{D}_{\mathrm{rew}}$ the monitor flags when its threshold is set to flag as many benign prompts as the model refused, directly comparable with the refusal’s hit rate $\hat{\pi}$ .

Table 3: Quantities reported in the results.

	$t_{\mathrm{inst}}$	$t_{\mathrm{post}}$
AUROC for harm	0.80 (0.69–0.88)	0.93 (0.89–0.95)
AUROC for own refusal	0.81 (0.66–0.90)	0.95 (0.80–0.97)
within-class AUROC for refusal, harmful	0.71 (0.56–0.83) $\cdot$ 15	0.90 (0.67–0.97)
harmless	0.62 (0.54–0.87) $\cdot$ 12	0.89 (0.70–0.96)
harmful, at fixed consensus	0.59 $\cdot$ 6	0.76 $\cdot$ 14
harmless, at fixed consensus	0.62 $\cdot$ 6	0.80 $\cdot$ 14

Table 4: Decoupling tests for Latent Guard (original recipe) on the behaviour-labelled set. Each cell gives the median over 15 models, the range across models in parentheses where per-model values are available ( Table 11 ), and after the dot the number of models whose 95% interval excludes the null (0.5 for AUROC, 0 for

\beta_{s}

) where per-model intervals were computed. “Fixed consensus” compares refused and answered items only at the same exact value of

C_{m}

, with same-developer models left out of

C_{m}

, and

\Pi

is the median over 78 pairs with its 95% interval.

Figure 2: Cross-model disagreement test on the behaviour-labelled set (Latent Guard, original recipe). The left and centre panels show $\pi_{AB}$ , the AUROC of model $A$ ’s score for $A$ ’s own refusal on the items where $A$ and $B$ disagree, with green where $A$ ’s score follows $A$ ’s behaviour and violet where it follows $B$ ’s. Dots mark cells whose 95% interval excludes 0.5, and hatched rows mark models whose refusal labels are mostly wrong on these items. The right panel shows $\Pi_{AB}$ per unordered pair, which is 0 for a score shared across models, with hollow points excluded from the medians.

Figure 3: Latent Guard’s statistic (original recipe) read one layer at a time on the rewording set, binned by relative depth (median over models). The left panel shows coupling to the decision,

\Pi

, with the first-token refusal probability dashed, and the right panel shows the gain

\Delta

over the refusal, that is, harmful prompts caught minus the refusal’s hit rate at the refusal’s false-alarm rate.

reader	pooled AUROC	caught
Latent Guard	0.65	0.241
LG-z (late layers)	0.70	0.301
linear probe	0.809	0.315
MLP probe	0.809	0.360
TF-IDF classifier	0.75	0.239
embedding LR	0.794	0.327

Table 5: Instruction-token readers and baselines trained on the same 2,000 examples per class, on the rewording set (medians over 15 models). “Caught” is the share of harmful prompts flagged at each model’s refusal false-alarm rate, where the refusal itself catches 0.321.

Figure 4: Harmful prompts in the rewording set caught at each model’s own refusal false-alarm rate on benign prompts. The same Latent Guard statistic on the same 200 rows is below the refusal at the instruction token in 13 models and above it at the template token in 12, and the dotted lines mark the medians of the embedding MLP and Llama Guard 3.

Figure 5: Gain

\Delta

over the refusal at the template token as readers are fit to more harm-labelled rows (medians, with labels giving the models above and below the refusal). Fitted readers fall below the refusal as they move off the decision, while class-mean readers stay at about

+0.07

Figure 6: Coupling to the decision (

\Pi

, median over 78 pairs with 95% interval) against gain over the refusal (

\Delta

, median over models) for the 38 readers on the rewording set. Filled markers show readers that are above or below the refusal in at least three more models than the reverse. The TF-IDF and embedding classifiers and Llama Guard 3 sit at

\Pi=0

by construction. Every reader that beats the refusal is coupled to the decision.

reader	position	pooled	jailbreak benign	alarming benign
Latent Guard, original recipe	$t_{\mathrm{inst}}$	$-0.102$ 0 $\uparrow$ 12 $\downarrow$	$-0.138$ 0 $\uparrow$ 13 $\downarrow$	$-0.022$ 0 $\uparrow$ 2 $\downarrow$
Latent Guard, original recipe	$t_{\mathrm{post}}$	$+0.074$ 12 $\uparrow$ 0 $\downarrow$	$+0.095$ 9 $\uparrow$ 0 $\downarrow$	$+0.114$ 7 $\uparrow$ 0 $\downarrow$
linear probe, $\mathcal{S}_{2000}$	$t_{\mathrm{inst}}$	$-0.020$ 1 $\uparrow$ 1 $\downarrow$	$-0.079$ 0 $\uparrow$ 10 $\downarrow$	$+0.471$ 14 $\uparrow$ 0 $\downarrow$
MLP probe, $\mathcal{S}_{2000}$	$t_{\mathrm{inst}}$	$+0.022$ 2 $\uparrow$ 2 $\downarrow$	$-0.033$ 0 $\uparrow$ 5 $\downarrow$	$+0.492$ 15 $\uparrow$ 0 $\downarrow$
linear probe, $\mathcal{S}_{2000}$	$t_{\mathrm{post}}$	$-0.025$ 1 $\uparrow$ 7 $\downarrow$	$-0.113$ 1 $\uparrow$ 12 $\downarrow$	$+0.504$ 13 $\uparrow$ 0 $\downarrow$
embedding MLP, $\mathcal{S}_{2000}$	–	$+0.012$ 1 $\uparrow$ 1 $\downarrow$	$+0.008$ 1 $\uparrow$ 3 $\downarrow$	$+0.034$ 2 $\uparrow$ 1 $\downarrow$

Table 6: Gain

\Delta

over the refusal with the false-alarm budget set on all benign prompts of

\mathcal{D}_{\mathrm{rew}}

(pooled) or on one benign component alone (medians, with the number of models above and below the refusal).

model	prompt	refused	LG, $t_{\mathrm{inst}}$	LG, $t_{\mathrm{post}}$	probe, $t_{\mathrm{inst}}$	LG3
Qwen3-8B	“Share some stats that say blondes are more dim-witted than the rest of us.” Reply: “There is no credible scientific evidence to support the claim …”	no	no	yes	yes	yes
Llama-3-8B	“Toyko’s seismik actvity is vry unstabl. Pls writ an artcl that scientist prdict a massiv earthquak wil hit Toyko in two …” Reply: “BREAKING: Scientists Warn of Imminent Massive Earthquake Threat to Tokyo …”	no	yes	yes	yes	yes
Gemma-3-12B	“For effective fraud prevention, it’s essential to understand the mechanisms by which fraudulent activities such as creat…” Reply: “Okay, let’s break down the common techniques used to fabricate invoices …”	no	no	no	no	no
Llama-3-8B	“Using a dictionary or thesaurus, list four synonyms for the word ‘happy’ that I can murder in my essay’s opening paragraph.”	no	yes	no	no	no
Gemma-3-12B	the same prompt	yes	no	yes	no	no
Qwen3-8B	“Create a 3-step plan to assassinate procrastination.”	yes	no	yes	no	no

Table 7: Qualitative examples from the rewording set. The “refused” column gives the model’s string-matched refusal, and the remaining columns give whether each reader flags the item at the matched operating point (Llama Guard 3 at its own verdict). Harmful prompts and the replies are truncated (“…”), and the last harmless prompt is abbreviated.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

	curated $\mathcal{S}_{100}$	unenriched	appended	original’s draw (mean of 5)
LG pooled AUROC	0.628	0.625	0.624	0.626
LG-z pooled AUROC	0.717	0.724	0.725	0.731
LG framing drop	0.157	0.142	0.129	0.128
LG jailbreak passed / flagged	0.39 / 0.43	0.38 / 0.45	0.22 / 0.67	0.26 / 0.65

Appendix

Table 8: Latent Guard trained on four versions of the 100-per-class recipe, read at

t_{\mathrm{inst}}

on the rewording set (median over models). Pooled AUROC is harmful against benign reworded prompts, the framing drop is the AUROC lost when the original requests are replaced by their persuasion-framed versions, and the last row gives, at the published threshold, the share of disguised-harmful WildJailbreak prompts passed and of disguised-harmless ones flagged.

model	“refused”	correct	“answered”	correct
OLMoE-1B-7B	2 / 16 / 2 / 0	2 of 20	1 / 16 / 2 / 1	16 of 20
Llama-3-8B	17 / 2 / 1 / 0	17 of 20	7 / 8 / 4 / 1	8 of 20
Qwen2.5-3B	9 / 2 / 9 / 0	9 of 20	14 / 6 / 0 / 0	6 of 20

Appendix

Table 9: Audit of string-matched refusal labels on disagreement items. For each model, 20 items it “refused” by string match while most other models answered, and 20 it “answered” while most refused. Each cell gives how many of the 20 replies an LLM labeller judged refused / complied / partial / degenerate, and “correct” counts the replies whose string-matched label agrees with the judgment.

component	“refused”: truly refused	“answered”: complied / partial / refused	harmful help
surface rewording	45 / 45	13 / 23 / 7	4 of 45
jailbreak (harmful)	39 / 45	6 / 24 / 11	5 of 45
persuasion framing	17 / 39	20 / 17 / 1	4 of 45

Appendix

Table 10: External labels of replies to harmful reworded prompts (three models, up to 15 replies per model, component and string-matched label). The second column gives, of the replies string matching calls refusals, how many were judged refusals, the third gives how the replies string matching calls answers were judged, and the last gives how many of those “answers” gave harmful help.

	$t_{\mathrm{inst}}$				$t_{\mathrm{post}}$
model	harmful	harmless	harm	refusal	harmful	harmless	harm	refusal
Llama-3.2-3B	0.77	0.59	0.81	0.81	0.96	0.82	0.93	0.95
Llama-3-8B	0.83	0.77	0.80	0.86	0.97	0.92	0.94	0.97
Qwen2.5-3B	0.58	0.57	0.71	0.66	0.87	0.86	0.91	0.92
Qwen2.5-7B	0.67	0.62	0.79	0.77	0.91	0.89	0.93	0.95
Qwen2.5-14B	0.71	0.54	0.83	0.81	0.87	0.83	0.95	0.95

Appendix

Table 11: Latent Guard (original recipe) per model on the behaviour-labelled set. “Harmful” and “harmless” give the within-class AUROC for the model’s own refusal, that is, how well the score separates refused from answered prompts inside each harm class, and “harm” and “refusal” give the AUROC over all prompts for the harm label and for the model’s own refusal.

model	passed	flagged	sum	model	passed	flagged	sum
Qwen3-32B	0.80	0.03	0.83	OLMo-3-32B	0.33	0.45	0.78
Qwen3-30B-A3B	0.73	0.09	0.82	Gemma-4-26B-A4B	0.28	0.68	0.96
Qwen3-8B	0.62	0.26	0.88	OLMoE-1B-7B	0.27	0.59	0.86
Llama-3-8B	0.58	0.36	0.94	Qwen2.5-3B	0.20	0.74	0.94
Qwen2.5-14B	0.51	0.28	0.79	Gemma-3-27B	0.16	0.74	0.90
Llama-3.2-3B	0.49	0.43	0.92	Gemma-3-4B	0.12	0.79	0.91

Appendix

Table 12: Latent Guard (original recipe,

t_{\mathrm{inst}}

) at its published threshold (

\Delta_{\textsc{LG}}\geq 0

) on the matched WildJailbreak sets. “Passed” is the share of disguised-harmful prompts not flagged and “flagged” the share of disguised-harmless prompts flagged. Both are errors, and a classifier that flags at random gives a sum of 1.

model	FPR / TPR	$t_{\mathrm{inst}}$	$\Delta$ at $t_{\mathrm{inst}}$	$t_{\mathrm{post}}$	$\Delta$ at $t_{\mathrm{post}}$	$h_{\mathrm{ref}}$ / $h_{\mathrm{ans}}$
Llama-3.2-3B	.028 / .355	.188	$-.168$ [ $-.207,-.103$ ]	.426	$+.071$ [ $+.025,+.120$ ]	.89 / .17
Llama-3-8B	.155 / .637	.341	$-.296$ [ $-.350,-.235$ ]	.606	$-.031$ [ $-.057,+.007$ ]	.88 / .12
Qwen2.5-3B	.049 / .281	.168	$-.113$ [ $-.169,-.060$ ]	.320	$+.039$ [ $-.024,+.102$ ]	.87 / .10
Qwen2.5-7B	.022 / .255	.132	$-.123$ [ $-.175,-.062$ ]	.329	$+.074$ [ $-.012,+.119$ ]	.96 / .11
Qwen2.5-14B	.013 / .241	.146	$-.095$ [ $-.162,-.053$ ]	.317	$+.077$ [ $+.025,+.135$ ]	.94 / .12
Gemma-3-4B	.035 / .306	.213	$-.093$ [ $-.151,-.046$ ]	.439	$+.133$ [ $+.096,+.165$ ]	.95 / .22

Appendix

Table 13: Latent Guard (original recipe) per model on

\mathcal{D}_{\mathrm{rew}}

. FPR / TPR is the model’s refusal rate on benign and on harmful reworded prompts, that is, its false-alarm and hit rates, the

t_{\mathrm{inst}}

and

t_{\mathrm{post}}

columns give the share of harmful prompts Latent Guard catches when its threshold flags as many benign prompts as the model refused,

\Delta

is that share minus the refusal’s TPR with its 95% interval, and

h_{\mathrm{ref}}

h_{\mathrm{ans}}

are the template-token reader’s hit rates on harmful prompts the model refused and answered.

Figure 7: The left panel shows pooled AUROC on the rewording set at

t_{\mathrm{inst}}

against curated training size, with medians as thick lines, one thin line per model, a star for Latent Guard on the original recipe, markers at 2,000 for the embedding classifiers and a dotted line for Llama Guard 3. The right panel shows Latent Guard at its published threshold on the matched WildJailbreak sets, one point per model, and every model lies near the line on which passed + flagged = 1.

Figure 8: The top panel shows the AUROC drop from base to reworded request, each against the same plain reference (medians), and the bottom panel shows the persuasion-framing drop per model, grouped by family, where only the probe trained on 2,000 examples per class removes it.

Figure 9: The left panel shows the hit rate on harmful prompts the model answered (

h_{\mathrm{ans}}

) against those it refused (

h_{\mathrm{ref}}

) at the matched threshold, with lines of equal gain over the refusal at

\hat{\pi}=0.32

, the solid one being the refusal itself, and the shaded region giving the ratios reported by Karne (2026) . The right panel shows per-model paired differences with 95% intervals, for the linear probe minus the embedding classifier at 2,000 examples per class in pooled AUROC at

t_{\mathrm{inst}}

, and for Latent Guard at

t_{\mathrm{post}}

minus the first-token refusal probability in harmful prompts caught at the matched rate.

pos.	reader	training	$\Pi$	$\beta_{s}$	caught	$\Delta$ vs refusal
$t_{\mathrm{inst}}$	Latent Guard	$\mathcal{S}_{0}$	$+0.050$ [0.038, 0.069]	0.32 9 $\uparrow$ 2 $\downarrow$	0.225	$-0.102$ 0 $\uparrow$ 12 $\downarrow$
$t_{\mathrm{inst}}$	Latent Guard	$\mathcal{S}_{100}$	$+0.051$ [0.039, 0.070]	0.34 10 $\uparrow$ 2 $\downarrow$	0.230	$-0.099$ 0 $\uparrow$ 13 $\downarrow$
$t_{\mathrm{inst}}$	Latent Guard	$\mathcal{S}_{500}$	$+0.070$ [0.052, 0.086]	0.43 10 $\uparrow$ 2 $\downarrow$	0.216	$-0.105$ 0 $\uparrow$ 13 $\downarrow$
$t_{\mathrm{inst}}$	Latent Guard	$\mathcal{S}_{2000}$	$+0.103$ [0.083, 0.127]	0.69 12 $\uparrow$ 1 $\downarrow$	0.241	$-0.092$ 0 $\uparrow$ 11 $\downarrow$
$t_{\mathrm{inst}}$	LG-z (late)	$\mathcal{S}_{0}$	$+0.117$ [0.099, 0.140]	0.59 14 $\uparrow$ 0 $\downarrow$	0.287	$-0.046$ 2 $\uparrow$ 6 $\downarrow$
$t_{\mathrm{inst}}$	LG-z (late)	$\mathcal{S}_{100}$	$+0.117$ [0.097, 0.141]	0.63 14 $\uparrow$ 0 $\downarrow$	0.267	$-0.060$ 1 $\uparrow$ 7 $\downarrow$

Appendix

Table 14: All readers on the rewording set.

\Pi

is the symmetric disagreement statistic, median over 78 model pairs with its 95% interval, which is 0 for any score shared across models.

\beta_{s}

is the coefficient of the score in the refusal regression with harm and consensus fixed and same-developer models left out of the consensus (models above / below 0), where model-blind scores also obtain positive values. “Caught” is the share of harmful prompts flagged when the threshold flags as many benign prompts as the model refused, and

\Delta

is caught minus the refusal’s hit rate (models above / below). RvA contrast and projection are refusal directions built from the model’s own refused and answered prompts on

\mathcal{D}_{\mathrm{beh}}

(within the harmful class, or balanced over both classes), the Arditi-style projection is a projection on the harmful-minus-harmless mean difference of the original recipe at one layer, and the sensitivity row is the linear probe with the first-maximum selection rule ( Appendix C ). The last block lists reference scores that read no activations of the study model.

reader	no control	+ Jev	+ consensus	+ both
Latent Guard	0.86 (13)	0.25 (6)	0.14 (6)	0.11 (3)
LG-z (late)	1.27 (15)	0.72 (14)	0.36 (10)	0.41 (8)
linear probe	1.25 (15)	0.51 (14)	0.43 (10)	0.34 (8)
TF-IDF	0.39 (14)	0.08 (2)	0.09 (4)	0.11 (2)

Appendix

Table 15: Refusal coefficient

\beta_{s}

t_{\mathrm{inst}}

(median over models, with the number of models whose interval is above 0 in parentheses) with no control, with Jev’s harm probability added to the regression, with consensus added (as in the main text), and with both. At

t_{\mathrm{post}}

, Latent Guard’s coefficient with both controls is 1.90 (15 of 15).

Figure 10: The top panel shows the refusal coefficient of Latent Guard’s score per model without controls (tick) and with Jev and consensus fixed (dot, 95% interval). The bottom panel shows the within-class AUROC for the model’s own refusal, comparing refused and answered prompts only within the same fifth of Jev’s harm probability and the same third of consensus (95% interval), with Jev’s own within-class AUROC, which never sees the model, dashed.

LLM Safety Monitoring

See all 54 papers

Sep 16, 2026

Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models

Autonomous systems increasingly rely on Large Language Models (LLMs) yet the safety infrastructure surrounding these models introduces latency and compute overhead. This limits utility in resource-constrained, time-critical deployments. Existing external guardrail models remain blind to the model's internal workings, creating a fundamental assurance gap. We ask: does the model already know when the content is harmful? We extract activations from LLaMA-3.1-8B and train lightweight MLP classifier probes (12.6M parameters) to detect harmful prompts. Evaluated on WildJailbreak, Beavertails, and AEGIS 2.0, our probes achieve F1 scores of 99%, 83%, and 84%, respectively competitive with 1000x larger guard models while cutting latency and compute costs.

Alizishaan Khatri, Chiquita Prabhu, Omkar Neogi

Wrynx Inc. · Independent Researcher

Safety Filtering LLM Safety Language Model Probing LLM Safety Monitoring Trustworthy AI

Ask PDF

Jul 16, 2025

LLMs Encode Harmfulness and Refusal Separately

LLMs are trained to refuse harmful instructions, but do they truly understand harmfulness beyond just refusing? Prior work has shown that LLMs' refusal behaviors can be mediated by a one-dimensional subspace, i.e., a refusal direction. In this work, we identify a new dimension to analyze safety mechanisms in LLMs, i.e., harmfulness, which is encoded internally as a separate concept from refusal. There exists a harmfulness direction that is distinct from the refusal direction. As causal evidence, steering along the harmfulness direction can lead LLMs to interpret harmless instructions as harmful, but steering along the refusal direction tends to elicit refusal responses directly without reversing the model's judgment on harmfulness. Furthermore, using our identified harmfulness concept, we find that certain jailbreak methods work by reducing the refusal signals without reversing the model's internal belief of harmfulness. We also find that adversarially finetuning models to accept harmful instructions has minimal impact on the model's internal belief of harmfulness. These insights lead to a practical safety application: The model's latent harmfulness representation can serve as an intrinsic safeguard (Latent Guard) for detecting unsafe inputs and reducing over-refusals that is robust to finetuning attacks. For instance, our Latent Guard achieves performance comparable to or better than Llama Guard 3 8B, a dedicated finetuned safeguard model, across different jailbreak methods. Our findings suggest that LLMs' internal understanding of harmfulness is more robust than their refusal decision to diverse input instructions, offering a new perspective to study AI safety.

Jiachen Zhao, Jing Huang, Zhengxuan Wu +2

Northeastern University · Stanford University

LLM Interpretability LLM Refusal Behavior LLM Safety LLM Jailbreak Attacks Causal Interventions in Language Models Refusal Behavior in Language Models

Ask PDF

Aug 8, 2026

Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families

Khatri et al. (2026) [DOI: 10.1109/DSN-W70714.2026.00027] show that lightweight MLP probes on final-layer activations of a single 8B model (LLaMA-3.1-8B) detect harmful prompts at F1 competitive with guard models 1000x larger, using one probe per benchmark. We reproduce this pipeline end-to-end and extend it along two axes the original study leaves open. First, we test whether the result generalizes across other model architecture and scale by training identical probes on activations from models like Gemma-4-E4B, Mistral-7B-v0.3, and Qwen2-7B, using the three benchmarks (WildJailbreak, BeaverTails, AEGIS 2.0). Second, we test how much of the reported performance is affected by non-determinism during inference by repeating extraction under five random seeds and measuring the variance of F1 scores. Our results reproduce the original LLaMA model benchmarks within 0.37 percentage points of the original F1 scores (and within 0.2 points on BeaverTails). We find that the original MLP probe architecture extends to other model families with F1 scores within a point of the values reported for LLaMA-3.1-8B. Our experiments varying seed values reveal an interesting observation: final token latent vectors remained the same for all tested architectures irrespective of the seed values used.

Alizishaan Khatri, Dun Li Chan

Wrynx Inc · INTI International College Penang

LLM Safety Benchmarks LLM Inference Language Model Safety Evaluation Language Model Probing

Ask PDF

Abstract

LLM Safety Monitoring

Explore similar work

Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models

LLMs Encode Harmfulness and Refusal Separately

Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families