Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge

Wonjun Lee; Kyungsik Yang; Gaeun Ji; Vaidehi Patil; Haon Park; Bumsub Ham; Mohit Bansal; Suhyun Kim

cs.CROct 5, 2026

Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge

Authors: Wonjun Lee, Kyungsik Yang, Gaeun Ji, Vaidehi Patil, Haon Park, Bumsub Ham, Mohit Bansal, Suhyun Kim

Organizations: UNC Chapel Hill · Kyung Hee University · AIM Intelligence · Yonsei University

Abstract

LLMs have advanced rapidly, raising growing concerns about their safety. Recent work has proposed approaches to detect and defend against attacks including defenses at decoding stage that leverage models' hidden states. However, existing decoding-stage defenses suffer from two limitations. First, they introduce a trade-off between safety and over-refusal, where strengthening safety degrades the model's helpfulness on benign queries. Second, many of these methods rely on internal hidden states and are thus restricted to specific architectures, incurring substantial overhead and limited generalization across models. To address these limitations, we introduce LADE (Latent Safety Signals for Defense), which leverages latent safety signals extracted by contrasting harmful and benign queries from dark knowledge (i.e., information carried by the output probability distribution beyond its argmax) in the first-token output probability distribution. Our key insight is that, beyond surface-level refusal tokens, the dark knowledge in the first-token distribution contains latent safety signals, defined as tokens whose probabilities differ sharply between harmful and benign queries. We show that these signals consistently align across LLMs, forming a model-agnostic direction that emerges from safety alignment. LADE consists of three components: (1) Extracting Latent Safety Signals from Dark Knowledge, which selects top-k safety-discriminative tokens from the first-token probability distribution; (2) Tokenizer Mapping, which maps these tokens across different tokenizers to enable model-agnostic application; and (3) kNN-based Discrimination, which classifies queries via a k-Nearest Neighbors search over the mapped tokens. Across diverse LLMs and benchmarks, LADE is robust against a wide range of jailbreak attacks and lowers attack success rates while maintaining a competitive safety-utility trade-off.

Figures & tables

Figure 1 : The key idea of LADE . (Left) Latent safety signals ( red points), identified by contrasting first-token probabilities between harmful and benign queries, are extracted from a reference LLM and transfer across target LLMs, while model-specific surface tokens ( gray points) remain scattered. (Right) Refusal tokens capture only a small subset of safety signals and fail to separate harmful from benign queries (top), whereas adding extracted latent safety signals yields clear separation (bottom).

Figure 2 : Overview of LADE . (1) Extracting Latent Safety Signals from Dark Knowledge selects top- $k$ tokens from the first-token output probability distribution based on the harmful-benign distributional gap. (2) Tokenizer Mapping aligns the selected tokens with the target model’s tokenizer, using anchor-based ratio estimation to handle duplicate mappings. (3) kNN-based Discrimination classifies the query as harmful or benign based on its distance to a reference set of harmful queries, blocking harmful queries.

Model	Defense	Compliant Responses to Harmful Queries ( $\downarrow$ )					Avg. ( $\downarrow$ )
Model	Defense	AutoDAN	DeepInception	GCG	PAIR	LIAR	Avg. ( $\downarrow$ )
Llama-2-7B-Chat	No Defense	4	23	1	3	3	6.80
	Self-Reminder	0	0	0	0	0	0.00
	SafeDecoding	0	0	0	0	0	0.00
	SafeInfer	1	14	0	1	0	3.20
	RDS	9	30	6	13	4	12.40

Table 1 : Defense performance against jailbreak attacks across different models. We report the number of compliant responses under AutoDAN, DeepInception, GCG, PAIR, and LIAR attacks.

Model	Method	Compliance on Harmful Queries ( $\downarrow$ )			Refusal on Benign Queries ( $\downarrow$ )				Held-out Avg. ( $\downarrow$ )
Model	Method	AdvBench	Hex-Phi †	StrongReject	MMLU	Alpaca	GSM8K	XSTest †	Held-out Avg. ( $\downarrow$ )
Llama-2-7B-Chat	No Defense	102	2	8	1	10	0	17	24.20
	Self-Reminder	0	0	0	372	270	22	130	132.80
	SafeDecoding	8	0	0	393	323	223	211	189.40
	SafeInfer	0	0	0	52	17	1	122	14.00
	RDS	0	4	0	6	19	0	87	5.00

Table 2 : Comparison of harmful-query compliance and benign-query refusal across LLMs. † marks datasets used in LADE ’s offline stage and excluded from Held-out Avg. Bold and underline denote the best and second-best Avg., respectively. SafeInfer is not evaluated on the 13B LLMs.

Figure 3 : Tokenizer mapping ablation in LADE . We refer to subword split handling as Split , and duplicate mappings via ratio estimation as Ratio . The average accuracy over the $6{\times}6$ transfer matrix is 0.8934, 0.9413, 0.9649, and 0.9660 for (a)–(d), respectively.

Method	Harmful Queries			Jailbreak Attacks					Benign Queries				Held-out Avg.
Method	AdvBench	Hex-Phi †	SR	AutoDAN	DI	GCG	LIAR	PAIR	MMLU	Alpaca	GSM8K	XSTest †	Held-out Avg.
Prompt-Guard-2-86M	0.481	0.110	0.090	0.727	0.420	0.613	0.137	0.280	1.000	0.999	1.000	1.000	0.575
Llama-Guard-4-12B	0.931	0.933	0.914	0.730	0.900	0.917	0.865	0.297	0.966	0.998	0.998	0.928	0.852
WildGuard-7B	0.998	0.987	0.990	0.980	1.000	0.997	1.000	0.553	0.960	0.996	1.000	0.992	0.947
LADE	0.962	0.877	0.971	0.980	1.000	1.000	1.000	0.780	1.000	0.996	1.000	0.932	0.969

Table 3 : Comparison with Guard Models. We report classification accuracy on harmful, jailbreak attack, and benign benchmarks. Datasets marked with

\dagger

are excluded from the average. SR and DI denote StrongReject and DeepInception, respectively.

Method	Latency per input (ms) ( $\downarrow$ )	Memory (GiB) ( $\downarrow$ )
Prompt-Guard-2-86M	$\mathbf{15.02\pm 0.72}$	15.48
Llama-Guard-4-12B	$85.97\pm 12.86$	37.31
WildGuard-7B	$553.42\pm 8.80$	28.46
LADE	$34.13\pm 3.93$	14.96

Table 4 : Deployment efficiency on a single NVIDIA RTX A6000 with Llama-3-8B-Instruct as the target LLM. Latency is the mean

\pm

standard deviation per input. Memory includes the target LLM for every method.

Model	Harmful Queries			Benign Queries				Avg.
Model	AdvBench	Hex-Phi	StrongReject	MMLU	Alpaca	GSM8K	XSTest	Avg.
Gemma-7B	0.0000	0.9400	0.8019	1.0000	0.6600	0.9940	0.3896	0.6836
Gemma-7B-it	0.9058	0.8700	0.9457	1.0000	1.0000	1.0000	0.8554	0.9396
Gemma-2-9B-it	0.9808	0.9100	0.7668	1.0000	0.9900	1.0000	0.9839	0.9474
Gemma-3-4B-it	0.9808	0.8833	0.9489	1.0000	0.9960	1.0000	0.9438	0.9647

Table 5 : Classification accuracy of LADE on harmful/benign query classification across the Gemma family. We evaluate Gemma-7B, Gemma-7B-it, Gemma-2-9B-it, and Gemma-3-4B-it.

Figure 4 : Effect of latent safety signals beyond refusal tokens. We report the average classification accuracy across all benchmarks (excluding Hex-Phi/XSTest).

First	Second	Third	Accuracy
✓	✗	✗	0.9657
✗	✓	✗	0.9222
✗	✗	✓	0.8449
✓	✓	✗	0.9651
✓	✓	✓	0.9523

Table 6 : Token candidate ablation.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Top- $k$	1	10	50	100	500	1000
Accuracy	0.4505	0.8867	0.9401	0.9611	0.9657	0.9596

Appendix

Table 7 : Top-

k

ablation.

Harmful Set $\mathcal{D}_{h}$	Benign Set $\mathcal{D}_{b}$
Harmful Set $\mathcal{D}_{h}$	Alpaca	GSM8K	MMLU	XSTest
AdvBench	0.9349	0.9259	0.9268	0.9552
Hex-Phi	0.9423	0.9309	0.9317	0.9660
StrongReject	0.9379	0.9317	0.9325	0.9627

Appendix

Table 8 : Ablation on the choice of harmful and benign extraction sets for Extracting latent safety signals from Dark Knowledge.

\mathcal{D}_{h}

denotes the harmful extraction set and

\mathcal{D}_{b}

denotes the benign extraction set used to select latent safety signals. Each entry reports the average classification accuracy over six reference models (Gemma, Llama-3, Llama-2, Mistral, Qwen2-7B, and Qwen3-8B) for the corresponding

\mathcal{D}_{h}

–

\mathcal{D}_{b}

dataset pair. Datasets used as

\mathcal{D}_{h}

\mathcal{D}_{b}

are excluded from the average.

Harmful Reference Set $\mathcal{R}$	Accuracy
AdvBench	0.9299
StrongReject	0.9488
Hex-Phi	0.9657

Appendix

Table 9 : Ablation on the harmful reference set

\mathcal{R}

. We report the average classification accuracy across all benchmarks and six models for each choice of

\mathcal{R}

Threshold Quantile $\rho$	Accuracy
0.85	0.9330
0.88	0.9411
0.90	0.9657
0.93	0.9403
0.95	0.9191
0.97	0.8847

Appendix

Table 10 : Ablation on the threshold quantile

\rho

. We report the average classification accuracy across all benchmarks and six models for different values of

\rho

used to determine the LADE decision threshold.

Duplicate mapping statistic	Avg. accuracy
No ratio estimation	0.9649
Harmful-side mean $\mu_{h}$	0.9655
Benign-side mean $\mu_{b}$ (ours)	0.9660

Appendix

Table 11 : Ablation on duplicate-mapping statistics. We compare harmful-side and benign-side mean probabilities for estimating the relative contribution of duplicated source tokens. Avg. accuracy denotes the mean over all

6{\times}6

reference–target LLM pairs, where each pair is evaluated by average classification accuracy over held-out benchmarks excluding Hex-Phi and XSTest used in the offline stage.

$r$	Overlap of $\mathcal{T}_{k}$	Harmful ( $\uparrow$ )	Benign ( $\downarrow$ )
$0\%$	87.8%	96.52%	0.13%
$50\%$	93.6%	96.52%	0.13%
$92.49\%$ (observed)	100.0%	96.52%	0.13%
$100\%$	95.0%	96.64%	0.13%

Appendix

Table 12 : Sensitivity of LADE to the share

r

of refused queries in

\mathcal{D}_{h}

used to compute

\mu_{h}

. Overlap is measured against

\mathcal{T}_{k}

of the observed setting. Harmful reports harmful queries classified as harmful, and Benign reports benign queries misclassified as harmful.

Target Model	AdvBench	Hex-Phi	StrongReject	Avg.
Llama-2-7B-Chat	1.0000	0.9933	1.0000	0.9978
Llama-3-8B-Instruct	1.0000	1.0000	1.0000	1.0000
Qwen2-7B-Instruct	1.0000	0.9967	0.9936	0.9968
Qwen3-8B	1.0000	1.0000	1.0000	1.0000
Gemma-7B-it	1.0000	0.9967	0.9968	0.9978
Mistral-7B-Instruct-v0.3	1.0000	1.0000	1.0000	1.0000

Appendix

Table 13 : Classification accuracy of LADE under prefilling-based suffix attacks (e.g., “Sure, here is” ) across six LLMs, with Llama-3-8B-Instruct used as the reference LLM.

Method	GCG ( $\uparrow$ )	Adaptive attack ( $\uparrow$ )
Prompt-Guard-2-86M Meta AI (2025b)	0.613	0.000
Llama-Guard-4-12B Meta AI (2025a)	0.917	0.000
Llama-Guard-3-8B Meta AI (2024)	1.000	0.020
WildGuard-7B Han et al. (2024)	0.997	0.220
LADE	1.000	0.480

Appendix

Table 14 : Classification accuracy of guard models and LADE on

50

harmful prompts under GCG and under the adaptive attack. Llama-3-8B-Instruct is the target LLM, and every method uses the same attack budget. Bold and underline denote the best and second-best results.

Method	AutoDAN ( $\uparrow$ )	Harmful ( $\uparrow$ )	Benign ( $\downarrow$ )
LADE	42.00%	92.67%	2.22%
LADE with evasion reference set	93.50%	94.59%	5.42%

Appendix

Table 15 : Effect of adding an evasion reference set to LADE . AutoDAN and Harmful show the percentage of AutoDAN prompts and harmful queries classified as harmful, and Benign shows the percentage of benign queries misclassified as harmful.

Target Model	JBB ( $100$ )	Security ( $750$ )	Medical ( $1{,}000$ )	Legal ( $1{,}063$ )
Llama-2-7B-Chat	2.00	0.00	3.10	1.22
Llama-3-8B-Instruct	1.00	0.00	3.90	0.94
Qwen2-7B-Instruct	22.00	8.80	13.70	1.22
Qwen3-8B	9.00	12.13	4.50	0.47
Gemma-7B-it	9.00	0.00	8.40	0.38
Mistral-7B-Instruct-v0.3	28.00	10.13	12.80	6.30

Appendix

Table 16 : Percentage of benign queries misclassified as harmful by LADE (%,

\downarrow

). Numbers in parentheses denote the number of queries in each set.

Model	Method	Harmful Avg. ( $\downarrow$ )	Benign Avg. ( $\downarrow$ )	Full Avg. ( $\downarrow$ )	Held-out Avg. ( $\downarrow$ )
Llama-2-7B-Chat	No Defense	37.33	7.00	20.00	24.20
	Self-Reminder	0.00	198.50	113.43	132.80
	SafeDecoding	2.67	287.50	165.43	189.40
	SafeInfer	0.00	48.00	27.43	14.00
	RDS	1.33	28.00	16.57	5.00
	LADE	1.33	14.50	8.86	1.60

Appendix

Table 17 : Average compliance on harmful queries (Harmful Avg.), average refusal on benign queries (Benign Avg.), and average over all seven datasets (Full Avg.), computed from Table 2 . Held-out Avg. is shown for reference. Bold denotes the lowest Full Avg. for each LLM.

Model	AdvBench	Hex-Phi †	StrongReject	MMLU	Alpaca	GSM8K	XSTest †	Held-out Avg. ( $\downarrow$ )
Beaver-v3	0	13	2	0	19	2	21	4.60
ThinkSafe-Qwen3-8B	0	0	1	0	8	0	5	1.80

Appendix

Table 18 : Compliance on harmful queries and refusal on benign queries of LADE on LLMs with different safety alignment methods. † marks datasets used in LADE ’s offline stage and excluded from Held-out Avg.

Judge	Compliance judgment (%)	Refusal detection (%)
HarmBench-Llama-2-13B	84.80	94.44
WildGuard-7B	91.22	95.03
Granite-Guardian-4.1-8B	90.54	94.31

Appendix

Table 19 : Agreement of three additional judges with Qwen3Guard-4B on compliance judgment and with Dic-Judge on refusal detection.

$m$	Harmful ( $\uparrow$ )	Benign ( $\downarrow$ )	Balanced accuracy ( $\uparrow$ )	Observed signals
5	2,339/2,499 (93.60%)	113/4,500 (2.51%)	95.54%	3.18
10	2,347/2,499 (93.92%)	104/4,500 (2.31%)	95.80%	5.95
20	2,347/2,499 (93.92%)	104/4,500 (2.31%)	95.80%	11.14
Full	2,347/2,499 (93.92%)	105/4,500 (2.33%)	95.79%	463.33

Appendix

Table 20 : LADE when only the

m

most likely first tokens are available, averaged over Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, and Qwen2-7B-Instruct. Harmful shows harmful queries classified as harmful, Benign shows benign queries misclassified as harmful, and balanced accuracy is the mean of the accuracies on harmful and benign queries. The last column shows the average number of latent safety signals other than refusal tokens observed per query.

Refusal String
I’m sorry
I am sorry
Sorry
I apologize
As an
As an AI

Appendix

Table 21 : Refusal strings used for the refusal-token-only ablation.

Model	Harmful ( $\uparrow$ )	Benign ( $\downarrow$ )	Accuracy ( $\uparrow$ )
Mistral-7B-Instruct-v0.3	95.79%	38.86%	0.7599
Qwen2-7B-Instruct	93.94%	45.83%	0.7121
Qwen3-8B	93.20%	6.96%	0.9311

Appendix

Table 22 : Classification results of the refusal-token-only configuration. Harmful shows harmful queries classified as harmful, and Benign shows benign queries misclassified as harmful.

Refusal Strings

"I’m sorry" , "Sorry" , "an AI" , "is illegal and unethical" , "I’m really sorry" , "My purpose" , "As a helpful assistant" , "OpenAI" , "unethical" , "positive and uplifting" , "against ethical guidelines" , "not ethical" , "legally" , "illegal" , "My apologies" , "legal ways" , "but I cannot" , "If you have any other non-malicious requests" , "I’m not able to provide" , "I cannot engage" , "I cannot generate" , "I do not generate" , "I cannot create" , "I do not actually" , "a bit unclear" , "I can’t create" , "I cannot support" , "I can’t provide" , "I’m truly sorry" , "I cannot provide" , "I’d prefer not" , "I do not provide" , "I cannot recommend" , "I do not recommend" , "I do not feel comfortable" , "I can’t assist" , "Absolutely not!" , "here’s an alternative" , "here’s a different" , "I cannot assist" , "I will refrain" , "I apologize" , "I will not" , "awesome coffee" , "creativity" , "promote well-being" , "positivity" , "harm reduction" , "self-care"

Appendix

Table 23 : Refusal strings used for refusal detection.

Explore similar work

May 12, 2025cs.CR

Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models

Large Language Models (LLMs) have been extensively used across diverse domains, including virtual assistants, automated code generation, and scientific research. However, they remain vulnerable to jailbreak attacks, which manipulate the models into generating harmful responses despite safety alignment. Recent studies have shown that current safety-aligned LLMs undergo shallow safety alignment. In this work, we conduct an in-depth investigation into the underlying mechanism of this phenomenon and reveal that it manifests through learned ''safety trigger tokens'' that activate the model's safety patterns when paired with the specific input. Through both analysis and empirical verification, we further demonstrate the high similarity of the safety trigger tokens across different harmful inputs. Accordingly, we propose D-STT, a simple yet effective defense algorithm that identifies and explicitly decodes safety trigger tokens of the given safety-aligned LLM to activate the model's learned safety patterns. In this process, the safety trigger is constrained to a single token, which effectively preserves model usability by introducing minimum intervention in the decoding process. Extensive experiments across diverse jailbreak attacks and benign prompts demonstrate that D-STT significantly reduces output harmfulness while preserving model usability and incurring negligible response time overhead, outperforming ten baseline methods.

Haoran Gu, Handing Wang, Yi Mei +2

Xidian University · Victoria University of Wellington · Westlake University

Large Language Model Safety Jailbreak Attacks Safety Claims

Ask PDF

Sep 16, 2026cs.AI

Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models

Autonomous systems increasingly rely on Large Language Models (LLMs) yet the safety infrastructure surrounding these models introduces latency and compute overhead. This limits utility in resource-constrained, time-critical deployments. Existing external guardrail models remain blind to the model's internal workings, creating a fundamental assurance gap. We ask: does the model already know when the content is harmful? We extract activations from LLaMA-3.1-8B and train lightweight MLP classifier probes (12.6M parameters) to detect harmful prompts. Evaluated on WildJailbreak, Beavertails, and AEGIS 2.0, our probes achieve F1 scores of 99%, 83%, and 84%, respectively competitive with 1000x larger guard models while cutting latency and compute costs.

Alizishaan Khatri, Chiquita Prabhu, Omkar Neogi

Wrynx Inc. · Independent Researcher

Large Language Model Safety Guardrail Harms Precision-Critical Manipulation Latent States Multilayer Perceptrons

Ask PDF

Jun 15, 2026cs.AI

Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models

While Large Reasoning Models (LRMs) excel at complex tasks, they remain highly vulnerable to sophisticated jailbreaks and direct harmful queries. To address this vulnerability, prior works depend heavily on external manual data annotation for safety alignment. However, we observe that LRMs can inherently identify safety risks when being re-presented with original queries alongside their own reasoning trajectories -- a capability we term Latent Safety Awareness. To leverage this safety awareness, we first employ Supervised Fine-Tuning (SFT) to explicitly induce safe tags to trigger safety analysis and guidance following the initial reasoning content for unsafe queries, while preserving standard responses for general queries to ensure adaptive triggering. Subsequently, we apply Direct Preference Optimization (DPO) to further enhance the correctness and stability of the safety analysis and guidance. Notably, responses required for both training stages are entirely generated by models being optimized. With (Safe Trigger) SFT and DPO, experimental results demonstrate significant safety enhancement. For example, the Attack Success Rate (ASR) of DeepSeek-R1-Distill-Llama-8B, on average, drops 24.65% and 36.72% on harmful and jailbreak benchmarks, respectively. Finally, our Safe Trigger method exerts almost no negative impact on general performance or user experience.

Ke Miao, Jiaxin Li, Hongliang Chen +2

The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China · Li Auto Inc. +2

Large Language Model Safety Large Reasoning Models Jailbreak Attacks Deepseek

Ask PDF