SelfGrader: LLM Jailbreak Detection via Anchored Token-Level Logits

Zikai Zhang; Rui Hu; Olivera Kotevska; Jiahao Xu

cs.CRApr 1, 2026

SelfGrader: LLM Jailbreak Detection via Anchored Token-Level Logits

Authors: Zikai Zhang, Rui Hu, Olivera Kotevska, Jiahao Xu

Organizations: Department of Computer Science and Engineering, University of Nevada, Reno, Reno, USA · Oak Ridge National Laboratory, Oak Ridge, USA

Abstract

Large Language Models (LLMs) are powerful tools for answering user queries, yet they remain highly vulnerable to jailbreak attacks. Existing guardrail methods typically rely on internal features or textual responses to detect malicious queries, which either introduce substantial latency or suffer from randomness in text generation. To overcome these limitations, we propose SelfGrader, a lightweight guardrail method that formulates jailbreak detection as a numerical grading problem using anchored token-level logits. Specifically, SelfGrader evaluates the safety of a user query within a compact set of numerical tokens (NTs) (e.g., 0-9) and interprets their logit distribution as an internal safety signal. To align these signals with the target safety rubric, SelfGrader constructs Probably Approximately Correct-guided ICL anchor examples and introduces a dual-perspective scoring rule that considers both the maliciousness and benignness of the query, yielding a stable and interpretable score that reflects harmfulness and reduces the false positive rate simultaneously. Extensive experiments across diverse jailbreak benchmarks, adaptive attacks, benign prompt benchmarks, multiple LLMs, and state-of-the-art guardrail baselines demonstrate that SelfGrader achieves strong robustness with low false positive rates, memory overhead, and latency.

Figures & tables

Guardrails	Manual (IJP)	GCG	AutoDAN	DrAttack	MultiJail	ActorAttack	Average	Latency	Memory
Guardrails	Manual (IJP)	GCG	AutoDAN	DrAttack	MultiJail	ActorAttack	Average	(Sec.)	Overhead (MB)
No Defense	7.80/-	13.00/-	2.00/-	10.00/-	4.44/-	22.66/-	9.98/-	0.86	-
Perplexity Filter	7.80 $\pm$ 0.00 /100.00 $\pm$ 0.00	10.00 $\pm$ 0.00 /62.00 $\pm$ 0.00	2.00 $\pm$ 0.00 /100.00 $\pm$ 0.00	10.00 $\pm$ 0.00 /100.00 $\pm$ 0.00	4.44 $\pm$ 0.00 /100.00 $\pm$ 0.00	22.67 $\pm$ 0.00 /100.00 $\pm$ 0.00	9.49/93.67	0.20	21489.17
GradSafe	7.70 $\pm$ 0.00 /61.00 $\pm$ 0.00	13.00 $\pm$ 0.00 /78.00 $\pm$ 0.00	1.00 $\pm$ 0.00 /5.00 $\pm$ 0.00	10.00 $\pm$ 0.00 /42.00 $\pm$ 0.00	4.44 $\pm$ 0.00 /91.75 $\pm$ 0.00	22.67 $\pm$ 0.00 /99.33 $\pm$ 0.00	9.80/62.85	3.23	50586.82
GradientCuff	2.15 $\pm$ 1.27 /7.10 $\pm$ 4.03	7.25 $\pm$ 2.17 /14.25 $\pm$ 3.70	0.50 $\pm$ 0.50 /2.50 $\pm$ 1.12	2.50 $\pm$ 1.66 /5.75 $\pm$ 3.49	1.59 $\pm$ 1.21 /16.11 $\pm$ 6.72	0.33 $\pm$ 0.39 /0.33 $\pm$ 0.39	2.39/7.67	39.76	1805.36
Token Highlighter	3.78 $\pm$ 0.29 /10.48 $\pm$ 0.75	7.25 $\pm$ 0.83 /14.00 $\pm$ 1.58	0.75 $\pm$ 0.43 /6.75 $\pm$ 0.83	6.50 $\pm$ 0.50 /9.50 $\pm$ 1.50	1.75 $\pm$ 0.27 / 4.37 $\pm$ 0.61	0.00 $\pm$ 0.00 /0.17 $\pm$ 0.00	3.34/7.55	27.78	29283.44

Table 1: Comparison of different defense methods against common jailbreak attacks and their variants on Llama-3-8B-Instruct. Defense performance is reported as ASR (

\downarrow

) / PGR (

\downarrow

) in %. Gray

\pm

values denote standard deviations across four runs with different seeds. (

\star

) denotes the method shows over-refusal and has nearly 100% FPR.

Figure 1: Average ASR vs. FPR of different defense methods on LLama-3-8B-Instruct model.

Guardrails	RS	Emoji	PIA	FITD
Llama Guard (Pre)	5.70/47.10	6.72/90.48	1.33/100.00	32.50/65.00
SelfDefend (Direct)	2.30/29.20	3.76/43.06	0.00 /19.33	0.00 / 0.00
SelfGrader	0.00 / 0.00	0.36 / 2.18	0.00 / 5.33	0.00 / 0.00

Table 2: Robustness evaluation of SelfGrader. Defense performance is reported as ASR (

\downarrow

) / PGR (

\downarrow

) in %.

Method	ASR	PGR	FPR
SelfGrader	0.11	2.51	1.91
w/o ICL Anchor Examples	7.32	70.27	0.70
w/o Benign View	0.05	1.16	29.26
w/o Malicious View	8.08	48.47	0.17

Table 3: Ablation on ICL anchors and DPL scoring.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	# Prompts	Jailbreak Methods
JailbreakHub Shen et al. (2024)	1,000	IJP Shen et al. (2024) , Emoji Attack Wei et al. (2024)
JailbreakBench Chao et al. (2024)	100	GCG Zou et al. (2023) , AutoDAN Liu et al. (2023) , TAP Mehrotra et al. (2024) , LLM-Fuzzer Yu et al. (2024) , DrAttack Li et al. (2024) , X-Teaming Rahman et al. (2025) , Emoji Attack Wei et al. (2024) , Foot-in-the-door-Jailbreak (FITD) Weng et al. (2025) , Random Search Andriushchenko et al. (2025)
MultiJail Deng et al. (2023)	315	MultiJail Deng et al. (2023) , Emoji Attack Wei et al. (2024)
SafeMTData Ren et al. (2024)	600	ActorAttack Ren et al. (2024) , Emoji Attack Wei et al. (2024)
AlpacaEval Li et al. (2023)	805	Benign prompts (Instruction-Following)
OR-Bench Cui et al. (2024)	1,000	Benign prompts (Over-Refusal)

Appendix

Table 4: The details of our collected benchmarks.

Anchors	MBR ( $\uparrow$ )		FPR ( $\downarrow$ )
Anchors	MedQA	PubMedQA	AlpacaEval	GSM8K
General	0.42	0.67	1.36	0.00
Medical	94.31	92.67	1.40	0.00

Appendix

Table 5: Case study under a stricter medical safety rubric. Re-anchoring with medical anchors substantially increases medical blocking while preserving non-medical FPR.

Guardrails	Manual (IJP)	GCG	AutoDAN	DrAttack	MultiJail	ActorAttack	Average	Latency (Sec.)	Memory Overhead (MB)
Qwen2.5-7B-Instruct (No Defense)	25.70/-	13.00/-	10.00/-	19.00/-	5.40/-	16.50/-	14.93/-	5.29	-
Perplexity Filter	25.70/100.00	5.00/62.00	10.00/100.00	19.00/100.00	5.40/100.00	16.50/100.00	14.93/93.67	0.10	13134.88
GradSafe	OOM	OOM	OOM	OOM	OOM	OOM	-	-	-
GradientCuff	10.20/27.70	3.00/12.00	8.00/24.00	16.00/52.00	5.40/88.25	16.50/97.54	9.85/50.25	80.70	1672.50
Token Highlighter	3.20/18.90	7.00/15.00	0.50/6.50	6.00/58.00	1.90/45.00	0.00/0.20	3.10/23.93	25.40	29500.00
Prompt Guard	0.00/0.00	0.00/8.00	7.00/42.00	19.00/94.00	5.40/100.00	16.50/100.00	7.98/57.33	8.48	1154.58

Appendix

Table 6: Comparison of different defense methods against multiple common jailbreak attacks on Qwen2.5-7B-Instruct. Defense performance is reported as ASR (

\downarrow

) / PGR (

\downarrow

) in %. OOM denotes out of memory. The average is computed only when all attack results are available. The number of NTs is

Q=100

Guardrails	Manual (IJP)	GCG	AutoDAN	DrAttack	MultiJail	ActorAttack	Average	Latency (Sec.)	Memory Overhead (MB)
Qwen3.5-9B (No Defense)	0.00/-	0.00/-	0.00/-	0.00/-	0.32/-	0.33/-	0.11/-	9.50	-
Perplexity Filter	0.00/100.00	0.00/62.00	0.00/100.00	0.00/100.00	0.32/100.00	0.33/100.00	0.11/93.67	0.10	13073.10
GradSafe	OOM	OOM	OOM	OOM	OOM	OOM	-	-	-
GradientCuff	0.00/100.00	0.00/100.00	0.00/100.00	0.00/100.00	0.32/100.00	0.33/100.00	0.10/100.00	73.69	1427.98
Token Highlighter	0.00/100.00	0.00/100.00	0.00/100.00	0.00/100.00	0.32/100.00	0.33/100.00	0.11/100.00	12.84	29280.00
Prompt Guard	0.00/0.00	0.00/8.00	0.00/42.00	0.00/94.00	0.32/100.00	0.33/97.55	0.10/56.92	7.77	1174.32

Appendix

Table 7: Comparison of different defense methods against multiple common jailbreak attacks on Qwen3.5-9B. Defense performance is reported as ASR (

\downarrow

) / PGR (

\downarrow

) in %. OOM denotes out of memory. The number of NTs is

Q=100

Guardrails	Manual (IJP)	GCG	AutoDAN	DrAttack	MultiJail	ActorAttack	Average	Latency (Sec.)	Memory Overhead (MB)
Vicuna-13B-v1.5 (No Defense)	47.40/-	89.00/-	66.00/-	78.00/-	25.40/-	23.83/-	54.94/-	3.20	-
Perplexity Filter	47.40/100.00	3.00/4.00	66.00/100.00	78.00/100.00	25.40/100.00	23.83/100.00	40.61/84.00	0.17	13214.01
GradSafe	OOM	OOM	OOM	OOM	OOM	OOM	-	-	-
GradientCuff	OOM	6.00/8.00	56.00/78.00	9.00/13.00	0.63 / 1.90	13.83/59.00	-	40.58	2818.34
Token Highlighter	18.60/87.40	34.00/92.00	29.00/88.00	41.00/90.00	12.70/86.35	10.83/84.20	24.36/87.99	24.80	28650.00
Prompt Guard	0.00 / 0.00	2.00/2.00	24.00/37.00	77.00/99.00	25.40/100.00	12.66/41.16	23.51/46.53	15.11	1064.73

Appendix

Table 8: Comparison of different defense methods against multiple common jailbreak attacks on Vicuna-13B-v1.5. Defending performance are presented as ASR (

\downarrow

) / PGR (

\downarrow

) in %. OOM means Out-of-Memory. The number of NTs

Q=10

Guardrails	GSM8K	HumanEval	AlpacaEval	OR-Bench	Average
Perplexity Filter	0.00	0.00	0.62	0.00	0.16
GradSafe	0.00	0.00	0.49	3.20	0.92
GradientCuff	15.33	51.21	24.09	10.00	25.16
Token Highlighter	99.33	100.00	99.25	98.20	99.20
Prompt Guard	0.00	0.60	0.62	2.50	0.93
Llama Guard (Pre)	0.00	0.00	0.37	5.40	1.44

Appendix

Table 9: False positive rate on benign prompts for LLama-3-8B-Instruct. Results are reported in %.

Guardrails	GSM8K	HumanEval	AlpacaEval	OR-Bench	Average
Perplexity Filter	0.00	0.00	0.60	0.00	0.15
GradSafe	OOM	OOM	OOM	OOM	-
GradientCuff	0.00	0.00	0.00	3.50	0.88
Token Highlighter	99.35	99.92	99.18	98.26	99.18
Prompt Guard	0.00	0.60	0.80	2.50	0.98
Llama Guard (Pre)	0.00	0.00	0.40	5.40	1.45

Appendix

Table 10: False positive rate on benign prompts for Qwen2.5-7B-Instruct. Results are reported in %.

Guardrails	GSM8K	HumanEval	AlpacaEval	OR-Bench	Average
Perplexity Filter	0.00	0.00	0.60	0.00	0.15
GradSafe	OOM	OOM	OOM	OOM	-
GradientCuff	0.00	0.00	0.00	0.00	0.00
Token Highlighter	99.28	99.96	99.31	98.17	99.18
Prompt Guard	0.00	0.60	0.80	2.50	0.98
Llama Guard (Pre)	0.00	0.00	0.40	5.40	1.45

Appendix

Table 11: False positive rate on benign prompts for Qwen3.5-9B. Results are reported in %.

Guardrails	GSM8K	HumanEval	AlpacaEval	OR-Bench	Average
Perplexity Filter	0.00	0.00	0.62	0.00	0.16
GradSafe	OOM	OOM	OOM	OOM	-
GradientCuff	18.00	0.00	13.91	12.40	11.08
Token Highlighter	99.34	99.91	99.22	98.26	99.18
Prompt Guard	0.00	0.60	0.62	2.50	0.93
Llama Guard (Pre)	0.00	0.00	0.37	5.40	1.44

Appendix

Table 12: False positive rate on benign prompts for Vicuna-13B-v1.5. Results are reported in %. The number of NTs

Q=10

Guardrails	TAP	LLM-Fuzzer	X-Teaming	Average	Latency (Sec.)	Memory Overhead (MB)
LLama-3-8B-Instruct (No Defense)	14.00/-	49.00/-	91.00/-	51.33/-	1.47	-
Perplexity Filter	14.00/100.00	49.00/100.00	91.00/100.00	51.33/100.00	0.49	13571.94
GradSafe	9.00/53.00	OOM	OOM	-	-	-
GradientCuff	5.00/10.00	0.00 / 0.00	1.00/1.00	2.00 / 3.67	26.57	2063.76
Token Highlighter	5.00/7.00	OOM	OOM	-	-	-
Prompt Guard	14.00/93.00	0.00 / 0.00	0.00 / 0.00	4.67/31.00	22.67	1864.11

Appendix

Table 13: Comparison of different defense methods against three adaptive attacks on LLama-3-8B-Instruct. Defense performance are reported as ASR (

\downarrow

) / PGR (

\downarrow

) in %.

Guardrails	RS
Llama Guard (Pre)	5.70/47.10
SelfDefend (Direct)	2.30/29.20
SelfGrader	0.00/0.00

Appendix

Table 14: Robustness to guardrail-adaptive Random Search. Results are reported as ASR/PGR (%).

Guardrails	Manual (IJP)	GCG	AutoDAN	DrAttack	MultiJail	ActorAttack	Average
Llama Guard (Pre)	6.40/62.40	5.00/87.00	2.00/95.00	16.00/99.00	6.90/100.00	4.00/99.50	6.72/90.48
SelfDefend (Direct)	0.90/27.00	1.00/8.00	2.00/14.00	9.00/44.00	6.98/89.20	2.66/76.16	3.76/43.06
SelfGrader	0.20/9.10	2.00/4.00	0.00/0.00	0.00/0.00	0.00/0.00	0.00/0.00	0.36/2.18

Appendix

Table 15: Comparison of different defense methods against Emoji Attack on LLama-3-8B-Instruct. Defense performance are reported as ASR (

\downarrow

) / PGR (

\downarrow

) in %.

Methods	BIPIA	Latency (Sec.)	Memory Overhead (MB)
LLama-3-8B-Instruct (No Defense)	1.33/-	-	-
Perplexity Filter	1.33/100.00	0.09	13112.46
GradSafe	OOM	-	-
GradientCuff	0.33/39.33	10.49	827.33
Prompt Guard	0.00 / 0.33	9.22	1077.71
Llama Guard (Pre)	1.33/100.00	0.22	13571.94

Appendix

Table 16: Comparison of different defense methods against prompt injection attack on LLama-3-8B-Instruct model. Defense performance are reported as ASR (

\downarrow

) / PGR (

\downarrow

) in %.

Guardrails	Manual (IJP)	GCG	AutoDAN	DrAttack	MultiJail	ActorAttack	Average	GSM8K	HumanEval	AlpacaEval	OR-Bench	Average
LLama-3-8B-Instruct (No Defense)	7.80/-	13.00/-	2.00/-	10.00/-	4.44/-	22.66/-	9.98/-	-	-	-	-	-
SelfGrader	0.00/1.30	0.00/2.00	0.00/0.00	0.00/0.00	0.63/11.75	0.00/0.00	0.11/2.51	0.00	0.00	1.36	6.30	1.91
SelfGrader (w/o ICL Anchor)	2.00/48.80	10.00/62.00	2.00/83.00	10.00/71.00	4.44/99.68	15.50/57.16	7.32/70.27	0.00	0.00	2.73	0.10	0.70
SelfGrader (w/o Benign View)	0.00/0.90	0.00/1.00	0.00/0.00	0.00/0.00	0.31/5.07	0.00/0.00	0.05/1.16	4.00	77.43	14.03	21.60	29.26
SelfGrader (w/o Malicious View)	6.70/80.50	9.00/49.00	0.00/43.00	10.00/100.00	4.44/98.41	18.33/81.16	8.08/48.47	0.00	0.00	0.49	0.20	0.17

Appendix

Table 17: Evaluation of ICL anchor and DPL scoring.

Guardrails	Manual (IJP)	OR-Bench
LLama-3-8B-Instruct (No Defense)	7.80/-	-
SelfGrader ( $k=10$ )	0.60/8.50	3.80
SelfGrader ( $k=20$ )	0.00/1.30	6.30
SelfGrader ( $k=40$ )	0.80/7.90	5.40
SelfGrader ( $k=60$ )	0.70/7.50	5.00
SelfGrader ( $k=80$ )	0.70/7.70	5.10

Appendix

Table 18: Performance of SelfGrader with different tail trimming parameter

k

Guardrails	Manual (IJP)	OR-Bench
LLama-3-8B-Instruct (No Defense)	7.80/-	-
SelfGrader ( $\lambda=0.3$ )	5.30/59.80	1.10
SelfGrader ( $\lambda=0.4$ )	3.50/36.90	1.10
SelfGrader ( $\lambda=0.5$ )	0.00/1.30	6.30
SelfGrader ( $\lambda=0.6$ )	0.00/2.00	13.80
SelfGrader ( $\lambda=0.7$ )	0.00/1.40	16.40

Appendix

Table 19: Performance of SelfGrader with different DPL coefficient

\lambda

Guardrails	Manual (IJP)	GCG	AutoDAN	DrAttack	MultiJail	ActorAttack	Average	Latency (Sec.)	Memory Overhead (MB)
SelfGrader ( $Q=2$ )	0.80/8.40	1.00/10.00	1.00/6.00	1.00/5.00	4.75/33.50	0.50/3.50	1.51/11.07	0.77	640.15
SelfGrader ( $Q=10$ )	0.30/4.20	0.50/5.00	0.50/2.00	0.50/2.00	2.50/21.00	0.20/1.50	0.75/5.95	0.77	640.15
SelfGrader ( $Q=101$ )	0.00/1.30	0.00/2.00	0.00/0.00	0.00/0.00	0.63/11.75	0.00/0.00	0.11/2.51	0.77	640.15
SelfGrader ( $Q=1000$ )	0.00/1.00	0.00/1.50	0.00/0.00	0.00/0.00	0.50/10.50	0.00/0.00	0.08/2.17	0.83	660.20
SelfGrader (+SelfDefend, $Q=10$ )	1.20/12.30	0.00/0.00	1.00/7.00	7.00/41.00	0.00/4.76	14.16/44.66	3.89/18.29	1.31	14283.17
SelfGrader (+GuardReasoner, $Q=10$ )	0.80/10.20	0.00/1.00	0.00/4.00	0.00/0.00	1.90/21.59	17.33/71.66	3.34/18.08	1.30	15695.03

Appendix

Table 20: Defensive performance of SelfGrader under different configurations against multiple common jailbreak attacks on LLama-3-8B-Instruct model. Defense performance are reported as ASR (

\downarrow

) / PGR (

\downarrow

) in %.

Guardrails	Manual (IJP)	GCG	AutoDAN	DrAttack	MultiJail	ActorAttack	Average	Latency (Sec.)	Memory Overhead (MB)
SelfGrader ( $Q=2$ )	35.40/73.40	35.00/41.00	4.00/8.00	74.00/96.00	20.00/72.00	2.50/8.33	28.48/49.79	0.73	2765.64
SelfGrader ( $Q=10$ )	25.10/55.20	0.00/0.00	4.00/8.00	0.00/0.00	11.74/36.82	3.83/11.00	7.45/18.50	0.74	2765.65
SelfGrader (+SelfDefend, $Q=10$ )	0.30/0.90	0.00/0.00	66.00/99.00	0.00/1.00	0.00/0.00	5.00/11.00	11.88/18.65	1.50	14257.13
SelfGrader (+GuardReasoner, $Q=10$ )	0.00/0.70	2.00/2.00	1.00/1.00	0.00/0.00	2.85/15.55	10.33/52.83	2.70/12.01	1.13	15658.06

Appendix

Table 21: Defensive performance of SelfGrader under different configurations against multiple common jailbreak attacks on Vicuna-13B-v1.5 model. Defending performance are presented as ASR (

\downarrow

) / PGR (

\downarrow

) in %.

Figure 2: Visualization of NT-based logit distributions under AutoDAN attacks with different NT granularities.

Figure 3: Comparison of different guardrail methods, including generation-based, classification-based, and the proposed SelfGrader.

LLM Guardrails

Large language model guardrails detect or constrain harmful outputs, prompt attacks, and unsafe agent actions before they cause harm.

See all 194 papers

SAFEGuard: Detect Optimization-Based Jailbreak Attacks Through Harmful Semantic Analysis and Fluency Measurement
Sep 5, 2026Quoc Viet Vo, Trung Le, Damith C. Ranasinghe +1LLM Security Jailbreak Attacks
Australian Institute for Machine Learning · Adelaide University · Monash University
SafeDream: Safety World Model for Proactive Early Jailbreak Detection
Apr 18, 2026Bo Yan, Weikai Lin, Yada Zhu +1Language Model Safety Evaluation World Models
University of Central Florida · University of Rochester · IBM Research
Re-Triggering Safeguards within LLMs for Jailbreak Detection
May 11, 2026Zheng Lin, Zhenxing Niu, Haoxuan Ji +2LLM Jailbreak Attacks Jailbreak Detection
1Xidian University · 2Xi’an Jiaotong University

Sep 5, 2026J/K move · Enter open · S save

SAFEGuard: Detect Optimization-Based Jailbreak Attacks Through Harmful Semantic Analysis and Fluency Measurement

Quoc Viet Vo, Trung Le, Damith C. Ranasinghe, Ehsan Abbasnejad

Australian Institute for Machine Learning · Adelaide University · Monash University

LLM Security Jailbreak Attacks LLM Jailbreak Attacks Language Model Safety Evaluation

Open paper AskPDF arXiv

Despite the significant efforts devoted to aligning large language models (LLMs) with human values and ensuring safe deployment, recent work has revealed that LLMs remain vulnerable to adversarial jailbreak attacks that can bypass safety guardrails and elicit harmful responses. Many defense methods are proposed to detect jailbreaks but they are limited in their effectiveness to counter wide-range optimization-based jailbreak mechanisms that can yield highly fluency-optimized or harmful semantic obfuscated prompts. To tackle this challenge, we propose a unified detection framework SAFEGuard which incorporates a hybrid fluency measurement based on cross-layer distribution distance and perplexity, and the analysis of harmful semantics through gradient matching. Our method is grounded in a paramount observation: high fluency prompts maintain their malicious intention close to harmful prompts while harmful semantic obfuscated prompts often inject gibberish token sequences. Our evaluation demonstrates that SAFEGuard consistently outperforms state-of-the-art baselines and achieves significant improvement in accuracy across different optimization-based jailbreaks. This underscores the effectiveness of SAFEGuard against evolving jailbreak attacks.

Abstract

LLM Guardrails

Explore similar work

SAFEGuard: Detect Optimization-Based Jailbreak Attacks Through Harmful Semantic Analysis and Fluency Measurement

SafeDream: Safety World Model for Proactive Early Jailbreak Detection

Re-Triggering Safeguards within LLMs for Jailbreak Detection