Gradient-Controlled Decoding: A Safety Guardrail for LLMs with Dual-Anchor Steering

Purva Chiniya; Kevin Scaria; Sagar Chaturvedi

cs.CLApr 6, 2026

Gradient-Controlled Decoding: A Safety Guardrail for LLMs with Dual-Anchor Steering

Authors: Purva Chiniya, Kevin Scaria, Sagar Chaturvedi

Organizations: Amazon Alexa · Amazon AGI

Abstract

Large language models (LLMs) remain susceptible to jailbreak and direct prompt-injection attacks, yet the strongest defensive filters frequently over-refuse benign queries and degrade user experience. Previous work on jailbreak and prompt injection detection such as GradSafe, detects unsafe prompts with a single "accept all" anchor token, but its threshold is brittle and it offers no deterministic guarantee that harmful content will not be emitted once decoding begins. We introduce Gradient-Controlled Decoding (GCD), a training-free guardrail that combines an acceptance anchor token ("Sure") and refusal anchor token ("Sorry") tightening the decision boundary and significantly lowering false positives. In the mitigation stage, if a prompt is flagged, GCD preset-injects one or two refusal tokens ("Sorry, I can't...") before autoregressive decoding resumes, guaranteeing first-token safety regardless of sampling strategy. On ToxicChat, XSTest-v2, and AdvBench, GCD reduces false positives by 52% vs. GradSafe at comparable recall, lowers attack success rate by up to 10% vs. the strongest decoding-only baseline, adds under 15-20 ms latency on an average on V100 instances, transfers to LLaMA-2-7B, Mixtral-8x7B, and Qwen-2-7B, and requires only 20 demonstration templates.

Figures & tables

Figure 1: Overview of our proposed approach Gradient Controlled Decoding (GCD) - three-phase gradient-based safety evaluation framework. Phase 1 computes and averages safe and unsafe template gradients from multiple samples. Phase 2 processes a given prompt (e.g., "What is the capital of France") to generate gradients with "Sure" and "Sorry" responses, which are compared against the template gradients using cosine similarity. A threshold check determines the safety level. Phase 3 routes the prompt to either a regular LLM inference path or generates a "Sorry" response based on the safety evaluation. This approach enables dynamic safety assessment of prompts during inference.

ToxicChat
Dataset	GradSafe	Our Method	SCANS	Safe-Decoding	Greedy	Top-k	Top-p
Precision (%)	75.46	67.34	-	9.58	68.37	68.49	73.07
Recall (%)	66.39	72.67	-	92.62	67.67	50.50	57.57
F1 (%)	70.64	69.91	-	17.37	68.02	58.14	64.41
FP (%)	1.55	2.54	-	62.89	15.50	11.51	10.51
ASR (%)	33.60	27.32	-	7.38	32.32	49.49	42.42

Table 1: Comparison of performance across different datasets and decoding strategies, averaged across model families. SCANS metrics are derived from its reported Refusal Rates.

Figure 2: Precision-Recall curves for the “Sure” (compliance) and “Sorry” (refusal) gradient anchors on ToxicChat (left pair) and XSTest (right pair). The operating point maximising F1 defines the selected thresholds $t_{\text{Sure}}$ and $t_{\text{Sorry}}$ . See Section 3.3 for detailed analysis.

Dataset	Attack Samples	Non-Attack Samples	Total Samples
XSTest	200	250	450
ToxicChat	366	4717	5083
AdvBench	520	0	520

Table 2: Summary of datasets used.

	Avg. Time-To-First-Token
Llama2-7b	86.60 ms
+GCD	99.89 ms
Mixtral-8x7b	101.34 ms
+GCD	119.07 ms
Qwen-2-7b	80.32 ms
+GCD	94.51 ms

Table 3: Inference speed of GCD applied to Llama2-7b, Mixtral-8x7B and Qwen-2-7B models.

Configuration	$n=2$	$n=5$	$n=10$
Varying Unsafe Prompt	$0.911\pm 0.042$	$0.928\pm 0.022$	$0.932$
Varying Safe Prompt	$0.934\pm 0.002$	$0.935\pm 0.001$	$0.934$

Table 4: Ablation study of varying numbers (

n

) of reference prompts sampled from the unsafe/safe prompt pool on XSTest in terms of AUPRC (Mean

\pm

SD over 10 runs) Xie et al. (2024) .

Model	Dataset	Precision (%)	Recall (%)	F1 (%)	FP (%)	ASR (%)
Llama-2-7b-chat	AdvBench	100.00	99.81	99.90	0.00	0.19
	ToxicChat	80.63	84.62	82.57	7.40	15.38
	XsTest	92.39	91.00	91.69	3.33	9.00
Llama-3.2-3b	AdvBench	100.00	99.23	99.61	0.00	0.77
	ToxicChat	77.16	68.68	72.67	7.40	31.32
	XsTest	88.63	93.50	91.00	5.33	6.50

Table 5: Evaluation results across models and datasets for gradient controlled decoding

LLM Guardrails

Large language model guardrails detect or constrain harmful outputs, prompt attacks, and unsafe agent actions before they cause harm.

See all 194 papers

Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models
May 12, 2025Haoran Gu, Handing Wang, Yi Mei +2LLM Safety Alignment LLM Safety
Xidian University · Victoria University of Wellington · Westlake University
Grammar-Constrained Decoding Can Jailbreak LLMs into Generating Malicious Code
Jun 10, 2026Yitong Zhang, Shiteng Lu, Jia LiLLM Safety Alignment LLM Jailbreak Attacks
College of AI Tsinghua University Beijing, China · Yingcai Honors College University of Electronic Science and Technology of China Chengdu, China
Controlled Decoding Attacks on Black-Box LLMs
Sep 29, 2026Jesson Wang, Shawn Li, Wei Yang +4Constrained Decoding Adversarial Attacks on LLMs
University of Southern California · Adobe · Amazon

May 12, 2025J/K move · Enter open · S save

Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models

Haoran Gu, Handing Wang, Yi Mei, Mengjie Zhang+1

Xidian University · Victoria University of Wellington · Westlake University

LLM Safety Alignment LLM Safety LLM Jailbreak Attacks Jailbreak Defense

Open paper AskPDF arXiv

Large Language Models (LLMs) have been extensively used across diverse domains, including virtual assistants, automated code generation, and scientific research. However, they remain vulnerable to jailbreak attacks, which manipulate the models into generating harmful responses despite safety alignment. Recent studies have shown that current safety-aligned LLMs undergo shallow safety alignment. In this work, we conduct an in-depth investigation into the underlying mechanism of this phenomenon and reveal that it manifests through learned ''safety trigger tokens'' that activate the model's safety patterns when paired with the specific input. Through both analysis and empirical verification, we further demonstrate the high similarity of the safety trigger tokens across different harmful inputs. Accordingly, we propose D-STT, a simple yet effective defense algorithm that identifies and explicitly decodes safety trigger tokens of the given safety-aligned LLM to activate the model's learned safety patterns. In this process, the safety trigger is constrained to a single token, which effectively preserves model usability by introducing minimum intervention in the decoding process. Extensive experiments across diverse jailbreak attacks and benign prompts demonstrate that D-STT significantly reduces output harmfulness while preserving model usability and incurring negligible response time overhead, outperforming ten baseline methods.

Abstract

LLM Guardrails

Explore similar work

Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models

Grammar-Constrained Decoding Can Jailbreak LLMs into Generating Malicious Code

Controlled Decoding Attacks on Black-Box LLMs