Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings

Istiaque Ahmed; Afia Anjum Borsha; Ranat Das Prangon; Abu-fuad Ahmad; Thi Hong Tran

cs.CRAug 18, 2026

Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings

Authors: Istiaque Ahmed, Afia Anjum Borsha, Ranat Das Prangon, Abu-fuad Ahmad, Thi Hong Tran

Organizations: Graduate School of Informatics, Osaka Metropolitan University, Osaka 558-8585, Japan · Dept. of Computer Science and Engineering, BRAC University, Dhaka 1212, Bangladesh · Dept. of Chemical Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka 1000, Bangladesh · New Mexico State University, Las Cruces NM 88001, USA

Abstract

Large Language Models (LLMs) in real-world applications often face the risks of specially crafted prompts designed to bypass the safety controls. Existing guardrail methods, such as LLM-as-a-judge and cloud-based safety APIs are able to detect unsafe content. However, they often add a delay of about 250-900 ms to each request. This delay is too high for real-time applications, when the system usually needs to respond in less than 100 ms. Furthermore, routing user prompts through external moderation endpoints raises significant data privacy concerns. This paper introduces Reflex-Guard, a lightweight guardrail that runs locally. It uses jailbreak-aware preprocessing, compact sentence-transformer embeddings, and seven fast binary classifiers. Together, these components enable high-accuracy prompt safety filtering with much lower latency than existing solutions. Through systematic evaluation on a strategically balanced dataset of 30,568 samples drawn from five complementary sources, we demonstrate that Reflex-Guard achieves 95.9% recall on harmful prompts at 37.6 ms end-to-end latency. It is faster than existing baselines, including Llama Guard 2 at 255 ms and SafeDecoding at 723 ms. It can detect 100% of GCG suffix attacks and Base64-encoded prompts using the default threshold. However, DrAttack structured prompts required lowering the threshold to 0.03 for optimal detection, as they produced a distinct probability distribution. Reflex-Guard achieves Reflex Efficiency Score (RES) scores up to 16.79, significantly outperforming Llama Guard 2 (11.90) and SafeDecoding (9.80). This analysis offers practical deployment advice and shows that different attack types occupy distinct regions in the embedding probability space.

Figures & tables

Approach	Sub-100 ms	Local	Encoded Attack	Structured Attack	Unified Metric
	Latency	Inference	Detection	Detection
Llama Guard 2 [ 8 , 16 ]	$\times$	$\checkmark$	$\times$	$\times$	$\times$
SafeDecoding [ 9 ]	$\times$	$\checkmark$	$\times$	$\times$	$\times$
OpenAI Moderation API [ 20 ]	$\times$	$\times^{*}$	$\sim$	$\times$	$\times$
Embedding-Only [ 28 ]	$\checkmark$	$\checkmark$	$\times$	$\times$	$\times$
Reflex-Guard (This Work)	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$

TABLE I: Comparison of Guardrail Approaches Across Key Dimensions

Fig. 1: Overall Reflex-Guard pipeline architecture.

Fig. 2: Jailbreak-aware preprocessing workflow.

Fig. 3: Embedding and Feature Engineering.

Classifier	Key Hyperparameters
Logistic Regression	solver=saga, max_iter=1000, class_weight=balanced
Random Forest	n_estimators=500, min_samples_split=5, min_samples_leaf=2, class_weight=balanced
XGBoost	n_estimators=250, max_depth=4, lr=0.05, subsample=0.8, scale_pos_weight=auto
LightGBM	n_estimators=500, lr=0.05, class_weight=balanced
HistGradientBoosting	max_iter=500, lr=0.05
AdaBoost	n_estimators=300, lr=0.5

TABLE II: Classifier Hyperparameter Specifications

Source	Role	Count	Label
AlpacaEval [ 30 ]	Benign baseline	15,000	Benign (0)
Anthropic Red-Team [ 12 ]	Training (T)	13,000	Harmful (1)
JailbreakHub/DAN [ 6 ]	Evaluation (E)	968	Harmful (1)
JailbreakBench [ 5 ]	Evaluation (E)	800	Harmful (1)
MultiJail Bengali [ 7 ]	Evaluation (E)	800	Harmful (1)
Total	—	30,568	—

TABLE III: Dataset Composition After Strategic Balancing.

Fig. 4: Dataset Preparation for proposed Reflex-Guard.

Classifiers	Accuracy	Precision	Recall	F1	ROC-AUC
HistGradientBoosting	0.954	0.952	0.959	0.955	0.990
LightGBM	0.953	0.951	0.958	0.954	0.990
Logistic Regression	0.940	0.936	0.947	0.942	0.984
XGBoost	0.940	0.940	0.943	0.941	0.985
Random Forest	0.939	0.930	0.952	0.941	0.984
AdaBoost	0.925	0.928	0.925	0.927	0.977

TABLE IV: In-Distribution Classification Performance at

\tau=0.25

. Models are ranked by F1 score.

Fig. 5: Testing and Evaluation Pipeline.

Fig. 6: Classifier performance on the test set at $\tau=0.25$ .

Attack Type	Samples	$\tau=0.25$ Recall	$\tau=0.03$ Recall	Mean Prob.
GCG Suffix	60	100%	100%	0.79
Base64 Encoded	100	100%	100%	0.63
DrAttack Structured	60	0%	100%	0.05

TABLE V: Out-of-Distribution Attack Detection at default (

\tau=0.25

) and sensitive (

\tau=0.03

) thresholds.

Fig. 7: Predicted probability distributions for GCG, Base64, DrAttack, and benign prompts.

Fig. 8: Component level latency breakdown across classifier architectures.

Model	Mean (ms)	Median/P50 (ms)	P99 (ms)
Logistic Regression	53.17	40.85	325.80
LightGBM	54.88	41.71	284.73
HistGradientBoosting	69.81	53.34	463.53
XGBoost	85.90	65.60	540.61
KNN	88.67	71.81	491.72
AdaBoost	116.93	99.52	355.44

TABLE VI: End-to-End Pipeline Latency (ms) Over 100 Prompts.

Model	Recall (%)	Mean Latency (ms)	RES
Logistic Regression	94.73	48.92	16.79
HistGradientBoosting	95.89	52.87	16.67
XGBoost	94.32	56.85	16.11
LightGBM	95.76	65.24	15.83
KNN	85.97	71.45	13.91
AdaBoost	92.52	113.28	13.53

TABLE VII: Reflex Efficiency Score (RES) comparison for security-latency trade-off.

Fig. 9: Security-latency trade-off showing recall against end-to-end latency on a logarithmic axis.

Threshold ( $\tau$ )	DrAttack Recall	General Recall*
0.01	100%	99.2%
0.02	100%	98.9%
0.03	100%	98.7%
0.04	75%	98.1%
0.05	58%	97.6%
0.06	33%	97.0%

TABLE VIII: DrAttack Detection Rate Across Threshold Settings.

Fig. 10: DrAttack detection rate as a function of the decision threshold.

Attack Type	Optimal $\tau$	Recall at Optimal	Mean Probability
GCG Suffix	0.25	100%	0.79
Base64 Encoded	0.25	100%	0.63
DrAttack Structured	0.03	100%	0.05
General Purpose	0.25	98.7%*	—

TABLE IX: Attack-Specific Optimal Threshold Recommendations.

Explore similar work

kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrail
Jul 2, 2026Mahmoud Abdelfattah, Hamid Nasiri, Peter GarraghanLarge Language Model Safety Adversarial Prompts
Lancaster University · Mindgard
Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection
May 24, 2026Lixing Lin, Juli You, Yue Li +4Large Language Model Safety Adversarial Prompts
yaleYale University, New Haven, CT, USA · indIndependent Researcher · columbiaColumbia University, New York, NY, USA +2
Robust and Efficient Guardrails with Latent Reasoning
May 27, 2026Siddharth Sai, Xiaofei Wen, Muhao ChenLarge Language Model Safety Guardrail
University of California, Davis

Jul 2, 2026, cs.LGJ/K move · Enter open · S save

kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrail

Mahmoud Abdelfattah, Hamid Nasiri, Peter Garraghan

Lancaster University · Mindgard

Large Language Model Safety Adversarial Prompts Model Activations

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly deployed in domains requiring guardrails to detect unsafe, off-topic, or adversarial prompts. Existing guardrails predominantly rely on fine-tuning to build classifiers, which often suffer from low generalization and high inference latency. We present kNNGuard, a training-free guardrail that utilizes the activation space of an off-the-shelf LLM. Given a small bank of 50 safe and unsafe prompts, kNNGuard extracts hidden activations and performs multi-layer kNN fusing activation-space and embedding-space scores for classification. Across six domains spanning topical and security prompts, kNNGuard achieves competitive or superior F1 compared to fine-tuned state-of-the-art guardrails while running 2.7x faster than the best comparable guardrail, and 10x faster than a fine-tuned safety classifier without gradient updates or fine-tuning. Domain adaptation requires only updating the labeled bank, which can be constructed in under 10 seconds and several orders of magnitude faster than established guardrails. We also analyze the impact of system prompts, layer selection, and integration into production LLM pipelines as a configurable, low-latency guardrail.

Abstract

Explore similar work

kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrail

Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection

Robust and Efficient Guardrails with Latent Reasoning