Efficient Safety Guardrails through Latent Dual-Hypothesis Reasoning with On-Demand Trace Reconstruction

Md Asiful Islam; Fahmida Alam; Mihai Surdeanu

cs.AIOct 13, 2026

Efficient Safety Guardrails through Latent Dual-Hypothesis Reasoning with On-Demand Trace Reconstruction

Authors: Md Asiful Islam, Fahmida Alam, Mihai Surdeanu

Organizations: Department of Computer Science University of Arizona, USA

Abstract

Guardrail approaches that rely on reasoning-based large language models (LLM) improve safety decisions, but come with substantial inference overhead due to the explicit reasoning. We introduce LR-Guard (Latent Reasoning Guardrail), an efficient and explainable guardrail that jointly learns structured latent reasoning and on-demand reasoning reconstruction. Our method distills a frozen teacher's analysis of safe and unsafe prompt interpretations and their comparison into compact latent representations. Joint training optimizes these representations for both safety prediction and reconstruction of the teacher's explicit reasoning traces, enabling human-readable explanations without requiring intermediate text generation for every decision. Experiments show that LR-Guard outperforms its explicit-reasoning teacher in most in-domain and out-of-domain settings on safety classification and explanation extraction, while achieving a 19.4x inference speedup at batch size one. Further, reconstructed reasoning traces from the latent information exhibit high semantic similarity to the teacher's explicit reasoning.

Figures & tables

Figure 1: High-level overview of LR-Guard . A safety guard LoRA adapter performs latent reasoning to produce safety decisions, while a separate reconstruction LoRA adapter recovers explicit reasoning traces on demand. Both adapters share a frozen LLM backbone.

Figure 2: Overview of LR-Guard . (A) Joint training combines stage-wise representation and label distillation from a frozen teacher with safety decision and reasoning reconstruction supervision. (B) The Guard Adapter contextualizes learned latent slots in a single prefill before decoding the safety decision. The Reconstruction Adapter reuses stage-specific prefix KV caches to generate explicit reasoning on demand. Both adapters share a frozen LLM backbone.

Model	Test sets
Model	AEGIS- 2.0	Wild- GuardMix	Toxic- Chat0124
OpenAI Moderation API	37.8	12.1	61.41
LLAMAGUARD2-8B	76.8	70.9	-
LLAMAGUARD3-1B	49.6	47.2	-
LLAMAGUARD3-8B	77.3	76.8	-
Llama Prompt Guard 2-22M	7.69	32.91	32.13

Table 1: Prompt classification performance of LR-Guard compared with other methods, reported using F1 scores for the unsafe class. LR-Guard results are averaged over three random seeds, with standard deviations reported in Table 10 . Gray ( ) cells indicate in-domain performance, while white ( ) cells indicate out-of-domain performance. The baseline comparison is adapted from Islam et al. (2026) , with additional latent-reasoning baselines and new LR-Guard results.

Test dataset	BLEU			ROUGE			Sim.
Test dataset	B-1	B-2	B-3	R-1	R-2	R-L	Sim.
LR-Guard -aegis2.0
AEGIS2.0	68.50	60.75	55.47	68.55	53.87	61.16	0.86
WildGuardMix	69.62	62.28	57.17	69.72	55.76	62.87	0.88
ToxicChat0124	69.68	62.32	57.25	69.88	55.87	63.00	0.86
LR-Guard -wildguardmix

Table 2: Detailed reasoning reconstruction performance of LR-Guard across models and test datasets, averaged over three random seeds (32, 42, and 52). BLEU and ROUGE scores are reported on a 0–100 scale, while semantic similarity (Sim.) is measured using cosine similarity on a

[-1,1]

scale. The sample standard deviations range from 0.11 to 1.97 for BLEU, 0.13 to 1.85 for ROUGE, and 0.0008 to 0.0113 for semantic similarity. Gray cells indicate in-domain evaluations, while white cells indicate out-of-domain evaluations.

Model	Test sets
Model	AEGIS- 2.0	Wild- GuardMix	Toxic- Chat0124
Trained on AEGIS2.0
LEG-1.0-aegis2.0-base	76.95	60.40	59.78
LEG-1.0-aegis2.0-large	79.60	66.66	63.18
ARBITER-aegis2.0	76.39	72.38	67.59
LR-Guard -aegis2.0	77.65	72.91	68.64

Table 3: Explainability classification performance of LR-Guard compared with baseline models, reported using F1 scores for the unsafe class. LR-Guard results are averaged over three random seeds, with standard deviations reported in Table 10 . Gray ( ) cells indicate in-domain performance, while white ( ) cells indicate out-of-domain performance. The baseline comparison is adapted from Islam et al. (2026) , with new LR-Guard results.

Figure 3: Effect of latent-state replacement on final-JSON label confidence. Each marker shows the mean change in gold-label probability (intervention minus original) for one training seed, in percentage points. Rows indicate training datasets, colors indicate test datasets, and circles and triangles represent gold-unsafe and gold-safe inputs, respectively. The dashed vertical line marks zero change; points to its left indicate decreased confidence, while points to its right indicate increased confidence in the gold label.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Inference mode	Batch size 1		Batch size 50
Model	Inference mode	Latency (ms/input)	Throughput (inputs/s)	Latency (ms/input)	Throughput (inputs/s)
ARBITER	Winner only	9360.36	0.11	557.57	1.79
ARBITER	Winner + Final JSON	9820.91	0.10	664.26	1.51
LR-Guard	Winner only (No reconstruction)	70.33	14.22	24.67	40.53
	Winner + Final JSON (No reconstruction)	505.12	1.98	61.16	16.35
	Winner + Final JSON + Reasoning trace reconstruction	2799.64	0.36	145.39	6.88

Appendix

Table 4: Inference efficiency of ARBITER and LR-Guard on the WildGuardMix test set, using models trained on WildGuardMix and evaluated on an NVIDIA H100 80GB GPU. Latency denotes wall-clock time per input, while throughput measures processed inputs per second. Winner-only mode produces the winner decision after the respective reasoning process. ARBITER generates explicit reasoning traces, whereas LR-Guard performs latent reasoning and reconstructs explicit reasoning traces only on demand.

Model	Model Size	GPU	Batch Size	Latency (ms/input)	Speedup
CoLaGuard-3B	3B	H100 80GB	-	318.90	$11.90\times$
CoLaGuard-8B	8B	H100 80GB	-	342.00	$12.90\times$
LatentGuard-1B	1B	Not reported	8	38.00	$11.20\times$
LatentGuard-3B	3B	Not reported	8	69.00	$9.00\times$
LatentGuard-8B	8B	Not reported	8	89.00	$8.90\times$
LPG-4B	4B	A100 80GB	1	748.00	$11.20\times$

Appendix

Table 5: Reported inference efficiency of latent-reasoning guardrails. Results for CoLaGuard, LatentGuard, and LPG are taken from Sai et al. (2026) , Liu et al. (2026) , and Li et al. (2026) , respectively. For LR-Guard , we report winner-only inference, since existing latent-reasoning guardrails primarily produce safety decisions without the additional structured JSON explanations generated by our model. Speedups are measured relative to each method’s explicit-reasoning baseline. For LR-Guard , speedups are calculated relative to ARBITER’s winner-only inference at the same batch size. Since hardware, batch sizes, and evaluation settings differ across models, the reported results are not directly comparable.

Assessment Type	Condition	Rate (%)
Reasoning Reconstruction	Safe-hypothesis reasoning is preserved	89.0
	Unsafe-hypothesis reasoning is preserved	95.0
	Deliberation reasoning is preserved	84.0
	All three reasoning sections are preserved	72.0
Decision Consistency	Deliberation decision agrees with the final predicted label	85.0

Appendix

Table 6: Human evaluation of latent reasoning reconstruction quality on 100 WildGuardMix test examples. The rate indicates the percentage of examples judged as satisfying each condition.

Model	Precision	Recall	F1	FPR	FNR
AprielGuard ( Kasundra et al., 2026 )	97.00	96.00	96.00	31.00	-
ARBITER-wildguardmix	99.17	96.15	97.64	7.62	3.85
LR-Guard -wildguardmix	99.03	96.70	97.85	9.05	3.30

Appendix

Table 7: Comparison of LR-Guard -wildguardmix with baseline guardrails on the WildJailbreak dataset. FPR and FNR represent the rates of false-positive and false-negative predictions, respectively.

Model	Precision	Recall	F1	FPR	FNR
LlamaGuard3-8B	95.20	80.00	87.00	–	–
LlamaGuard4-12B	90.10	77.50	83.30	–	–
GradSafe (Llama-3.1-8B)	88.60	93.50	91.00	–	–
RLShield (Llama-2-7B)	86.90	96.50	91.50	–	–
RLShield (Llama-3.1-8B)	95.30	91.00	93.10	–	–
ARBITER-wildguardmix	94.17	97.00	95.57	4.80	3.00

Appendix

Table 8: Comparison of LR-Guard -wildguardmix with baseline guardrails on the XSTest benchmark. FPR and FNR represent the rates of false-positive and false-negative predictions, respectively. Baseline results are taken from Islam et al. (2026) .

Unsafe Category	ARBITER	LR-Guard -wildguardmix
Violence and Physical Harm	98.73	98.73
Cyberattack	98.95	98.95
Fraud Assisting Illegal Activities	88.89	88.89
Private Information Individual	89.80	89.80
Sensitive Information Organization Government	90.11	92.47
Causing Material Harm by Disseminating Misinformation	98.88	98.88

Appendix

Table 9: Unsafe-class F1 scores (%) across different categories in the WildGuardMix test set. Both models are trained on WildGuardMix and evaluated using seed 32. The ARBITER results are taken from Islam et al. (2026) .

Model	Prompt classification			Explainability classification
Model	AEGIS2.0	WildGuardMix	ToxicChat0124	AEGIS2.0	WildGuardMix	ToxicChat0124
LR-Guard -aegis2.0	0.55	0.62	0.53	0.22	1.00	1.24
LR-Guard -wildguardmix	0.15	0.19	1.05	0.72	0.40	2.72
LR-Guard -toxicchat0124	0.51	0.54	0.42	1.04	0.49	0.61

Appendix

Table 10: Sample standard deviations across seeds 32, 42, and 52 corresponding to the LR-Guard F1 scores reported in Tables 1 and 3 . All values are reported on the 0–100 scale.

Model	AEGIS2.0					WildGuardMix					ToxicChat0124
Model	P	R	F1	FPR	FNR	P	R	F1	FPR	FNR	P	R	F1	FPR	FNR
LR-Guard -aegis2.0	86.49	90.26	88.33	16.46	9.74	79.19	87.92	83.32	17.92	12.08	59.88	86.19	70.67	4.43	13.81
LR-Guard -wildguardmix	73.27	94.05	82.37	40.11	5.95	91.71	86.60	89.09	6.08	13.40	56.55	90.61	69.64	5.34	9.39
LR-Guard -toxicchat0124	87.19	72.16	78.96	12.39	27.84	88.06	62.60	73.18	6.59	37.40	82.62	85.36	83.97	1.38	14.64

Appendix

Table 11: Detailed prompt classification results for LR-Guard variants trained on AEGIS2.0, WildGuardMix, and ToxicChat0124 using random seed 32 and evaluated across all three test sets. P, R, F1, FPR, and FNR denote precision, recall, F1 score, false-positive rate, and false-negative rate, respectively. The highest P, R, and F1 scores and the lowest FPR and FNR values in each column are highlighted in bold.

LLM Guardrails

Large language model guardrails detect or constrain harmful outputs, prompt attacks, and unsafe agent actions before they cause harm.

See all 194 papers

Robust and Efficient Guardrails with Latent Reasoning
May 27, 2026Siddharth Sai, Xiaofei Wen, Muhao ChenLLM Guardrails Content Moderation
University of California, Davis
$\mathrm{R}{\scriptstyle\mathrm{EAL}}\mathrm{G}{\scriptstyle\mathrm{UARD}}$ : Reliable, Efficient, and Auditable LLM Safeguards via Latent Reasoning
Aug 4, 2026Zhinan Liu, Jie Li, Mingyu Kang +1LLM Auditing LLM Guardrails
Xiamen University · University of Science and Technology of China
YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models
Jan 22, 2026Junyu Lin, Meizhen Liu, Xiufeng Huang +12LLM Guardrails LLM Interpretability
Alibaba-AAIG

May 27, 2026J/K move · Enter open · S save

Robust and Efficient Guardrails with Latent Reasoning

Siddharth Sai, Xiaofei Wen, Muhao Chen

University of California, Davis

LLM Guardrails Content Moderation LLM Safety Efficient Language Model Inference Latent Reasoning

Open paper AskPDF arXiv

Maintaining the safety of large language models (LLMs) is crucial as they are increasingly deployed in real-world applications. Existing safety guardrails typically rely on single-pass classification or, more recently, distilled reasoning. Reasoning-based guardrails significantly outperform classification-only baselines, but they incur substantial query latency and token overhead that make them impractical for highthroughput deployment. To address this challenge, we propose COLAGUARD, a guardrail model that transfers multi-step safety reasoning into a continuous latent space through a stage-wise training curriculum, enabling direct hidden-state propagation at inference. Evaluated on ten prompt- and response-moderation settings spanning eight safety benchmarks, COLAGUARD improves macro-F1 by 8.24 points over Llama Guard 3 and matches our explicit reasoning baseline, GuardReasoner, in macroF1 while delivering a 12.9X speedup and 22.4X reduction in token usage. Our results suggest that latent reasoning offers a practical alternative to explicit rationale generation for deployable guardrails, jointly improving safety robustness and inference efficiency rather than treating them as competing objectives.

Abstract

LLM Guardrails

Explore similar work

Robust and Efficient Guardrails with Latent Reasoning

REALGUARD\mathrm{R}{\scriptstyle\mathrm{EAL}}\mathrm{G}{\scriptstyle\mathrm{UARD}}REALGUARD: Reliable, Efficient, and Auditable LLM Safeguards via Latent Reasoning

YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models

$\mathrm{R}{\scriptstyle\mathrm{EAL}}\mathrm{G}{\scriptstyle\mathrm{UARD}}$ : Reliable, Efficient, and Auditable LLM Safeguards via Latent Reasoning