Jev-Guard: Fast Routing and Auditing Agent Actions over Editable Policy Trees

Wenbin Hu; Huihao Jing; Haochen Shi; Yuxuan Liu; Changxuan Fan; Sirui Zhang; Kawai Chung; Hanyu Yang

cs.CROct 13, 2026

Jev-Guard: Fast Routing and Auditing Agent Actions over Editable Policy Trees

Authors: Wenbin Hu, Huihao Jing, Haochen Shi, Yuxuan Liu, Changxuan Fan, Sirui Zhang, Kawai Chung, Hanyu Yang,

Organizations: HKUST · ModeIO · CUHK (Shenzhen)

Abstract

Agent harnesses require a low-latency guardrail model together with customizable audit policies. However, deploying an auto-regressive guardrail model remains challenging: such models incur high inference latency and are often poorly aligned with customized policies. To address these problems, we present Jev-Guard, a unified model for safety classification and policy routing. We further construct a policy tree over which Jev-Guard performs routing, spanning diverse domains including content moderation, cybersecurity, privacy, and legal compliance. Specifically, Jev-Guard first navigates the policy tree to select the applicable audit policy given the caller's audit context, and then performs safety classification conditioned on the retrieved policies. On 7,033 evaluation cases, Jev-Guard achieves 93.72% safety accuracy, a mean latency of 58ms, and 93.10% hierarchical routing accuracy. The full routing-and-classification pipeline averages 240ms. Furthermore, we integrate Jev-Guard into the Hermes harness framework. Across agent trajectories, its separate permission gate cuts mean review time per task from 4.80s for an SFT-LLM judge to 1.33s, with no policy violations are observed for either guard.

Figures & tables

Figure 1: Policy tree, routed diagnosis, and execution gate. (a) The tree is editable without retraining. (b) The router picks one child per node, then the leaf review diagnoses. (c) A separate permission call decides execution without reading the diagnosis. S and C mark the heads. Scores and edits are illustrative.

Model	Content	Cyber	Privacy	Compliance	Overall	Latency (ms)
Matched test cases	2,180	3,694	655	504	7,033	—
Official Jev 1.13 (API)	69.08	82.27	80.46	52.58	75.89	—
Qwen3.5-2B SFT	99.91	89.52	91.45	95.63	93.36	513.94
Jev-Guard (ours)	97.29	91.66	91.60	96.03	93.72	58.31

Table 1: Four-domain baseline comparison: safety accuracy (%). The first row gives matched cases per domain, and the other rows are the systems compared on the same 7,033 text records. Latency is the local per-case median without routing or permission checks. Prompts and training histories differ across systems.

Domain / view	Evaluation task	$N$	$N_{s}$	$N_{c}$	Safety acc.	Category F1
Content	All native views	11,909	11,909	9,806	85.20	61.86
	AEGIS	1,679	1,679	0	84.69	—
	SafeMT	997	997	997	100.00	84.35
	Nemotron-3.5	1,494	1,494	1,489	56.02	36.87
	VLGuard	925	925	925	43.03	24.97
Cyber	All native views	61,972	57,478	10,164	93.77	46.96

Table 2: Native classification by domain and task. Bold rows are domains covering all their native views, and indented rows are selected views, with all 109 in the appendix.

N_{s}

N_{c}

count safety/category supervision, dashes mean missing supervision, and category F1 is micro-averaged.

Per stage, in-process
	Mean	p50	p95
Routing	87.85	72.34	153.13
Leaf review	71.32	75.67	116.86
Permission judgment	74.50	71.29	104.95
Explicit rules	0.01	0.00	0.01
End to end

Table 3: Review latency in ms. Rows are review stages and end-to-end totals, with quantiles computed per stage. In-process rows use 200 states, HTTP uses 647.

Rule edits, 30 targets and 30 controls
Condition	Correct	Acc. (%)
Choice head, targets	21/30	70.00
Choice head, controls	30/30	100.00
Safety head, targets	30/30	100.00
Safety head, controls	24/30	80.00
Category edits, 24 actions

Table 4: Frozen-model edit results. Rows are edit conditions or scored heads, and indented rows restrict scoring to affected cases.

Condition	Safety	Cat. F1	Strict	Unsupp.	ms
Tree	66.28	24.73	19.66	12/128	205.22
Flat leaf	68.60	24.44	18.80	15/128	152.20
Oracle leaf	67.44	27.86	23.08	0/128	73.66
Full flat	79.07	20.65	2.56	115/128	725.08

Table 5: Policy presentation on 128 cases at 32K. Rows are the conditions in the text. Cat. F1 is observed category F1, Unsupp. is unsupported outputs.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	Native train	Validation library
Content	212,773	1,003
Cyber	74,193	43,764
Privacy	29,581	16,954
Compliance	61,383	3,809
Total	377,930	65,530

Appendix

Table 6: Native records by domain, excluding router examples.

Domain	Leaves	Prediction types
Content	1	multi-label
Privacy	1	multi-label
Cyber	10	choice, multi-label, ordinal
Compliance	61	choice
Total	73

Appendix

Table 7: Leaves of the evaluated policy tree by domain. Each compliance leaf corresponds to one law or provider policy.

Domain	Task cases	Safety cases	Safety acc.	Category F1	Safe $\to$ unsafe	Unsafe $\to$ safe
Content	11,909	11,909	85.20	61.86	54.31	1.76
Cyber	61,972	57,478	93.77	46.96	3.19	21.17
Privacy	69,051	5,889	82.92	78.40	28.08	10.87
Compliance	4,711	4,711	95.31	58.17	2.48	7.04

Appendix

Table 8: Native evaluation in the original test-label space. Scores and conditional error rates are percentages. Category F1 is micro-averaged within each domain. Category-labeled counts are 9,806 (content), 10,164 (cyber), 68,318 (privacy), and 4,707 (compliance). No common-category aggregate is implied.

Benchmark / view	$N$	$N_{s}$	$N_{c}$	S-acc.	C-acc.	C-F1	$t_{50}$ ms
Content
aegis prompt safety	1,679	1,679	0	84.69	—	—	53.49
CSSBench	202	202	202	93.56	88.12	88.12	54.89
ChineseSafe	649	649	649	92.91	97.23	97.23	54.99
HADES	750	750	750	100.00	74.13	74.13	75.03
JailBench	226	226	226	100.00	98.67	98.67	54.86

Appendix

Table 9: Complete native results in the original test-label space.

Table 9 continued
Benchmark / view	$N$	$N_{s}$	$N_{c}$	S-acc.	C-acc.	C-F1	$t_{50}$ ms
Cyber (continued)
primevul code	24,777	24,777	545	96.86	0.00	20.45	55.19
primevul paired	862	862	431	50.00	0.00	20.26	56.19
prompt injection	116	116	0	93.97	—	—	55.15
Privacy

Appendix

Table 11

Table 9 continued
Benchmark / view	$N$	$N_{s}$	$N_{c}$	S-acc.	C-acc.	C-F1	$t_{50}$ ms
Compliance (continued)
google: dev site policy	7	7	7	100.00	85.71	85.71	55.17
google: gemma prohibition	5	5	5	100.00	60.00	60.00	55.57
google: gen ai terms	5	5	5	100.00	40.00	40.00	55.39
google: google privacy policy	45	45	45	100.00	55.56	55.56	55.33

Appendix

Table 12

Rule family	Choice jointly correct
External recipient	6/6
Publication review	6/6
Regional storage	6/6
Security-work authorization	3/6
Retention limit	0/6
Total affected actions	21/30

Appendix

Table 10: Both the original and edited choice predictions must be correct. Each family also has six unrelated controls. The native safety head’s 30/30 target result does not repair its 6/30 control errors.

Domain	Records	Tree	Flat leaf
Content	100	100.00	100.00
Cyber	312	80.45	77.24
Privacy	100	98.00	96.00
Compliance	488	98.77	99.80

Appendix

Table 11: Full-path leaf correctness (%) by domain on the paired routing protocol, for the tree and flat-leaf conditions.

Domain	Condition	$N$	$N_{s}$	Safety acc.	Category F1	Strict set acc.
Content	Tree	7	7	100.00	40.00	25.00
Content	Flat leaf	7	7	100.00	40.00	25.00
Content	Oracle leaf	7	7	100.00	40.00	25.00
Content	Full flat	7	7	85.71	33.33	25.00
Cyber	Tree	40	33	87.88	57.14	46.88
Cyber	Flat leaf	40	33	90.91	54.17	43.75

Appendix

Table 12: Domain-level scores from the matched 32K comparison. Unknown categories are masked. Strict accuracy additionally penalizes unsupported outputs.

Domain	$N$	Official	SFT	Ours
Content	2,180	1,506	2,178	2,121
Cyber	3,694	3,039	3,307	3,386
Privacy	655	527	599	600
Compliance	504	265	482	484
Overall	7,033	5,337	6,566	6,591

Appendix

Table 13: Correct counts per system for Table 1 . All cases, including three official API failures, remain in the denominator.

LLM Guardrails

Large language model guardrails detect or constrain harmful outputs, prompt attacks, and unsafe agent actions before they cause harm.

See all 194 papers

AdaGuard: Enhancing Safety and Policy Compliance with Reasoning-Enabled LLM-As-A-Judge Guardrails
Oct 6, 2026Melissa Kazemi Rad, Sihui Dai, Isha Slavin +6LLM Guardrails LLM Safety
Capital One
LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails
May 17, 2026Nanxi Li, Zhengyue Zhao, Chaowei XiaoLanguage Model Safety Evaluation LLM Guardrails
Johns Hopkins University
AdaGuard: An Adaptive Guard Model with User-defined Policies
Sep 28, 2026Yunhao Feng, Yifan Ding, Yuxiang Xie +4LLM Guardrails AI Agent Safety
1National University of Defense Technology · 2Fudan University

Oct 6, 2026J/K move · Enter open · S save

AdaGuard: Enhancing Safety and Policy Compliance with Reasoning-Enabled LLM-As-A-Judge Guardrails

Melissa Kazemi Rad, Sihui Dai, Isha Slavin, Kushal Chawla+5

Capital One

LLM Guardrails LLM Safety Content Moderation LLM-as-a-Judge

Open paper AskPDF arXiv

Enterprise generative AI applications require robust safety mechanisms that can accommodate diverse risk postures, evolving policies, and varying latency constraints. Current guardrail solutions often suffer from rigidity, relying on fixed policy sets and offering limited transparency or reasoning flexibility. We present Adaguard, an adaptive LLM-as-a-Judge framework designed to address these challenges through dynamic policy enforcement and adaptive reasoning-budget allocation. Built using supervised fine-tuning (SFT) and reinforcement learning (GRPO), AdaGuard generalizes to user-defined safety and compliance policies at runtime without requiring frequent model updates. A core innovation of our approach is the ability to dynamically infer the complexity of input-policy pairs, allowing the model to switch between high-speed black-box inference and explainable, reasoning-enabled moderation. This flexibility enables developers to balance stringent latency requirements with the need for actionable transparency. This adaptive capability allows AdaGuard to rival other guardrail and frontier models several times its size, while its auto-reasoning mode recovers the accuracy of always-on reasoning at a fraction of the latency

Abstract

LLM Guardrails

Explore similar work

AdaGuard: Enhancing Safety and Policy Compliance with Reasoning-Enabled LLM-As-A-Judge Guardrails

LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails

AdaGuard: An Adaptive Guard Model with User-defined Policies