Agent harnesses require a low-latency guardrail model together with customizable audit policies. However, deploying an auto-regressive guardrail model remains challenging: such models incur high inference latency and are often poorly aligned with customized policies. To address these problems, we present Jev-Guard, a unified model for safety classification and policy routing. We further construct a policy tree over which Jev-Guard performs routing, spanning diverse domains including content moderation, cybersecurity, privacy, and legal compliance. Specifically, Jev-Guard first navigates the policy tree to select the applicable audit policy given the caller's audit context, and then performs safety classification conditioned on the retrieved policies. On 7,033 evaluation cases, Jev-Guard achieves 93.72% safety accuracy, a mean latency of 58ms, and 93.10% hierarchical routing accuracy. The full routing-and-classification pipeline averages 240ms. Furthermore, we integrate Jev-Guard into the Hermes harness framework. Across agent trajectories, its separate permission gate cuts mean review time per task from 4.80s for an SFT-LLM judge to 1.33s, with no policy violations are observed for either guard.
Figures & tables
Figure 1: Policy tree, routed diagnosis, and execution gate. (a) The tree is editable without retraining. (b) The router picks one child per node, then the leaf review diagnoses. (c) A separate permission call decides execution without reading the diagnosis. S and C mark the heads. Scores and edits are illustrative.
Model
Content
Cyber
Privacy
Compliance
Overall
Latency (ms)
Matched test cases
2,180
3,694
655
504
7,033
—
Official Jev 1.13 (API)
69.08
82.27
80.46
52.58
75.89
—
Qwen3.5-2B SFT
99.91
89.52
91.45
95.63
93.36
513.94
Jev-Guard (ours)
97.29
91.66
91.60
96.03
93.72
58.31
Table 1: Four-domain baseline comparison: safety accuracy (%). The first row gives matched cases per domain, and the other rows are the systems compared on the same 7,033 text records. Latency is the local per-case median without routing or permission checks. Prompts and training histories differ across systems.
Domain / view
Evaluation task
N
Ns
Nc
Safety acc.
Category F1
Content
All native views
11,909
11,909
9,806
85.20
61.86
AEGIS
1,679
1,679
0
84.69
—
SafeMT
997
997
997
100.00
84.35
Nemotron-3.5
1,494
1,494
1,489
56.02
36.87
VLGuard
925
925
925
43.03
24.97
Cyber
All native views
61,972
57,478
10,164
93.77
46.96
Table 2: Native classification by domain and task. Bold rows are domains covering all their native views, and indented rows are selected views, with all 109 in the appendix. Ns / Nc count safety/category supervision, dashes mean missing supervision, and category F1 is micro-averaged.
Mean
p50
p95
Per stage, in-process
Routing
87.85
72.34
153.13
Leaf review
71.32
75.67
116.86
Permission judgment
74.50
71.29
104.95
Explicit rules
0.01
0.00
0.01
End to end
Table 3: Review latency in ms. Rows are review stages and end-to-end totals, with quantiles computed per stage. In-process rows use 200 states, HTTP uses 647.
Condition
Correct
Acc. (%)
Rule edits, 30 targets and 30 controls
Choice head, targets
21/30
70.00
Choice head, controls
30/30
100.00
Safety head, targets
30/30
100.00
Safety head, controls
24/30
80.00
Category edits, 24 actions
Table 4: Frozen-model edit results. Rows are edit conditions or scored heads, and indented rows restrict scoring to affected cases.
Condition
Safety
Cat. F1
Strict
Unsupp.
ms
Tree
66.28
24.73
19.66
12/128
205.22
Flat leaf
68.60
24.44
18.80
15/128
152.20
Oracle leaf
67.44
27.86
23.08
0/128
73.66
Full flat
79.07
20.65
2.56
115/128
725.08
Table 5: Policy presentation on 128 cases at 32K. Rows are the conditions in the text. Cat. F1 is observed category F1, Unsupp. is unsupported outputs.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Domain
Native train
Validation library
Content
212,773
1,003
Cyber
74,193
43,764
Privacy
29,581
16,954
Compliance
61,383
3,809
Total
377,930
65,530
Appendix
Table 6: Native records by domain, excluding router examples.
Domain
Leaves
Prediction types
Content
1
multi-label
Privacy
1
multi-label
Cyber
10
choice, multi-label, ordinal
Compliance
61
choice
Total
73
Appendix
Table 7: Leaves of the evaluated policy tree by domain. Each compliance leaf corresponds to one law or provider policy.
Domain
Task cases
Safety cases
Safety acc.
Category F1
Safe → unsafe
Unsafe → safe
Content
11,909
11,909
85.20
61.86
54.31
1.76
Cyber
61,972
57,478
93.77
46.96
3.19
21.17
Privacy
69,051
5,889
82.92
78.40
28.08
10.87
Compliance
4,711
4,711
95.31
58.17
2.48
7.04
Appendix
Table 8: Native evaluation in the original test-label space. Scores and conditional error rates are percentages. Category F1 is micro-averaged within each domain. Category-labeled counts are 9,806 (content), 10,164 (cyber), 68,318 (privacy), and 4,707 (compliance). No common-category aggregate is implied.
Benchmark / view
N
Ns
Nc
S-acc.
C-acc.
C-F1
t50 ms
Content
aegis prompt safety
1,679
1,679
0
84.69
—
—
53.49
CSSBench
202
202
202
93.56
88.12
88.12
54.89
ChineseSafe
649
649
649
92.91
97.23
97.23
54.99
HADES
750
750
750
100.00
74.13
74.13
75.03
JailBench
226
226
226
100.00
98.67
98.67
54.86
Appendix
Table 9: Complete native results in the original test-label space.
Table 9 continued
Benchmark / view
N
Ns
Nc
S-acc.
C-acc.
C-F1
t50 ms
Cyber (continued)
primevul code
24,777
24,777
545
96.86
0.00
20.45
55.19
primevul paired
862
862
431
50.00
0.00
20.26
56.19
prompt injection
116
116
0
93.97
—
—
55.15
Privacy
Appendix
Table 11
Table 9 continued
Benchmark / view
N
Ns
Nc
S-acc.
C-acc.
C-F1
t50 ms
Compliance (continued)
google: dev site policy
7
7
7
100.00
85.71
85.71
55.17
google: gemma prohibition
5
5
5
100.00
60.00
60.00
55.57
google: gen ai terms
5
5
5
100.00
40.00
40.00
55.39
google: google privacy policy
45
45
45
100.00
55.56
55.56
55.33
Appendix
Table 12
Rule family
Choice jointly correct
External recipient
6/6
Publication review
6/6
Regional storage
6/6
Security-work authorization
3/6
Retention limit
0/6
Total affected actions
21/30
Appendix
Table 10: Both the original and edited choice predictions must be correct. Each family also has six unrelated controls. The native safety head’s 30/30 target result does not repair its 6/30 control errors.
Domain
Records
Tree
Flat leaf
Content
100
100.00
100.00
Cyber
312
80.45
77.24
Privacy
100
98.00
96.00
Compliance
488
98.77
99.80
Appendix
Table 11: Full-path leaf correctness (%) by domain on the paired routing protocol, for the tree and flat-leaf conditions.
Domain
Condition
N
Ns
Safety acc.
Category F1
Strict set acc.
Content
Tree
7
7
100.00
40.00
25.00
Content
Flat leaf
7
7
100.00
40.00
25.00
Content
Oracle leaf
7
7
100.00
40.00
25.00
Content
Full flat
7
7
85.71
33.33
25.00
Cyber
Tree
40
33
87.88
57.14
46.88
Cyber
Flat leaf
40
33
90.91
54.17
43.75
Appendix
Table 12: Domain-level scores from the matched 32K comparison. Unknown categories are masked. Strict accuracy additionally penalizes unsupported outputs.
Domain
N
Official
SFT
Ours
Content
2,180
1,506
2,178
2,121
Cyber
3,694
3,039
3,307
3,386
Privacy
655
527
599
600
Compliance
504
265
482
484
Overall
7,033
5,337
6,566
6,591
Appendix
Table 13: Correct counts per system for Table 1 . All cases, including three official API failures, remain in the denominator.
Enterprise generative AI applications require robust safety mechanisms that can accommodate diverse risk postures, evolving policies, and varying latency constraints. Current guardrail solutions often suffer from rigidity, relying on fixed policy sets and offering limited transparency or reasoning flexibility. We present Adaguard, an adaptive LLM-as-a-Judge framework designed to address these challenges through dynamic policy enforcement and adaptive reasoning-budget allocation. Built using supervised fine-tuning (SFT) and reinforcement learning (GRPO), AdaGuard generalizes to user-defined safety and compliance policies at runtime without requiring frequent model updates. A core innovation of our approach is the ability to dynamically infer the complexity of input-policy pairs, allowing the model to switch between high-speed black-box inference and explainable, reasoning-enabled moderation. This flexibility enables developers to balance stringent latency requirements with the need for actionable transparency. This adaptive capability allows AdaGuard to rival other guardrail and frontier models several times its size, while its auto-reasoning mode recovers the accuracy of always-on reasoning at a fraction of the latency
Guardrails are a critical safety layer for modern AI systems, but their operating regime is changing. As LLMs are deployed as customized assistants, safety policies are increasingly specified at inference time by users, organizations, or regulatory contexts. This makes safety enforcement fundamentally dynamic: the guardrail should adapt to changing safety policies without retraining. Yet this requirement creates a fundamental tension: faithfully judging complex policy contexts demands reasoning capability, while practical deployment requires low-latency responses. We introduce Latent Policy Guardrail (LPG), a guardrail framework that learnssemantic latent deliberation over dynamic policies. LPG compresses the internal deliberation needed for intent interpretation and policy grounding into continuous states supervised by decision-relevant semantics. At inference time, it generates only a compact verdict anchored to the violated policy clauses, preserving auditability while avoiding the latency of explicit reasoning. Across policy guardrail benchmarks, LPG-4B reaches 84.5% average safety accuracy and 77.9% F1 by compressing deliberation into just 10 latent tokens, outperforming the strongest dynamic baseline while running roughly 11 times faster than Qwen3-4B-Thinking under the single-sample evaluation setup. Code and data are available at https://github.com/SaFo-Lab/Latent_Policy_Guard.
Guard models support the safe deployment of language model agents, but fixed risk taxonomies limit their ability to accommodate requirements that vary across applications and tasks. Under user-defined policies, detecting violations requires interpreting both the applicable rules and the agent's behavior, since identical actions can receive different judgments under different policies. To support learning this capability, we introduce AdaptiveSafety, a dataset of 10,939 training examples and 1,000 test examples covering policies with 1--100 rules. The dataset combines trajectories from multiple sources with policy and behavioral counterfactuals, pairing each example with an explanation and the complete set of violated rules. These counterfactuals expose changes that alter compliance, while structural augmentations provide supervision for consistency under rule reordering and identifier remapping. Building on this supervision, we propose SafePO, a reinforcement learning algorithm for refining violation identification while balancing explanatory reasoning and final verdicts. SafePO uses structured rewards to assess prediction correctness, retains group-relative advantages at the response level, and employs a separately trained value model to modulate token weights within explanation and verdict regions. Separate normalization controls their relative contribution to training despite differences in length. Through supervised initialization followed by SafePO, we develop AdaGuard, a family of 0.6B, 4B, and 8B guard models that assess agent trajectories under policies supplied at inference time. Our 4B model achieves binary accuracies of 89.30% on AdaptiveSafety and 71.82% on DynaBench. The project repository is available at https://github.com/Yunhao-Feng/AdaGuard
Yunhao Feng, Yifan Ding, Yuxiang Xie +4
1National University of Defense Technology · 2Fudan University