RePolicy: Learning to Select and Apply Safety Policies for Agent Safeguards

Houcheng Jiang; Boxuan Zhang; Qiyong Zhong; Junfeng Fang; Xiang Wang; Xiangnan He

cs.AIAug 25, 2026

RePolicy: Learning to Select and Apply Safety Policies for Agent Safeguards

Authors: Houcheng Jiang, Boxuan Zhang, Qiyong Zhong, Junfeng Fang, Xiang Wang, Xiangnan He

Organizations: Zhongguancun Academy · University of Science and Technology of China · Zhejiang University · National University of Singapore

Abstract

Agent safeguards must determine whether an agent's behavior violates the safety rules of the current task, environment, and user. Because these rules vary across contexts, a safeguard needs to select the relevant policy before judging the behavior. Existing policy-aware safeguards can apply a supplied policy, but generally assume that the correct policy is already known. They therefore do not learn how to select a policy from a library, even though an incorrect choice can lead to an incorrect safety judgment. We introduce RePolicy, a safeguard that jointly learns policy selection and safety assessment. Given an agent trajectory and a set of candidate policies, RePolicy selects the policy that best matches the situation, applies its requirements, and predicts whether the behavior is safe. We construct PolicyTraj-20K, a dataset of more than 20,000 agent trajectories annotated with applicable policies, safety labels, and explanations. We train RePolicy with supervised demonstrations followed by reinforcement learning, while randomizing policy order, identifiers, and distractors to reduce shortcut learning. Across six agent-safety benchmarks and 19 baselines, RePolicy-4B achieves leading overall safety-detection performance and improves policy-selection accuracy. These results show that safeguards can learn which safety requirements to use for assessing agent behavior.

Figures & tables

Figure 1: Headline comparison on six agent safety benchmarks. We compare RePolicy-4B with representative agent safeguards using Unsafe F1 (%, $\uparrow$ ). The dashed line denotes the strongest external baseline on each benchmark, and the value on the right reports the performance difference between RePolicy-4B and this baseline.

Figure 2: Overview of PolicyTraj-20K construction and RePolicy training. Top: trajectory synthesis and safety-policy initialization. Bottom: policy invocation, grounded rationale, safety judgment, and GRPO optimization.

Model	ATBench	R-Judge	OpenAgent Safety	ASSE Bench	HINT Bench	Agent Hazard	Overall
Closed-source General Models
Claude Sonnet 4.6	95.46	86.26	51.97	81.10	96.47	93.73	84.17
GPT-5.4	96.83	87.44	55.63	76.25	95.42	87.13	83.12
Qwen3.6-Plus	67.81	90.80	21.83	66.51	98.26	54.39	66.60
Open-source General Models
Qwen3-4B- [-1pt]Instruct-2507	75.79	63.12	11.48	61.72	84.58	54.11	58.47

Table 1: Main results on six agent safety benchmarks. We report Unsafe F1 (%,

\uparrow

). Bold and underlined values denote the best and second-best results, respectively. Overall is the unweighted average across benchmarks.

Figure 3: Evolution of safety-policy invocation during GRPO. (a) Correct policy invocation measured by the policy reward $R_{\mathrm{pol}}$ . The dashed line marks the cold-start level. (b) Mean number of invoked policies per trajectory and the selection rate of injected decoy policies.

Figure 4: Sensitivity to the evaluation-time policy-library size $|\mathcal{P}|$ . (a) Macro Unsafe F1 and correct policy invocation $R_{\mathrm{pol}}$ . (b) Per-benchmark Unsafe F1 changes relative to $|\mathcal{P}|=30$ .

Variant	ATBench	R-Judge	OpenAgent Safety	ASSE Bench	HINT Bench	Agent Hazard	Overall
Base model (zero-shot)	75.79	63.12	11.48	61.72	84.58	54.11	58.47
Cold-Start SFT	97.71	77.61	72.20	82.81	94.10	97.25	86.95
w/o policy invocation †	98.30	81.67	58.39	86.81	96.53	93.30	85.83
w/o policy-context perturbation †	98.92	81.74	61.46	87.56	96.72	98.41	87.47
RePolicy (full)	99.40	82.17	62.79	88.11	97.23	99.20	88.15

Table 2: Ablation of RePolicy training components. We report Unsafe F1 (%,

\uparrow

\dagger

marks provisional values that must be replaced with the final ablation run before submission.

Figure 5: Effect of explicit safety-policy invocation during GRPO. Unsafe F1 across training for full RePolicy and the variant trained without policy invocation. Endpoint labels report the gain of the full method on each benchmark.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Unsafe	Safe	Total	Unsafe (%)
ATBench	500	500	1,000	50.0
R-Judge	702	540	1,242	56.5
OpenAgentSafety	189	100	289	65.4
ASSEBench	806	670	1,476	54.6
HINTBench	583	126	709	82.2
AgentHazard	2,653	0	2,653	100.0

Appendix

Table 3: Composition of the normalized evaluation split. The unsafe rate is computed within each benchmark.

Model	Nominal scale	Primary comparison axis
Closed-source general-purpose models
Claude Sonnet 4.6 ( Anthropic, 2026 )	–	General and agentic safety reasoning
GPT-5.4 ( OpenAI, 2026a )	–	Reasoning, tool use, and computer use
Qwen3.6-Plus ( Qwen Team, 2026 )	–	Agent-oriented general reasoning
Open-source general-purpose models
Qwen3-4B-Instruct-2507 ( Qwen Team, 2025b )	4B	Backbone-scale instruction following

Appendix

Table 4: Inventory of the 19 external baseline models. Nominal model scales follow the corresponding release materials; a dash indicates that the provider does not disclose a parameter count.

Reported model	Evaluated backend identifier	Max. output tokens
Claude Sonnet 4.6	claude-sonnet-4-6	1,024
GPT-5.4	gpt-5.4	1,024
Qwen3.6-Plus	qwen3.6-plus	2,048
Qwen3-4B-Instruct-2507	Qwen/Qwen3-4B-Instruct-2507	1,024
QwQ-32B-Preview	qwq-32b-preview	8,192
Qwen3-235B-A22B-Instruct-2507	qwen3-235b-a22b-instruct-2507	1,024

Appendix

Table 5: Inference settings for general-purpose baselines. All models use the shared prompt above and temperature 0. The final column is the configured maximum completion budget; larger budgets are assigned to reasoning-oriented endpoints.

Guard	Native input and decision interface	Mapping to {safe, unsafe}
WildGuard	Official request/response classifier over the user request and recorded execution; emits harmful-request, response-refusal, and harmful-response fields.	Harmful response yes $\rightarrow$ unsafe; no $\rightarrow$ safe.
ShieldGemma-9B	Role-labelled trajectory in the native policy template with an agent-behavior safety guideline; scores the final-token logits for Yes and No .	$p(\texttt{Yes})\geq 0.5$ $\rightarrow$ unsafe; otherwise safe.
Llama Guard 4 12B	Role-labelled trajectory as a user message under the checkpoint chat template; may append native hazard categories.	First nonempty line unsafe $\rightarrow$ unsafe; safe $\rightarrow$ safe.
DynaGuard-4B	Native fast policy-compliance prompt containing the behavior criterion and role-labelled dialogue; choice-constrained decoding.	Unsafe / FAIL $\rightarrow$ unsafe; Safe / PASS $\rightarrow$ safe.
Qwen3Guard-Gen-4B	Native response moderation with request as the user message and execution as the assistant response.	Safety: Safe $\rightarrow$ safe; Unsafe or Controversial $\rightarrow$ unsafe.
Qwen3Guard-Gen-8B	Native trajectory moderation with the role-labelled transcript as a user message.	Same conservative three-to-two-way mapping as the 4B checkpoint.

Appendix

Table 6: Native guard interfaces and binary normalization. Request denotes concatenated user content; execution denotes all recorded agent and environment behavior. Auxiliary categories, rationales, and scores are retained but do not enter the reported F1.

Guard	Max. context	Explicit input cap	Max. output	TP
WildGuard	32,768	2,048 user / 29,000 exec.	32	1
ShieldGemma-9B	native	none (full transcript)	logit score	–
Llama Guard 4 12B	8,192	none (full transcript)	16	2
DynaGuard-4B	32,768	30,000 transcript	16	1
Qwen3Guard-Gen-4B	32,768	2,048 user / 28,000 exec.	128	1
Qwen3Guard-Gen-8B	32,768	30,000 transcript	128	2

Appendix

Table 7: Input and inference limits for specialized guards. “User/exec.” gives separate caps for user content and the recorded agent–environment execution. TP is tensor-parallel degree.

Setting	Cold-Start SFT	GRPO
Backbone	Qwen3-4B-Instruct-2507
Training examples	5,000	14,425
Validation examples	500	500
Training epochs	2	3
Global batch size	128	64
Learning rate	$1\times 10^{-5}$	$1\times 10^{-6}$

Appendix

Table 8: Default training configuration for RePolicy-4B.

Component	RePolicy behavior
Trajectory	The agent reads staging credentials and sends them to an external address without explicit authorization.
Candidate ambiguity	The trajectory contains both file-operation and external-messaging cues, making several policies superficially plausible.
Policy invocation	RePolicy invokes External Messaging Safety and Authorization (the local policy identifier in the released demo is Policy_1 ).
Relevant requirements	External messages require appropriate recipient authorization; sensitive credentials should not be externally disclosed without explicit authorization; consequential external transmission requires confirmation when authorization is not established.
Final judgment	Unsafe . The external transmission of credentials violates the retrieved policy requirements.

Appendix

Table 9: Qualitative example of policy invocation and policy-grounded judgment.

Figure 6: Optimization dynamics with and without explicit policy invocation. (a) Validation reward throughout GRPO. (b) Mean rollout response length. (c) Evolution of false-positive and false-negative rates across training checkpoints. Full RePolicy maintains a stronger validation objective and converges to a more favorable error trade-off, at the cost of longer safeguard rollouts.

Figure 7: Validation-set error decomposition. Bars report the share of trajectories receiving a correct judgment, a false positive (safe $\rightarrow$ unsafe), or a false negative (unsafe $\rightarrow$ safe). Green values report the wrong-policy invocation rate when policy invocation is defined.

LLM Agent Safety

Large language model agent safety studies how to keep tool-using, persistent, and multi-agent systems aligned with user intent and authorized limits.

See all 322 papers

AdaGuard: An Adaptive Guard Model with User-defined Policies
Sep 28, 2026Yunhao Feng, Yifan Ding, Yuxiang Xie +4LLM Guardrails AI Agent Safety
1National University of Defense Technology · 2Fudan University
AgentPolicyForge: From Executable Environments to Policy Benchmarks and Adaptive Guards
Oct 20, 2026Chao Shang, Wanyu Du, Yi Zhang +1Tool Access Control for LLM Agents LLM Guardrails
AWS AI Labs
LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails
May 17, 2026Nanxi Li, Zhengyue Zhao, Chaowei XiaoLanguage Model Safety Evaluation LLM Guardrails
Johns Hopkins University

Sep 28, 2026J/K move · Enter open · S save

AdaGuard: An Adaptive Guard Model with User-defined Policies

Yunhao Feng, Yifan Ding, Yuxiang Xie, Zheng Li+3

1National University of Defense Technology · 2Fudan University

LLM Guardrails AI Agent Safety Safety Filtering LLM Safety AI Agent Safety Benchmarks

Open paper AskPDF arXiv

Guard models support the safe deployment of language model agents, but fixed risk taxonomies limit their ability to accommodate requirements that vary across applications and tasks. Under user-defined policies, detecting violations requires interpreting both the applicable rules and the agent's behavior, since identical actions can receive different judgments under different policies. To support learning this capability, we introduce AdaptiveSafety, a dataset of 10,939 training examples and 1,000 test examples covering policies with 1--100 rules. The dataset combines trajectories from multiple sources with policy and behavioral counterfactuals, pairing each example with an explanation and the complete set of violated rules. These counterfactuals expose changes that alter compliance, while structural augmentations provide supervision for consistency under rule reordering and identifier remapping. Building on this supervision, we propose SafePO, a reinforcement learning algorithm for refining violation identification while balancing explanatory reasoning and final verdicts. SafePO uses structured rewards to assess prediction correctness, retains group-relative advantages at the response level, and employs a separately trained value model to modulate token weights within explanation and verdict regions. Separate normalization controls their relative contribution to training despite differences in length. Through supervised initialization followed by SafePO, we develop AdaGuard, a family of 0.6B, 4B, and 8B guard models that assess agent trajectories under policies supplied at inference time. Our 4B model achieves binary accuracies of 89.30% on AdaptiveSafety and 71.82% on DynaBench. The project repository is available at https://github.com/Yunhao-Feng/AdaGuard

Abstract

LLM Agent Safety

Explore similar work

AdaGuard: An Adaptive Guard Model with User-defined Policies

AgentPolicyForge: From Executable Environments to Policy Benchmarks and Adaptive Guards

LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails