VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent Guardrails

Duy Nguyen; Lesly Miculicich; Bhavana Dalvi Mishra; Shuo Li; Krishnamurthy Dj Dvijotham; Mohit Bansal; Tomas Pfister; Long T. Le

cs.AIOct 8, 2026

VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent Guardrails

Authors: Duy Nguyen, Lesly Miculicich, Bhavana Dalvi Mishra, Shuo Li, Krishnamurthy Dj Dvijotham, Mohit Bansal, Tomas Pfister, Long T. Le

Organizations: Google Cloud AI Research · University of North Carolina at Chapel Hill · Google DeepMind

Abstract

LLM agents increasingly operate over sensitive resources, such as private documents and mutable system states, which can create serious safety and security risks. Runtime guardrails should therefore make inspectable, reproducible, and deterministic allow or refuse decisions before potentially irreversible agent actions are executed. However, safety and security requirements are rarely complete before deployment, so guardrails need to keep adapting as new risks emerge. Existing adaptive guardrails learn from deployment experience, but they usually apply safety policies through LLM inference, which can lead to inconsistent decisions and ambiguous policy interpretations. Formal guardrails, on the other hand, enforce policies deterministically, but they usually rely on pre-deployment or task-specific policies and have no mechanism to adapt to newly observed risks. We introduce VEPA, a framework for the verified evolution of formal agent guardrails. VEPA represents the guardrail as a formal policy of executable symbolic rules over typed agent events. During deployment, we use an LLM guided by deployment feedback to propose policy patches that fix newly observed failures while preserving previously correct behavior. Then, a deterministic verifier only accepts policies that are structurally valid and conflict-free. Finally, an evidence-aware authority layer activates a verified rule only when it has enough supporting evidence, so uncertain updates cannot directly affect guardrail decisions. Across ASSEBench-Safety, ASSEBench-Security, and ATBench, VEPA consistently outperforms baselines while expanding the scope of deterministic formal-policy enforcement. These gains also hold under more challenging deployment conditions, including mixed streams, corrupted feedback, and longer horizons.

Figures & tables

Figure 1: Motivation for evolving formal policy. Static formal policy cannot adapt. Across a sequence of deployment tasks, natural-language memory can accumulate ambiguous or conflicting guidance and overgeneralize from feedback, causing regression on previously correct behavior. In contrast, VEPA represents policy as executable symbolic rules, verifies new rules for conflict-freedom, and preserves previously correct rules while adapting to newly observed cases.

Figure 2: Overview of VEPA. Layer 1 uses an LLM to propose formal-policy patches, while deterministic checks admit only well-formed, conflict-free candidates and a Pareto archive balances failure repair against regression. Layer 2 separates verification from deployment authority by activating only rules with sufficient supporting evidence.

Figure 3: Decision quality on events covered by the active formal policy, with all baselines evaluated on the same examples before each deployment update.

Figure 4: Growth of active formal-policy coverage across deployment rounds, measuring the fraction of unseen events for which VEPA makes a symbolic allow-or-refuse decision.

Figure 5: Performance under realistic deployment conditions. (A) Mixed-stream evaluation combining three benchmarks, testing adaptation across heterogeneous and shifting risk types. (B) Robustness to noisy feedback, where revealed adaptation labels are independently corrupted before each update. (C) Full-stream hybrid deployment, where events covered by the active formal policy receive its deterministic decision and uncovered events are routed to the deployment-time LLM fallback.

Figure 6: Ablation study of VEPA on the mixed deployment stream experiment

Figure 7: Performance over a 20-round mixed deployment stream.

Figure 8: Mixed-stream performance when replacing the update-time policy proposer with Gemini-3.6-Flash.

Figure 9: Mixed-stream performance when replacing the deployment-time parser and inference model with Gemini-3.5-Flash-Lite.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Neural and memory-based guardrails
Method	Persistent evolution	Executable symbolic policy	Mechanically verified updates	Explicit non-regression	Evidence-gated policy/memory	Deterministic enforcement
Llama Guard / ShieldGemma / WildGuard ( Inan et al., 2023 ; Zeng et al., 2024 ; Han et al., 2024 )	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$
AGrail / Membrane ( Luo et al., 2025c ; Choi et al., 2026b )	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$
PolicyBank / SafeHarbor ( Choi et al., 2026a ; Liu et al., 2026 )	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$
LiSA ( Kim et al., 2026b )	$\checkmark$	$\times$	$\times$	$\times$	$\checkmark$	$\times$
Hybrid neural–symbolic/programmatic guardrails

Appendix

Table 1: Comparison of representative LLM-agent guardrail methods.

\checkmark

indicates that a method satisfies the corresponding property, and

\times

indicates otherwise. Grouped methods have the same properties under the definitions below.

Dataset	Total	allow	refuse
ASSEBench-Safety	1476	670	806
ASSEBench-Security	817	408	409
ATBench	1000	503	497

Appendix

Table 2: Dataset statistics used in our experiments.

Feedback Ratio	Hoeffding Tree	RIPPER	AGrail	LiSA	VEPA (Ours)
$0.05$	$0.62\pm 0.11$	$0.74\pm 0.08$	$0.73\pm 0.07$	$0.74\pm 0.04$	$\mathbf{0.79\pm 0.04}$
$0.10$	$0.66\pm 0.15$	$0.76\pm 0.04$	$0.75\pm 0.05$	$0.76\pm 0.04$	$\mathbf{0.81\pm 0.05}$
$0.25$	$0.76\pm 0.05$	$0.76\pm 0.05$	$0.79\pm 0.04$	$0.76\pm 0.04$	$\mathbf{0.83\pm 0.03}$
$0.50$	$0.76\pm 0.05$	$0.79\pm 0.02$	$0.78\pm 0.01$	$0.80\pm 0.02$	$\mathbf{0.86\pm 0.02}$

Appendix

Table 3: Final-round performance under sparse deployment feedback on the mixed deployment stream combining ASSEBench-Safety, ASSEBench-Security, and ATBench. Values report mean F1

\pm

standard deviation across seeds.

Figure 10: Performance under targeted adversarial feedback on the mixed deployment stream. Corrupted feedback is concentrated on prompt-injection examples.

Figure 11: Adaptation-regression trade-off across methods on the mixed deployment stream.

Figure 12: Growth of the verified rule archive and the subset of rules granted active authority.

Figure 13: Performance of different methods using Qwen-3.8-27B.

Figure 14: F1 versus relative inference latency. Latency is reported relative to the deployment-time reference used in the experiment.

Method	Final-round F1
LLM No Memory	$0.71\pm 0.04$
Hoeffding Tree	$0.77\pm 0.07$
RIPPER	$0.78\pm 0.07$
AGrail	$0.80\pm 0.06$
LiSA	$0.81\pm 0.06$
VEPA (Ours)	$\mathbf{0.85\pm 0.07}$

Appendix

Table 4: Final-round performance on the 30-step deployment streams. Results report mean F1

\pm

standard deviation across seeds.

LLM Guardrails

Large language model guardrails detect or constrain harmful outputs, prompt attacks, and unsafe agent actions before they cause harm.

See all 185 papers

DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model
Aug 6, 2026Wenhao Lin, Chenyu Yu, Xingwei Lin +6LLM World Models LLM Guardrails
Zhejiang University · Nanjing University of Posts and Telecommunications · Sun Yat-sen University
LiSA: Lifelong Safety Adaptation via Conservative Policy Induction
May 14, 2026Minbeom Kim, Lesly Miculicich, Bhavana Dalvi Mishra +6Continual Learning for LLM Agents LLM Guardrails
Google Cloud AI Research · Google · Seoul National University
LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails
May 17, 2026Nanxi Li, Zhengyue Zhao, Chaowei XiaoLanguage Model Safety Evaluation LLM Guardrails
Johns Hopkins University

Aug 6, 2026J/K move · Enter open · S save

DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model

Wenhao Lin, Chenyu Yu, Xingwei Lin, Sicong Cao+5

Zhejiang University · Nanjing University of Posts and Telecommunications · Sun Yat-sen University

LLM World Models LLM Guardrails World Models Runtime Enforcement for AI Agents LLM Agent Safety Long-Horizon LLM Agents

Open paper AskPDF arXiv

As large language model (LLM) agents increasingly invoke external tools and interact with real-world systems, unsafe actions may cause irreversible consequences on external states, user data, and downstream services. Recent runtime guardrails mitigate such risks by checking proposed actions before execution, but many remain reactive: they primarily assess the apparent safety of the current action, lacking an explicit model of how risk evolves across the trajectory. This limitation creates a critical blind spot for long-horizon risks, where individually benign-looking actions can gradually drift the agent toward hazardous states. In response, we propose DreamGuard, a proactive guardrail for LLM agents built around a risk-aware world model. The world model maintains a compact recurrent latent state over the trajectory and predicts future latent states from which DreamGuard derives immediate-hazard and prefix-risk evidence. It then fuses these multi-horizon signals into intervention decisions before execution. Experiments across four benchmarks and an online guardrail evaluation show that DreamGuard outperforms generic, reactive, and proactive guardrail baselines, achieves the best safety-utility trade-off among evaluated guardrails, and maintains an average end-to-end latency of 25 ms per call.

Abstract

LLM Guardrails

Explore similar work

DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model

LiSA: Lifelong Safety Adaptation via Conservative Policy Induction

LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails