cs.AIOct 8, 2026

VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent Guardrails

Authors: Duy Nguyen, Lesly Miculicich, Bhavana Dalvi Mishra, Shuo Li, Krishnamurthy Dj Dvijotham, Mohit Bansal, Tomas Pfister, Long T. Le

Organizations: Google Cloud AI Research · University of North Carolina at Chapel Hill · Google DeepMind

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

LLM agents increasingly operate over sensitive resources, such as private documents and mutable system states, which can create serious safety and security risks. Runtime guardrails should therefore make inspectable, reproducible, and deterministic allow or refuse decisions before potentially irreversible agent actions are executed. However, safety and security requirements are rarely complete before deployment, so guardrails need to keep adapting as new risks emerge. Existing adaptive guardrails learn from deployment experience, but they usually apply safety policies through LLM inference, which can lead to inconsistent decisions and ambiguous policy interpretations. Formal guardrails, on the other hand, enforce policies deterministically, but they usually rely on pre-deployment or task-specific policies and have no mechanism to adapt to newly observed risks. We introduce VEPA, a framework for the verified evolution of formal agent guardrails. VEPA represents the guardrail as a formal policy of executable symbolic rules over typed agent events. During deployment, we use an LLM guided by deployment feedback to propose policy patches that fix newly observed failures while preserving previously correct behavior. Then, a deterministic verifier only accepts policies that are structurally valid and conflict-free. Finally, an evidence-aware authority layer activates a verified rule only when it has enough supporting evidence, so uncertain updates cannot directly affect guardrail decisions. Across ASSEBench-Safety, ASSEBench-Security, and ATBench, VEPA consistently outperforms baselines while expanding the scope of deterministic formal-policy enforcement. These gains also hold under more challenging deployment conditions, including mixed streams, corrupted feedback, and longer horizons.

Figures & tables

Figure 1: Motivation for evolving formal policy. Static formal policy cannot adapt. Across a sequence of deployment tasks, natural-language memory can accumulate ambiguous or conflicting guidance and overgeneralize from feedback, causing regression on previously correct behavior. In contrast, VEPA represents policy as executable symbolic rules, verifies new rules for conflict-freedom, and preserves previously correct rules while adapting to newly observed cases.
Figure 2: Overview of VEPA. Layer 1 uses an LLM to propose formal-policy patches, while deterministic checks admit only well-formed, conflict-free candidates and a Pareto archive balances failure repair against regression. Layer 2 separates verification from deployment authority by activating only rules with sufficient supporting evidence.
Figure 3: Decision quality on events covered by the active formal policy, with all baselines evaluated on the same examples before each deployment update.
Figure 4: Growth of active formal-policy coverage across deployment rounds, measuring the fraction of unseen events for which VEPA makes a symbolic allow-or-refuse decision.
Figure 5: Performance under realistic deployment conditions. (A) Mixed-stream evaluation combining three benchmarks, testing adaptation across heterogeneous and shifting risk types. (B) Robustness to noisy feedback, where revealed adaptation labels are independently corrupted before each update. (C) Full-stream hybrid deployment, where events covered by the active formal policy receive its deterministic decision and uncovered events are routed to the deployment-time LLM fallback.
Figure 6: Ablation study of VEPA on the mixed deployment stream experiment
Figure 7: Performance over a 20-round mixed deployment stream.
Figure 8: Mixed-stream performance when replacing the update-time policy proposer with Gemini-3.6-Flash.
Figure 9: Mixed-stream performance when replacing the deployment-time parser and inference model with Gemini-3.5-Flash-Lite.
Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

MethodPersistent evolutionExecutable symbolic policyMechanically verified updatesExplicit non-regressionEvidence-gated policy/memoryDeterministic enforcement
Neural and memory-based guardrails
Llama Guard / ShieldGemma / WildGuard ( Inan et al., 2023 ; Zeng et al., 2024 ; Han et al., 2024 )×\times×\times×\times×\times×\times×\times
AGrail / Membrane ( Luo et al., 2025c ; Choi et al., 2026b )✓\checkmark×\times×\times×\times×\times×\times
PolicyBank / SafeHarbor ( Choi et al., 2026a ; Liu et al., 2026 )×\times×\times×\times×\times×\times×\times
LiSA ( Kim et al., 2026b )✓\checkmark×\times×\times×\times✓\checkmark×\times
Hybrid neural–symbolic/programmatic guardrails
Appendix
Table 1: Comparison of representative LLM-agent guardrail methods. ✓\checkmark indicates that a method satisfies the corresponding property, and ×\times indicates otherwise. Grouped methods have the same properties under the definitions below.
DatasetTotalallowrefuse
ASSEBench-Safety1476670806
ASSEBench-Security817408409
ATBench1000503497
Appendix
Table 2: Dataset statistics used in our experiments.
Feedback RatioHoeffding TreeRIPPERAGrailLiSAVEPA (Ours)
0.050.050.62±0.110.62\pm 0.110.74±0.080.74\pm 0.080.73±0.070.73\pm 0.070.74±0.040.74\pm 0.040.79±0.04\mathbf{0.79\pm 0.04}
0.100.100.66±0.150.66\pm 0.150.76±0.040.76\pm 0.040.75±0.050.75\pm 0.050.76±0.040.76\pm 0.040.81±0.05\mathbf{0.81\pm 0.05}
0.250.250.76±0.050.76\pm 0.050.76±0.050.76\pm 0.050.79±0.040.79\pm 0.040.76±0.040.76\pm 0.040.83±0.03\mathbf{0.83\pm 0.03}
0.500.500.76±0.050.76\pm 0.050.79±0.020.79\pm 0.020.78±0.010.78\pm 0.010.80±0.020.80\pm 0.020.86±0.02\mathbf{0.86\pm 0.02}
Appendix
Table 3: Final-round performance under sparse deployment feedback on the mixed deployment stream combining ASSEBench-Safety, ASSEBench-Security, and ATBench. Values report mean F1 ±\pm standard deviation across seeds.
Figure 10: Performance under targeted adversarial feedback on the mixed deployment stream. Corrupted feedback is concentrated on prompt-injection examples.
Figure 11: Adaptation-regression trade-off across methods on the mixed deployment stream.
Figure 12: Growth of the verified rule archive and the subset of rules granted active authority.
Figure 13: Performance of different methods using Qwen-3.8-27B.
Figure 14: F1 versus relative inference latency. Latency is reported relative to the deployment-time reference used in the experiment.
MethodFinal-round F1
LLM No Memory0.71±0.040.71\pm 0.04
Hoeffding Tree0.77±0.070.77\pm 0.07
RIPPER0.78±0.070.78\pm 0.07
AGrail0.80±0.060.80\pm 0.06
LiSA0.81±0.060.81\pm 0.06
VEPA (Ours)0.85±0.07\mathbf{0.85\pm 0.07}
Appendix
Table 4: Final-round performance on the 30-step deployment streams. Results report mean F1 ±\pm standard deviation across seeds.
More in this topic

LLM Guardrails

Large language model guardrails detect or constrain harmful outputs, prompt attacks, and unsafe agent actions before they cause harm.

See all 185 papers

Explore similar work

CardsList
Aug 6, 2026

DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model

As large language model (LLM) agents increasingly invoke external tools and interact with real-world systems, unsafe actions may cause irreversible consequences on external states, user data, and downstream services. Recent runtime guardrails mitigate such risks by checking proposed actions before execution, but many remain reactive: they primarily assess the apparent safety of the current action, lacking an explicit model of how risk evolves across the trajectory. This limitation creates a critical blind spot for long-horizon risks, where individually benign-looking actions can gradually drift the agent toward hazardous states. In response, we propose DreamGuard, a proactive guardrail for LLM agents built around a risk-aware world model. The world model maintains a compact recurrent latent state over the trajectory and predicts future latent states from which DreamGuard derives immediate-hazard and prefix-risk evidence. It then fuses these multi-horizon signals into intervention decisions before execution. Experiments across four benchmarks and an online guardrail evaluation show that DreamGuard outperforms generic, reactive, and proactive guardrail baselines, achieves the best safety-utility trade-off among evaluated guardrails, and maintains an average end-to-end latency of 25 ms per call.

Wenhao Lin, Chenyu Yu, Xingwei Lin +6

Zhejiang University · Nanjing University of Posts and Telecommunications · Sun Yat-sen University

LLM World ModelsLLM GuardrailsWorld ModelsRuntime Enforcement for AI AgentsLLM Agent SafetyLong-Horizon LLM Agents
AskPDF
May 14, 2026

LiSA: Lifelong Safety Adaptation via Conservative Policy Induction

As AI agents move from chat interfaces to systems that read private data, call tools, and execute multi-step workflows, guardrails become a last line of defense against concrete deployment harms. In these settings, guardrail failures are no longer merely answer-quality errors: they can leak secrets, authorize unsafe actions, or block legitimate work. The hardest failures are often contextual: whether an action is acceptable depends on local privacy norms, organizational policies, and user expectations that resist pre-deployment specification. This creates a practical gap: guardrails must adapt to their own operating environments, yet deployment feedback is typically limited to sparse, noisy user-reported failures, and repeated fine-tuning is often impractical. To address this gap, we propose LiSA (Lifelong Safety Adaptation), a conservative policy induction framework that improves a fixed base guardrail through structured memory. LiSA converts occasional failures into reusable policy abstractions so that sparse reports can generalize beyond individual cases, adds conflict-aware local rules to prevent overgeneralization in mixed-label contexts, and applies evidence-aware confidence gating via a posterior lower bound, so that memory reuse scales with accumulated evidence rather than empirical accuracy alone. Across PrivacyLens+, ConFaide+, and AgentHarm, LiSA consistently outperforms strong memory-based baselines under sparse feedback, remains robust under noisy user feedback even at 20% label-flip rates, and pushes the latency--performance frontier beyond backbone model scaling. Ultimately, LiSA offers a practical path to secure AI agents against the unpredictable long tail of real-world edge risks.

Minbeom Kim, Lesly Miculicich, Bhavana Dalvi Mishra +6

Google Cloud AI Research · Google · Seoul National University

Continual Learning for LLM AgentsLLM GuardrailsAI Agent SafetyMemory-Augmented Language Model Agents
AskPDF
May 17, 2026

LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails

Guardrails are a critical safety layer for modern AI systems, but their operating regime is changing. As LLMs are deployed as customized assistants, safety policies are increasingly specified at inference time by users, organizations, or regulatory contexts. This makes safety enforcement fundamentally dynamic: the guardrail should adapt to changing safety policies without retraining. Yet this requirement creates a fundamental tension: faithfully judging complex policy contexts demands reasoning capability, while practical deployment requires low-latency responses. We introduce Latent Policy Guardrail (LPG), a guardrail framework that learnssemantic latent deliberation over dynamic policies. LPG compresses the internal deliberation needed for intent interpretation and policy grounding into continuous states supervised by decision-relevant semantics. At inference time, it generates only a compact verdict anchored to the violated policy clauses, preserving auditability while avoiding the latency of explicit reasoning. Across policy guardrail benchmarks, LPG-4B reaches 84.5% average safety accuracy and 77.9% F1 by compressing deliberation into just 10 latent tokens, outperforming the strongest dynamic baseline while running roughly 11 times faster than Qwen3-4B-Thinking under the single-sample evaluation setup. Code and data are available at https://github.com/SaFo-Lab/Latent_Policy_Guard.

Nanxi Li, Zhengyue Zhao, Chaowei Xiao

Johns Hopkins University

Language Model Safety EvaluationLLM GuardrailsLatent Reasoning
AskPDF