cs.AINov 2, 2026

Cumulative and Audited Guardrail Tool Evolution for LLM Agent Safety

Authors: Xiao Yang, Aishan Liu, Siyuan Liang, Zonghao Ying, Le Wang, Jiakai Wang, Xianglong Liu, Dacheng Tao

Organizations: School of Computer Science and Engineering, Beihang University, Beijing, China. · State Key Laboratory of Complex & Critical Software Environment, Beihang University, Beijing, China. · College of Computing & Data Science, Nanyang Technological University, Singapore. · Zhongguancun Laboratory, Beijing, China.

Abstract

Large language model (LLM) agents increasingly operate in open-world environments where safety risks cannot be fully predefined. However, most existing guardrail mechanisms remain static, relying on fixed rules, prompts, or classifiers that generalize poorly to emerging risks. Here we present EVOLVE, a tool-level self-evolving defense framework that formulates agent security as a process of cumulative security evolution. Inspired by how human societies transform experiences into transferable protective knowledge, EVOLVE externalizes security knowledge into executable guardrail tools that are continuously inherited, independently audited, and iteratively refined through lifelong interaction. Across multiple benchmarks, backbone models, and embodied environments, EVOLVE consistently outperforms existing defenses across diverse risk scenarios with +22% average F1 score. We further validate the framework in real-world robotic and autonomous agent systems. Our results suggest that scalable agent safety may require continuously evolving defense systems rather than static guardrails.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety

    Oct 14, 2026Chenxu Wang, Xiaoyu Wen, Han Qi +4LLM Agent SafetyAdversarial Training

  2. SHE: Trajectory-driven Safety Harness Evolution for LLM Agents

    Aug 10, 2026Wanying Qu, Qinghua Mao, Yu Li +12Agent Harness OptimizationLLM Agent Safety

  3. Beyond Handcrafted Security: Towards Self-Evolving Defense for LLM Agents

    Aug 13, 2026Jiajun Ruan, Peiyang Li, Yukun Chen +2LLM Agent SecurityLLM Agent Safety