cs.CRJun 22, 2026

Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies

Authors: Ruixiao LinXinhao DengQingming LiJianan MaYunhao FengYuqi QingZhenyuan LiYechao Zhang+7 more

Organizations: 1Zhejiang University · 2Ant Group · 3Tsinghua University · 4Hangzhou Dianzi University · 5Nanyang Technological University · 6Fudan University

Abstract

Self-evolving LLM agent systems, which autonomously update their model parameters, memory, tools, and architectures, introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded, self-amplify across generations, and propagate through populations without sustained attacker access. We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface (MLAS) matrix, which decomposes the attack surface into five functional modules (Brain, Cognitive Resource, Execution, Self-Design, Collective) ×\times five lifecycle stages (Bootstrap, Propose, Evaluate, Commit, Serve). Analysis of the resulting 25 cells reveals that 17 face critical threats for which no effective partial mitigation. We identify seven cross-cutting amplification effects that interact synergistically and cannot be addressed by securing individual modules in isolation. Comparative case studies of two open-source frameworks demonstrate that evolution-native design activates 3.5×3.5\times more attack surface cells and achieves a 100% attack persistence rate (40/40 payloads across all CIA+Privacy categories), while co-located security scanners block only 2.5% of attacks. Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent, gives rise to entirely new attack classes, and renders static defenses structurally inadequate, motivating evolution-aware security frameworks and formal verification for self-modifying systems.

Explore similar work

CardsList