cs.CRMar 8, 2026

From Thinker to Society: Security in Hierarchical Autonomy Evolution of AI Agents

Authors: Xiaolei Zhang, Lu Zhou, Xiaogang Xu, Jiafei Wu, Tianyu Du, Heqing Huang, Hao Peng, Zhe Liu

Organizations: Nanjing University of Aeronautics and Astronautics, China · Collaborative Innovation Center of Novel Software Technology and Industrialization, China · The Chinese University of Hong Kong, China · The University of Hong Kong, China · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Artificial Intelligence (AI) agents have evolved from passive predictive tools into active entities capable of autonomous decision-making and environmental interaction, driven by the reasoning capabilities of Large Language Models (LLMs). However, this evolution has introduced critical security vulnerabilities that existing frameworks fail to address. The Hierarchical Autonomy Evolution (HAE) framework organizes agent security into three tiers: Cognitive Autonomy (L1) targets internal reasoning integrity; Execution Autonomy (L2) covers tool-mediated environmental interaction; Collective Autonomy (L3) addresses systemic risks in multi-agent ecosystems. We present a taxonomy of threats spanning cognitive manipulation, physical environment disruption, and multi-agent systemic failures, and evaluate existing defenses while identifying key research gaps. The findings aim to guide the development of multilayered, autonomy-aware defense architectures for trustworthy AI agent systems.

Figures & tables

PerspectiveRepresentativeTaxonomyLimitationsHAE Advantages
LifecycleWang et al. ( wang2025comprehensive )Data, Training, DeploymentStatic model focus; overlooks runtime interaction risks.Centers on real-time evolutionary risks in open-world interactions.
Trustworthy AttributesYu et al. ( yu2025survey )Safety, Privacy, Fairness, RobustnessFragments causal chains; misses cross-attribute cascading.Reveals capability-threat symbiosis and cross-layer propagation.
ComponentDeng et al. ( deng2025ai )Brain, Memory, Tools, PerceptionTreats components as isolated; lacks emergent perspective.Integrates component interactions; reveals qualitative changes from combinations.
Autonomy StructuralSu et al. ( su2025survey )L1-L5 autonomy levels; internal architectural fragilitiesSingle-agent focused; insufficient on multi-agent societal risks.Explicitly proposes L3 Collective Autonomy; extends to systemic governance.
HAE (Ours)This workL1 Cognition – L2 Execution – L3 Collective—Unifies micro-cognition and macro-society evolution; fills societal-level security gap.
Table 1. Comparison between HAE Framework and Existing Papers
Figure 1. Illustration of the HAE Framework. This framework delineates the co-evolution of agent capabilities and emergent threats across three distinct levels. Specifically, L1 (Cognitive Autonomy) is associated with internal risks targeting the Brain and Memory, such as Cognitive Hijacking and Memory Corruption (Section ). L2 (Executional Autonomy) is linked to external risks arising from Tool use and Environment interactions, including Confused Deputy attacks and Unsafe Action Chains (Section ). L3 (Collective Autonomy) concerns systemic threats in Multi-Agent Systems (MAS), such as Malicious Collusion and Viral Infection (Section ). Illustration of the HAE Framework.
Figure 2. Agent architecture showing perception, brain, memory, and action modules with security risks. Red triangles indicate critical security risks across perception, cognition, memory, and action, including prompt injection, cognitive manipulation, memory poisoning, and unsafe tool execution, illustrating how untrusted inputs may propagate into real-world consequences.
More in this topic

AI Agent Security

Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.

See all 508 papers

Explore similar work

CardsList

  1. The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey

    Mar 11, 2026Juhee Kim, Xiaoyuan Liu, Zhun Wang +4AI Agent SecurityLLM Agent Security

    UC Berkeley / Seoul National University, USA · UC Berkeley, USA · University of Illinois Urbana-Champaign, USA +1

  2. Security Attack and Defense Strategies for Autonomous Agent Frameworks: A Layered Review with OpenClaw as a Case Study

    Apr 30, 2026Luyao Xu, Xiang ChenAI Agent SecurityLLM Agent Security

    School of Artificial Intelligence and Computer Science, Nantong University, Nantong, China · State Key Lab. for Novel Software Technology, Nanjing University, Nanjing, China

  3. A Security Analysis of Long-Horizon Agentic AI Systems: Threats, Evaluation, and Framework Development

    Jun 12, 2026Ahmed Mohammed Almalki, Mehedi MasudAI Agent Security

    Department of Computer Science College of Computers and Information Technology Taif University, KSA