LLM Agent Security

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 76% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 234

All topics

One Goal, Many Commands: Characterizing Denylist Fragility in AI Agents
Jun 14, 2026Chuyang Chen, Zhiqiang LinAI Agent Security LLM Agent Security
The Ohio State University
Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems
Jun 13, 2026Yi Xie, Jiawei Du, Yu Cheng +2LLM Agent Security AI Agent Security Benchmarks
East China Normal University, Shanghai, China · Centre for Frontier AI Research A*STAR, Singapore · Shanghai Innovation Institute, Shanghai, China
Securing Multi-Agent GIS Systems: Risk Evaluation and Prompt Hardening Optimization
Jun 13, 2026Kyle Gao, Pranavi Kotta, Linlin Xu +2Adversarial Robustness LLM Red Teaming
Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada · Department of Mechatronics Engineering, University of Waterloo, Waterloo, ON, Canada · Department of Geomatics Engineering, University of Calgary, Calgary, AB, Canada +1
When Should Agent Trust Be Conditional? Characterizing and Attacking Skill-Conditional Reputation in Agent Swarms
Jun 12, 2026Yihan Xia, Taotao WangAgent Reliability LLM Agent Security
Shenzhen University, China
SEVRA-BENCH: Social Engineering of Vulnerabilities in Review Agents
Jun 11, 2026Rui Melo, Riccardo Fogliato, Sean Zhou +2Software Vulnerability Detection AI Agent Security
1Carnegie Mellon University · 2Microsoft Core AI · 3Independent Researcher +1
Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents
Jun 11, 2026Zihao Wang, Yiming Li, Yutong Wu +8AI Agent Evaluation LLM Agent Security
Nanyang Technological University, Singapore · ST Engineering, Singapore · IBM Research, USA +1
Smarter Saboteurs, Better Fixers: Scaling & Security in Linear Multi-Agent Workflows
Jun 10, 2026Timothy McAllister, Sina Abdidizaji, Ivan Garibay +1AI Agent Security LLM Agent Security
Department of Computer Science, University of Central Florida, Orlando, FL, USA · Department of Industrial Engineering, University of Central Florida, Orlando, FL, USA
SAIGuard: Communication-State Simulation for Proactive Defense of LLM Multi-Agent Systems
Jun 10, 2026Ruxue Shi, Yili Wang, Mengnan Du +4LLM Agent Security Multi-Agent System Security
School of Artificial Intelligence, Jilin University, Changchun, China · The Chinese University of Hong Kong, Shenzhen, China · School of Information and Communication Technology, Griffith University, Australia
Runtime Skill Audit: Targeted Runtime Probing for Agent Skill Security
Jun 10, 2026Tu Lan, Chaowei XiaoAI Agent Auditing LLM Agent Security
Johns Hopkins University
ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity
Jun 9, 2026Andrew Bo Liu, Samira Nedungadi, Bryce Cai +3LLM Agent Security AI Agent Security Benchmarks
SecureBio, Cambridge, MA, USA · Active Site, Cambridge, MA, USA
Understanding and mitigating the risks of OpenClaw for non-technical users: A practical guide with Skill
Jun 9, 2026Junchang Zheng, Junfeng Tan, Jialiang LinCybersecurity AI Agent Security
School of Computer Science and Engineering, Guangzhou Institute of Science and Technology, Guangzhou, China · Science and Education Evaluation Lab, Guangzhou Institute of Science and Technology, Guangzhou, China
Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation
Jun 9, 2026Yuchen Ling, Shengcheng Yu, Zhenyu Chen +1Prompt Injection LLM Agent Security
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Technical University of Munich, Heilbronn, Germany
Assessing Automated Prompt Injection Attacks in Agentic Environments
Jun 9, 2026David Hofer, Edoardo Debenedetti, Florian TramèrAdversarial Prompt Generation LLM Agent Security
ETH Zurich Zurich, Switzerland
SafeGEO: Understanding Generative Engine Optimization Risks in Recommendation Agents
Jun 8, 2026Qianfeng Wen, Yifan Simon Liu, Xin Liu +4Generative Engine Optimization LLM Agent Security
Department of Computer Science, University of Toronto · Department of Mechanical & Industrial Engineering, University of Toronto · Faculty of Information, University of Toronto +3
SecureClaw: Clawing Back Control of LLM Agents
Jun 8, 2026Yuhan Ma, Stefan SchmidData Leakage LLM Agent Security
1TU Berlin
Brain-Prompt Injection: A Route-Safety Audit for BCI-LLM Agents
Jun 8, 2026Jianwei TaiBrain-Computer Interfaces LLM Agent Security
School of Internet, Anhui University
Context-Fractured Decomposition Attacks on Tool-Using LLM Agents: Exploiting Artifact Provenance Gaps
Jun 8, 2026Xiaofeng Lin, Yukai Yang, Daniel Guo +3Jailbreak Attacks LLM Agent Security
University of California, Los Angeles, USA · 2Computer Science and Engineering Department, Texas A&M University, USA · 3Cisco, USA +1
Data Agents Under Attack: Vulnerabilities in LLM-Driven Analytical Systems
Jun 7, 2026Kuncan Wang, Ziting Wang, Peizhuo Lv +4LLM Agent Evaluation Cybersecurity
Nanyang Technological University, Singapore · The Hong Kong Polytechnic University · Tsinghua University
AgentTrust: A Self-Improving Trust Layer for AI-Agent Actions
Jun 7, 2026Chenglin YangAI Agent Reliability LLM-as-a-Judge
Independent Researcher
The Cold-Start Safety Gap in LLM Agents
Jun 5, 2026Chung-En Sun, Linbo Liu, Tsui-Wei WengLLM Agent Security AI Agent Safety Benchmarks
University of California, San Diego
Hierarchical Certified Semantic Commitment for Byzantine-Resilient LLM-Agent Collaboration
Jun 5, 2026Haoran Xu, Lei Zhang, Iadh Ounis +1Multi-Agent LLM Systems LLM Agent Security
University of Glasgow Glasgow, United Kingdom · University of Western Ontario London, Canada
Caught in the Act(ivation): Toward Pre-Output and Multi-Turn Detection of Credential Exfiltration by LLM Agents
Jun 2, 2026Kargi Chauhan, Pratibha RevankarData Leakage LLM Agent Security
University of California, Santa Cruz.
Ghost Tool Calls: Issue-Time Privacy for Speculative Agent Tools
Jun 1, 2026Bardia Mohammadi, Lars Klein, Akhil Arora +1Data Leakage LLM Agent Security
Max Planck Institute for Software Systems · EPFL · Aarhus University
SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agents
Jun 1, 2026Hao Cheng, Changtao Miao, Tianle Song +21AI Agent Evaluation LLM Agent Security
1The Hong Kong University of Science and Technology · 4The Hong Kong University of Science and Technology (Guangzhou) · 2Ant Digital Technologies, Ant Group +8
AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations
Jun 1, 2026Hiskias Dingeto, William LeeneyLLM Red Teaming LLM Agent Security
StackOne Technologies
Defenses & Enablers For Skill Injection Attacks on Terminal Based Agents
Jun 1, 2026Yoshinari Fujinuma, Varun Gangal, Traian Rebedea +4Adversarial Attacks on LLMs LLM Agent Security
Patronus AI · NVIDIA
"I Strongly Suspect This Website Is a Scam": Benchmarking PII Leakage and Detection without Defense in Autonomous Web Agents
May 30, 2026Soham Roy, Sarthakbrata Halder, Arya Bharaty +5Data Leakage LLM Agent Security
KIIT Bhubaneshwar · 2BITS Pilani · 3Lam Research
When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems
May 30, 2026Su Wang, Pin Qian, Yihang Chen +6Software Security AI Agent Security
Carnegie Mellon University · Georgia Institute of Technology · University of Glasgow +2
Stateful Online Monitoring Catches Distributed Agent Attacks
May 29, 2026Davis Brown, Samarth Bhargav, Arav Santhanam +7LLM Agent Security AI Agent Monitoring
University of Pennsylvania · 2Carnegie Mellon University
From Prompt Injection to Persistent Control: Defending Agentic Harness Against Trojan Backdoors
May 29, 2026Jiejun Tan, Zhicheng Dou, Xinyu Yang +4Prompt Injection LLM Agent Security
Gaoling School of Artificial Intelligence, Renmin University of China

Jun 14, 2026, cs.CRJ/K move · Enter open · S save

One Goal, Many Commands: Characterizing Denylist Fragility in AI Agents

Chuyang Chen, Zhiqiang Lin

The Ohio State University

AI Agent Security LLM Agent Security

Open paper AskPDF arXiv

The adoption of AI agents is increasing rapidly. Terminal AI agents, i.e., AI agents that run in terminal environments, are a widely used type of AI agents. Terminal AI agents rely heavily on shell command execution to interact with the host systems. They adopt a three-list command-gating mechanism to mitigate security risks introduced by command execution, with denylists serving as the load-bearing component. However, modern operating systems often ship a large, ever-expanding set of shell commands with complex functionalities. Our observation is that even a built-in denylist of Claude Code, well-maintained by its developers, can overlook bypass commands that invalidate its effectiveness. Such negligence leads to fragile command denylists that cannot even block operations that practitioners expect them to block. This paper presents the first systematic characterization of command denylist fragility in terminal AI agents. The paper formalizes the command denylist fragility problem and proposes an LLM-driven pipeline, ShellSieve, to detect such fragility. It prompts the LLM to propose possible bypasses and iteratively repairs them using feedback from a validator that executes them in a sandbox. In the evaluation, we applied ShellSieve to 1,709 real-world command denylists (containing 13,332 denylist rules) collected from GitHub. The evaluation shows several key findings, including that 69.0--98.6% of the denylists are fragile, that this fragility occurs consistently across projects and agents, and the validity of several possible root causes for this fragility. Our pipeline and findings will hopefully facilitate future research and practice regarding the command denylists used by AI agents.

Momentum

Related topics

Latest papers 234