LLM Agent Security

LLM: Large Language Model

The field examines how agents can be manipulated through indirect prompt injection, malicious tools or skills, poisoned memory, excessive permissions, and multi-agent interactions, sometimes leading to data leaks or harmful actions. Researchers evaluate these risks with red-team attacks and benchmarks that track agent trajectories, tool calls, and real effects—not just final answers. Defenses include runtime authorization and least privilege, sandboxing, provenance and information-flow controls, monitoring, and security-focused training. A central challenge is protecting agents over long, changing workflows without blocking legitimate tasks, while making evaluations realistic enough to capture adaptive attacks and deployment risks.

Latest papers

332 papers44 papers in the last four weeks, up 33% on the four weeks before. 0.3% of all new papers.

Pre-Crime for Agents, at the Last Second: Probing When a Recruited Agent Gives In
Oct 29, 2026Leonidas RaghavLLM Agent Security AI Agent Monitoring
Independent
Scan Before You Install: Detecting and Mitigating Local and Composite Vulnerabilities in Skills
Oct 27, 2026Parth A. Gandhi, Avital Aviv, Prasanna N. Wudali +2Agent Skill Security Auditing Software Vulnerability Detection
Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Beer-Sheva, Israel
CoSA: The Conflict of Skills Attack in Real-World LLM Agent Ecosystems
Oct 26, 2026Yujia Zheng, Tianhao Li, Shangzhi Xu +6LLM Agent Security LLM Agent Safety
University of Science and Technology Beijing · ZODA · Fangcun Leap +5
LLM WikiLeaks: Uncovering URL-Based Capability Extension and Security Circumvention in Agentic Message Boards
Oct 25, 2026Orel Navi, Elad Feldman, Ben NassiLLM Agent Security Web Agents
Tel Aviv University, Tel Aviv, Israel
ProbeID: Verifying MCP Tool Identity for LLM Agents under Registry Drift
Oct 24, 2026Guanxing Chen, Sining Zhang, Haohuai He +3LLM Agent Verification AI Agent Security
City University of Hong Kong (Dongguan), Dongguan, China · The Hong Kong Polytechnic University, Hong Kong SAR, China · Zhengzhou University, Zhengzhou, China +1
A Deterministic Security Runtime for LLM Agents on MPI and Batch-Scheduled HPC Systems: Consensus-Governed, Hardware-Rooted Supervision of Every Rank
Oct 23, 2026Alfredo MetereLLM Agent Security Runtime Enforcement for AI Agents
Enclawed, Inc., California, USA
AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack
Oct 22, 2026Justin Tan, Miao Li, Ding ZhaoLLM Agent Security AI Agent Security Benchmarks
Carnegie Mellon University
Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems
Oct 21, 2026Daniel Ruiz, Luca Luceri, Emilio FerraraLLM Agent Security Privacy Leakage in Language Models
Information Sciences Institute University of Southern California Los Angeles, CA, USA
Disclosure Begets Disclosure: A Self-Reinforcing Privacy Loop in LLM Agents with Persistent Memory
Oct 20, 2026Masahiro Kaneko, Hiromi Arai, Timothy BaldwinLLM Agent Security Privacy Protection for LLM Agents
MBZUAI · Third Intelligence · RIKEN AIP
The Show-Up Attack: When Subagents Win Over Their Supervisor
Oct 20, 2026Maosen Zhang, Jianshuo Dong, Yutong Zhang +3LLM Agent Security Multi-Agent System Security
Tsinghua University · SiliconProspect AI · Nanyang Technological University
PROVISO: Provenance-Informed Action Selection for Secure LLM Agents
Oct 19, 2026Bo Yan, Qi Zhang, Ziwen Kan +2LLM Agent Security LLM Agents
University of Central Florida · Independent Researcher · University of Tennessee, Knoxville
Unbounded Losses, Unchecked Inputs: Measuring the Safeguards of LLM Options-Trading Agents
Oct 16, 2026Mengxiao Wang, Nitesh SaxenaLLM Agent Security AI Agent Security Benchmarks
Texas A&M University
SPIDES: Information-Flow Control for Browser Agents
Oct 16, 2026Egor Zverev, Juhee Kim, Leon Trampert +4LLM Agent Security Information Flow Control
ISTA · Microsoft · CISPA Helmholtz Center for Information Security
MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection
Oct 16, 2026Songping Wang, Peiyang Li, Yi Huang +5LLM Agent Security AI Agent Security Benchmarks
AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents
Oct 13, 2026Chengquan Guo, Zhaorun Chen, Zinan Lin +2Privacy Protection for LLM Agents Contextual Integrity
University of Chicago · Microsoft Research · University of Illinois Urbana-Champaign
Kairos: Transactional Security Enforcement at the System Boundary for LLM Agents
Oct 13, 2026Kristoffer Hempel, Nils Loose, Lorenzo Cavallaro +1LLM Agent Security Runtime Enforcement for AI Agents
Universität zu Lübeck · University College London · Mohamed bin Zayed University of Artificial Intelligence
Cost-Efficient Privacy Enforcement for Email-Generating LLM Agents under Strategic Information Requests
Oct 13, 2026Qikun Cai, Tao Huang, Chen HouPrivacy Protection for LLM Agents LLM Agent Security
ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities?
Oct 13, 2026Fasheng Miao, Zhaoyu Hu, Xiang Li +5AI Agent Security Benchmarks LLM Agent Security
Tsinghua University · Alibaba Group · Nankai University +1
From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents
Oct 11, 2026Binhong Wang, Haidong Kang, Yanfeng Zhang +3LLM Agent Security Edge AI
Northeastern University, China · Department of Computer Science, Hong Kong Baptist University, Hong Kong · Institute of Intelligent Manufacturing, Nanjing Tech University, Nanjing, China +1
APEX-Harm: Evaluating Agent Safety in Shared Professional Workspaces
Oct 9, 2026Ziyi Yang, Adel Bibi, Guohao Li +2AI Agent Security LLM Agent Security
King Abdullah University of Science and Technology · University of Oxford
One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails
Oct 8, 2026Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud PedramLLM Guardrails LLM Agent Security
University of Southern California (USC), USA
Loose Lips: Understanding Privacy Leakage in Agentic Dialogue
Oct 7, 2026Soumya Sharma, Ayana Hussain, Heber Hwang Arcolezi +4Privacy Leakage in Language Models LLM Agent Security
Mila – Quebec AI Institute · McGill University · Simon Fraser University +3
From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage
Oct 7, 2026Saimon Amanuel Tsegai, Alex Kantchelian, Danfeng +2LLM Agent Security AI Agent Reliability
Daphne · North Carolina State University · Virginia Tech
Semantic Behavioral Watermarking: Paraphrase-Robust and Forgery-Resistant Provenance for LLM Agents
Oct 6, 2026Suxin Ji, Hungtao Wan, Shaoxuan Chen +1LLM Watermarking Robust Watermarking
University of Pennsylvania · Independent Researcher · University of Massachusetts Amherst +1
CredLeakBench: Evaluating Credential Leakage and Recovery in LLM Agents
Oct 6, 2026Rafid Ahmed, Joseph Fioresi, Mubarak Shah +1AI Agent Security LLM Agent Security
Institute of Artificial Intelligence, University of Central Florida
What May an Agent Change About Itself? A Containment Floor for Self-Configuring Agent Runtimes
Oct 5, 2026Sajib Hossain, Moeen Uddin MahmudLLM Agent Security Runtime Enforcement for AI Agents
North South University · HelicanHQ
Where Did the Repair First Go Wrong? Localizing the Origins of Silent Failures in Agentic Vulnerability Repair
Oct 5, 2026Wenji Bai, Muhammad Waseem, Zeeshan Rasheed +2Agent Failure Analysis Automated Program Repair
Tampere University, Tampere, Finland
AgentSpy: Making AI Agent Behavior Observable
Oct 5, 2026Christoph Bühler, Matteo Biagiola, Luca Di Grazia +1AI Agent Reliability AI Agent Auditing
University of St. Gallen, St. Gallen, SG, Switzerland · University of St. Gallen and Università della Svizzera italiana (USI), St. Gallen and Lugano, SG and TI, Switzerland
Runaway Reaction: When Benign Skills Compose into Malicious Behavior
Oct 5, 2026Zunlong Zhou, Ziyuan Yang, Mengyu Sun +1AI Agent Security LLM Agent Security
Sichuan University
The Same Zero: Why Identical ASR Can Imply Different Guarantees in LLM-Agent Security
Oct 3, 2026YaJie YinLLM Agent Security LLM Agent Evaluation

Oct 29, 2026J/K move · Enter open · S save

Pre-Crime for Agents, at the Last Second: Probing When a Recruited Agent Gives In

Leonidas Raghav

Independent

LLM Agent Security AI Agent Monitoring LLM Guardrails Linear Probing

Open paper AskPDF arXiv

When AI agents share channels and credentials, one agent can press another to break its operator's rule, a first step towards collusion. Output filters check only what an agent has already written. Can an overseer tell beforehand that an agent is about to give in? We show that a linear probe on the agent's activations can, in a controlled multi-agent setting. Across six scenarios, a peer presses a target for up to 20 rounds to reveal a secret, such as a deploy token; the target often refuses, then gives in. Read at each model call before its output is written, the probe picks out the one that leaks. On held-out Qwen3-32B episodes, at a threshold set to flag 5% of benign calls, it catches 82% of leaking calls. At a similar flag rate, a zero-shot decision-model monitor catches 26%. On gpt-oss-20b, a probe on its reasoning reaches AUROC 0.91. A bag-of-words classifier trained on the same calls comes close, so most of the gain comes from training on labels, not activations. The probe also flags permitted disclosures: it reads the coming disclosure more than the violation. The warning is short-range, and halting on any alarm would stop most benign episodes, which a second text check does not fix. A warning raised before generation could let an overseer stop an agent that is about to give in, ahead of others acting on its messages or recruitment spreading through a swarm.