LLM Security

LLM: Large Language Model

LLM security examines how models and the systems around them can be manipulated, made to leak information, or induced to take unsafe actions. Attacks include prompt injection and jailbreaks, poisoned training data and backdoors, model extraction, side channels in shared serving systems, and vulnerabilities in generated code or tool-using agents. Defenses range from safer training and fine-tuning to runtime detectors, access controls, secure serving, and application-level isolation; evaluations increasingly use adaptive attacks and realistic agent, coding, and deployment benchmarks. A central tension is that model-level safeguards can fail under new inputs or system interactions, while stronger controls may reduce utility, privacy, or efficiency, making end-to-end testing and clear threat models essential.

Latest papers

327 papers45 papers in the last four weeks, up 45% on the four weeks before. 0.3% of all new papers.

Omission Constraints Decay While Commission Constraints Persist in Long-Context LLM Agents
Apr 22, 2026Yeran GamageAI Agent Monitoring LLM Security
University of South Florida · Independent AI Security Researcher
STAR-Teaming: A Strategy-Response Multiplex Network Approach to Automated LLM Red Teaming
Apr 21, 2026MinJae Jung, YongTaek Lim, Chaeyun Kim +3LLM Security LLM Jailbreak Attacks
DATUMO INC.
An Empirical Study of Multi-Generation Sampling for Jailbreak Detection in Large Language Models
Apr 20, 2026Hanrui Luo, Shreyank N GowdaLanguage Model Safety Evaluation LLM Auditing
School of Computer Science, University of Nottingham, Nottingham, United Kingdom.
Understanding Secret Leakage Risks in Code LLMs: A Tokenization Perspective
Apr 20, 2026Meifang Chen, Zhe Yang, Huang Nianchen +4Data Leakage Memorization in Language Models
†The Chinese University of Hong Kong · ‡Nanyang Technological University · §University of Southern California +1
Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems
Apr 19, 2026Yuji Yamamoto, Satoshi MatsuuraLLM Serving Adversarial Attacks on LLMs
Institute of Science Tokyo, Tokyo, Japan
CASCADE: A Component Ablation and Corpus Audit of a Layered Local Defense for MCP-Based Systems
Apr 18, 2026İpek Abasıkeleş Turgut, Edip GümüşMCP Security LLM Security
Department of Computer Engineering, Faculty of Engineering and Natural Sciences, Iskenderun Technical University, Hatay, Türkiye · Department of Computer Engineering, Institute of Graduate Studies, Iskenderun Technical University, Hatay, Türkiye
SafeLM: Unified Privacy-Aware Optimization for Trustworthy Federated Large Language Models
Apr 17, 2026Noor Islam S. Mohammad, Uluğ BayazıtLLM Hallucination Mitigation Privacy-Preserving Language Models
Istanbul Technical University
Hijacking Large Audio-Language Models via Context-Agnostic and Imperceptible Auditory Prompt Injection
Apr 16, 2026Meng Chen, Kun Wang, Li Lu +2Audio-Language Models LLM Security
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security · National University of Singapore +1
TEMPLATEFUZZ: Fine-Grained Chat Template Fuzzing for Jailbreaking and Red Teaming LLMs
Apr 14, 2026Qingchao Shen, Zibo Xiao, Lili Huang +3Adversarial Attacks on LLMs LLM Security
School of Computer Software, Tianjin University, Tianjin, China · Monash University, Monash, Australia
The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systems
Apr 13, 2026Yihao Zhang, Kai Wang, Jiangrong Wu +7LLM Security LLM Jailbreak Attacks
School of Mathematical Sciences, Peking University · Sun Yat-sen University · Wuhan University +3
Machine Learning-Based Detection of MCP Attacks
Apr 12, 2026Tobias Mattsson, Samuel Nyberg, Anton Borg +1Classification MCP Security
Blekinge Institute of Technnology, Sweden · Department of Computer Science, Blekinge Institute of Technology, Sweden · Ericsson, Sweden +1
The Defense Trilemma: Why Prompt Injection Defense Wrappers Fail?
Apr 7, 2026Manish Bhatt, Sarthak Munshi, Vineeth Sai Narajala +6Prompt Injection Defense LLM Security
OWASP · Amazon Leo · UNO
Broken by Default: A Formal Verification Study of Security Vulnerabilities in AI-Generated Code
Apr 7, 2026Dominik Blain, Maxime NoiseuxLLM Security LLM Code Generation
Cobalt AI Gatineau, QC, Canada
SALLIE: Generation-Free Hidden-State Detection of Jailbreaks and Prompt Injections Across Text and Vision
Apr 6, 2026Guy Azov, Ofer Rivlin, Guy ShtarPrompt Injection Detection LLM Security
Intuit
A Patch-based Cross-view Regularized Framework for Backdoor Defense in Multimodal Large Language Models
Apr 6, 2026Tianmeng Fang, Yong Wang, Zetai Kong +4Backdoor Defense in LLMs LLM Security
School of Computing and Information Systems, Singapore Management University, Singapore, 178902, , Singapore. · School of Artificial Intelligence, China University of Mining and Technology, Beijing, 102206, Beijing, PR China. · Faculty of Arts, The University of Melbourne, Melbourne, Carlton VIC 3053, Australia. +3
CoopGuard: Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Round Attacks
Apr 5, 2026Siyuan Li, Zehao Liu, Xi Lin +6LLM Security Adversarial Robustness of Language Models
School of Computer Science, Shanghai Jiao Tong University · Department of Computer Science, University of Illinois Urbana-Champaign · Big Data Institute, Central South University
Emergent Inference-Time Semantic Contamination via In-Context Priming
Apr 5, 2026Marcin AbramLLM Security In-Context Learning
Los Angeles, California 90089, USA
An Independent Safety Evaluation of Kimi K2.5
Apr 3, 2026Zheng-Xin Yong, Parv Mahajan, Andy Wang +12Language Model Safety Evaluation LLM Security
Constellation · Brown University · Georgia Institute of Technology +7
Understanding the Effects of Safety Unalignment on Large Language Models
Apr 2, 2026John T. HalloranLLM Safety Alignment LLM Security
Leidos
TRACE-Bot: Detecting Emerging LLM-Driven Social Bots via Implicit Semantic Representations and AIGC-Enhanced Behavioral Patterns
Apr 2, 2026Zhongbo Wang, Zhiyu Lin, Zhu Wang +1Social Bot Detection LLM Security
School of Cyber Science and Engineering, Sichuan University, Sichuan 610207, China · School of Software and Microelectronics, Peking University, Beijing 102600, China · College of Computer Science, Sichuan University, Sichuan 610065, China +1
SelfGrader: LLM Jailbreak Detection via Anchored Token-Level Logits
Apr 1, 2026Zikai Zhang, Rui Hu, Olivera Kotevska +1LLM Guardrails LLM Security
Department of Computer Science and Engineering, University of Nevada, Reno, Reno, USA · Oak Ridge National Laboratory, Oak Ridge, USA
Evaluation and Hardening of LLM System Instructions Against Extraction via Encoding Attacks
Apr 1, 2026Anubhab Sahu, Diptisha Samanta, Reza SoosahabiLLM Security Privacy Leakage in Language Models
Keysight Technologies, India · Keysight Technologies, USA
Security in LLM-as-a-Judge: A Comprehensive SoK
Mar 31, 2026Aiman Al Masoud, Antony Anju, Marco Arazzi +6LLM-as-a-Judge LLM Security
Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy · Department of Computer Applications, Cochin University of Science and Technology, Kerala, India
CivicShield: A Cross-Domain Defense-in-Depth Framework for Securing Government-Facing AI Chatbots Against Multi-Turn Adversarial Attacks
Mar 30, 2026KrishnaSaiReddy PatilLLM Security Cybersecurity
Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
Mar 30, 2026Bhavuk Jain, Sercan Ö. Arık, Hardeo K. ThakurAdversarial Attacks on Unified Multimodal Models LLM Security
Google · Bennett University, India
Prompt Attack Detection with LLM-as-a-Judge and Mixture-of-Models
Mar 26, 2026Hieu Xuan Le, Benjamin Goh, Quy Anh TangPrompt Injection Detection LLM Guardrails
GovTech, Singapore
Secure On-Premise Deployment of Open-Weights Large Language Models in Radiology: An Isolation-First Architecture with Prospective Pilot Evaluation
Mar 25, 2026Sebastian Nowak, Jann-Frederick Laß, Narine Mesropyan +8LLM Serving LLM Security
Department of Diagnostic and Interventional Radiology, University Hospital Bonn · Department of Research-IT, University Hospital Bonn
How Vulnerable Are Edge LLMs?
Mar 25, 2026Ao Ding, Hongzong Li, Zi Liang +5LLM Security
China University of Geoscience Beijing · Hong Kong University of Science and Technology · Hong Kong Polytechnic University +4
Not All Tokens Are Created Equal: Query-Efficient Jailbreak Fuzzing for LLMs
Mar 24, 2026Wenyu Chen, Xiangtao Meng, Chuanchao Zang +6LLM Security LLM Jailbreak Attacks
Shandong University
Chain-of-Authorization: Embedding authorization into large language models
Mar 24, 2026Yang Li, Yule Liu, Xinlei He +3LLM Security LLM Safety
Department of Computer Science and Technology, Tsinghua University . · Data Science and Analytic Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou) . · Institude for Math and AI, Wuhan University .

Apr 22, 2026J/K move · Enter open · S save

Omission Constraints Decay While Commission Constraints Persist in Long-Context LLM Agents

Yeran Gamage

University of South Florida · Independent AI Security Researcher

AI Agent Monitoring LLM Security Instruction Following LLM Safety Evaluation LLM Agent Safety

Open paper AskPDF arXiv

LLM agents deployed in production operate under operator-defined behavioral policies (system-prompt instructions such as prohibitions on credential disclosure, data exfiltration, and unauthorized output) that safety evaluations assume hold throughout a conversation. Prohibition-type constraints decay under context pressure while requirement-type constraints persist; we term this asymmetry Security-Recall Divergence (SRD). In a 4,416-trial three-arm causal study across 12 models and 8 providers at six conversation depths, omission compliance falls from 73% at turn 5 to 33% at turn 16 while commission compliance holds at 100% (Mistral Large 3,

p < 10^{-33}

). In the two models with token-matched padding controls, schema semantic content accounts for 62-100% of the dilution effect. Re-injecting constraints before the per-model Safe Turn Depth (STD) restores compliance without retraining. Production security policies consist of prohibitions such as never revealing credentials, never executing untrusted code, and never forwarding user data. Commission-type audit signals remain healthy while omission constraints have already failed, leaving the failure invisible to standard monitoring.