LLM Security

LLM: Large Language Model

LLM security examines how models and the systems around them can be manipulated, made to leak information, or induced to take unsafe actions. Attacks include prompt injection and jailbreaks, poisoned training data and backdoors, model extraction, side channels in shared serving systems, and vulnerabilities in generated code or tool-using agents. Defenses range from safer training and fine-tuning to runtime detectors, access controls, secure serving, and application-level isolation; evaluations increasingly use adaptive attacks and realistic agent, coding, and deployment benchmarks. A central tension is that model-level safeguards can fail under new inputs or system interactions, while stronger controls may reduce utility, privacy, or efficiency, making end-to-end testing and clear threat models essential.

Latest papers

327 papers45 papers in the last four weeks, up 45% on the four weeks before. 0.3% of all new papers.

InferD: A Honeypot Framework for AI Inference Infrastructure
Oct 29, 2026Hicham Hammouchi, Gabriele LenziniAI Agent Security LLM Security
SnT, University of Luxembourg Luxembourg, Luxembourg
Discovering Jailbreaks via Curriculum of Models
Oct 28, 2026Michał Burzyński, Noah T. Rossi, Shomit Basu +1LLM Security LLM Jailbreak Attacks
University of Warsaw · Independent · Northeastern University
Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness
Oct 27, 2026Igor Shilov, Xiaoxue Yang, Alexander Panfilov +6LLM Security Prompt Injection Defense
Imperial College London · Tübingen AI Center · ELLIS Institute Tübingen +2
Detecting and Mitigating Query-Based System Prompt Leakage Attacks
Oct 26, 2026Tushar Nayan, Pratik Poudel, Qian Lou +3LLM Security Privacy Leakage in Language Models
Florida International University · University of Central Florida · University of Texas at Arlington
Trustworthy Recursive Self-Improvement in the Era of Large Models: Foundation, Risks, Evaluation and Open Challenges
Oct 26, 2026Jiayue Pu, Ang Gao, Jinyi Bai +9Recursive Self-Improvement Language Model Self-Improvement
Gaoling School of Artificial Intelligence, Renmin University of China · University of Chinese Academy of Sciences
TrustJev: Separating Policy Authority from Untrusted Evidence in LLM-Based Decisions
Oct 26, 2026Zhiwei Xu, Xuyang Liu, Xiaolin Peng +8LLM Decision-Making LLM Security
KLISS, BNRist, School of Software, Tsinghua University, Beijing, China · ChinaBond Finance and Information Technology Co., Ltd., Beijing, China
All-in-One: A Composable Attack on LLM Transformations
Oct 23, 2026Xiaohua Zhan, Kazuki Egashira, Robin Staab +2LLM Security LLM Backdoor Attacks
ETH Zurich
Amplifying Latency and Cost of LLM Inference via Rowhammer Attack
Oct 22, 2026Mrityunjay Shukla, Shubham Roy, Biswabandan PandaLLM Security LLM Inference Efficiency
Indian Institute of Technology Bombay, Mumbai, India
Beyond End-to-End Recovery: Towards a Better Understanding of LLM Steganography via Rolewise Decomposition
Oct 21, 2026Om Tailor, Yucheng Pan, Solaleh Mohammadi +2Language Model Steganography LLM Security
University of Maryland, College Park
Combining Long-Term Memory and LLMs Comes at a High Cost: From Prompt Injection to Persistent Compromise
Oct 20, 2026Pedro H. Barcha Correia, Matheus Azevedo de Sá, Paulo H. Gomes de Senna +4LLM Security Prompt Injection Attacks on LLMs
Contextual Candour: Role-Aware Graduated Disclosure for LLMs over Mixed-Sensitivity Contexts
Oct 20, 2026Devansh Srivastav, David Pape, Samira Abedini +1LLM Security Privacy Leakage in Language Models
CISPA Helmholtz Center for Information Security, Saarbrücken, Germany
Fake News, Real Crashes: Information Injection Attacks on LLM Financial Trading Agents
Oct 19, 2026Mengxiao Wang, Nitesh SaxenaAdversarial Attacks on LLMs AI Agent Security
Texas A&M University
SPLASH: Contrastive Audit Pressure for Detecting and Localizing Security Hallucinations in Generated Code
Oct 19, 2026Mirza Masfiqur Rahman, MD Ajwad Akil, Imtiaz Karim +3LLM Security Software Vulnerability Detection
Purdue University, USA · University of Texas at Dallas, USA · Cisco Research, USA
MapSteal: Map-based Autonomous Knowledge Stealing Attack Against Large Language Models
Oct 19, 2026Zhengao Li, Chuxu Zhang, Yushun DongKnowledge Augmentation for Language Models LLM Training
University of Connecticut, Storrs, CT, USA · Florida State University, Tallahassee, FL, USA
LLM-Attack Suite: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM Systems
Oct 15, 2026Georgios Politis, Evangelos PappasLLM Security Privacy Leakage in Language Models
BiShield-TEE: On the (In-)Security of Unilateral Weight Obfuscation in On-Device TEE-Shielded LLM Partition
Oct 15, 2026Zhiyuan Wang, Yuezhi Zhou, Ziqi Zhang +3LLM Security On-Device Language Model Inference
Department of Computer Science and Technology, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China · Zhongguancun Laboratory, Beijing, China · University of Illinois Urbana-Champaign, IL, USA
Stronger Judges Do Not See More: Deployment-Integrity Blindness in LLM Evaluation
Oct 15, 2026Jianchang Su, Dawei Xiang, Yifan Zhang +1LLM Security LLM Agent Evaluation
University of Connecticut
From Training to Escalation: Forensic Analysis and Local Recreation of RL/RLHF-Induced Privilege Escalation in AI Agents (CVE-2026-65616)
Oct 14, 2026Alicia Colmenero-FernándezAI Agent Security LLM Security
IA-ismo LAB, Spain
Beyond Static Cues: Unified Dynamic Detection of MLLM Backdoors via Dual-Flow Trajectory Analysis
Oct 14, 2026Yinzhen Huang, Jiahe Xie, Yitong Sun +2Backdoor Detection Backdoor Defense in LLMs
Beihang University, Beijing, Beijing, China
What You Sign Is Not What You Run: The Verifiability Gap in Quantized LLM Supply Chains
Oct 14, 2026Wei-Liang Liu, Cheng-Yen TsaiLLM Security ML Reproducibility
National Yang Ming Chiao Tung University · National Taiwan University of Science and Technology
Rethinking Quality in Large Language Model Routing: Security-Conditioned Model-and-Budget Selection
Oct 13, 2026Yichi Zhang, Zhenxiang Huang, Yuyang Zhang +1LLM Routing LLM Security
The Pennsylvania State University · The University of Hong Kong
SpecCloak: Probing Harmful Code Generation through Engineering Specifications
Oct 13, 2026Wenbin Chen, Jiexin Wang, Guanting Wu +2LLM Security LLM Code Generation
South China University of Technology
Logit Leakage through Floating-Point Bias Addition
Oct 12, 2026Shaomin Chen, Xiaorui GongPrivacy Leakage in Language Models LLM Security
What the Model Cannot See: Covert and Provable Text Invisibility in LLMs
Oct 11, 2026Mark RussinovichLLM Security LLM Backdoor Attacks
Microsoft Azure
Security in LLM-Assisted Coding: A Systematic Literature Review
Oct 11, 2026Liang Yuan, Maryam Khanian Najafabadi, Mashud Rana +1LLM Security Software Security
Australian Catholic University, North Sydney, New South Wales, Australia
DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design
Oct 9, 2026Yuzhen Wang, Chunxiao Li, Baichen Le +6LLM Security AI Agent Safety Benchmarks
Shanghai Artificial Intelligence Laboratory · Fudan University
LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defense
Oct 8, 2026Luman Zhao, Minghui Xu, Yue Zhang +1LLM Security Prompt Injection Defense
Shandong University, Shandong Academy of Artificial Intelligence
SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routing
Oct 7, 2026Hui Zhang, Yachao Yuan, Jiayun Wang +3LLM Security LLM Safety
North China Electric Power University · Southeast University · Soochow University +1
Sensitive-Topic Leakage Through LLM Routing Metadata: Measurement and Mitigation
Oct 7, 2026Teng-Ruei ChenPrivacy Leakage in Language Models LLM Routing
Krixvon, Taipei 100, Taiwan
Secure Speculative Decoding for Large Language Models
Oct 6, 2026Yichi Zhang, Zhiqi Wang, Neil Gong +1Adversarial Attacks on LLMs Speculative Decoding
The Pennsylvania State University · Duke University

Oct 29, 2026J/K move · Enter open · S save

InferD: A Honeypot Framework for AI Inference Infrastructure

Hicham Hammouchi, Gabriele Lenzini

SnT, University of Luxembourg Luxembourg, Luxembourg

AI Agent Security LLM Security LLM Serving

Open paper AskPDF arXiv

Large language model serving stacks, commercial inference APIs, vector databases, and agent orchestration gateways are now ex- posed on the public internet and indexed by commodity scanners within hours of deployment. The threat characteristics of this sur- face are only beginning to be measured, and the honeypot frame- works that could measure it were designed for conventional net- work services rather than for AI infrastructure. In fact, emulating an AI service is a distinct deception problem: the observable behavior of the service, including model inventories, token formats, tool-call protocols, and authorization flows, must be reproduced faithfully, while the underlying computation, model inference and tool ex- ecution, must never occur. We present InferD, a low-interaction honeypot framework organized around a single principle we call deception without inference: every request an adversary can observe is answered from static response pools, and nothing an adversary submits is ever executed. InferD decomposes the emulation into four deception planes, transport, service protocol, vulnerability surface, and agentic session, adds a provenance plane that links a planted credential to evidence of downstream model-mediated processing.