LLM Security

LLM: Large Language Model

LLM security examines how models and the systems around them can be manipulated, made to leak information, or induced to take unsafe actions. Attacks include prompt injection and jailbreaks, poisoned training data and backdoors, model extraction, side channels in shared serving systems, and vulnerabilities in generated code or tool-using agents. Defenses range from safer training and fine-tuning to runtime detectors, access controls, secure serving, and application-level isolation; evaluations increasingly use adaptive attacks and realistic agent, coding, and deployment benchmarks. A central tension is that model-level safeguards can fail under new inputs or system interactions, while stronger controls may reduce utility, privacy, or efficiency, making end-to-end testing and clear threat models essential.

Latest papers

327 papers45 papers in the last four weeks, up 45% on the four weeks before. 0.3% of all new papers.

OptiLeak: Efficient Prompt Reconstruction via Reinforcement Learning in Multi-tenant LLM Services
Feb 24, 2026Longxiang Wang, Xiang Zheng, Xuhao Zhang +3Privacy Leakage in Language Models LLM Security
City University of Hong Kong · ByteDance Inc.
LLM-enabled Applications Require System-Level Threat Monitoring
Feb 23, 2026Yedi Zhang, Haoyu Wang, Xianglin Yang +2LLM Security LLM Reliability
Department of Computer Science, National University of Singapore, Singapore · School of Computing and Information Systems, Singapore Management University, Singapore
Hiding in Plain Text: Detecting Concealed Jailbreaks via Activation Disentanglement
Feb 23, 2026Amirhossein Farzam, Majid Behabahani, Mani Malek +2LLM Security Representation Disentanglement
Duke University · Princeton University · Morgan Stanley +2
TFL: Targeted Bit-Flip Attack on Large Language Model
Feb 19, 2026Jingkai Guo, Chaitali Chakrabarti, Deliang FanAdversarial Attacks Adversarial Attacks on LLMs
School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ, USA
DAVE: A Policy-Enforcing LLM Spokesperson for Secure Multi-Document Data Sharing
Feb 19, 2026René Brinkhege, Prahlad MenonCybersecurity LLM Security
Fraunhofer ISST Dortmund, Germany · Fraunhofer CMA Riverdale Park, MD, USA
Large-scale online deanonymization with LLMs
Feb 18, 2026Simon Lermen, Daniel Paleka, Joshua Swanson +3LLM Security
MATS · Anthropic
Recursive language models for jailbreak detection: a procedural defense for tool-augmented agents
Feb 18, 2026Doron ShavitJailbreak Detection LLM Security
ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Models
Feb 17, 2026Mitchell Piehl, Zhaohan Xi, Zuobin Xiong +2Adversarial Attacks on LLMs LLM Security
Department of Computer Science, The University of Iowa · School of Computing, State University of New York at Binghamton · Department of Computer Science, University of Nevada, Las Vegas +1
TensorCommitments: A Lightweight Verifiable Inference for Large Language Models
Feb 13, 2026Oguzhan Baser, Elahe Sadeghi, Eric Wang +5LLM Inference LLM Security
Electrical and Computer Engineering, The University of Texas at Austin · Theseus AI Labs · Electrical and Computer Engineering, McGill University +1
Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model
Feb 8, 2026Tianyi Wang, Huawei Fan, Yuanchao Shu +2LLM Security Adversarial Attacks on LLMs
Zhejiang University
Verifiable Manifest Signing and Transparency Enforcement for Secure MCP-Based LLM Pipelines
Jan 30, 2026Saeid Jamshidi, Kawser Wazed Nafi, Arghavan Moradi Dakhel +2LLM Auditing MCP Security
SWAT Laboratory, Polytechnique Montr´eal, Montr´eal, Canada · SæT Laboratory, Polytechnique Montr´eal, Montr´eal, Canada
From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching
Jan 30, 2026Zhixiang Zhang, Zesen Liu, Yuchong Xie +2Adversarial Attacks LLM Agent Security
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology · Fudan University
HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation
Jan 20, 2026Qirui Chen, Jingxian Shuai, Shuangwu Chen +8LLM Security Secure Code Generation
University of Science and Technology of China, China · Xi’an Jiaotong University, China · Nanjing University, China +1
SecureCode: A Production-Grade Multi-Turn Dataset for Training Security-Aware Code Generation Models
Dec 20, 2025Scott ThorntonSoftware Security LLM Security
One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs
Dec 14, 2025Yixin Tan, Zhe Yu, Rui Wen +1LLM Security Adversarial Attacks on LLMs
Institute of Science Tokyo, Tokyo, Japan · RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan
SELF: A Robust Singular Value and Eigenvalue Approach for LLM Fingerprinting
Dec 3, 2025Hanxiu Zhang, Qiang Xu, Yue ZhengLanguage Model Fingerprinting LLM Security
The Chinese University of Hong Kong, Shenzhen
Epistemic Bias Injection: Manipulating LLM Opinion via Selective Context Retrieval
Nov 30, 2025Hao Wu, Prateek SaxenaAdversarial Attacks on LLMs LLM Security
National University of Singapore
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
Oct 17, 2025Hanbin Hong, Shuang Wu, Shuya Feng +6LLM Security LLM Jailbreak Attacks
University of Connecticut · University of Alabama at Birmingham · Independent
MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation
Oct 11, 2025Wentian Zhu, Zhen Xiang, Wei Niu +1Adversarial Attacks on LLMs LLM Security
University of Georgia
From Construction to Injection: Edit-Based Fingerprints for Large Language Models
Sep 3, 2025Yue Li, Xin Yi, Dongsheng Shi +3Language Model Fingerprinting LLM Security
East China Normal University · Hasso Plattner Institute/University of Potsdam
Data Security in Large Language Models: Risks, Defense, and Directions
Aug 4, 2025Kang Chen, Xiuze Zhou, Yuanhui Yu +4Adversarial Attacks on LLMs LLM Security
School of Computer Engineering, Jimei University, Xiamen, 361021, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 511453, China · School of Informatics, Xiamen University, Xiamen, 361102, China +1
GPT, But Backwards: Exactly Inverting Language Model Outputs
Jul 2, 2025Adrians Skapars, Edoardo Manino, Youcheng Sun +1Privacy Leakage in Language Models LLM Security
Department of Computer Science, University of Manchester, Manchester, United Kingdom · Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates · Federal University of Amazonas, Manaus, Brazil
LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge
Jun 11, 2025Sahar Abdelnabi, Aideen Fay, Ahmed Salem +22LLM Security Indirect Prompt Injection
Microsoft · ISTA · Trend Micro +5
Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks
Apr 24, 2025Yang Liu, Kejia Zhang, Bingjie Yan +11Privacy-Preserving Language Models LLM Fine-Tuning
The Hong Kong Polytechnic University, China · Institute for AI Industry Research, Tsinghua University, China · Institute for AI Industry Research, Tsinghua University, China and Shanghai Jiao Tong University, China +3
The Challenge of Identifying the Origin of Black-Box Large Language Models
Mar 6, 2025Ziqing Yang, Yixin Wu, Yun Shen +3Language Model Fingerprinting LLM Security
CISPA Helmholtz Center for Information Security · NetApp · TikTok Inc.
UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models
Feb 18, 2025Huawei Lin, Yingjie Lao, Tony Geng +2LLM Backdoor Attacks Adversarial Attacks on LLMs
Rochester Institute of Technology · Tufts University · University of Rochester +1
BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models
Jun 3, 2024Jiaqi Xue, Mengxin Zheng, Yebowen Hu +3LLM Backdoor Attacks RAG Poisoning Attacks
University of Central Florida · Emory University · Samsung Research America

Feb 24, 2026J/K move · Enter open · S save

OptiLeak: Efficient Prompt Reconstruction via Reinforcement Learning in Multi-tenant LLM Services

Longxiang Wang, Xiang Zheng, Xuhao Zhang, Yao Zhang+2

City University of Hong Kong · ByteDance Inc.

Privacy Leakage in Language Models LLM Security Adversarial Attacks on LLMs

Open paper AskPDF arXiv

Multi-tenant LLM serving frameworks widely adopt shared Key-Value caches to enhance efficiency. However, this creates side-channel vulnerabilities enabling prompt leakage attacks. Prior studies identified these attack surfaces yet focused on expanding attack vectors rather than optimizing attack performance, reporting impractically high attack costs that underestimate the true privacy risk. We propose OptiLeak, a reinforcement learning-enhanced framework that maximizes prompt reconstruction efficiency through two-stage fine-tuning. Our key insight is that domain-specific ``hard tokens'' -- terms difficult to predict yet carrying sensitive information -- can be automatically identified via likelihood ranking and used to construct preference pairs for Direct Preference Optimization, eliminating manual annotation. This enables effective preference alignment while avoiding the overfitting issues of extended supervised fine-tuning. Evaluated on three benchmarks spanning medical and financial domains, OptiLeak achieves up to

12.48\times

reduction in average requests per token compared to baseline approaches, with consistent improvements across model scales from 3B to 14B parameters. Our findings demonstrate that cache-based prompt leakage poses a more severe threat than previously reported, underscoring the need for robust cache isolation in production deployments.