Adversarial Attacks on LLMs
LLM: Large Language Model
Overview
This field studies how attackers can make large language models produce unsafe or unreliable outputs, leak information, manipulate judgments, or misuse tools. Common approaches include black-box prompt search and multi-turn red teaming, gradient-based suffix optimization, reinforcement-learning attackers, and interventions on model activations or weights. Researchers evaluate these attacks on benchmarks such as HarmBench, AdvBench, and JailbreakBench, as well as in multilingual, multimodal, and tool-using agent settings. A central challenge is that adaptive attacks often expose weaknesses missed by static tests, while defenses must improve robustness without excessive false refusals, lost capability, or new system-level vulnerabilities.
Latest papers
385 papers30 papers in the last four weeks, down 40% on the four weeks before. 0.2% of all new papers.
From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
Diffusion LLMs are Natural Adversaries for any LLM
Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks
MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation
Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization
NonTextual Target Attack
There are Levels to It: Red Teaming Dialogue With Hierarchical Reinforcement Learning
Data Security in Large Language Models: Risks, Defense, and Directions
Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Models
Logit-Gap Steering: A Forward-Pass Diagnostic for Alignment Robustness
In-Context Bias Propagation in LLM-Based Tabular Data Generation
Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models
GPT-2, Llama-3, Gemma-2, Qwen2.5, and commercial models GPT-4.1 and GPT-5-Chat. This work advances the field of LLM safety by providing a systematic and effective approach to automated red-teaming, ultimately supporting the responsible deployment of LLMs. Our code is available at https://github.com/lamda-bbo/QDRT.