LLM Red Teaming

LLM: Large Language Model

LLM red teaming studies how adversarial prompts and interactions expose failures in language models, from harmful text generation to unsafe tool use by agents. Approaches include jailbreak prompts, multi-turn dialogue, prompt-injection attacks, reinforcement learning, evolutionary search, and human-guided testing, with evaluations spanning general safety, coding, health, finance, and cybersecurity. As systems gain tools and longer workflows, tests increasingly track executed actions, attack trajectories, severity, and transfer across models rather than relying only on single-turn pass/fail scores. A central tension is that adaptive attacks uncover failures missed by static benchmarks, while limited coverage, judge disagreement, and uneven transfer make it difficult to infer real-world safety from a clean evaluation.

Latest papers

63 papers6 papers in the last four weeks, up 50% on the four weeks before. 0.0% of all new papers.

Red-Teaming Large Language Models in Low-Resource African Languages: A Case Study of Yoruba and Nigerian Pidgin
Oct 29, 2026Paula Chikaodinaka Anyanwu, Valdini Douglace Lemofouet, Blessing Ngozi Uzor +7Language Model Safety Evaluation LLM Red Teaming
African Institute for Mathematical Sciences (AIMS), Cameroon · Independent Researcher · Brown University +5
The Target Agent Is Its Own Best Critic but a Poor Proposer: Self-Improved Red Teaming
Oct 26, 2026Mintong Kang, Stephen Rawls, Melissa Kazemi Rad +2LLM Red Teaming Automated Red Teaming
University of Illinois Urbana-Champaign · Capital One
Black-Box Red-Teaming of Multi-Agent Systems via Reinforcement Learning
Oct 20, 2026Yuqi Jia, Hangsheng Zhang, Chenxi Liu +2LLM Red Teaming Black-Box Adversarial Attacks
Duke University · ByteDance · University of Maryland, College Park
Budgeted Attack Scheduling for LLM Red-Teaming: The Roles of Static Priors, Semantic Guidance, and Online Feedback
Oct 19, 2026Zhicheng Fang, Han Sun, Jiachen Lei +5LLM Red Teaming Language Model Safety Evaluation
Shanghai Qi Zhi Institute · East China Normal University · The Chinese University of Hong Kong +3
Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming
Oct 14, 2026Gosha Kozhevnikov, Tyler TracyLLM Red Teaming Automated Red Teaming
Redwood Research
RedDarwin: Evolving Jailbreak Skills against Frontier LLMs
Oct 14, 2026Zhouyuan Ma, Yutao Wu, Hanxun Huang +3LLM Jailbreak Attacks LLM Red Teaming
Fudan University · Deakin University · University of Melbourne +2
ATTCK-R: Knowledge-Aware Long-Horizon Jailbreaking for Protected LLMs
Oct 12, 2026Md Mushfiqur Rahman, Tobias Rohde, Rahul Tripathi +2LLM Jailbreak Attacks LLM Red Teaming
George Mason University · Amazon
No One Architecture Fits All: A Cross-Environment Evaluation of Hierarchical Red Team Agents
Sep 30, 2026Ayan Javeed Shaikh, Arunesh Sinha, Nathaniel D. Bastian +1LLM Red Teaming AI Agent Evaluation
Indiana University Bloomington, IN, USA · Rutgers University Piscataway, NJ, USA · Johns Hopkins University Baltimore, MD, USA
Climbing the Hill: Prompt Injection Red-Teaming Against Frontier Models with Curriculum Reinforcement Learning
Sep 27, 2026Chenlong Yin, Xiaolong Jin, Wei Zou +2Adversarial Attacks on LLMs LLM Red Teaming
The Pennsylvania State University · Purdue University
SkillDRE: Dual-Stage Red-Team Evolution of Agent Skills via Pre-Execution and Runtime Feedback
Sep 26, 2026Pengyu Zhu, Jingyi Yang, Yi Liu +2Automated Red Teaming LLM Red Teaming
Beijing University of Posts and Telecommunications · North China Electric Power University · Chongqing University of Posts and Telecommunications
ACEA: An Adversarial Co-Evolution Arena for Head-to-Head Red-Team and Blue-Team LLM Testing
Sep 8, 2026Yi Ting Shen, Kentaroh Toyoda, Alex LeungLLM Safety Benchmarks Language Model Safety Evaluation
Vulcan Research, AIFT Singapore
EvoFlint: An Evolutionary Atlas of Multi-Turn LLM Vulnerabilities
Aug 31, 2026Feitong Qiao, Liren Peng, Shiming Ren +7Language Model Safety Evaluation LLM Red Teaming
Reinforce Labs USA
REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems
Aug 11, 2026Zixing Chen, Xingyuan Liu, Jie Zhu +6LLM Red Teaming LLM Agents
Fudan University · The Hong Kong University of Science and Technology · Qwen DianJin Team, Alibaba Cloud Computing +1
Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)
Aug 5, 2026Ryozo Masukawa, Ian Bryant, Armita Kazeminajafabadi +6Autonomous Cyber Defense LLM Red Teaming
University of California, Irvine · Northeastern University · Johns Hopkins University
GPT-Red: Automated Red Teaming via Self-Play at Scale
Jul 28, 2026Eric Wallace, Christopher A. Choquette-Choo, Nikhil Kandpal +15Adversarial Training LLM Red Teaming
OpenAI
What AI Red-Team Evaluations Can and Cannot Prove
Jul 23, 2026Bandana KaurLLM Safety Benchmarks LLM Red Teaming
APIsec Research Labs
AMT-X: Phase-Structured Multi-Turn Red-Teaming with Checklist-Gated Evaluation
Jul 13, 2026Yi Ting Shen, Kentaroh Toyoda, Alex LeungLLM Safety Benchmarks Adversarial Attacks on LLMs
Vulcan Research, AIFT, Singapore
Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents
Jul 8, 2026Harry Owiredu-AshleyLLM Red Teaming AI Agent Safety
Independent Researcher New Jersey, USA
CONTRA: Red-Teaming Configurations of Personalizable Agents
Jul 3, 2026Jonathan Nöther, Adish Singla, Goran RadanovicLLM Red Teaming AI Agent Security
Max Planck Institute for Software Systems, Germany
MIRROR: Novelty-Constrained Memory-Guided MCTS Red-Teaming for Agentic RAG
Jun 25, 2026Inderjeet Singh, Andrés Murillo, Motoyoshi Sekiya +2Multimodal RAG Tool-Augmented Language Model Agents
Fujitsu Research of Europe, United Kingdom · Fujitsu Limited, Japan
AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability
Jun 23, 2026Khanak KhandelwalAdversarial Prompt Generation Adversarial Attacks on LLMs
Indian Institute of Technology Jodhpur
Distributed Quality-Diversity Search for Toxicity in Large Language Models
Jun 23, 2026Onkar Shelar, Travis DesellAdversarial Prompt Generation LLM Red Teaming
Rochester Institute of Technology, Rochester NY 14623, USA
OTTER: A Red-Teaming System for Toxicity-Evading Jailbreak Prompt Optimization
Jun 19, 2026Jerry Wang, Hsin-Ling Hsu, Yi-Cheng Lai +2Adversarial Prompt Generation Adversarial Attacks on LLMs
1National Chengchi University, Taipei, Taiwan
NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms
Jun 18, 2026Hanwool Lee, Dasol Choi, Bokyeong Kim +2Multi-Agent LLM Systems Safety-Critical Control
AIM Intelligence · Republic of Korea · KAERI
FinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming
Jun 18, 2026Daeyoung Park, Chaeyun Kim, Junghwan Kim +4LLM Safety Benchmarks Financial Services
Financial Security Institute (FSI), Republic of Korea · Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea · DATUMO INC., Republic of Korea
A Red-Team Study of Anthropic Fable 5 & Opus 4.8 Models
Jun 16, 2026Nicola FrancoLanguage Model Safety Evaluation Neural Network Robustness
Head of AI Security Lab The Italian Institute of Artificial Intelligence (AI4I) Corso Castelfidardo 22, 10129 Turin, Italy
Securing Multi-Agent GIS Systems: Risk Evaluation and Prompt Hardening Optimization
Jun 13, 2026Kyle Gao, Pranavi Kotta, Linlin Xu +2Adversarial Robustness LLM Red Teaming
Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada · Department of Mechatronics Engineering, University of Waterloo, Waterloo, ON, Canada · Department of Geomatics Engineering, University of Calgary, Calgary, AB, Canada +1
PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
Jun 10, 2026Pengfei He, Lesly Miculicich, Vishesh Sharma +5AI Agent Auditing LLM Auditing
Google Cloud AI Research · Google · Michigan State University
Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO
Jun 8, 2026Blake Bullwinkel, Eugenia Kim, Amanda Minnich +1Adversarial Training Adversarial Attacks on LLMs
Microsoft AI Red Team · Microsoft Azure
Culturally-Adapted Red-Teaming Across East and Southeast Asian Contexts: A Methodological and Comparative Analysis
Jun 8, 2026Hyeji Choi, Yongtaek Lim, Minwoo KimMultilingual Language Model Evaluation Language Model Safety Evaluation
AI Safety Team, DATUMO.INC, Seoul, South Korea.

Oct 29, 2026J/K move · Enter open · S save

Red-Teaming Large Language Models in Low-Resource African Languages: A Case Study of Yoruba and Nigerian Pidgin

Paula Chikaodinaka Anyanwu, Valdini Douglace Lemofouet, Blessing Ngozi Uzor, Ifeoluwa Dorcas Awotunde+6

African Institute for Mathematical Sciences (AIMS), Cameroon · Independent Researcher · Brown University+5

Language Model Safety Evaluation LLM Red Teaming Multilingual Language Model Evaluation Cross-Lingual Consistency in Language Models African Language Processing

Open paper AskPDF arXiv

As large language models (LLMs) become more widespread in Africa, their safety in low-resource languages remains largely untested since most red-teaming work focuses on English. This study evaluates six state-of-the-art LLMs using 400 explicit adversarial prompts in Yoruba and Nigerian Pidgin, categorised into four harm categories. The responses were scored by native-speaker annotators and reliability was assessed using Cohen's

κ

. While models that rejected up to 69% of English prompts rejected as few as 0% in Yoruba, this was not because they were safer, but because they failed to understand the prompts. Attack success rates increased for most models in the Nigerian languages, with GPT-4o Mini demonstrating the most significant cross-lingual safety discrepancy (+0.23 in Yoruba). When models did engage with adversarial prompts, many of the responses were harmful, with rates of harmful content reaching 0.63 in English and 0.50 in Nigerian Pidgin. The most difficult harm category to detect was cultural stereotypes and discrimination (ASR = 0.44 in Nigerian Pidgin). Comprehension in Yoruba was low across most models, and this directly suppressed apparent attack success without indicating actual safety. Gemini 2.0 Flash was the most consistent model across languages. Despite its explicit training in African languages, Aya 101 had the highest attack success rate in English (0.60) and one of the lowest Yoruba comprehension rates (0.28). These results demonstrate that English-centric safety alignment does not transfer to Nigerian languages, highlighting the urgent need for language-specific safety evaluations.