LLM Red Teaming

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 47

All topics

No One Architecture Fits All: A Cross-Environment Evaluation of Hierarchical Red Team Agents
Sep 30, 2026Ayan Javeed Shaikh, Arunesh Sinha, Nathaniel D. Bastian +1LLM Red Teaming AI Agent Evaluation
Indiana University Bloomington, IN, USA · Rutgers University Piscataway, NJ, USA · Johns Hopkins University Baltimore, MD, USA
Climbing the Hill: Prompt Injection Red-Teaming Against Frontier Models with Curriculum Reinforcement Learning
Sep 27, 2026Chenlong Yin, Xiaolong Jin, Wei Zou +2Adversarial Attacks on LLMs LLM Red Teaming
The Pennsylvania State University · Purdue University
ACEA: An Adversarial Co-Evolution Arena for Head-to-Head Red-Team and Blue-Team LLM Testing
Sep 8, 2026Yi Ting Shen, Kentaroh Toyoda, Alex LeungLLM Safety Benchmarks Language Model Safety Evaluation
Vulcan Research, AIFT Singapore
EvoFlint: An Evolutionary Atlas of Multi-Turn LLM Vulnerabilities
Aug 31, 2026Feitong Qiao, Liren Peng, Shiming Ren +7Language Model Safety Evaluation LLM Red Teaming
Reinforce Labs USA
REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems
Aug 11, 2026Zixing Chen, Xingyuan Liu, Jie Zhu +6LLM Red Teaming LLM Agents
Fudan University · The Hong Kong University of Science and Technology · Qwen DianJin Team, Alibaba Cloud Computing +1
Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)
Aug 5, 2026Ryozo Masukawa, Ian Bryant, Armita Kazeminajafabadi +6Autonomous Cyber Defense LLM Red Teaming
University of California, Irvine · Northeastern University · Johns Hopkins University
GPT-Red: Automated Red Teaming via Self-Play at Scale
Jul 28, 2026Eric Wallace, Christopher A. Choquette-Choo, Nikhil Kandpal +15Adversarial Training LLM Red Teaming
OpenAI
What AI Red-Team Evaluations Can and Cannot Prove
Jul 23, 2026Bandana KaurLLM Safety Benchmarks LLM Red Teaming
APIsec Research Labs
AMT-X: Phase-Structured Multi-Turn Red-Teaming with Checklist-Gated Evaluation
Jul 13, 2026Yi Ting Shen, Kentaroh Toyoda, Alex LeungLLM Safety Benchmarks Adversarial Attacks on LLMs
Vulcan Research, AIFT, Singapore
Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents
Jul 8, 2026Harry Owiredu-AshleyLLM Red Teaming AI Agent Safety
Independent Researcher New Jersey, USA
CONTRA: Red-Teaming Configurations of Personalizable Agents
Jul 3, 2026Jonathan Nöther, Adish Singla, Goran RadanovicLLM Red Teaming AI Agent Security
Max Planck Institute for Software Systems, Germany
MIRROR: Novelty-Constrained Memory-Guided MCTS Red-Teaming for Agentic RAG
Jun 25, 2026Inderjeet Singh, Andrés Murillo, Motoyoshi Sekiya +2Multimodal RAG Tool-Augmented Language Model Agents
Fujitsu Research of Europe, United Kingdom · Fujitsu Limited, Japan
AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability
Jun 23, 2026Khanak KhandelwalAdversarial Prompt Generation Adversarial Attacks on LLMs
Indian Institute of Technology Jodhpur
Distributed Quality-Diversity Search for Toxicity in Large Language Models
Jun 23, 2026Onkar Shelar, Travis DesellAdversarial Prompt Generation LLM Red Teaming
Rochester Institute of Technology, Rochester NY 14623, USA
OTTER: A Red-Teaming System for Toxicity-Evading Jailbreak Prompt Optimization
Jun 19, 2026Jerry Wang, Hsin-Ling Hsu, Yi-Cheng Lai +2Adversarial Prompt Generation Adversarial Attacks on LLMs
1National Chengchi University, Taipei, Taiwan
NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms
Jun 18, 2026Hanwool Lee, Dasol Choi, Bokyeong Kim +2Multi-Agent LLM Systems Safety-Critical Control
AIM Intelligence · Republic of Korea · KAERI
FinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming
Jun 18, 2026Chaeyun Kim, Daeyoung Park, Junghwan Kim +4LLM Safety Benchmarks Financial Services
DATUMO INC., Republic of Korea · Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea · Financial Security Institute (FSI), Republic of Korea
A Red-Team Study of Anthropic Fable 5 & Opus 4.8 Models
Jun 16, 2026Nicola FrancoLanguage Model Safety Evaluation Neural Network Robustness
Head of AI Security Lab The Italian Institute of Artificial Intelligence (AI4I) Corso Castelfidardo 22, 10129 Turin, Italy
Securing Multi-Agent GIS Systems: Risk Evaluation and Prompt Hardening Optimization
Jun 13, 2026Kyle Gao, Pranavi Kotta, Linlin Xu +2Adversarial Robustness LLM Red Teaming
Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada · Department of Mechatronics Engineering, University of Waterloo, Waterloo, ON, Canada · Department of Geomatics Engineering, University of Calgary, Calgary, AB, Canada +1
PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
Jun 10, 2026Pengfei He, Lesly Miculicich, Vishesh Sharma +5AI Agent Auditing LLM Auditing
Google Cloud AI Research · Google · Michigan State University
Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO
Jun 8, 2026Blake Bullwinkel, Eugenia Kim, Amanda Minnich +1Adversarial Training Adversarial Attacks on LLMs
Microsoft AI Red Team · Microsoft Azure
Culturally-Adapted Red-Teaming Across East and Southeast Asian Contexts: A Methodological and Comparative Analysis
Jun 8, 2026Hyeji Choi, Yongtaek Lim, Minwoo KimMultilingual Language Model Evaluation Language Model Safety Evaluation
AI Safety Team, DATUMO.INC, Seoul, South Korea.
VATS: Exploiting Implicit Authority in Error-Path Injection via Systematic Mutation
Jun 6, 2026Harshil Patel, Kunal PaiLLM Red Teaming AI Agent Security
Department of Computer Science, University of California, Davis, CA, USA.
Beyond Pass/Fail: Using Process Mining to Understand How LLMs Resist (and Fail) Red Team Attacks
Jun 5, 2026Zvi TopolLanguage Model Safety Evaluation Adversarial Attacks on LLMs
MuyVentive, LLC
CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning
Jun 4, 2026Rahul Markasserithodi, Aditya Joshi, Yuekang Li +3Adversarial Training Adversarial Prompt Generation
University of New South Wales, Australia
Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing
Jun 1, 2026Alexandre Cristovão MaioranoLLM Red Teaming AI Agent Security Benchmarks
Lumytics
AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations
Jun 1, 2026Hiskias Dingeto, William LeeneyLLM Red Teaming LLM Agent Security
StackOne Technologies
Cross-Generational Transfer of Adversarial Attacks Reveals Non-Monotonic Safety Alignment in LLMs
May 30, 2026Subhadip MitraAdversarial Attacks on LLMs LLM Red Teaming
Research Lead, Rota Labs
Understanding Safety-Sensitive Expert Behavior in Mixture-of-Experts LLMs
May 28, 2026Zhibo Zhang, Yuxi Li, Zhen Ouyang +2Mixture-of-Experts Language Models LLM Auditing
Huazhong University of Science and Technology, Wuhan, China · Nanyang Technological University, Singapore
Constitutional Arms Races in the Public Goods Game: Co-Evolving LLM Constitutions Under Cooperation-Defection Pressure
May 26, 2026Ujwal Kumar, Arth Singh, Hershraj Niranjani +5LLM Red Teaming Multi-Agent LLMs
Shibaura Institute of Technology · NIT Agartala · UC Berkeley +2

Sep 30, 2026, cs.CRJ/K move · Enter open · S save

No One Architecture Fits All: A Cross-Environment Evaluation of Hierarchical Red Team Agents

Ayan Javeed Shaikh, Arunesh Sinha, Nathaniel D. Bastian, Ankit Shah

Indiana University Bloomington, IN, USA · Rutgers University Piscataway, NJ, USA · Johns Hopkins University Baltimore, MD, USA

LLM Red Teaming AI Agent Evaluation AI Agent Security Benchmarks LLMs for Cybersecurity Automated Red Teaming

Open paper AskPDF arXiv

Autonomous red team agents increasingly stress-test AI-enabled cyber defenses by planning strategy and executing multistage attacks. Reinforcement learning (RL) and large language models (LLMs) offer complementary mechanisms for the planning and execution such agents require, and prior work has combined them in hybrid hierarchies. Yet a given architecture is typically developed and evaluated within a single environment, leaving open whether an observed advantage reflects a generally stronger decision mechanism or merely alignment with a particular setting. We address this gap with a controlled cross-environment comparison of two homogeneous hierarchical red team architectures: an RL planner with an RL executor (RL+RL) and an LLM planner with an LLM executor (LLM+LLM). We evaluate both against expert autonomous defenders in CybORG CAGE-4 and in Cyberwheel at two network scales, across 18 configurations under one unified disruption metric. We find a pronounced environment-dependent inversion. RL+RL wins the compact, densely rewarded CAGE-4 (78.5% disruption success versus 18.0% for the strongest LLM configuration) and the 100-host Cyberwheel network (81.0% versus 50.5%), while a pretrained cybersecurity LLM agent wins the larger, escalation-gated 1010-host Cyberwheel network (55.0% versus 0.0% for RL). A kill-chain analysis explains the inversion through architecture-specific bottlenecks that aggregate success rates conceal.In the 1010-host Cyberwheel network, RL discovers and compromises hosts but stalls at privilege escalation, whereas in CAGE-4, LLM agents obtain privileged access but rarely convert it into operational impact. These results indicate that conclusions drawn in a single environment may not generalize, and that hybrid planner-executor designs should be motivated by specific failure modes rather than the assumption that one architecture is universally preferable.

Momentum

Related topics

Latest papers 47