Overview

Automated red teaming studies how to generate adversarial tests that reveal harmful, insecure, or unreliable behavior in AI systems, including language models, multimodal and web agents, and multi-agent workflows. Common approaches use adaptive prompt search, reinforcement learning, self-play, evolutionary strategies, and agents that learn reusable attack skills from execution feedback; tests increasingly include multi-turn interactions, tool use, memory, and realistic environments. Benchmarks and evaluation methods measure attack success, risk severity, coverage, and whether attacks transfer across models or evade monitors, while adversarial testing is also used to improve defenses and harden benchmarks. A central challenge is making evaluations realistic and reproducible while ensuring that discovered attacks generalize, represent meaningful harm, and keep pace with defenses that can themselves adapt.

Latest papers

CardsList

66 papers11 papers in the last four weeks, down 21% on the four weeks before. 0.1% of all new papers.

  1. Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services

    Mar 11, 2026Fabrizio Dimino, Bhaskarjit Sarmah, Stefano PasqualiLanguage Model Safety EvaluationLLM Safety

  2. A Systematic Review of Algorithmic Red Teaming Methodologies for Assurance and Security of AI Applications

    Feb 24, 2026Shruti Srivastava, Kiranmayee Janardhan, Shaurya JauhariAutomated Red TeamingCybersecurity

  3. FERRET: Framework for Expansion Reliant Red Teaming

    Feb 17, 2026Ninareh Mehrabi, Vitor Albiero, Maya Pavlova +1Automated Red TeamingAdversarial Scenario Generation

  4. Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models

    Dec 21, 2025Zhang Wei, Hanxuan Chen, Peilu Hu +19Adversarial Robustness of Language ModelsLLM Safety Evaluation

  5. A Survey on Agentic Security: Applications, Threats and Defenses

    Oct 7, 2025Asif Shahriar, Md Nafiu Rahman, Sadif Ahmed +2AI Agent SecurityAutonomous Cyber Defense

  6. RedCoder: Automated Multi-Turn Red Teaming for Code LLMs

    Jun 25, 2025Wenjie Jacky Mo, Qin Liu, Xiaofei Wen +5Adversarial Prompt GenerationLLM Red Teaming