Overview

Automated red teaming studies how to generate adversarial tests that reveal harmful, insecure, or unreliable behavior in AI systems, including language models, multimodal and web agents, and multi-agent workflows. Common approaches use adaptive prompt search, reinforcement learning, self-play, evolutionary strategies, and agents that learn reusable attack skills from execution feedback; tests increasingly include multi-turn interactions, tool use, memory, and realistic environments. Benchmarks and evaluation methods measure attack success, risk severity, coverage, and whether attacks transfer across models or evade monitors, while adversarial testing is also used to improve defenses and harden benchmarks. A central challenge is making evaluations realistic and reproducible while ensuring that discovered attacks generalize, represent meaningful harm, and keep pace with defenses that can themselves adapt.

Latest papers

CardsList

66 papers11 papers in the last four weeks, down 21% on the four weeks before. 0.1% of all new papers.

  1. MobileProbe: Adaptive Security Evaluation of Mobile Agents via Trajectory-Aware Injection

    Nov 12, 2026Yingbin Jin, Ji'an Wang, HaoYang Chai +7AI Agent AuditingMobile GUI Agents

  2. On the Framing Effect of AI Red-Teaming Instructions: Evidence from Prior Exercises and Controlled Simulations

    Oct 30, 2026Michael Feffer, Alice Qian, Anusha Sinha +6LLM Red TeamingAI Safety Evaluation

  3. ANGLER: Verifier-Grounded Benchmarking with an Adaptive Attacker for Web Agents

    Oct 30, 2026Yini Huang, Aiwen LU, Zhen Sun +3AI Agent Security BenchmarksAutomated Red Teaming

  4. Program-Level Adversarial Co-Adaptive Self-Play for Robotic Security

    Oct 26, 2026Leo Benaharon, Chi Zhang, Wei Zheng +3Robot Policy AdaptationSelf-Evolving Agents

  5. The Target Agent Is Its Own Best Critic but a Poor Proposer: Self-Improved Red Teaming

    Oct 26, 2026Mintong Kang, Stephen Rawls, Melissa Kazemi Rad +2LLM Red TeamingAutomated Red Teaming

  6. SEEK: Skill Extraction with Evolving Knowledge against Frontier Science Agents

    Oct 25, 2026Cheng Gui, Xiang Zheng, Yutao Wu +5AI Agent SecurityAgent Skill Security Auditing

  7. AutoRedClaw: Automated Red-Teaming for Claw-Style Agents

    Oct 19, 2026Hao Li, Tongyan Hu, Weidi Luo +8AI Agent SecurityAutomated Red Teaming

  8. Red-Teaming Multi-Agent Attacks for Loss-of-Control Risks

    Oct 16, 2026John Kitaoka, Thao Amelia Pham, Carlos Rafael Giudice +1AI Agent SafetyMulti-Agent System Security

  9. CoRedSkill: Stealthy Skill Composition Attacks via AST-Guided Injection

    Oct 14, 2026Qichen Zhao, Shengfang Zhai, Qingni Shen +2AI Agent SecurityAutomated Red Teaming

  10. RedDarwin: Evolving Jailbreak Skills against Frontier LLMs

    Oct 14, 2026Zhouyuan Ma, Yutao Wu, Hanxun Huang +3LLM Jailbreak AttacksLLM Red Teaming

  11. X-AgentRed: Red-Teaming Personal Agents in Evolving Multi-Service Ecosystems

    Oct 9, 2026Yeonsung Jung, Zhiyuan Wang, Zifan Wang +5Automated Red TeamingAI Agent Security

  12. Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models

    Oct 4, 2026Tongyan Hu, Hao Li, Xiaogeng Liu +8Test-Time TrainingLLM Jailbreak Attacks

  13. HASTE: Evolving Agent Harnesses Against Emerging Attacks Using Sparse Evidence

    Oct 2, 2026Xiqiao Xiong, Moxin Li, Zhixin Ma +4Agent Harness Self-EvolutionLLM Agent Safety

  14. No One Architecture Fits All: A Cross-Environment Evaluation of Hierarchical Red Team Agents

    Sep 30, 2026Ayan Javeed Shaikh, Arunesh Sinha, Nathaniel D. Bastian +1LLM Red TeamingAI Agent Evaluation

  15. SkillDRE: Dual-Stage Red-Team Evolution of Agent Skills via Pre-Execution and Runtime Feedback

    Sep 26, 2026Pengyu Zhu, Jingyi Yang, Yi Liu +2Automated Red TeamingLLM Red Teaming

  16. AgentXploit: Autonomous Repository-to-Runtime Red-Teaming for AI Agents

    Sep 25, 2026Weida Liang, Shi Qiu, Zhun Wang +6AI Agent AuditingAutomated Red Teaming

  17. Stealth Apart, Harm Together: Skill Cascading Attacks on Skill-Based Agent Systems

    Sep 24, 2026Zihao Zhu, Siwei Lyu, Adel Bibi +1Adversarial AttacksLLM Agent Security

  18. Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents

    Sep 17, 2026Alex Remedios, Simon Storf, Fabien Roger +1AI Agent SecurityAI Agent Monitoring

  19. EvoSkill Injection: Red-Teaming Autonomous Skill Generation and Evolution in Self-Evolving Agents

    Aug 31, 2026Doyun Kim, Chanwoo Kim, Sugyeong Eo +2AI Agent SecurityAI Agent Security Benchmarks

  20. SIR: Self-improving Red-teaming for Compute Use Agents

    Aug 31, 2026Chen Xiong, Zhiyuan He, Pin-Yu Chen +2Computer-Use AgentsIndirect Prompt Injection

  21. RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution

    Aug 27, 2026Junjie Zhang, Hui Liu, Kecheng Chen +3Automated Red TeamingLLM Agent Security