Red-Teaming

Follow this topic Ask about this topic

Recent momentum

-70%

3 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Red-Teaming.

Period ending 2026-09-07

2 new papers

A weekly snapshot of new work published in Red-Teaming.

67 papers

Latest in Red-Teaming

Open your feed →

Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Sep 10, 2026Divyanshu Kumar, Nitin Aravind Birur, Tanay Baswa +2Artificial Intelligence Risk Agentic Evaluations
EvoFlint: An Evolutionary Atlas of Multi-Turn LLM Vulnerabilities
Aug 31, 2026Feitong Qiao, Liren Peng, Shiming Ren +7Language Model Evasion Attacks Red-Teaming
SIR: Self-improving Red-teaming for Compute Use Agents
Aug 31, 2026Chen Xiong, Zhiyuan He, Pin-Yu Chen +2Red-Teaming Indirect Prompt Injection
REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems
Aug 11, 2026Zixing Chen, Xingyuan Liu, Jie Zhu +6Red-Teaming Large Language Model Safety
Generating Attacks for LLMs with GFlowNets
Aug 10, 2026Berkay Ozcam, Irem Onen, Mehmet Fatih Amasyali +1Language Model Evasion Attacks Large Language Model Safety
From Inaudible Inputs to Model Failures: Low-Frequency Safety Risks in LALMs
Aug 10, 2026Yuanhe Zhang, Weiliu Wang, Jie Ren +7Large Audio Language Models Large Language Model Safety
Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)
Aug 5, 2026Ryozo Masukawa, Ian Bryant, Armita Kazeminajafabadi +6Red-Teaming Agentic Reinforcement Learning
OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution
Aug 1, 2026Yunhao Chen, Xin Wang, Yixu Wang +6Self-Evolving Agents Agentic Evaluations
Symbolic Attack Chain Generation from Atomic Red Team Techniques: An Empirical Study of Predicate Representation Granularity
Jul 31, 2026Ramya VarunsegarAttack Surface Red-Teaming
GPT-Red: Automated Red Teaming via Self-Play at Scale
Jul 28, 2026Eric Wallace, Christopher A. Choquette-Choo, Nikhil Kandpal +15Red-Teaming Language Model Evasion Attacks
What AI Red-Team Evaluations Can and Cannot Prove
Jul 23, 2026Bandana KaurSafety Benchmarks Artificial Intelligence Evaluation
Code Monitor Red Teaming for Public-Test-Passing Code
Jul 23, 2026Junchi Liao, Jiawen Deng, Fuji RenRed-Teaming Adversarial Evaluation
An Early Warning of Emerging Biosecurity Risks in Frontier LLMs
Jul 20, 2026Zhida He, Xia Hu, Baichen Le +20Large Language Model Safety Life Sciences Research
Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming
Jul 13, 2026Xutao Mao, Xiang Zheng, Cong WangRed-Teaming Vulnerability Detection & Repair
Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety
Jul 8, 2026Yujiao ChenRed-Teaming Artificial Intelligence Safety
Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents
Jul 8, 2026Harry Owiredu-AshleyRed-Teaming Attack-Success Rate
CONTRA: Red-Teaming Configurations of Personalizable Agents
Jul 3, 2026Jonathan Nöther, Adish Singla, Goran RadanovicRed-Teaming Malicious Instructions
Online Safety Monitoring for LLMs
Jul 2, 2026Mona Schirmer, Metod Jazbec, Alexander Timans +3Large Language Model Safety AI Safety Alignment
A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation
Jun 24, 2026Abrar Alotaibi, Raed Mughus, Moataz AhmedRed-Teaming Adversarial Prompts
RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems
Jun 22, 2026Yarin Yerushalmi Levi, Roy Betser, Amit Giloni +5Red-Teaming Production Agentic Systems
REALM: A Unified Red-Teaming Benchmark for Physical-World VLMs
Jun 22, 2026Yifei Zhao, Qian Lou, Mengxin ZhengAdversarial Evaluation Red-Teaming
OTTER: A Red-Teaming System for Toxicity-Evading Jailbreak Prompt Optimization
Jun 19, 2026Jerry Wang, Hsin-Ling Hsu, Yi-Cheng Lai +2Toxicity Red-Teaming
NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms
Jun 18, 2026Hanwool Lee, Dasol Choi, Bokyeong Kim +2Large Language Model Safety Red-Teaming
FinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming
Jun 18, 2026Chaeyun Kim, Daeyoung Park, Junghwan Kim +4Red-Teaming Large Language Model Safety
Learning Red Agent Policy from Observations for Neurosymbolic Autonomous Cyber Agents
Jun 16, 2026Ankita Samaddar, Sandeep Neema, Daniel Balasubramanian +1Safe Reinforcement Learning Red-Teaming
A Red-Team Study of Anthropic Fable 5 & Opus 4.8 Models
Jun 16, 2026Nicola FrancoLarge Language Models Fail Red-Teaming
MAStrike: Shapley-Guided Collusive Red-Teaming on Multi-Agent Systems
Jun 11, 2026Chejian Xu, Zhaorun Chen, Jingyang Zhang +5Decentralized Multi-Agent System Red-Teaming
PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
Jun 10, 2026Pengfei He, Lesly Miculicich, Vishesh Sharma +5Prompt Injection Detection Language Model Evasion Attacks
RedAct: Redacting Agent Capability Traces for Procedural Skill Protection
Jun 9, 2026Shuwen Xu, Zhitao He, Yi R. FungTraces Red-Teaming
Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO
Jun 8, 2026Blake Bullwinkel, Eugenia Kim, Amanda Minnich +1Red-Teaming Defence
Diffuse AI Control on Fuzzy Tasks
Jun 8, 2026Mikhail Terekhov, Caglar Gulcehre, Vivek Hebbar +1Artificial Intelligence Safety Sabotage
Beyond Pass/Fail: Using Process Mining to Understand How LLMs Resist (and Fail) Red Team Attacks
Jun 5, 2026Zvi TopolRed-Teaming Attack-Success Rate
FoeGlass: Simple In-Context Learning Is Enough for Red Teaming Audio Deepfake Detectors
Jun 3, 2026Sepehr Dehdashtian, Jacob H Seidman, Vishnu N Boddeti +1Audio Deepfake Detection Audio Generation and Processing
Attack Selection in Agentic AI Control Evaluations Meaningfully Decreases Safety
Jun 3, 2026Catherine Ge-Wang, Tyler Crosse, Benjamin Hadad +3Adversarial Attack Strategies Agentic Evaluations
Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming
Jun 3, 2026Nicholas SabanRed-Teaming Attack-Success Rate
AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations
Jun 1, 2026Hiskias Dingeto, William LeeneyRed-Teaming Indirect Prompt Injection
Cross-Generational Transfer of Adversarial Attacks Reveals Non-Monotonic Safety Alignment in LLMs
May 30, 2026Subhadip MitraSafety Alignment Red-Teaming
Quality-Diversity Evolution for Discovering Diverse Vulnerabilities in LLM Safety
May 30, 2026Subhadip MitraLanguage Model Evasion Attacks Large Language Model Safety
Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South
May 18, 2026Charvi Rastogi, Mukul Bhutani, Minsuk Kahng +13Multilingual Safety Red-Teaming
Proteus: A Self-Evolving Red Team for Agent Skill Ecosystems
May 12, 2026Zhaojiacheng ZhouRed-Teaming Agent Skill Ecosystems
IPI-proxy: An Intercepting Proxy for Red-Teaming Web-Browsing AI Agents Against Indirect Prompt Injection
May 12, 2026Chia-Pei, Chen, Kentaroh Toyoda +2Indirect Prompt Injection Web Agents
Persona-Conditioned Adversarial Prompting: Multi-Identity Red-Teaming for Adversarial Discovery and Mitigation
May 12, 2026Cristian Morasso, Anisa Halimi, Muhammad Zaid Hameed +1Red-Teaming
Beyond Red-Teaming: Formal Guarantees of LLM Guardrail Classifiers
May 11, 2026Nikita Kezins, Urbas Ekka, Pascal Berrang +1Large Language Model Safety Threat Models
Red-Teaming Agent Execution Contexts: Open-World Security Evaluation on OpenClaw
May 11, 2026Hongwei Yao, Yiming Liu, Yiling He +1Openclaw Vulnerability Detection & Repair
MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring
May 10, 2026Monika Jotautaitė, Maria Angelica Martinez, Ollie Matthews +1Red-Teaming Agentic Benchmarks
The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
May 9, 2026Ismail Hossain, Tanzim Ahad, Md Jahangir Alam +3Large Language Model Jailbreaks Jailbreaks
OTora: A Unified Red Teaming Framework for Reasoning-Level Denial-of-Service in LLM Agents
May 9, 2026Xinyu Li, Ronghui Mu, Lin Li +2Language Model Evasion Attacks LLM Reasoning Methods
LoopTrap: Termination Poisoning Attacks on LLM Agents
May 7, 2026Huiyu Xu, Zhibo Wang, Wenhui Zhang +4Large Language Model Agents Data Poisoning
PersonaTeaming: Supporting Persona-Driven Red-Teaming for Generative AI
May 7, 2026Wesley Hanwen Deng, Mingxi Yan, Sunnie S. Y. Kim +5Red-Teaming Human-Ai Collaboration
DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents
May 6, 2026Zhaorun Chen, Xun Liu, Haibo Tong +14Red-Teaming Artificial Intelligence Agents
Redefining AI Red Teaming in the Agentic Era: From Weeks to Hours
May 5, 2026Raja Sekhar Rao Dheekonda, Will Pearce, Nick LandersRed-Teaming Open-Source
ContextualJailbreak: Evolutionary Red-Teaming via Simulated Conversational Priming
May 4, 2026Mario Rodríguez Béjar, Francisco J. Cortés-Delgado, S. Braghin +1Large Language Model Jailbreaks Red-Teaming
Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance
May 1, 2026Minchan Kwon, Sunghyun Baek, Minseo Kim +3Generative Flow Networks Large Language Model Safety
Training a General Purpose Automated Red Teaming Model
Apr 24, 2026Aishwarya Padmakumar, Leon Derczynski, Traian Rebedea +1Red-Teaming Language Model Evasion Attacks
RedVLA: Physical Red Teaming for Vision-Language-Action Models
Apr 24, 2026Yuhao Zhang, Borong Zhang, Jiaming Fan +4Diffusion-Based Vision-Language-Actions Vision-Language-Action
AutoRISE: Agent-Driven Strategy Evolution for Red-Teaming Large Language Models
Apr 23, 2026Tanmay Gautam, Alireza Bahramali, Sandeep AtluriRed-Teaming Language Model Evasion Attacks
Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture the Flag Challenges
Apr 21, 2026Ali Al-Kaswan, Maksim Plotnikov, Maxim Hájek +3Red-Teaming Language Model Evasion Attacks
STAR-Teaming: A Strategy-Response Multiplex Network Approach to Automated LLM Red Teaming
Apr 21, 2026MinJae Jung, YongTaek Lim, Chaeyun Kim +3Red-Teaming Language Model Evasion Attacks
ARES: Adaptive Red-Teaming and End-to-End Repair of Policy-Reward System
Apr 20, 2026Jiacheng Liang, Yao Ma, Tharindu Kumarage +5Reinforcement Learning From Human Feedback Red-Teaming
Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection
Apr 9, 2026Wenkui Yang, Chao Jin, Haisu Zhu +7Desktop Graphical User Interface Agents Red-Teaming

Sep 10, 2026 · cs.AIJ/K move · Enter open · S save

Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery

Divyanshu Kumar, Nitin Aravind Birur, Tanay Baswa, Sahil Agarwal+1

Agentic systems are rapidly moving to production, where they read untrusted inputs, call tools with real permissions, and act autonomously, expanding the security surface beyond chat-only models. Yet standard evaluations remain single-turn and fail to capture multi-step agent vulnerabilities. We present a systematic black-box framework for risk-aware agent evaluation requiring only basic system descriptions. Our approach introduces: (1) a seven-domain taxonomy mapping observable behaviors to risk categories, (2) fully automated SAGE-RT red teaming producing 120 adversarial scenarios per domain, and (3) human-validated evaluation using LLM judges. Empirical validation across two agent architectures (CrewAI and AutoGen) with four base models reveals alarming patterns: 56.25% average governance risk, 65% privacy risk in multi-agent configurations, and agent behavior vulnerabilities reaching 85%. Our black-box approach effectively identifies critical architectural vulnerabilities without privileged access, providing a scalable path toward safer agent deployments.

Artificial Intelligence Risk Agentic Evaluations Red-Teaming Production Agentic Systems Adversarial Evaluation Vulnerability Detection & Repair AI Risk Assessment Black Box

Open paper Ask about it PDF