Adversarial Prompt Generation

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 53

All topics

RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Models
Sep 28, 2026Dmitrii Kharlapenko, Sergei Bratchikov, Konstantin Korolev +1Adversarial Prompt Generation T2I Generation Evaluation
White Circle
ENDOPROMPT: Victim-Side Pseudo-References for Utility Degradation
Sep 24, 2026Qingyu Wu, Zeyu Feng, Yongda Yu +2Adversarial Prompt Generation Prompt Injection Attacks
Defense Innovation Institute, Academy of Military Science, Beijing, China · Nanjing University, Nanjing, China
Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions
Sep 23, 2026Tiantong Wu, Wei Yang Bryan LimAdversarial Prompt Generation Prompt Injection Attacks
Nanyang Technological University Singapore
Generating Attacks for LLMs with GFlowNets
Aug 10, 2026Berkay Ozcam, Irem Onen, Mehmet Fatih Amasyali +1Adversarial Prompt Generation Adversarial Attacks on LLMs
Cybersecurity R&D Turkcell Istanbul, Turkey · Department of Computer Engineering Yıldız Technical University Istanbul, Turkey
Mind the Gap: Zero-Query Jailbreaks via Filter-Generator Discrepancy in Text-to-Image Systems
Aug 2, 2026Wanguang Li, Zhaoxin Wang, Handing WangAdversarial Prompt Generation T2I Generation
School of Artificial Intelligence, Xidian University, Shaanxi, China
Adversarial Prompts for Acceptance Collapse in Speculative Decoding
Jul 23, 2026Run Wang, Chaoyi Zhou, Xi Liu +7Adversarial Prompt Generation Speculative Decoding
Clemson University · Wayne State University · University of Washington
Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models
Jul 20, 2026Dongdong Yang, Deyue Zhang, Zhao Liu +5Adversarial Prompt Generation Adversarial Examples
AI Security Lab, Beijing, China · School of Artificial Intelligence, Beihang University, Beijing, China
Adversarial Prompting Framework for AI Safety Assessment
Jul 15, 2026Yash Bhatnagar, Kunal Banerjee, Anirban ChatterjeeAdversarial Prompt Generation Adversarial Attacks on LLMs
Microsoft · Bengaluru, Karnataka, India · Walmart Global Tech
Seduced by the Narrative: Assessing Rule Adherence in Semi-Open Textual Sandboxes
Jul 2, 2026Weiying Chen, Junlong Shen, Zhanyuan Guo +1LLM Safety Benchmarks Adversarial Prompt Generation
University of Alberta, Edmonton, AB, Canada · N-Dice Association · Qilu University of Technology, Shandong, China
AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability
Jun 23, 2026Khanak KhandelwalAdversarial Prompt Generation Adversarial Attacks on LLMs
Indian Institute of Technology Jodhpur
Distributed Quality-Diversity Search for Toxicity in Large Language Models
Jun 23, 2026Onkar Shelar, Travis DesellAdversarial Prompt Generation LLM Red Teaming
Rochester Institute of Technology, Rochester NY 14623, USA
Adversarial Domain Prompt Tuning and Generation for Single Domain Generalization
Jun 19, 2026Zhipeng Xu, De Cheng, Xinyang Jiang +3Synthetic Data Augmentation Adversarial Prompt Generation
Xidian University · Microsoft Research Asia · Chongqing University of Posts and Telecommunications
OTTER: A Red-Teaming System for Toxicity-Evading Jailbreak Prompt Optimization
Jun 19, 2026Jerry Wang, Hsin-Ling Hsu, Yi-Cheng Lai +2Adversarial Prompt Generation Adversarial Attacks on LLMs
1National Chengchi University, Taipei, Taiwan
Automated jailbreak attack targeting multiple defense strategies
Jun 15, 2026Qi Wang, Chengcheng Wan, Weijia He +4Adversarial Prompt Generation LLM Jailbreak Attacks
East China Normal University Shanghai, China · East China Normal University, Shanghai Innovation Institute Shanghai, China · University of Southampton Southampton, England +2
GAS-Leak-LLM: Genetic Algorithm-Based Suffix Optimization for Black-Box LLM Jailbreaking
Jun 14, 2026Aman Anifer, Vignesh Kumar Kembu, Vishnu M +4Adversarial Prompt Generation LLM Safety
Department of Computer Applications, Cochin University of Science and Technology, Kerala, India. · Department of Electrical, Computer and Biomedical Engineering, University of Pavia, A. Ferrata, 5, Pavia, 27100, Italy.
Greedy Coordinate Diffusion: Effective and Semantically Coherent Adversarial Attacks via Diffusion Guidance
Jun 14, 2026Bohdan Turbal, Blossom Metevier, Max Springer +1Adversarial Prompt Generation Adversarial Attacks
Princeton University, Princeton, NJ, USA.
JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization
Jun 9, 2026Ge Shi, Jun Yin, Donglin Xie +3Adversarial Prompt Generation Large Language Model-Guided Optimization
University of California, Davis · The Renmin University of China · Independent Researcher +3
Assessing Automated Prompt Injection Attacks in Agentic Environments
Jun 9, 2026David Hofer, Edoardo Debenedetti, Florian TramèrAdversarial Prompt Generation LLM Agent Security
ETH Zurich Zurich, Switzerland
Diffuse AI Control on Fuzzy Tasks
Jun 8, 2026Mikhail Terekhov, Caglar Gulcehre, Vivek Hebbar +1Adversarial Prompt Generation Language Model Safety Evaluation
Anthropic Fellows Program (via MATS) · EPFL · Redwood Research +1
CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning
Jun 4, 2026Rahul Markasserithodi, Aditya Joshi, Yuekang Li +3Adversarial Training Adversarial Prompt Generation
University of New South Wales, Australia
Voices Across Registers: Corpus-Conditioned Vernacular Jailbreaks against Aligned LLMs via Fanfiction Subgenres
Jun 3, 2026Zhongze Luo, Ruihe Shi, Zhenshuai Yin +3Adversarial Prompt Generation Language Model Safety Evaluation
School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen) · School of Microelectronics, Xi’an Jiaotong University · 2The Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen) +1
Dive into Ambiguity: A*-Inspired Multi-Agents Commonsense Obfuscation Attack on LLM Prompts
May 31, 2026Boxuan Wang, Zhuoyun Li, Xiaowei Huang +1Adversarial Prompt Generation Adversarial Attacks
University of Liverpool Merseyside, United Kingdom
Quality-Diversity Evolution for Discovering Diverse Vulnerabilities in LLM Safety
May 30, 2026Subhadip MitraAdversarial Prompt Generation Adversarial Attacks on LLMs
Research Lead, Rota Labs
Automatically Attacking Software Reverse Engineering AI Agents
May 28, 2026Brian Crawford, Justin Phillips, Patrick McClureSoftware Engineering Adversarial Prompt Generation
Naval Postgraduate School, Monterey, CA, United States
Harmless Yet Harmful: Neutral Prompting Attacks for Stealthy Hallucination Steering in Agent Skills
May 28, 2026Chia-Yi Hsu, Chia-Mu Yu, Chun-Ying Huang +1Adversarial Prompt Generation AI Coding Agents
Department of Computer Science National Yang Ming Chiao Tung University · Department of Electronics and Electrical Engineering National Yang Ming Chiao Tung University · School of Computing Institute of Science Tokyo
Evolving and Detecting Multi-Turn Deception using Geometric Signatures
May 26, 2026Surender Suresh Kumar, Mary L. CummingsAdversarial Prompt Generation Deception in Language Models
George Mason University Fairfax, USA
PAST2HARM: A Simple Adaptive Past Tense Attack for Jailbreaking Multimodal AI
May 26, 2026Snehasis MukhopadhyayAdversarial Prompt Generation Multimodal Jailbreak Attacks
Indian Institute of Information Technology, Kalyani
Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Models
May 25, 2026Arian Komaei Koma, Seyed Amir Kasaei, AmirMahdi Sadeghzadeh +1Adversarial Prompt Generation Adversarial Attacks
Department of Computer Engineering Sharif University of Technology
LASH: Adaptive Semantic Hybridization for Black-Box Jailbreaking of Large Language Models
May 20, 2026Abdullah Al Nomaan Nafi, Fnu Suya, Swarup Bhunia +1Adversarial Prompt Generation LLM Jailbreak Attacks
University of Maine · University of Tennessee, Knoxville · University of Florida
PQR: A Framework to Generate Diverse and Realistic User Queries that Elicit QA Agent Failures
May 15, 2026Yunan Lu, Luigi Liu, Omar Yahia +2AI Agent Reliability Adversarial Prompt Generation
Columbia University · University of California San Diego · Walmart
The Great Pretender: A Stochasticity Problem in LLM Jailbreak
May 14, 2026Jean-Philippe Monteuuis, Cong Chen, Jonathan PetitLLM Safety Benchmarks LLM Evaluation
Qualcomm Technologies, Inc.
LLM-Based Persuasion Enables Guardrail Override in Frontier LLMs
May 13, 2026Rodrigo Nogueira, Thales Sales Almeida, Giovana Kerche Bonás +7Adversarial Prompt Generation LLM Guardrails
Maritaca AI · JusBrasil
REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations
May 12, 2026Buyun Liang, Jinqi Luo, Liangzu Peng +6Adversarial Prompt Generation Adversarial Attacks
University of Pennsylvania
Persona-Conditioned Adversarial Prompting: Multi-Identity Red-Teaming for Adversarial Discovery and Mitigation
May 12, 2026Cristian Morasso, Anisa Halimi, Muhammad Zaid Hameed +1Adversarial Training Adversarial Prompt Generation
IBM Research · Trinity College Dublin, Ireland
The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
May 9, 2026Ismail Hossain, Tanzim Ahad, Md Jahangir Alam +3Adversarial Prompt Generation Language Model Safety Evaluation
Department of Computer Science, University of Texas at El Paso, TX, USA 79902 · School of Computing, Southern Illinois University Carbondale, IL, USA 62901 · University of Illinois Urbana-Champaign, IL, USA
LLM-Agnostic Semantic Representation Attack
May 9, 2026Jiawei Lian, Jianhong Pan, Lefan Wang +4Adversarial Prompt Generation Adversarial Attacks on LLMs
School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, China · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR · Hong Kong Institute of AI for Science, City University of Hong Kong, Hong Kong SAR
OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
May 8, 2026Jianming Chen, Yawen Wang, Junjie Wang +3Adversarial Prompt Generation T2I Generation
Institute of Software, Chinese Academy of Sciences, Beijing, China · Science & Technology on Integrated Information System Laboratory, Beijing, China · State Key Laboratory of Complex System Modeling and Simulation Technology, Beijing, China +1
PersonaTeaming: Supporting Persona-Driven Red-Teaming for Generative AI
May 7, 2026Wesley Hanwen Deng, Mingxi Yan, Sunnie S. Y. Kim +5Adversarial Prompt Generation LLM Red Teaming
Carnegie Mellon University Pittsburgh, Pennsylvania, USA · Apple Seattle, Washington, USA · Apple Cupertino, California, USA +1
MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
May 3, 2026Jialin Song, Xiaodong Liu, Weiwei Yang +4LLM Safety Benchmarks Adversarial Prompt Generation
Simon Fraser University · Microsoft Research
A Theoretical Game of Attacks via Compositional Skills
May 1, 2026Xinbo Wu, Huan Zhang, Abhishek Umrawal +1Adversarial Prompt Generation Adversarial Attacks on LLMs
University of Illinois Urbana-Champaign, Urbana, IL, USA · Stony Brook University, Stony Brook, NY, USA.
SafeReview: Defending LLM-based Review Systems Against Adversarial Hidden Prompts
Apr 29, 2026Yuan Xin, Yixuan Weng, Minjun Zhu +5Adversarial Training Adversarial Prompt Generation
1CISPA · 2Westlake University · 3Southern University of Science and Technology +1
Adaptive Prompt Embedding Optimization for LLM Jailbreaking
Apr 27, 2026Miles Q. Li, Benjamin C. M. Fung, Boyang Li +2Adversarial Prompt Generation Adversarial Attacks on LLMs
School of Information Studies, McGill University, Montreal, QC, Canada · Department of Computer Science, Kean University, Union, NJ, United States · Mila - Quebec AI Institute, Montreal, QC, Canada +1
Agentic Adversarial Rewriting Exposes Architectural Vulnerabilities in Black-Box NLP Pipelines
Apr 26, 2026Mazal Bethany, Kim-Kwang Raymond Choo, Nishant Vishwamitra +1Fake News Detection Adversarial Prompt Generation
Department of Information Systems and Cybersecurity, University of Texas at San Antonio, San Antonio, TX 78249 USA
CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoning
Apr 25, 2026Shuxu Chen, Yitian Zhou, Jiaquan Zhang +6Adversarial Prompt Generation LLM Prompting
Department of Electronics and Information Convergence Engineering, Kyung Hee University, Republic of Korea · University of Electronic Science and Technology of China, Chengdu, China · East China Jiaotong University, Nanchang, China +1
Adaptive Instruction Composition for Automated LLM Red-Teaming
Apr 22, 2026Jesse Zymet, Andy Luo, Swapnil Shinde +2Adversarial Prompt Generation LLM Red Teaming
Capital One, AI Foundations
EvoJail: Evolutionary Diverse Jailbreak Prompt Generation for Large Language Models
Apr 22, 2026Rui Tang, Kaiyu Xu, Pengsen Cheng +3Adversarial Prompt Generation LLM Jailbreak Attacks
aSchool of Cyber Science and Engineering, Sichuan University, Chengdu 610065, China · dCollege of Computer Science, Sichuan University, Chengdu 610065, China · aSchool of Cyber Science and Engineering, Sichuan University, Chengdu 610065, China bKey Laboratory of Data Protection and Intelligent Management (Sichuan University), Ministry of Education, Chengdu 610065, China
Reasoning-targeted Jailbreak Attacks on Large Reasoning Models via Semantic Triggers and Psychological Framing
Apr 17, 2026Zehao Wang, Lanjun WangAdversarial Prompt Generation Jailbreak Attacks
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China. · School of New Media and Communication, Tianjin University, Tianjin 300072, China. · Shanghai Key Laboratory of Data Science,[cs.LG] Shanghai 200438, China.
Route to Rome Attack: Directing LLM Routers to Expensive Models via Adversarial Suffix Optimization
Apr 16, 2026Haochun Tang, Yuliang Yan, Jiahua Lu +2Adversarial Prompt Generation Cost-Aware Inference
1Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University · 2The Hong Kong University of Science and Technology (Guangzhou)
T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search
Mar 21, 2026Hyomin Lee, Sangwoo Park, Yumin Choi +3Adversarial Prompt Generation LLM Red Teaming
KAIST · University of California, Los Angeles
"Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers
Nov 3, 2025Qin Zhou, Zhexin Zhang, Zhi Li +1Adversarial Prompt Generation Indirect Prompt Injection
Institute of Information Engineering, CAS, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · The Conversational AI (CoAI) group, DCST, Tsinghua University
NonTextual Target Attack
Oct 3, 2025Xinzhe Huang, Wenjing Hu, Tianhang Zheng +6Adversarial Prompt Generation Adversarial Attacks on LLMs
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou 310007, China · Hangzhou High-Tech Zone, Binjiang Institute of Blockchain and Data Security, Hangzhou 310053, China · School of Cyberspace Security, Nanjing University of Science and Technology, Nanjing 210094, China +3
RedCoder: Automated Multi-Turn Red Teaming for Code LLMs
Jun 25, 2025Wenjie Jacky Mo, Qin Liu, Xiaofei Wen +5Adversarial Prompt Generation LLM Red Teaming
University of California, Davis · University of Southern California
Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models
Mar 8, 2025Thomas Winninger, Boussad Addad, Katarzyna KapustaAdversarial Prompt Generation Adversarial Attacks on LLMs
*Work started during internship at Thales.

Sep 28, 2026, cs.AIJ/K move · Enter open · S save

RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Models

Dmitrii Kharlapenko, Sergei Bratchikov, Konstantin Korolev, Aleksandr Nikolich

White Circle

Adversarial Prompt Generation T2I Generation Evaluation

Open paper AskPDF arXiv

On modern production text-to-image systems, successful policy violations are rare, and previously effective human-written seeds are often patched out. Current automated red-teamers are poorly matched to this regime in two ways: unreliable success measurement and poor exploration. First, we find that judges widely used in prior T2I red-teaming work are unreliable under vague unsafe-content targets: they either miss true violations or reward benign borderline images on hardened APIs. We therefore define strict category-specific success criteria and calibrate strong VLM judges against human labels. Second, we show that broadly used prompt-modification pipelines do not solve the exploration problem: on harder guardrail settings they remain tied to seed prompts, fail to transfer, or cannot bootstrap positive examples. We introduce RISE, which evolves reusable strategies used to generate prompts rather than rewriting them one by one. The best discovered strategies are then reused to generate attacks across new scenarios. On DALL-E 3, Nano Banana 2 (Google) and GPT-Image-2, RISE reaches up to 13% human-verified ASR; under the same calibrated evaluation, prior methods with reported ASR as high as roughly 30% fall to near zero.

Momentum

Related topics

Latest papers 53