Adversarial Prompt Generation

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 55

All topics

RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Models
Sep 28, 2026Dmitrii Kharlapenko, Sergei Bratchikov, Konstantin Korolev +1Adversarial Prompt Generation T2I Generation Evaluation
White Circle
ENDOPROMPT: Victim-Side Pseudo-References for Utility Degradation
Sep 24, 2026Qingyu Wu, Zeyu Feng, Yongda Yu +2Adversarial Prompt Generation Prompt Injection Attacks
Defense Innovation Institute, Academy of Military Science, Beijing, China · Nanjing University, Nanjing, China
Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions
Sep 23, 2026Tiantong Wu, Wei Yang Bryan LimAdversarial Prompt Generation Prompt Injection Attacks
Nanyang Technological University Singapore
ASCII Attack: Recontextualising Harmful Requests as Artistic Critique in Large Language Models
Sep 2, 2026Da Cheng Gu, Yifei Dong, Xinghao Yang +2LLM Jailbreak Attacks LLM Safety Evaluation
Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo, NSW, 2007, Australia · China University of Petroleum, China · Shandong University, China
Breadth Beats Depth: Improving GCG-Based Jailbreak Optimization with Breadth-Oriented Suffix Search
Sep 2, 2026Shiliang Xiao, Jingsong Wei, Yuzhi Liang +3LLM Jailbreak Attacks Adversarial Prompt Generation
School of Information Science and Technology Guangdong University of Foreign Studies
Generating Attacks for LLMs with GFlowNets
Aug 10, 2026Berkay Ozcam, Irem Onen, Mehmet Fatih Amasyali +1Adversarial Prompt Generation Adversarial Attacks on LLMs
Cybersecurity R&D Turkcell Istanbul, Turkey · Department of Computer Engineering Yıldız Technical University Istanbul, Turkey
Mind the Gap: Zero-Query Jailbreaks via Filter-Generator Discrepancy in Text-to-Image Systems
Aug 2, 2026Wanguang Li, Zhaoxin Wang, Handing WangAdversarial Prompt Generation T2I Generation
School of Artificial Intelligence, Xidian University, Shaanxi, China
Adversarial Prompts for Acceptance Collapse in Speculative Decoding
Jul 23, 2026Run Wang, Chaoyi Zhou, Xi Liu +7Adversarial Prompt Generation Speculative Decoding
Clemson University · Wayne State University · University of Washington
Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models
Jul 20, 2026Dongdong Yang, Deyue Zhang, Zhao Liu +5Adversarial Prompt Generation Adversarial Examples
AI Security Lab, Beijing, China · School of Artificial Intelligence, Beihang University, Beijing, China
Adversarial Prompting Framework for AI Safety Assessment
Jul 15, 2026Yash Bhatnagar, Kunal Banerjee, Anirban ChatterjeeAdversarial Prompt Generation Adversarial Attacks on LLMs
Microsoft · Bengaluru, Karnataka, India · Walmart Global Tech
Seduced by the Narrative: Assessing Rule Adherence in Semi-Open Textual Sandboxes
Jul 2, 2026Weiying Chen, Junlong Shen, Zhanyuan Guo +1LLM Safety Benchmarks Adversarial Prompt Generation
University of Alberta, Edmonton, AB, Canada · N-Dice Association · Qilu University of Technology, Shandong, China
AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability
Jun 23, 2026Khanak KhandelwalAdversarial Prompt Generation Adversarial Attacks on LLMs
Indian Institute of Technology Jodhpur
Distributed Quality-Diversity Search for Toxicity in Large Language Models
Jun 23, 2026Onkar Shelar, Travis DesellAdversarial Prompt Generation LLM Red Teaming
Rochester Institute of Technology, Rochester NY 14623, USA
Adversarial Domain Prompt Tuning and Generation for Single Domain Generalization
Jun 19, 2026Zhipeng Xu, De Cheng, Xinyang Jiang +3Synthetic Data Augmentation Adversarial Prompt Generation
Xidian University · Microsoft Research Asia · Chongqing University of Posts and Telecommunications
OTTER: A Red-Teaming System for Toxicity-Evading Jailbreak Prompt Optimization
Jun 19, 2026Jerry Wang, Hsin-Ling Hsu, Yi-Cheng Lai +2Adversarial Prompt Generation Adversarial Attacks on LLMs
1National Chengchi University, Taipei, Taiwan
Automated jailbreak attack targeting multiple defense strategies
Jun 15, 2026Qi Wang, Chengcheng Wan, Weijia He +4Adversarial Prompt Generation LLM Jailbreak Attacks
East China Normal University Shanghai, China · East China Normal University, Shanghai Innovation Institute Shanghai, China · University of Southampton Southampton, England +2
GAS-Leak-LLM: Genetic Algorithm-Based Suffix Optimization for Black-Box LLM Jailbreaking
Jun 14, 2026Aman Anifer, Vignesh Kumar Kembu, Vishnu M +4Adversarial Prompt Generation LLM Safety
Department of Computer Applications, Cochin University of Science and Technology, Kerala, India. · Department of Electrical, Computer and Biomedical Engineering, University of Pavia, A. Ferrata, 5, Pavia, 27100, Italy.
Greedy Coordinate Diffusion: Effective and Semantically Coherent Adversarial Attacks via Diffusion Guidance
Jun 14, 2026Bohdan Turbal, Blossom Metevier, Max Springer +1Adversarial Prompt Generation Adversarial Attacks
Princeton University, Princeton, NJ, USA.
JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization
Jun 9, 2026Ge Shi, Jun Yin, Donglin Xie +3Adversarial Prompt Generation Large Language Model-Guided Optimization
University of California, Davis · The Renmin University of China · Independent Researcher +3
Assessing Automated Prompt Injection Attacks in Agentic Environments
Jun 9, 2026David Hofer, Edoardo Debenedetti, Florian TramèrAdversarial Prompt Generation LLM Agent Security
ETH Zurich Zurich, Switzerland
Diffuse AI Control on Fuzzy Tasks
Jun 8, 2026Mikhail Terekhov, Caglar Gulcehre, Vivek Hebbar +1Adversarial Prompt Generation Language Model Safety Evaluation
Anthropic Fellows Program (via MATS) · EPFL · Redwood Research +1
CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning
Jun 4, 2026Rahul Markasserithodi, Aditya Joshi, Yuekang Li +3Adversarial Training Adversarial Prompt Generation
University of New South Wales, Australia
Voices Across Registers: Corpus-Conditioned Vernacular Jailbreaks against Aligned LLMs via Fanfiction Subgenres
Jun 3, 2026Zhongze Luo, Ruihe Shi, Zhenshuai Yin +3Adversarial Prompt Generation Language Model Safety Evaluation
School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen) · School of Microelectronics, Xi’an Jiaotong University · 2The Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen) +1
Dive into Ambiguity: A*-Inspired Multi-Agents Commonsense Obfuscation Attack on LLM Prompts
May 31, 2026Boxuan Wang, Zhuoyun Li, Xiaowei Huang +1Adversarial Prompt Generation Adversarial Attacks
University of Liverpool Merseyside, United Kingdom
Quality-Diversity Evolution for Discovering Diverse Vulnerabilities in LLM Safety
May 30, 2026Subhadip MitraAdversarial Prompt Generation Adversarial Attacks on LLMs
Research Lead, Rota Labs
Automatically Attacking Software Reverse Engineering AI Agents
May 28, 2026Brian Crawford, Justin Phillips, Patrick McClureSoftware Engineering Adversarial Prompt Generation
Naval Postgraduate School, Monterey, CA, United States
Harmless Yet Harmful: Neutral Prompting Attacks for Stealthy Hallucination Steering in Agent Skills
May 28, 2026Chia-Yi Hsu, Chia-Mu Yu, Chun-Ying Huang +1Adversarial Prompt Generation AI Coding Agents
Department of Computer Science National Yang Ming Chiao Tung University · Department of Electronics and Electrical Engineering National Yang Ming Chiao Tung University · School of Computing Institute of Science Tokyo
Evolving and Detecting Multi-Turn Deception using Geometric Signatures
May 26, 2026Surender Suresh Kumar, Mary L. CummingsAdversarial Prompt Generation Deception in Language Models
George Mason University Fairfax, USA
PAST2HARM: A Simple Adaptive Past Tense Attack for Jailbreaking Multimodal AI
May 26, 2026Snehasis MukhopadhyayAdversarial Prompt Generation Multimodal Jailbreak Attacks
Indian Institute of Information Technology, Kalyani
Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Models
May 25, 2026Arian Komaei Koma, Seyed Amir Kasaei, AmirMahdi Sadeghzadeh +1Adversarial Prompt Generation Adversarial Attacks
Department of Computer Engineering Sharif University of Technology

Sep 28, 2026, cs.AIJ/K move · Enter open · S save

RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Models

Dmitrii Kharlapenko, Sergei Bratchikov, Konstantin Korolev, Aleksandr Nikolich

White Circle

Adversarial Prompt Generation T2I Generation Evaluation

Open paper AskPDF arXiv

On modern production text-to-image systems, successful policy violations are rare, and previously effective human-written seeds are often patched out. Current automated red-teamers are poorly matched to this regime in two ways: unreliable success measurement and poor exploration. First, we find that judges widely used in prior T2I red-teaming work are unreliable under vague unsafe-content targets: they either miss true violations or reward benign borderline images on hardened APIs. We therefore define strict category-specific success criteria and calibrate strong VLM judges against human labels. Second, we show that broadly used prompt-modification pipelines do not solve the exploration problem: on harder guardrail settings they remain tied to seed prompts, fail to transfer, or cannot bootstrap positive examples. We introduce RISE, which evolves reusable strategies used to generate prompts rather than rewriting them one by one. The best discovered strategies are then reused to generate attacks across new scenarios. On DALL-E 3, Nano Banana 2 (Google) and GPT-Image-2, RISE reaches up to 13% human-verified ASR; under the same calibrated evaluation, prior methods with reported ASR as high as roughly 30% fall to near zero.

Momentum

Related topics

Latest papers 55