Adversarial Prompt Generation

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 55

All topics

LASH: Adaptive Semantic Hybridization for Black-Box Jailbreaking of Large Language Models
May 20, 2026Abdullah Al Nomaan Nafi, Fnu Suya, Swarup Bhunia +1Adversarial Prompt Generation LLM Jailbreak Attacks
University of Maine · University of Tennessee, Knoxville · University of Florida
PQR: A Framework to Generate Diverse and Realistic User Queries that Elicit QA Agent Failures
May 15, 2026Yunan Lu, Luigi Liu, Omar Yahia +2AI Agent Reliability Adversarial Prompt Generation
Columbia University · University of California San Diego · Walmart
The Great Pretender: A Stochasticity Problem in LLM Jailbreak
May 14, 2026Jean-Philippe Monteuuis, Cong Chen, Jonathan PetitLLM Safety Benchmarks LLM Evaluation
Qualcomm Technologies, Inc.
LLM-Based Persuasion Enables Guardrail Override in Frontier LLMs
May 13, 2026Rodrigo Nogueira, Thales Sales Almeida, Giovana Kerche Bonás +7Adversarial Prompt Generation LLM Guardrails
Maritaca AI · JusBrasil
REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations
May 12, 2026Buyun Liang, Jinqi Luo, Liangzu Peng +6Adversarial Prompt Generation Adversarial Attacks
University of Pennsylvania
Persona-Conditioned Adversarial Prompting: Multi-Identity Red-Teaming for Adversarial Discovery and Mitigation
May 12, 2026Cristian Morasso, Anisa Halimi, Muhammad Zaid Hameed +1Adversarial Training Adversarial Prompt Generation
IBM Research · Trinity College Dublin, Ireland
The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
May 9, 2026Ismail Hossain, Tanzim Ahad, Md Jahangir Alam +3Adversarial Prompt Generation Language Model Safety Evaluation
Department of Computer Science, University of Texas at El Paso, TX, USA 79902 · School of Computing, Southern Illinois University Carbondale, IL, USA 62901 · University of Illinois Urbana-Champaign, IL, USA
LLM-Agnostic Semantic Representation Attack
May 9, 2026Jiawei Lian, Jianhong Pan, Lefan Wang +4Adversarial Prompt Generation Adversarial Attacks on LLMs
School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, China · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR · Hong Kong Institute of AI for Science, City University of Hong Kong, Hong Kong SAR
OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
May 8, 2026Jianming Chen, Yawen Wang, Junjie Wang +3Adversarial Prompt Generation T2I Generation
Institute of Software, Chinese Academy of Sciences, Beijing, China · Science & Technology on Integrated Information System Laboratory, Beijing, China · State Key Laboratory of Complex System Modeling and Simulation Technology, Beijing, China +1
PersonaTeaming: Supporting Persona-Driven Red-Teaming for Generative AI
May 7, 2026Wesley Hanwen Deng, Mingxi Yan, Sunnie S. Y. Kim +5Adversarial Prompt Generation LLM Red Teaming
Carnegie Mellon University Pittsburgh, Pennsylvania, USA · Apple Seattle, Washington, USA · Apple Cupertino, California, USA +1
MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
May 3, 2026Jialin Song, Xiaodong Liu, Weiwei Yang +4LLM Safety Benchmarks Adversarial Prompt Generation
Simon Fraser University · Microsoft Research
A Theoretical Game of Attacks via Compositional Skills
May 1, 2026Xinbo Wu, Huan Zhang, Abhishek Umrawal +1Adversarial Prompt Generation Adversarial Attacks on LLMs
University of Illinois Urbana-Champaign, Urbana, IL, USA · Stony Brook University, Stony Brook, NY, USA.
SafeReview: Defending LLM-based Review Systems Against Adversarial Hidden Prompts
Apr 29, 2026Yuan Xin, Yixuan Weng, Minjun Zhu +5Adversarial Training Adversarial Prompt Generation
1CISPA · 2Westlake University · 3Southern University of Science and Technology +1
Adaptive Prompt Embedding Optimization for LLM Jailbreaking
Apr 27, 2026Miles Q. Li, Benjamin C. M. Fung, Boyang Li +2Adversarial Prompt Generation Adversarial Attacks on LLMs
School of Information Studies, McGill University, Montreal, QC, Canada · Department of Computer Science, Kean University, Union, NJ, United States · Mila - Quebec AI Institute, Montreal, QC, Canada +1
Agentic Adversarial Rewriting Exposes Architectural Vulnerabilities in Black-Box NLP Pipelines
Apr 26, 2026Mazal Bethany, Kim-Kwang Raymond Choo, Nishant Vishwamitra +1Fake News Detection Adversarial Prompt Generation
Department of Information Systems and Cybersecurity, University of Texas at San Antonio, San Antonio, TX 78249 USA
CAP-CoT: Cycle Adversarial Prompt for Improving Chain of Thoughts in LLM Reasoning
Apr 25, 2026Shuxu Chen, Yitian Zhou, Jiaquan Zhang +6Adversarial Prompt Generation LLM Prompting
Department of Electronics and Information Convergence Engineering, Kyung Hee University, Republic of Korea · University of Electronic Science and Technology of China, Chengdu, China · East China Jiaotong University, Nanchang, China +1
Adaptive Instruction Composition for Automated LLM Red-Teaming
Apr 22, 2026Jesse Zymet, Andy Luo, Swapnil Shinde +2Adversarial Prompt Generation LLM Red Teaming
Capital One, AI Foundations
EvoJail: Evolutionary Diverse Jailbreak Prompt Generation for Large Language Models
Apr 22, 2026Rui Tang, Kaiyu Xu, Pengsen Cheng +3Adversarial Prompt Generation LLM Jailbreak Attacks
aSchool of Cyber Science and Engineering, Sichuan University, Chengdu 610065, China · dCollege of Computer Science, Sichuan University, Chengdu 610065, China · aSchool of Cyber Science and Engineering, Sichuan University, Chengdu 610065, China bKey Laboratory of Data Protection and Intelligent Management (Sichuan University), Ministry of Education, Chengdu 610065, China
Reasoning-targeted Jailbreak Attacks on Large Reasoning Models via Semantic Triggers and Psychological Framing
Apr 17, 2026Zehao Wang, Lanjun WangAdversarial Prompt Generation Jailbreak Attacks
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China. · School of New Media and Communication, Tianjin University, Tianjin 300072, China. · Shanghai Key Laboratory of Data Science,[cs.LG] Shanghai 200438, China.
Route to Rome Attack: Directing LLM Routers to Expensive Models via Adversarial Suffix Optimization
Apr 16, 2026Haochun Tang, Yuliang Yan, Jiahua Lu +2Adversarial Prompt Generation Cost-Aware Inference
1Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University · 2The Hong Kong University of Science and Technology (Guangzhou)
T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search
Mar 21, 2026Hyomin Lee, Sangwoo Park, Yumin Choi +3Adversarial Prompt Generation LLM Red Teaming
KAIST · University of California, Los Angeles
"Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers
Nov 3, 2025Qin Zhou, Zhexin Zhang, Zhi Li +1Adversarial Prompt Generation Indirect Prompt Injection
Institute of Information Engineering, CAS, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · The Conversational AI (CoAI) group, DCST, Tsinghua University
NonTextual Target Attack
Oct 3, 2025Xinzhe Huang, Wenjing Hu, Tianhang Zheng +6Adversarial Prompt Generation Adversarial Attacks on LLMs
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou 310007, China · Hangzhou High-Tech Zone, Binjiang Institute of Blockchain and Data Security, Hangzhou 310053, China · School of Cyberspace Security, Nanjing University of Science and Technology, Nanjing 210094, China +3
RedCoder: Automated Multi-Turn Red Teaming for Code LLMs
Jun 25, 2025Wenjie Jacky Mo, Qin Liu, Xiaofei Wen +5Adversarial Prompt Generation LLM Red Teaming
University of California, Davis · University of Southern California
Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models
Mar 8, 2025Thomas Winninger, Boussad Addad, Katarzyna KapustaAdversarial Prompt Generation Adversarial Attacks on LLMs
*Work started during internship at Thales.

May 20, 2026, cs.CLJ/K move · Enter open · S save

LASH: Adaptive Semantic Hybridization for Black-Box Jailbreaking of Large Language Models

Abdullah Al Nomaan Nafi, Fnu Suya, Swarup Bhunia, Prabuddha Chakraborty

University of Maine · University of Tennessee, Knoxville · University of Florida

Adversarial Prompt Generation LLM Jailbreak Attacks Black-Box Adversarial Attacks

Open paper AskPDF arXiv

Jailbreak attacks expose a persistent gap between the intended safety behavior of aligned large language models and their behavior under adversarial prompting. Existing automated methods are increasingly effective but each commits to a single attack family (e.g., one refinement loop, one tree search, one mutation space, or one strategy library) and no single family dominates: the best-performing method shifts across target models and harm categories, suggesting complementary strengths that per-prompt composition could exploit. We introduce LASH (LLM Adaptive Semantic Hybridization), a black-box framework that treats outputs from multiple base attacks as reusable seed prompts and adaptively composes them for each target request. Given a seed pool, LASH searches over seed subsets and softmax-normalized mixture weights; a composition module synthesizes a single candidate prompt, and a derivative-free genetic optimizer updates the weights using black-box target feedback and a two-stage fitness function combining keyword-based refusal detection with LLM-judge scoring. On JailbreakBench, which contains 100 harmful prompts across 10 categories, we evaluate LASH on six common target models. LASH achieves an average attack success rate of 84.5% under keyword-based evaluation and 74.5% under two-stage evaluation, where responses are first filtered for refusals and then scored by an LLM judge for whether they substantively fulfill the original harmful request. LASH outperforms five state-of-the-art baselines on both metrics with only 30 mean target queries. LASH also remains competitive under three defense mechanisms and induces more success-like internal representations. These results suggest that adaptive composition across heterogeneous jailbreak strategies is a promising direction for black-box red-teaming.

Momentum

Related topics

Latest papers 55