Step Attack

Step attacks, encompassing single-turn and multi-turn variations, aim to exploit vulnerabilities in machine learning models, particularly large language models (LLMs) and deep neural networks (DNNs), by crafting carefully designed inputs to elicit undesired outputs or behaviors. Current research focuses on developing more sophisticated attack strategies, such as crescendo attacks and context-based fusion attacks, alongside robust defenses, including adversarial training with adaptive step sizes and contrastive decoding methods. Understanding and mitigating the effectiveness of step attacks is crucial for ensuring the safety and reliability of AI systems across various applications, from conversational AI to medical image analysis and federated learning.

Papers

October 14, 2024

Derail Yourself: Multi-turn LLM Jailbreak Attack through Self-discovered Clues
Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, Jing Shao
Jailbreak Attack Attack Method Superficial Clue Step Attack Multi Step Attack

October 2, 2024

On Using Certified Training towards Empirical Robustness
Alessandro De Palma, Serge Durand, Zakaria Chihani, François Terrier, Caterina Urban
Adversarial Attack Training Data Adversarial Example Adversarial Training Step Attack Multi Step Attack

September 26, 2024

RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreaking
Yifan Jiang, Kriti Aggarwal, Tanmay Laud, Kashif Munir, Jay Pujara, Subhabrata Mukherjee
Language Model Adversarial Attack Jailbreak Attack Step Attack Query Enhanced Network

September 4, 2024

Well, that escalated quickly: The Single-Turn Crescendo Attack (STCA)
Alan Aqrawi
Adversarial Attack Adversarial Testing Multi Step Adversarial Attack Step Attack

August 28, 2024

FRACTURED-SORRY-Bench: Framework for Revealing Attacks in Conversational Turns Undermining Refusal Efficacy and Defenses over SORRY-Bench (Automated Multi-shot Jailbreaks)
Aman Priyanshu, Supriti Vijay
New Framework New Attack Adversarial Prompt Robust Defense Step Attack Refusal Training CodeRAG Bench

August 23, 2024

Protecting against simultaneous data poisoning attacks
Neel Alex, Shoaib Ahmed Siddiqui, Amartya Sanyal, David Krueger
Backdoor Attack Backdoor Defense Data Poisoning Attack Step Attack

August 11, 2024

Improving Adversarial Transferability with Neighbourhood Gradient Information
Haijing Guo, Jiafeng Wang, Zhaoyu Chen, Kaixun Jiang, Lingyi Hong, Pinxue Guo, Jinglun Li, Wenqiang Zhang
Adversarial Example Step Attack Mask Strategy Infrared Attack

August 8, 2024

Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles
Xiongtao Sun, Deyue Zhang, Dongdong Yang, Quanchen Zou, Hui Li
Jailbreak Attack First Principle Step Attack Context Injection

July 17, 2024

Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspective
Zhaoxin Wang, Handing Wang, Cong Tian, Yaochu Jin
Adversarial Attack Adversarial Example Adversarial Training Bi Level Optimization Fast Algorithm Fast Adversarial Training Catastrophic Overfitting Step Attack

July 4, 2024

Securing Multi-turn Conversational Language Models From Distributed Backdoor Triggers
Terry Tong, Jiashu Xu, Qin Liu, Muhao Chen
Large Language Model Backdoor Attack Backdoor Trigger Conversational Large Language Model Step Attack Stealthy Backdoor Attack Multimodal Backdoor

July 1, 2024

Purple-teaming LLMs with Adversarial Defender Training
Jingyan Zhou, Kun Li, Junan Li, Jiawen Kang, Minda Hu, Xixin Wu, Helen Meng
Guardrail Model Step Attack Rainbow Teaming

November 23, 2023

OASIS: Offsetting Active Reconstruction Attacks in Federated Learning
Tre' R. Jeter, Truc Nguyen, Raed Alharbi, My T. Thai
Attack Strategy Reconstruction Attack Gradient Inversion Step Attack

October 24, 2023

Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworks
Xiaojun Jia, Jianshu Li, Jindong Gu, Yang Bai, Xiaochun Cao
Adversarial Example Adversarial Training Fast Propagation Step Attack Efficient Adversarial Training Single Step Adversarial Training

October 21, 2023

The Hidden Adversarial Vulnerabilities of Medical Federated Learning
Erfan Darzi, Florian Dubost, Nanna. M. Sijtsema, P. M. A van Ooijen
Adversarial Attack Federated Learning Medical Image Analysis Adversarial Vulnerability Gradient Information Artificial Intelligence Security Step Attack

August 4, 2023

Multi-attacks: Many images $+$ the same adversarial attack $\to$ many target labels
Stanislav Fort
Adversarial Attack Adversarial Example Adversarial Perturbation Step Attack Target Class

July 29, 2023

You Can Backdoor Personalized Federated Learning
Tiandi Ye, Cen Chen, Yinggui Wang, Xiang Li, Ming Gao
Backdoor Attack Malicious Client Step Attack pFL Method Decoupling Degradation

May 27, 2023

Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RL
Xiangyu Liu, Souradip Chakraborty, Yanchao Sun, Furong Huang
Adversarial Attack Human Like RL Adversarial Policy Step Attack Provable Defense

June 28, 2022

Reasoning about Moving Target Defense in Attack Modeling Formalisms
Gabriel Ballot, Vadim Malvone, Jean Leneutre, Etienne Borde
Step Attack DEfense Mechanism Attack Model DAG Learning Moving Target Defense

June 6, 2022

Fast Adversarial Training with Adaptive Step Size
Zhichao Huang, Yanbo Fan, Chen Liu, Weizhong Zhang, Yong Zhang, Mathieu Salzmann, Sabine Süsstrunk, Jue Wang
Adversarial Attack Adversarial Training Step Size Fast Adversarial Training Catastrophic Overfitting Step Attack Multi Step Attack

February 27, 2022

The Art of Manipulation: Threat of Multi-Step Manipulative Attacks in Security Games
Thanh H. Nguyen, Arunesh Sinha
Strategic Manipulation Art Specific Information Threat Word Step Attack Security Game Multi Step Attack Stackelberg Security Game