Instruction Backdoor Attack

Instruction backdoor attacks exploit the vulnerabilities of instruction-tuned large language models (LLMs) by embedding malicious triggers within instructions, causing the model to generate harmful outputs upon encountering these triggers. Current research focuses on developing both sophisticated attack methods, including multimodal attacks targeting visual-language models and compositional attacks that hide malicious intent within seemingly harmless instructions, and robust defense mechanisms, such as embedding-based adversarial removal and test-time mitigation strategies using defensive demonstrations. Understanding and mitigating these attacks is crucial for ensuring the safe and reliable deployment of LLMs in various applications, impacting both the security of AI systems and the trustworthiness of AI-driven services.

Papers

January 9, 2025

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency
Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Jialing Tao, YueFeng Chen, Hui Xue, Xingxing Wei
Jailbreak Attack Shuffle Model Malicious Instruction Instruction Backdoor Attack

December 31, 2024

A Method for Enhancing the Safety of Large Model Generation Based on Multi-dimensional Attack and Defense
Keke Zhai
Practical Method Large Model Human SAFETY Step Attack Generative Attack Instruction Backdoor Attack

June 24, 2024

BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models
Yi Zeng, Weiyu Sun, Tran Ngoc Huynh, Dawn Song, Bo Li, Ruoxi Jia
Backdoor Attack Backdoor Trigger Instruction Tuned Language Model Instruction Backdoor Attack Backdoor Attacker Adversarial Removal

February 21, 2024

VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models
Jiawei Liang, Siyuan Liang, Man Luo, Aishan Liu, Dongchen Han, Ee-Chien Chang, Xiaochun Cao
Instruction Tuning Trojan Attack Instruction Backdoor Attack Autoregressive Vision Language Model

February 14, 2024

Instruction Backdoor Attacks Against Customized LLMs
Rui Zhang, Hongwei Li, Rui Wen, Wenbo Jiang, Yuan Zhang, Michael Backes, Yun Shen, Yang Zhang
Instruction Backdoor Attack

January 21, 2024

Instructional Fingerprinting of Large Language Models
Jiashu Xu, Fei Wang, Mingyu Derek Ma, Pang Wei Koh, Chaowei Xiao, Muhao Chen
Full Model LLM Watermarking Instruction Backdoor Attack

January 2, 2024

Imperio: Language-Guided Backdoor Attacks for Arbitrary Model Control
Ka-Ho Chow, Wenqi Wei, Lei Yu
Backdoor Attack Instruction Backdoor Attack

November 16, 2023

Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations
Wenjie Mo, Jiashu Xu, Qin Liu, Jiongxiao Wang, Jun Yan, Chaowei Xiao, Muhao Chen
Noisy Demonstration Backdoor Defense Black Box Large Language Model Training Time Attack Instruction Backdoor Attack Test Time Defense

October 16, 2023

Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attacks
Shuyu Jiang, Xingshu Chen, Rui Tang
Large Language Model Stealthy Attack Unsafe Prompt Composite Attack Instruction Backdoor Attack

May 24, 2023

Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models
Jiashu Xu, Mingyu Derek Ma, Fei Wang, Chaowei Xiao, Muhao Chen
Backdoor Attack Instruction Tuning Human Instruction Instruction Tuned Model Data Poisoning Instruction Backdoor Attack

April 20, 2023

Safety Assessment of Chinese Large Language Models
Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng, Minlie Huang
Large Language Model Chinese Large Language Model Safety Evaluation Safety Benchmark Instruction Backdoor Attack

Instruction Backdoor Attack

Papers

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

A Method for Enhancing the Safety of Large Model Generation Based on Multi-dimensional Attack and Defense

BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models

VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models

Instruction Backdoor Attacks Against Customized LLMs

Instructional Fingerprinting of Large Language Models

Imperio: Language-Guided Backdoor Attacks for Arbitrary Model Control

Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations

Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attacks

Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models

Safety Assessment of Chinese Large Language Models