Training Time Attack

Training-time attacks exploit vulnerabilities in the machine learning model training process to inject malicious behavior, compromising model integrity and security. Current research focuses on various attack vectors, including backdoor insertion, data poisoning, and adversarial reward manipulation, across diverse model architectures like LLMs, reinforcement learning agents, and deep neural networks. These attacks pose significant risks to the reliability and trustworthiness of AI systems across numerous applications, driving intense investigation into robust defense mechanisms and verifiable training methods. The ultimate goal is to develop models resistant to manipulation during training, ensuring the safety and security of deployed AI systems.

Papers

September 19, 2024

August 20, 2024

Security Assessment of Hierarchical Federated Deep Learning
D Alqattan, R Sun, H Liang, G Nicosia, V Snasel, R Ranjan, V Ojha
Adversarial Attack Adversarial Training Inference Time Hierarchical Federated Learning Training Time Attack Security Evaluation

July 11, 2024

Model-agnostic clean-label backdoor mitigation in cybersecurity environments
Giorgio Severi, Simona Boboila, John Holodnak, Kendra Kratkiewicz, Rauf Izmailov, Michael J. De Lucia, Alina Oprea
Backdoor Defense Threat Model Training Time Attack Clean Label Backdoor Attack Clean Label Attack Label Poisoning Attack Cybersecurity Environment

July 9, 2024

Tracing Back the Malicious Clients in Poisoning Attacks to Federated Learning
Yuqi Jia, Minghong Fang, Hongbin Liu, Jinghuai Zhang, Neil Zhenqiang Gong
Poisoning Attack Malicious Client Training Time Attack

July 3, 2024

SOS! Soft Prompt Attack Against Open-Source Large Language Models
Ziqing Yang, Michael Backes, Yang Zhang, Ahmed Salem
Large Language Model Backdoor Attack Open Source Large Language Model Training Time Attack

June 17, 2024

FullCert: Deterministic End-to-End Certification for Training and Inference of Neural Networks
Tobias Lorenz, Marta Kwiatkowska, Mario Fritz
Neural Network Training Data Scientific Inference Inference Time Robustness Guarantee Training Time Attack Novel Certification Certification Algorithm

May 30, 2024

SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents
Ethan Rathbun, Christopher Amato, Alina Oprea
Reinforcement Learning Adversarial Attack Reinforcement Learning Agent Backdoor Poisoning Attack Training Time Attack Universal Backdoor

March 14, 2024

Optimistic Verifiable Training by Controlling Hardware Nondeterminism
Megha Srivastava, Simran Arora, Dan Boneh
Training Time Attack Verifiable Learning

January 5, 2024

Adaptive Discounting of Training Time Attacks
Ridhima Bector, Abhay Aradhya, Chai Quek, Zinovi Rabinovich
Reinforcement Learning Training Time Attack Time Varying Impedance Attack

December 20, 2023

Progressive Poisoned Data Isolation for Training-time Backdoor Defense
Yiming Chen, Haiwei Wu, Jiantao Zhou
Data Poisoning Training Time Attack Social Isolation Poisoned Data Input Level Backdoor Detection

November 16, 2023

Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations
Wenjie Mo, Jiashu Xu, Qin Liu, Jiongxiao Wang, Jun Yan, Chaowei Xiao, Muhao Chen
Noisy Demonstration Backdoor Defense Black Box Large Language Model Training Time Attack Instruction Backdoor Attack Test Time Defense

May 3, 2023

Defending against Insertion-based Textual Backdoor Attacks via Attribution
Jiazhao Li, Zhuofeng Wu, Wei Ping, Chaowei Xiao, V. G. Vinod Vydiswaran
Backdoor Attack Source Attribution Training Time Attack Textual Backdoor Attack Attribution Mask Insertion Based

March 23, 2023

Backdoor Defense via Adaptively Splitting Poisoned Dataset
Kuofeng Gao, Yang Bai, Jindong Gu, Yong Yang, Shu-Tao Xia
Backdoor Attack Backdoor Defense Training Time Attack Data Splitting Robust Backdoor Attack

March 22, 2023

Test-time Detection and Repair of Adversarial Samples via Masked Autoencoder
Yun-Yun Tsai, Ju-Chin Chao, Albert Wen, Zhaoyuan Yang, Chengzhi Mao, Tapan Shah, Junfeng Yang
Adversarial Attack Adversarial Training Adversarial Sample Masked AutoEncoder Third Position Repair Training Time Attack Test Time Defense

February 27, 2023

Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attacks
Mohammad Mohammadi, Jonathan Nöther, Debmalya Mandal, Adish Singla, Goran Radanovic
Poisoning Attack Training Time Attack Adversarial Policy Optimal Adversarial Optimal Attack

February 24, 2023

Defending Against Backdoor Attacks by Layer-wise Feature Analysis
Najeeb Moharram Jebreel, Josep Domingo-Ferrer, Yiming Li
Deep Neural Network Backdoor Attack Layer Wise Training Time Attack Trigger Attack

February 16, 2023

SplitOut: Out-of-the-Box Training-Hijacking Detection in Split Learning via Outlier Detection
Ege Erdogan, Unat Teksen, Mehmet Salih Celiktenyildiz, Alptekin Kupcu, A. Ercument Cicek
Deep Neural Network Data Detection Outlier Detection Split Learning Attack Strategy Split and Fit Training Time Attack Training Data Privacy Shelf Model

January 25, 2023

A Data-Centric Approach for Improving Adversarial Training Through the Lens of Out-of-Distribution Detection
Mohammad Azizmalayeri, Arman Zarei, Alireza Isavand, Mohammad Taghi Manzuri, Mohammad Hossein Rohban
Adversarial Training Distribution Detection Camera Lens Data Centric Imperceptible Adversarial Perturbation Training Time Attack Softmax Probability

November 2, 2022

Dormant Neural Trojans
Feisi Fu, Panagiota Kiourti, Wenchao Li
Backdoor Attack Backdoor Detection Training Time Attack Trojan Detection Neural Trojan