Adversarial Input

Adversarial input research focuses on developing and mitigating vulnerabilities in machine learning models, particularly large language models (LLMs) and deep neural networks (DNNs), by crafting inputs designed to elicit incorrect or harmful outputs. Current research emphasizes developing novel attack methods, such as prompt injection and image manipulation techniques, alongside robust defenses including adversarial training, invariance regularization, and prompt rewriting. This field is crucial for ensuring the safe and reliable deployment of AI systems across various applications, from autonomous vehicles to medical diagnosis, by improving model robustness and trustworthiness.

Papers

January 27, 2023

Targeted Attacks on Timeseries Forecasting
Yuvaraj Govindarajulu, Avinash Amballa, Pavan Kulkarni, Manojkumar Parmar
Adversarial Attack Time Series Time Series Forecasting Adversarial Input Targeted Attack Projected Gradient Descent Attack

January 5, 2023

gRoMA: a Tool for Measuring the Global Robustness of Deep Neural Networks
Natan Levy, Raz Yerushalmi, Guy Katz
Deep Neural Network BRIO Tool Adversarial Input Global Robustness

December 24, 2022

Frequency Regularization for Improving Adversarial Robustness
Binxiao Huang, Chaofan Tao, Rui Lin, Ngai Wong
Adversarial Training Adversarial Robustness Adversarial Input Imperceptible Adversarial Perturbation Robustness Problem Frequency Regularization

November 30, 2022

November 29, 2022

Interpretations Cannot Be Trusted: Stealthy and Effective Adversarial Perturbations against Interpretable Deep Learning
Eldor Abdukhamidov, Mohammed Abuhamad, Simon S. Woo, Eric Chan-Tin, Tamer Abuhmed
Deep Learning Deep Learning Model Adversarial Perturbation Adversarial Input Interpretable Deep Learning

November 23, 2022

Adversarial Attacks are a Surprisingly Strong Baseline for Poisoning Few-Shot Meta-Learners
Elre T. Oldewage, John Bronskill, Richard E. Turner
Adversarial Attack Adversarial Example Poisoning Attack Strong Baseline Adversarial Input Data Poisoning Attack Meta Learner Shot Meta Learning

November 4, 2022

An Adversarial Robustness Perspective on the Topology of Neural Networks
Morgane Goibert, Thomas Ricatte, Elvis Dohmatob
Neural Network Adversarial Robustness Adversarial Input Adversarial Vulnerability Topology Problem Adversary Agent Edge Optimization

October 31, 2022

Scoring Black-Box Models for Adversarial Robustness
Jian Vora, Pranay Reddy Samala
Adversarial Attack Native Robustness Adversarial Robustness Black Box Model Adversarial Input Score Based

October 10, 2022

July 31, 2022

DNNShield: Dynamic Randomized Model Sparsification, A Defense Against Adversarial Machine Learning
Mohammad Hossein Samavatian, Saikat Majumdar, Kristin Barber, Radu Teodorescu
Deep Neural Network Adversarial Attack Adversarial Learning Adversarial Input Dynamic Sparsity

July 21, 2022

Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testing
Michael Weiss, André García Gómez, Paolo Tonella
Adversarial Input Ambiguous Input Input Uncertainty Hidden Risk

July 8, 2022

Guiding the retraining of convolutional neural networks against adversarial inputs
Francisco Durán López, Silverio Martínez-Fernández, Michael Felderer, Xavier Franch
Convolutional Neural Network Deep Learning Model Adversarial Input Retraining Method Adequacy Metric

July 3, 2022

Tricking the Hashing Trick: A Tight Lower Bound on the Robustness of CountSketch to Adaptive Inputs
Edith Cohen, Jelani Nelson, Tamás Sarlós, Uri Stemmer
Native Robustness Lower Bound Adversarial Input Private Aggregation Input Adaptation Partitioned B Bit Hashing

June 23, 2022

BERT Rankers are Brittle: a Study using Adversarial Document Perturbations
Yumeng Wang, Lijun Lyu, Avishek Anand
Adversarial Attack Study Feature BERT Based Adversarial Input Token Embeddings Brittle Fracture Adversarial Text Perturbation

June 19, 2022

On the Limitations of Stochastic Pre-processing Defenses
Yue Gao, Ilia Shumailov, Kassem Fawaz, Nicolas Papernot
Adversarial Example Adversarial Robustness Fundamental Limitation Stochastic Way Adversarial Input Stochastic Defense

June 16, 2022

Using adversarial images to improve outcomes of federated learning for non-IID data
Anastasiya Danilenka, Maria Ganzha, Marcin Paprzycki, Jacek Mańdziuk
LeArning Abstract Imbalanced Data Adversarial Image Adversarial Input Multiple Outcome Non IID Data Skewed Non IID

June 15, 2022

Adversarial Input

Papers

Targeted Attacks on Timeseries Forecasting

gRoMA: a Tool for Measuring the Global Robustness of Deep Neural Networks

Frequency Regularization for Improving Adversarial Robustness

Extreme Image Transformations Affect Humans and Machines Differently

Efficient Adversarial Input Generation via Neural Net Patching

Interpretations Cannot Be Trusted: Stealthy and Effective Adversarial Perturbations against Interpretable Deep Learning

Adversarial Attacks are a Surprisingly Strong Baseline for Poisoning Few-Shot Meta-Learners

An Adversarial Robustness Perspective on the Topology of Neural Networks

Scoring Black-Box Models for Adversarial Robustness

Revisiting adapters with adversarial training

Certified Training: Small Boxes are All You Need

DNNShield: Dynamic Randomized Model Sparsification, A Defense Against Adversarial Machine Learning

Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testing

Guiding the retraining of convolutional neural networks against adversarial inputs

Tricking the Hashing Trick: A Tight Lower Bound on the Robustness of CountSketch to Adaptive Inputs

BERT Rankers are Brittle: a Study using Adversarial Document Perturbations

On the Limitations of Stochastic Pre-processing Defenses

Using adversarial images to improve outcomes of federated learning for non-IID data

Hardening DNNs against Transfer Attacks during Network Compression using Greedy Adversarial Pruning

Morphence-2.0: Evasion-Resilient Moving Target Defense Powered by Out-of-Distribution Detection