Adversarial Risk

Adversarial risk focuses on the vulnerability of machine learning models to malicious inputs designed to cause misclassification or other undesirable outputs. Current research investigates various attack methods, including test-time data poisoning and attacks targeting specific model components (e.g., face detectors, language models), and explores defense strategies such as adversarial training, robust model architectures (e.g., Vision Transformers), and novel loss functions. Understanding and mitigating adversarial risk is crucial for deploying reliable machine learning systems in security-sensitive applications, driving ongoing efforts to develop more robust models and rigorous evaluation methods.

Papers

November 6, 2024

A Fundamental Accuracy--Robustness Trade-off in Regression and Classification
Sohail Bahmani
Classification Code Adversarial Robustness Novel Regression Adversarial Risk Accuracy Robustness Ridge Function

October 26, 2024

Transferable Adversarial Attacks on SAM and Its Downstream Models
Song Xia, Wenhan Yang, Yi Yu, Xun Lin, Henghui Ding, Lingyu Duan, Xudong Jiang
Adversarial Attack Adversarial Example Segment Anything Model Transferable Adversarial Attack Adversarial Risk Downstream Model Transfer Based Adversarial Attack

October 7, 2024

On the Adversarial Risk of Test Time Adaptation: An Investigation into Realistic Test-Time Data Poisoning
Yongyi Su, Yushu Li, Nanqing Liu, Kui Jia, Xulei Yang, Chuan-Sheng Foo, Xun Xu
Test Time Adaptation Comprehensive Investigation Data Poisoning Adversarial Risk Adaptive Attack

September 23, 2024

Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI
Ambrish Rawat, Stefan Schoepf, Giulio Zizzo, Giandomenico Cornacchia, Muhammad Zaid Hameed, Kieran Fraser, Erik Miehling, Beat Buesser, Elizabeth M. Daly, Mark Purcell, Prasanna Sattigeri, Pin-Yu Chen, Kush R. Varshney
Adversarial Attack Technical Challenge Generative AI Visual Perspective Common Pitfall Red Teaming Generative AI System Adversarial Risk Adversarial Threat

April 26, 2024

Adversarial Consistency and the Uniqueness of the Adversarial Bayes Classifier
Natalie S. Frank
Generative Adversarial Adversarial Risk Adversarial Consistency Adversarial Surrogate

February 19, 2024

Manipulating hidden-Markov-model inferences by corrupting batch data
William N. Caballero, Jose Manuel Camacho, Tahir Ekin, Roi Naveiro
Adversarial Learning Hidden Markov Model Adversarial Behavior Adversarial Risk Batch Data

February 4, 2024

PROSAC: Provably Safe Certification for Machine Learning Models under Adversarial Attacks
Ziquan Liu, Zhuo Zhi, Ilija Bogunovic, Carsten Gerner-Beuerle, Miguel Rodrigues
Adversarial Attack Machine Learning Model Adversarial Risk Effective Adversarial

January 30, 2024

Towards Assessing the Synthetic-to-Measured Adversarial Vulnerability of SAR ATR
Bowen Peng, Bo Peng, Jingyuan Xia, Tianpeng Liu, Yongxiang Liu, Li Liu
Adversarial Attack Synthetic Aperture Radar Adversarial Vulnerability Transferable Attack Adversarial Risk

November 27, 2023

Attend Who is Weak: Enhancing Graph Condensation via Cross-Free Adversarial Training
Xinglin Li, Kun Wang, Hanhui Deng, Yuxuan Liang, Di Wu
Training Graph Synthetic Graph Adversarial Risk Graph Condensation General Adversarial Adversarial Clothes

September 7, 2023

How adversarial attacks can disrupt seemingly stable accurate classifiers
Oliver J. Sutton, Qinghua Zhou, Ivan Y. Tyukin, Alexander N. Gorban, Alexander Bastounis, Desmond J. Higham
Adversarial Attack Adversarial Example Adversarial Perturbation Adversarial Risk Stable Classifier

September 5, 2023

The Adversarial Implications of Variable-Time Inference
Dudi Biton, Aditi Misra, Efrat Levy, Jaidip Kotak, Ron Bitton, Roei Schuster, Nicolas Papernot, Yuval Elovici, Ben Nassi
Side Channel Adversarial Risk Decision Based Temporal Inference Time Varying Impedance Attack

September 2, 2023

Non-Asymptotic Bounds for Adversarial Excess Risk under Misspecified Models
Changyu Liu, Yuling Jiao, Junhui Wang, Jian Huang
Adversarial Loss Non Asymptotic Model Misspecification Adversarial Risk Adversarial Estimator

July 20, 2023

Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examples
Shaokui Wei, Mingda Zhang, Hongyuan Zha, Baoyuan Wu
Adversarial Example Backdoor Attack Adversarial Risk Backdoor Policy

June 22, 2023

Visual Adversarial Examples Jailbreak Aligned Large Language Models
Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, Prateek Mittal
Adversarial Example Jailbreak Attack Adversarial Vulnerability Adversarial Risk Man in the Middle Attack

June 16, 2023

You Don't Need Robust Machine Learning to Manage Adversarial Attack Risks
Edward Raff, Michel Benaroch, Andrew L. Farris
Machine Learning Native Robustness Adversarial Learning Adversarial Risk Adversarial Machine Learning

May 17, 2023

The Adversarial Consistency of Surrogate Risks for Binary Classification
Natalie Frank, Jonathan Niles-Weed
Binary Classification Adversarial Risk Surrogate Loss Adversarial Consistency Surrogate Safety Measure Consistent Surrogate

April 11, 2023

Simultaneous Adversarial Attacks On Multiple Face Recognition System Components
Inderjeet Singh, Kazuya Kakizaki, Toshinori Araki
Adversarial Example Adversarial Risk Adversarial Objective Face Recognition System

February 21, 2023

Generalization Bounds for Adversarial Contrastive Learning
Xin Zou, Weiwei Liu
Adversarial Attack Adversarial Training Generalization Bound Adversarial Risk Adversarial Contrastive Learning

February 14, 2023

On the Role of Randomization in Adversarially Robust Classification
Lucas Gnecco-Heredia, Yann Chevaleyre, Benjamin Negrevergne, Laurent Meunier, Muni Sreenivas Pydi
Adversarial Attack Adversarial Perturbation Integral Role Generative Adversarial Ensemble Classifier Adversarial Risk Randomization Program Deterministic Learner

January 31, 2023

Robust Linear Regression: Gradient-descent, Early-stopping, and Beyond
Meyer Scetbon, Elvis Dohmatob
Adversarial Attack Native Robustness Robust Model Robust Regression Early Stopping Adversarial Risk