Adversarial Evaluation

Adversarial evaluation assesses the robustness of machine learning models, particularly deep neural networks (DNNs) and large language models (LLMs), against malicious inputs designed to cause misclassification or undesirable behavior. Current research focuses on developing more robust architectures (e.g., incorporating specialized first layers or quantization techniques), improving adversarial training methods (e.g., through data pruning or adaptive perturbation strategies), and creating more comprehensive evaluation benchmarks that consider out-of-distribution data and diverse attack types. These efforts are crucial for ensuring the reliability and safety of AI systems in various applications, from medical image analysis to autonomous driving and cybersecurity, where vulnerabilities can have significant consequences.

Papers

October 11, 2024

Do Unlearning Methods Remove Information from Language Model Weights?
Aghyad Deeb, Fabien Roger
Large Language Model Unlearning Framework New Attack Adversarial Evaluation

August 21, 2024

First line of defense: A robust first layer mitigates adversarial attacks
Janani Suresh, Nancy Nayak, Sheetal Kalyani
Adversarial Attack Adversarial Training Adversarial Learning Convolutional Filter Best Fit Line Adversarial Evaluation

July 21, 2024

Adversarial Circuit Evaluation
Niels uit de Bos, Adrià Garriga-Alonso
Adversarial Text Large Scale Circuit Adversarial Evaluation Benign Input

June 25, 2024

Robustly Optimized Deep Feature Decoupling Network for Fatty Liver Diseases Detection
Peng Huang, Shu Hu, Bo Peng, Jiashu Zhang, Xi Wu, Xin Wang
Medical Image Classification Feature Decoupling Adversarial Evaluation Novel Deep Learning Framework Non Alcoholic Fatty Liver Disease Adaptive Adversarial Training

June 21, 2024

DataFreeShield: Defending Adversarial Attacks without Training Data
Hyeyoon Lee, Kanghyun Choi, Dain Kwon, Sunjong Park, Mayoore Selvarasa Jaiswal, Noseong Park, Jonghyun Choi, Jinho Lee
Adversarial Attack Training Data Adversarial Training Adversarial Robustness Adversarial Evaluation

June 20, 2024

Uniform Convergence of Adversarially Robust Classifiers
Rachel Morris, Ryan Murray
Adversarial Perturbation Adversarial Evaluation Optimal Classifier Uniform Convergence Adversarial Capability

June 6, 2024

Batch-in-Batch: a new adversarial training framework for initial perturbation and sample selection
Yinting Wu, Pai Peng, Bo Cai, Le Li, .
Adversarial Training Adversarial Sample Adversarial Framework Large Batch Adversarial Evaluation Adversarial Setting Sample Selection

May 29, 2024

Crowdsourcing with Difficulty: A Bayesian Rating Model for Heterogeneous Items
Seong Woo Han, Ozan Adıgüzel, Bob Carpenter
Crowdsourcing Context Difficulty Level Adversarial Evaluation Crowdsourcing System Consensus Label

May 11, 2024

Integrating Emotional and Linguistic Models for Ethical Compliance in Large Language Models
Edward Y. Chang
Adversarial Evaluation Emotional Impact Emotion Representation

March 8, 2024

The Impact of Quantization on the Robustness of Transformer-based Text Classifiers
Seyed Parsa Neshaei, Yasaman Boreshban, Gholamreza Ghassem-Sani, Seyed Abolghasem Mirroshandel
Adversarial Attack Native Robustness Transformer Based Adversarial Training Global Impact Transformer Based Model Quantization Operator Adversarial Evaluation

February 18, 2024

A Curious Case of Searching for the Correlation between Training Data and Adversarial Robustness of Transformer Textual Models
Cuong Dang, Dung D. Le, Thai Le
Training Data Adversarial Training Adversarial Robustness Search Query Total Correlation Adversarial Evaluation Adversarial Text Perturbation

February 1, 2024

Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection
Xinlin Peng, Ying Zhou, Ben He, Le Sun, Yingfei Sun
Text Generation Task Adversarial Evaluation Writing Process Text Perturbation

November 16, 2023

Whispers of Doubt Amidst Echoes of Triumph in NLP Robustness
Ashim Gupta, Rishanth Rajendhran, Nathan Stringham, Vivek Srikumar, Ana Marasović
Native Robustness Model Robustness Adversarial Input State of the Art Whisper Adversarial Evaluation Fidelity Reward Acoustic Echo Robustness Issue

November 15, 2023

How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities
Lingbo Mo, Boshi Wang, Muhao Chen, Huan Sun
Open Source Large Language Model Direct Assessment Security Vulnerability Adversarial Evaluation Trustworthy Deep Learning Malicious Content Adversarial Demonstration

October 19, 2023

OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shift
Lin Li, Yifei Wang, Chawin Sitawarin, Michael Spratling
New Benchmark Adversarial Robustness Distribution Shift Adversarial Evaluation Large Scale Analysis High OOD Robustness

October 17, 2023

The Efficacy of Transformer-based Adversarial Attacks in Security Domains
Kunyang Li, Kyle Domico, Jean-Charles Noirot Ferrand, Patrick McDaniel
Adversarial Example Adversarial Perturbation Transformer Architecture Adversarial Sample Adversarial Evaluation Transformer Based Generative Adversarial

July 27, 2023

NSA: Naturalistic Support Artifact to Boost Network Confidence
Abhijith Sharma, Phil Munz, Apurva Narayan
Adversarial Attack Adversarial Example C Gan Adversarial Evaluation Large Scale Naturalistic Corruption Emulation

July 14, 2023

Exploiting Counter-Examples for Active Learning with Partial labels
Fei Zhang, Yunjie Ye, Lei Feng, Zhongwen Rao, Jieming Zhu, Marcus Kalander, Chen Gong, Jianye Hao, Bo Han
Active Learning Explanatory Inference Adversarial Evaluation Partial Label Counter Example Pseudo Label Sampling

June 15, 2023

DiffAug: A Diffuse-and-Denoise Augmentation for Training Robust Classifiers
Chandramouli Sastry, Sri Harsha Dumpala, Sageev Oore
Better Robustness Robust Classifier Adversarial Evaluation Augmentation Approach Classifier Guided Diffusion Diffusion Based Augmentation

May 30, 2023

A Multilingual Evaluation of NER Robustness to Adversarial Inputs
Akshay Srinivasan, Sowmya Vajjala
Language Model Entity Recognition Named Entity Recognition Adversarial Input Multilingual Evaluation Adversarial Evaluation