Adversarial Datasets

Adversarial datasets are designed to expose vulnerabilities in machine learning models by including examples that are easily misinterpreted, even by state-of-the-art systems, but are readily understood by humans. Current research focuses on developing robust methods for creating these datasets, including automated generation techniques and improved annotation strategies to ensure diversity and challenge model robustness across various tasks (e.g., question answering, hate speech detection, image recognition). The development and analysis of adversarial datasets are crucial for evaluating model reliability and improving the safety and trustworthiness of AI systems across diverse applications, ultimately driving progress in model robustness and generalization.

Papers

October 14, 2024

ROSAR: An Adversarial Re-Training Framework for Robust Side-Scan Sonar Object Detection
Martin Aubard, László Antal, Ana Madureira, Luis F. Teixeira, Erika Ábrahám
Autonomous Underwater Vehicle Adversarial Patch Attack Side Scan Sonar Adversarial Datasets Adversarial Retraining

June 24, 2024

Is your benchmark truly adversarial? AdvScore: Evaluating Human-Grounded Adversarialness
Yoo Yeon Sung, Maharshi Gor, Eve Fleisig, Ishani Mondal, Jordan Lee Boyd-Graber
Adversarial Example Global Evaluation Visual Creation North Star Metric Adversarial Datasets Adversarial Question Adversarial Benchmark

May 27, 2024

Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization
Dixuan Wang, Yanda Li, Junyuan Jiang, Zepeng Ding, Guochao Jiang, Jiaqing Liang, Deqing Yang
Language Model Tokenization Matter Adversarial Datasets Class Token

March 29, 2024

Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets
Shadi Manafi, Nikhil Krishnaswamy
Entity Recognition Low Resource Language Cross Lingual Transfer Multilingual Language Model High Resource Language Adversarial Datasets

March 28, 2024

Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Dataset
Janis Goldzycher, Paul Röttger, Gerold Schneider
Adversarial Example Hate Speech Critical Lesson Individual Annotator Adversarial Data Collection Adversarial Datasets

December 19, 2023

Zero-Shot Fact-Checking with Semantic Triples and Knowledge Graphs
Zhangdie Yuan, Andreas Vlachos
Large Language Model Knowledge Graph Zero Shot Natural Language Inference Fact Checking Adversarial Datasets

November 14, 2023

AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications
Bhaktipriya Radharapu, Kevin Robinson, Lora Aroyo, Preethi Lahoti
Red Teaming Adversarial Testing Adversarial Datasets LLM Powered Application

November 6, 2023

Measuring Adversarial Datasets
Yuanchen Bai, Raoyi Huang, Vijay Viswanathan, Tzu-Sheng Kuo, Tongshuang Wu
Adversarial Example Adversarial Robustness Adversarial Datasets

September 20, 2023

PRAT: PRofiling Adversarial aTtacks
Rahul Ambati, Naveed Akhtar, Ajmal Mian, Yogesh Singh Rawat
Adversarial Attack Adversarial Example Adversarial Sample Adversarial Datasets

September 1, 2023

Curating Naturally Adversarial Datasets for Learning-Enabled Medical Cyber-Physical Systems
Sydney Pugh, Ivan Ruchkin, Insup Lee, James Weimer
Native Robustness Adversarial Example Adversarial Datasets Natural Adversarial Sample

February 19, 2023

X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection
Aishan Liu, Jun Guo, Jiakai Wang, Siyuan Liang, Renshuai Tao, Wenbo Zhou, Cong Liu, Xianglong Liu, Dacheng Tao
Adversarial Attack Adversarial Datasets Adversarial Texture Adversarial Object Item Detection

August 18, 2022

Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning
Olivia Wiles, Isabela Albuquerque, Sven Gowal
Image Generation Supervised ImageNet Text to Image Model Vision Model Generated Caption Large Scale Generative Model Adversarial Datasets Bug Finding

March 12, 2022

A Proposal to Study "Is High Quality Data All We Need?"
Swaroop Mishra, Anjana Arunkumar
Deep Neural Network High Quality Proposal Balance Refinement High Quality Data Adversarial Datasets

February 27, 2022

Overlooked Implications of the Reconstruction Loss for VAE Disentanglement
Nathan Michlo, Richard Klein, Steven James
Variational Autoencoder Variational Autoencoders Disentanglement Capability Reconstruction Loss Adversarial Datasets

February 24, 2022

Using calibrator to improve robustness in Machine Reading Comprehension
Jing Jin, Houfeng Wang
Native Robustness Calibration Performance Machine Reading Comprehension Systematic Generalization Adversarial Datasets Unified Generalization

November 16, 2021

Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair
Jason Phang, Angelica Chen, William Huang, Samuel R. Bowman
Adversarial Text Evaluation Datasets Evaluation Set Challenging Dataset Adversarial Datasets Adversary Model

Adversarial Datasets

Papers

ROSAR: An Adversarial Re-Training Framework for Robust Side-Scan Sonar Object Detection

Is your benchmark truly adversarial? AdvScore: Evaluating Human-Grounded Adversarialness

Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization

Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets

Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Dataset

Zero-Shot Fact-Checking with Semantic Triples and Knowledge Graphs

AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications

Measuring Adversarial Datasets

PRAT: PRofiling Adversarial aTtacks

Curating Naturally Adversarial Datasets for Learning-Enabled Medical Cyber-Physical Systems

X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection

Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning

A Proposal to Study "Is High Quality Data All We Need?"

Overlooked Implications of the Reconstruction Loss for VAE Disentanglement

Using calibrator to improve robustness in Machine Reading Comprehension

Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair