Adversarial Challenge

Adversarial challenges in machine learning focus on designing and mitigating attacks that exploit vulnerabilities in models, leading to incorrect or harmful outputs. Current research emphasizes improving model robustness against these attacks across various domains, including image generation (using diffusion models and graph neural networks), object detection, and question answering systems. These efforts are crucial for ensuring the reliability and safety of AI systems in high-stakes applications like healthcare and autonomous driving, and for advancing our understanding of model limitations and biases. The development of robust defenses and novel attack strategies is driving significant progress in both theoretical understanding and practical application of machine learning.

Papers

May 25, 2024

R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Model
Changhoon Kim, Kyle Min, Yezhou Yang
Text to Image Diffusion Model Text to Image Adversarial Framework Textual Adversarial Attack Adversarial Challenge

March 21, 2024

Adversary-Robust Graph-Based Learning of WSIs
Saba Heidari Gheshlaghi, Milan Aryal, Nasim Yahyasoltani, Masoud Ganji
Adversarial Attack Native Robustness Whole Slide Image Graph Based Learning Adversarial Challenge

March 4, 2024

Improving the Robustness of Object Detection and Classification AI models against Adversarial Patch Attacks
Roie Kazoom, Raz Birman, Ofer Hadar
Adversarial Attack Native Robustness Artificial Intelligence Model Adversarial Patch Attack Robust Defense Adversarial Challenge

February 14, 2024

Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation
Jessica Quaye, Alicia Parrish, Oana Inel, Charvi Rastogi, Hannah Rose Kirk, Minsuk Kahng, Erin van Liemt, Max Bartolo, Jess Tsang, Justin White, Nathan Clement, Rafael Mosquera, Juan Ciro, Vijay Janapa Reddi, Lora Aroyo
Text to Image Generation Generative AI Model Adversarial Prompt Red Teaming Potential Harm Unsafe Image Imperceptible Attack Adversarial Challenge

February 2, 2024

SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding
Chanho Park, Namyoon Lee
Adversarial Attack Convergence Rate Gradient Information Adversarial Scenario Adversarial Agent Adversarial Challenge Federated Data Sanitization Defense SignSGD MV

January 30, 2024

Improving QA Model Performance with Cartographic Inoculation
Allen Chen, Okan Tanrikulu
Adversarial Example QA Datasets Context Reasoning Adversarial Challenge QA Model

December 26, 2023

From Text to Multimodal: A Comprehensive Survey of Adversarial Example Generation in Question Answering Systems
Gulsum Yigit, Mehmet Fatih Amasyali
Adversarial Example Text Modality Comprehensive Survey Multimodal Phenomenon Question Answering System Adversarial Question Adversarial Challenge

November 27, 2023

Adversarial Doodles: Interpretable and Human-drawable Attacks Provide Describable Insights
Ryoya Nara, Yusuke Matsui
Adversarial Attack Adversarial Example DCU Insight AQ Imperceptible Attack Adversarial Challenge

September 20, 2023

Distilling Adversarial Prompts from Safety Benchmarks: Report for the Adversarial Nibbler Challenge
Manuel Brack, Patrick Schramowski, Kristian Kersting
Adversarial Prompt Adversarial Input Unsafe Content Text Conditioned Image Generation Safety Benchmark Adversarial Challenge

May 22, 2023

Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models
Alicia Parrish, Hannah Rose Kirk, Jessica Quaye, Charvi Rastogi, Max Bartolo, Oana Inel, Juan Ciro, Rafael Mosquera, Addison Howard, Will Cukierski, D. Sculley, Vijay Janapa Reddi, Lora Aroyo
Adversarial Attack Text to Image Model Human SAFETY Generative AI Model Adversarial Challenge

Adversarial Challenge

Papers

R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Model

Adversary-Robust Graph-Based Learning of WSIs

Improving the Robustness of Object Detection and Classification AI models against Adversarial Patch Attacks

Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation

SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding

Improving QA Model Performance with Cartographic Inoculation

From Text to Multimodal: A Comprehensive Survey of Adversarial Example Generation in Question Answering Systems

Adversarial Doodles: Interpretable and Human-drawable Attacks Provide Describable Insights

Distilling Adversarial Prompts from Safety Benchmarks: Report for the Adversarial Nibbler Challenge

Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models