Adversarial Agent

Adversarial agents represent a significant challenge in various machine learning applications, focusing on designing robust systems that can withstand malicious or unpredictable influences. Current research emphasizes developing algorithms and models, such as robust policy gradient methods and Byzantine-resilient gradient aggregation schemes, to mitigate the impact of adversarial behavior in federated learning, reinforcement learning, and other distributed systems. This work is crucial for enhancing the reliability and safety of AI systems deployed in real-world scenarios, particularly in high-stakes domains like autonomous vehicles and healthcare, where adversarial attacks could have severe consequences. The development of effective defenses against adversarial agents is a key area of ongoing research, with a focus on both theoretical guarantees and practical implementations.

Papers

November 5, 2024

Formal Logic-guided Robust Federated Learning against Poisoning Attacks
Dung Thuy Nguyen, Ziyan An, Taylor T. Johnson, Meiyi Ma, Kevin Leach
Poisoning Attack Adversarial Behavior Adversarial Agent Robust Federated Learning Model Level Defense

October 21, 2024

Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservation
Anh Bui, Long Vuong, Khanh Doan, Trung Le, Paul Montague, Tamas Abraham, Dinh Phung
Diffusion Model Adversarial Text Stable Diffusion Model Adversarial Agent Erasure Method

September 30, 2024

Fine-Tuning Personalization in Federated Learning to Mitigate Adversarial Clients
Youssef Allouah, Abdellah El Mrini, Rachid Guerraoui, Nirupam Gupta, Rafael Pinot
Generalization Performance Adversarial Behavior Personalization Performance Adversarial Agent Image Classification Benchmark

September 21, 2024

Adversarial and Reactive Traffic Agents for Realistic Driving Simulation
Joshua Ransiek, Philipp Reis, Eric Sax
Autonomous Vehicle Adversarial Text Traffic Simulation Adversarial Agent Driving Planning Simulated Driving Traffic Agent

August 15, 2024

AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents
Guhong Chen, Liyang Fan, Zihan Gong, Nan Xie, Zixuan Li, Ziqiang Liu, Chengming Li, Qiang Qu, Shiwen Ni, Min Yang
Adversarial Example Agent Smith Adversarial Agent Agent Self Evolution LLM Driven Agent

June 16, 2024

Toward Optimal LLM Alignments Using Two-Player Games
Rui Zheng, Hongyi Guo, Zhihan Liu, Xiaoying Zhang, Yuanshun Yao, Xiaojun Xu, Zhaoran Wang, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang, Hang Li, Yang Liu
LLM Alignment Adversarial Agent Two Player Standard Reinforcement Learning Two Agent Iterative Interaction

May 22, 2024

FACT or Fiction: Can Truthful Mechanisms Eliminate Federated Free Riding?
Marco Bornstein, Amrit Singh Bedi, Abdirisak Mohamed, Furong Huang
Adversarial Agent Multi FAct Truthful Space Aggregation Function Truthful Incentive Mechanism Agent Failure Fictional Language

May 8, 2024

AirGapAgent: Protecting Privacy-Conscious Conversational Agents
Eugene Bagdasarian, Ren Yi, Sahra Ghalebikesabi, Peter Kairouz, Marco Gruteser, Sewoong Oh, Borja Balle, Daniel Ramage
Sensitive Data Filling Gap Adversarial Agent QuEry Based Attack Privacy Assistant

March 15, 2024

Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries
Swetha Ganesh, Jiayu Chen, Gugan Thoppe, Vaneet Aggarwal
Policy Gradient Federated Reinforcement Learning Adversary Agent Adversarial Agent Global Convergence Guarantee

February 16, 2024

Assessing biomedical knowledge robustness in large language models by query-efficient sampling attacks
R. Patrick Xian, Alex J. Lee, Satvik Lolla, Vincent Wang, Qiming Cui, Russell Ro, Reza Abbasi-Asl
Zero Shot Adversarial Example Adversarial Agent Adversarial Setting Adversarial Question Biomedical Question

February 2, 2024

SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding
Chanho Park, Namyoon Lee
Adversarial Attack Convergence Rate Gradient Information Adversarial Scenario Adversarial Agent Adversarial Challenge SignSGD MV Federated Data Sanitization Defense

January 7, 2024

Data-Driven Subsampling in the Presence of an Adversarial Actor
Abu Shafin Mohammad Mahdee Jameel, Ahmed P. Mohamed, Jinho Yi, Aly El Gamal, Akshay Malhotra
Adversarial Attack Adversarial Robustness Speech Presence Subsampling Method Adversarial Agent Modulation Classification Automatic Modulation Classification

December 6, 2023

Dr. Jekyll and Mr. Hyde: Two Faces of LLMs
Matteo Gioele Collu, Tom Janssen-Groesbeek, Stefanos Koffas, Mauro Conti, Stjepan Picek
Large Language Model Chatbot Response Human Face Adversarial Agent Conversational Assistant Context Impersonation

July 23, 2023

Adversarial Agents For Attacking Inaudible Voice Activated Devices
Forrest McKee, David Noever
Adversarial Agent Stealthy Attack Voice Activated Inaudible Voice

July 13, 2023

Effective Prompt Extraction from Language Models
Yiming Zhang, Nicholas Carlini, Daphne Ippolito
Large Language Model Language Model Adversarial Agent Text Attack Prompt Extraction

June 19, 2023

Learning Models of Adversarial Agent Behavior under Partial Observability
Sean Ye, Manisha Natarajan, Zixuan Wu, Rohan Paleja, Letian Chen, Matthew C. Gombolay
Adversarial Attack Adversarial Learning Partial Observability Adversarial Behavior Learning Model Adversarial Agent

May 9, 2023

An Algorithm For Adversary Aware Decentralized Networked MARL
Soumajyoti Sarkar
Multi Agent Reinforcement Learning Practical Algorithm Multi Agent Heterogeneous Agent Adversarial Agent

May 4, 2023

Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policy
Xiang Zheng, Xingjun Ma, Shengjie Wang, Xinyu Wang, Chao Shen, Cong Wang
Reinforcement Learning Robustness Evaluation Adversarial Agent Adversarial Policy State Adversarial

April 19, 2023

Fundamental Limitations of Alignment in Large Language Models
Yotam Wolf, Noam Wies, Oshri Avnery, Yoav Levine, Amnon Shashua
Large Language Model Language Model Alignment Problem Fundamental Limitation Adversarial Agent Alignment Approach

April 7, 2023

Learning Multi-Pursuit Evasion for Safe Targeted Navigation of Drones
Jiaping Xiao, Mir Feroskhan
Adversarial Attack Deep Reinforcement Learning Various Fast Moving Drone Adversarial Agent Safe Navigation Pursuit Evasion Pursuit Evasion Game