Reward Poisoning Attack

Reward poisoning attacks manipulate the reward signals in reinforcement learning (RL) systems to induce undesirable agent behavior. Current research focuses on developing effective attack strategies across various RL settings, including online and offline learning, single-agent and multi-agent scenarios, and different model architectures like deep neural networks and multi-armed bandits. These attacks highlight vulnerabilities in RL algorithms and their applications, particularly in safety-critical systems and those involving human feedback, such as large language models. Understanding and mitigating these attacks is crucial for ensuring the robustness and reliability of RL systems in real-world deployments.

Papers

October 25, 2024

Robust Thompson Sampling Algorithms Against Reward Poisoning Attacks
Yinglun Xu, Zhiwei Wang, Gagandeep Singh
Optimal Regret Linear Bandit Thompson Sampling Reward Poisoning Attack

February 21, 2024

Stealthy Adversarial Attacks on Stochastic Multi-Armed Bandits
Zhiwei Wang, Huazheng Wang, Hongning Wang
Adversarial Attack Multi Armed Bandit Stochastic Way Stealthy Attack Reward Poisoning Attack

February 15, 2024

Universal Black-Box Reward Poisoning Attack against Offline Reinforcement Learning
Yinglun Xu, Rohan Gumaste, Gagandeep Singh
Contrastive Learning Offline Reinforcement Learning Offline RL Algorithm Reward Poisoning Attack

November 16, 2023

RLHFPoison: Reward Poisoning Attack for Reinforcement Learning with Human Feedback in Large Language Models
Jiongxiao Wang, Junlin Wu, Muhao Chen, Yevgeniy Vorobeychik, Chaowei Xiao
Reinforcement Learning Human Feedback Adversarial Learning Poisoning Attack RLHF V Reward Poisoning Attack Preference Poisoning

October 8, 2023

Adversarial Attacks on Combinatorial Multi-Armed Bandits
Rishab Balasubramanian, Jiawei Li, Prasad Tadepalli, Huazheng Wang, Qingyun Wu, Haoyu Zhao
Adversarial Attack Multi Armed Bandit Combinatorial Approach Combinatorial Multi Armed Bandit Bandit Instance Reward Poisoning Attack

May 18, 2023

Black-Box Targeted Reward Poisoning Attack Against Online Deep Reinforcement Learning
Yinglun Xu, Gagandeep Singh
Unknown Dynamic Attack Efficacy Multi Step Attack Reward Poisoning Attack

June 4, 2022

Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learning
Young Wu, Jeremy McMahan, Xiaojin Zhu, Qiaomin Xie
Multi Agent Nash Equilibrium Offline Multi Agent Reinforcement Learning Single Agent RL Reward Poisoning Attack

May 30, 2022

Efficient Reward Poisoning Attacks on Online Deep Reinforcement Learning
Yinglun Xu, Qi Zeng, Gagandeep Singh
DQN Agent Adversarial Markov Decision Process Reward Poisoning Attack

January 6, 2022

Admissible Policy Teaching through Reward Design
Kiarash Banihashem, Adish Singla, Jiarui Gan, Goran Radanovic
Reward Function Based Policy Reward Engineering Reward Design Reward Poisoning Attack