cs.LGNov 12, 2026

Bait: Mitigating Harmful Agentic Reinforcement Learning via Probing Reward Function

Authors: Tiansheng Huang, Ananth Balashankar, Virat Shejwalkar, Santosh Pande, Ling Liu

Organizations: Georgia Institute of Technology · Google DeepMind

Abstract

We expose a safety vulnerability on nascent agentic reinforcement-learning-as-a-service. Specifically, with a cost as low as $20, we show that safety alignment of a large language model (e.g., Inkling) can be completely subverted through rounds of harmful agentic reinforcement learning (agentic RL) with a toxic environment from its users. We also demonstrate that upon a successful harmful agentic RL attack via the service API, the attackers can potentially use the safety-degraded model to assist with real-world harmful activities, e.g., exploit software vulnerabilities and launch code re-use attacks. To mitigate harmful agentic RL, we propose Bait, a defense that mitigates the attack via proactively probing the reward function. Experimental results show that such a simple defense can sufficiently mitigate learning from a harmful reward function and can still learn from a benign reward function that promotes benign agentic tasks. Further analysis on both empirical and theoretical aspects shows that an adaptive attacker who assumes the knowledge of Bait defense cannot perfectly subvert it due to a systematic challenge. Source code is available at https://github.com/git-disl/Bait.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. CLEANER: Self-Purified Trajectories Boost Agentic Reinforcement Learning

    Jan 21, 2026Tianshi Xu, Yuteng Chen, Meng LiAgentic RLReinforcement Learning

  2. Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds

    Jun 13, 2026Ömer Veysel Çağatan, Xuandong ZhaoReward HackingLanguage Model Safety Evaluation

  3. Learning Agentic Policy from Action Guidance

    May 12, 2026Yuxiang Ji, Zengbin Wang, Yong Wang +6Agentic RLReinforcement Learning