Reward Hacking

Ask about this topic

Overview

Reward hacking occurs when a system optimizes a proxy—such as a reward model, judge, rubric, or benchmark verifier—without achieving the intended goal. Research studies this in settings including reinforcement learning from human feedback, verifiable-reward language-model training, tool-using agents, and image or robot generation. Common approaches include building controlled tests and adversarial audits, hardening verifiers, using uncertainty-aware or multiple reward signals, and monitoring behavior through outputs or internal activations. A central challenge is that optimization can expose new weaknesses or adapt around defenses, so high scores and low detector readings do not necessarily establish genuine task success or safety.

Latest papers

CardsList

182 papers43 papers in the last four weeks, up 105% on the four weeks before. 0.2% of all new papers.

  1. Audit Where Hacks Pay: Budgeted Strong-Verifier Audits Against Reward Hacking in GRPO

    Oct 26, 2026Shurun Li, Michael Hua Wang, Haibo Zeng +1Reward HackingReinforcement Learning with Verifiable Rewards

    REASONS Lab, Institute for Network Sciences and Cyberspace, Tsinghua University · Virginia Tech

  2. Eat Your Precision: Precision Objective Mitigates Reward Hacking in Rubric-based RL

    Oct 24, 2026Hyun Ryu, Jong-Heon Kim, Yiqing Xie +3Rubric-Based RLReward Design for RL

    KAIST AI · Carnegie Mellon University Language Technologies Institute

  3. Mitigating Reward Hacking in Realistic Scenarios Through Representation Engineering

    Oct 21, 2026Austin Meek, Kyle Cox, Matthew Nguyen +2Reward HackingActivation Steering

    University of Delaware · Independent

  4. SimHack: Predicting the onset of detectable reward hacking before RL training

    Oct 20, 2026Junwei Deng, Alex Bie, Natalia PonomarevaReward HackingRL for Language Models

    University of Illinois Urbana-Champaign · Google Research

  5. Reward Misspecification Degrades Chain-of-Thought Monitorability

    Oct 20, 2026Tianyi Lorena Yan, John HewittLLM Safety MonitoringCoT Monitoring

    Columbia University

  6. Intra-Rubric Reward Divergence: Mitigating Rubric Hacking through Latent Dimension Recovery and Balancing

    Oct 19, 2026Jianqiang Xia, Han Xu, Dianxi ShiReward HackingRubric-Based RL

    Shanghai Jiao Tong University · Baidu, Inc. · Intelligent Game and Decision Lab

  7. Mitigating Reward Hacking in Preference Reward Models with Evolving Penalties

    Oct 19, 2026Afonso Baldo, António Farinhas, Marcos Treviso +2Reward HackingRL for Language Models

    Sword Health · Instituto Superior Técnico, Universidade de Lisboa · Gandara AI +2

  8. LeakBench: Auditing Reward Hacking through Prediction-Time Leakage in Clinical Modeling Agents

    Oct 19, 2026Xiaoyuan Bao, Jingsi Jiang, Jiao Shan +4Data LeakageAI Agent Auditing

    Medical Informatics Center, Institute of Advanced Clinical Medicine, Peking University, Beijing, China · Department of Ultrasound, Peking University Third Hospital, Beijing, China · Department of Hospital-Acquired Infection Control, Beijing Jishuitan Hospital, Beijing, China +4

  9. When Can You Trust an Agent's Own Verifier? Stress-Testing Counterfactual Rationale Verification Under Search Pressure and Oracle Access

    Oct 16, 2026Jonghyun Lee, Moonsu Kim, Sunghwan ParkLLM Agent VerificationAutomated Agent Evaluation

    Yonsei University

  10. Specification Gaming and a Constrained Reinforcement Learning Mitigation in a Cleaning Gridworld

    Oct 15, 2026Hilda Adwubi OseiReward HackingReward Design for RL

    Department of Industrial Engineering, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana

  11. Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense

    Oct 15, 2026Yushu Li, Jiin Woo, Xiaoxiao Li +1Reward HackingRL for Language Models

    University of British Columbia · Amazon Annapurna Labs

  12. Recursive Reward Improvement via Adversarial Meta-Prompt Evolution

    Oct 15, 2026Wei Dai, Qianchu Liu, Guanghui Qin +4Recursive Self-ImprovementRubric-Based RL

    Massachusetts Institute of Technology · Microsoft Research · Johns Hopkins University

  13. Rent-Seeking Language Models: Reward Models Set the Prices, Frequent Audits Curb the Rent

    Oct 14, 2026Zixiao Wang, Xingbo Du, Duzhen Zhang +1Reward Over-OptimizationRL for Language Models

    Mohamed bin Zayed University of Artificial Intelligence

  14. Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following

    Oct 14, 2026Sangyeon Yoon, Wooseok Seo, Hamish Ivison +5Reward HackingInstruction Following

    Yonsei University · FuriosaAI · University of Washington

  15. Copying Is Not Grounding: Training Faithful Long-Form RAG with Authored-Sentence Credit

    Oct 13, 2026Hoang Pham, Duc Anh Vu, Duy Tung Doan +2RL for RAGRAG Citation Faithfulness

    Nanyang Technological University · Viettel AI, Viettel Group · VinUniversity

  16. Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR

    Oct 13, 2026Savyasachi Gupta, Aniruddha Srinivas JoshiReinforcement Learning with Verifiable RewardsRL for Language Model Reasoning

    Independent Researcher TX, USA · Independent Researcher CA, USA

  17. Gradient Routing Localizes and Removes Unintended Behaviors in RL

    Oct 12, 2026Jake Ward, Shawn Hu, Aria Wong +4Reward HackingRL Post-Training

    MATS · Stanford · FAR AI +1

  18. Self-Verifier Capture in Unified Visual Generators: Diagnosing and Repairing Closed-Loop Self-Reward

    Oct 12, 2026Binyan Chen, Tong PanReward HackingGenerative Modeling

    University of Science and Technology of China

  19. Scalable Weak-to-Strong Verifier Hardening for Agent Environments

    Oct 11, 2026Chong Zeng, Ziqian Zhong, Kexun Zhang +5AI Agent SecurityReward Hacking

    Stanford University · Carnegie Mellon University · Fewshot Corp +1

  20. Who Sets the Pace of Judge Hacking? Judge Capability versus the Policy-Judge Gap

    Oct 11, 2026Haosen WangReward HackingLLM-Guided RL

  21. Reward Hacking as a Transmissible Strategy in Multi-Agent Coding Systems

    Oct 9, 2026Yuepeng Hu, Pegah Ghahremani, Reza Aghajani +2Reward HackingMulti-Agent Systems

    Meta Superintelligence Labs · Duke University

  22. How to post-train on a surrogate: Envelope sampling mitigates reward hacking

    Oct 8, 2026Sanjit Dandapanthula, Shuvom Sadhuka, Samir Khan +3RL Post-TrainingReward Hacking

    Carnegie Mellon University Department of Statistics and Machine Learning Department · Abridge AI · MIT EECS +2

  23. RH-Detect: A Unified Benchmark for Reward Hacking Detection

    Oct 7, 2026Junwei Quan, Evgenii Opryshko, Rohan Subramani +1Reward HackingLLM Evaluation

    University of Toronto · Vector Institute · Trajectory Labs +1

  24. TestJack: Should you trust the results in coding benchmarks? Agentic Coding Benchmarks Auditing via Evaluator Evolution

    Oct 7, 2026Shuangjie Yao, Hao Wang, Koushik Sen +3LLM Agent EvaluationAI Agent Benchmarks

    Columbia University · University of California, Berkeley · George Mason University

  25. SpecGuard: Proving a Task Is Broken Before the Agent Cheats

    Oct 6, 2026Param Biyani, Krishnamurthy DvijothamAI Agent SafetySoftware Engineering Agents

    MATS · Google DeepMind

  26. Better Call Reward: Reward Hacking as Strategic Abstention in Legal Reasoning Models

    Oct 5, 2026Subramanyam Sahoo, Justin ShenkReward HackingLLM Auditing

    Horizon Research.

  27. RubricArmor: Adversarial Evolution Improves LLM-Based Rubric Generation

    Oct 4, 2026Haocheng Yang, Yuchao Zhang, Licheng Pan +8Reward HackingRubric-Based RL

    National University of Singapore · Beijing University of Chemical Technology · Zhejiang University +7

  28. Pressure, Context, and Machine Self-Control: A Criminological Test of Reward Hacking in Generative AI Models

    Oct 3, 2026Murat Ozer, Isaac Kofi NtiReward HackingAI Agent Safety

    School of Information Technology, University of Cincinnati, Cincinnati, OH, USA

  29. Can LLM Agents Automate Reinforcement Learning for Text-to-Speech?

    Oct 3, 2026Xuanjun Chen, Zixiong Su, Hao Shi +4TTS SynthesisAgentic RL

    National Taiwan University · Shanda Group · National Institute of Informatics +1

  30. Reinforcement Learning on the Discrete Composition Channel of a Crystal Generator: Validated Gains and Reward Hacking

    Oct 2, 2026Pawan Prakash, Philipp Höllmer, Addis Fuhr +4Crystal Structure GenerationReward Hacking

    University of Florida · New York University · Oak Ridge National Laboratory