RL for Code Generation

RL: Reinforcement Learning

This field trains language models to write, repair, and optimize programs by using execution environments, unit tests, compilers, formal verifiers, or renderers as feedback. Common approaches include reinforcement learning with verifiable rewards, often using GRPO or PPO, alongside synthetic tasks, curricula, and multi-turn tool-using agents. Newer methods add dense or token-level credit, reward models, self-correction, and search, with evaluation spanning HumanEval, MBPP, APPS, LiveCodeBench, SWE-bench, hardware kernels, and visual-to-code tasks. A central challenge is making feedback reliable and informative without reward hacking or excessive rollout costs, while ensuring gains transfer beyond the training tasks and execution harnesses.

Latest papers

150 papers24 papers in the last four weeks, up 26% on the four weeks before. 0.1% of all new papers.

Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable Rewards
Oct 27, 2026Bo Yang, Zishuai Zhang, Weihua Li +3RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University
RIPPLE: Recovery Internalization through Process-Level Policy Learning and Effect-Aware Credit for Long-Horizon Agentic Coding
Oct 22, 2026Jingxuan Xu, Zihang Xu, Yuechen Luo +9Code Generation RL for LLM Agents
National University of Singapore, Singapore · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Tsinghua University, Beijing, China +1
SimHack: Predicting the onset of detectable reward hacking before RL training
Oct 20, 2026Junwei Deng, Alex Bie, Natalia PonomarevaReward Hacking RL for Language Models
University of Illinois Urbana-Champaign · Google Research
Co-Evolving Policy and Reward for Code Generation: A Mechanistic Study of Single-Adapter Drift
Oct 19, 2026Amarsaikhan Batjargal, Noriko TomuroRL for Code Generation Reinforcement Learning
Jarvis College of Computing and Digital Media, DePaul University, Chicago, IL, USA
Over-Defense in Coding Agents Begins at Silent Forks
Oct 18, 2026Hanjun Luo, Xuecheng Liu, Xiucheng Zhang +5AI Coding Agents RL for Code Generation
New York University · New York University Abu Dhabi · Carnegie Mellon University +3
Peer Pressure: Execution-Verified Co-Evolution of Attacker and Defender Populations for Code Security
Oct 17, 2026Khushbu Pahwa, Yubin Ge, Daniele Bonadiman +2AI Agent Security Adversarial Training
AWS AI
Self-Evolving Coding Agents via Harness-Policy Optimization
Oct 16, 2026Zhepei Wei, Wenlin Yao, Baolin Peng +5AI Coding Agents Agent Harness Optimization
Microsoft Research · University of Virginia · Carnegie Mellon University
Verifier-Grounded Agentic RL for Provably Correct Code
Oct 13, 2026George Ma, Yuhao Zhang, Bing He +5RL for Code Generation Verified Code Generation
UC Berkeley · AWS AI Labs
AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards
Oct 13, 2026Xin Gao, Yifan Yang, Xin Li +13Code Generation RL for Code Generation
Yale University · Microsoft · Nanyang Technological University +3
PreciseCoder: Faithful and Generalizable Debugging Beyond Unit-Test Rewards
Oct 13, 2026Yunkai Zhan, Miaosen Chai, Shangshang Wang +5Automated Program Repair RL Post-Training
University of Southern California · University of Chicago · Google +1
CodeLift: Your Test Suite Is Secretly a Dense SWE Curriculum
Oct 12, 2026Junjielong Xu, Huachao Zhu, Alyssa Wu +2RL for Code Generation Software Engineering Agents
School of Data Science, The Chinese University of Hong Kong, Shenzhen · Tencent
ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs
Oct 11, 2026Yi Ge, Kangrui Ruan, Jifan Chen +6Code World Models Software Engineering Agents
University of Maryland, College Park · AWS AI Labs
Learning after Execution with Action-Specific Credit
Oct 10, 2026Yufeng Tian, Jiahao Tang, Minghao WuReinforcement Learning Credit Assignment
The Chinese University of Hong Kong, Shenzhen, China · Central South University, Changsha, China
SPROUT: Scaling Agentic Reinforcement Learning for Long-Horizon Software Engineering
Oct 9, 2026Mengyang Liu, Weiming Wang, Haixin Nan +6Agentic RL RL for Code Generation
Shanghai Jiao Tong University · Amazon · Nanjing University +2
Reward Hacking as a Transmissible Strategy in Multi-Agent Coding Systems
Oct 9, 2026Yuepeng Hu, Pegah Ghahremani, Reza Aghajani +2Reward Hacking Multi-Agent Systems
Meta Superintelligence Labs · Duke University
BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation
Oct 7, 2026Yingxiang Yang, Weihang Xiao, Zhunxuan Wang +2RL for Language Model Reasoning Group Relative Policy Optimization
Amazon AGI · Virginia Tech
AGAR: a reinforcement learning substrate for LLM program evolution
Oct 6, 2026Haoran Li, Zengle Ge, Xiaomin Yuan +11LLM-Based Program Synthesis RL for Code Generation
University of the Chinese Academy of Sciences · Baidu · Tsinghua University +2
FC-SWE: Failure-Conditioned RL for Long-Horizon Software Engineering Agents
Oct 6, 2026Jia Liufu, Bin Hu, Linglin Jing +5RL for Code Generation Software Engineering Agents
NVIDIA · Tsinghua University
CodeForge-MA: Execution-Verified Multi-Agent Learning with Language-Conditioned LoRA for Multilingual Code Generation
Oct 4, 2026Zhizhou Gu, Xianting Wu, Siyu Gu +2RL for Code Generation Synthetic Data Generation for Language Models
Northeastern University Bellevue, USA · Columbia University New York, USA · University of Pennsylvania Philadelphia, USA +2
Pareto-Dominant Clarification: Post-Training Coding LLMs via PPO-Lagrangian Budget Constraints
Oct 2, 2026Abhinav Rajput, Acey VogelsteinLLM Post-Training RL for Code Generation
Center for Data Science New York University New York, NY, USA
CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning
Oct 1, 2026Yafei Zhang, Songshuo Lu, Sicong Liao +2RL for Code Generation RL for Language Models
Moore Threads AI
Cross-Benchmark Transfer from RL on Agentic Coding Tasks
Oct 1, 2026Sushant Mehta, Logan Ritchie, Edwin ChenRL for Code Generation AI Agent Benchmarks
Surge AI
CATCH: A Controllable Analysis Testbed for Reward Hacking in Coding RL
Sep 30, 2026Shouli Wang, Yanfeng Jia, Zhihao Ou +6RL Benchmarks RL for Code Generation
Tsinghua University · Beihang University · Southern University of Science and Technology +2
RLTLDR: Self-improvement by Internalizing Self-generated Feedback
Sep 29, 2026Michael Kirchhof, Eleonora Gualdoni, Andrew Szot +6RL for Code Generation Reinforcement Learning with Verifiable Rewards
Apple
Reinforcement Learning from Intermediate Renders for Image-to-Code Generation
Sep 28, 2026Omri Kaduri, Kate Feingold, Phillip Isola +1RL for Code Generation Image-to-Code Generation
Weizmann Institute of Science · MIT
Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents
Sep 27, 2026Yuanhao Li, Hongbo Wang, Xuhong Chen +2RL for Code Generation Software Engineering Agents
State Key Laboratory of Networking and Switching Technology Beijing University of Posts and Telecommunications, Beijing 100876, China · University of Luxembourg, Luxembourg
CodeSkill: Latent Skill Abstraction for Long-Horizon Code Agents
Sep 27, 2026Song-Li Wu, Jingyi Wang, Zhaocheng Du +2LLM Agent Skill Learning Long-Horizon LLM Agents
Tsinghua University · Huawei Noah’s Ark Lab · Shanghai Jiao Tong University
KernelZero: Co-Evolving Proposer and Coder for Continuously Improved GPU Kernel Generation
Sep 27, 2026Changxin Ke, Rui Zhang, Zixiang Fang +8GPU Kernel Generation LLM-Based Program Synthesis
State Key Lab of Processors, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Intelligent Software Research Center, Institute of Software, CAS +1
Groupwise Agentic Grading and Advantage Redistribution for Code Agent RL
Sep 26, 2026Jinhao Dong, Liang Zhao, Zihao Yue +7RL for Code Generation LLM Agent Training
LLM Core, Xiaomi · Renmin University of China · Peking University +1
CompassPlay: Rewarding the Proposer for Where It Moves the Solver
Sep 26, 2026Sophia Xiao Pu, Ximeng Sun, Jiang Liu +4Self-Play RL Reward Design for RL
University of California, Santa Barbara · Advanced Micro Devices

Oct 27, 2026J/K move · Enter open · S save

Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable Rewards

Bo Yang, Zishuai Zhang, Weihua Li, Qinnan Zhang+2

Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University

RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards LLM Reasoning RL Exploration RL for Code Generation

Open paper AskPDF arXiv

Reinforcement learning with verifiable rewards (RLVR) has shown strong empirical gains in improving the reasoning ability of large language models (LLMs) on tasks such as mathematics and programming. However, recent studies have shown that RLVR rollouts often collapse onto a narrow set of reasoning trajectories, limiting the explored reasoning space. Motivated by findings that thinking tokens at critical steps serve as critical structural elements in LLM reasoning pathways, we propose Guided-RLVR, a novel approach that augments RLVR with a lightweight controller. The controller strategically injects semantically meaningful steering tokens into the sampling distribution, encouraging the model to approach problems from multiple reasoning perspectives and thereby expanding the exploration space. Experiments on mathematical reasoning and code generation benchmarks show that Guided-RLVR consistently outperforms existing RLVR methods while producing more diverse reasoning trajectories.