Reinforcement Learning from Verifiable Rewards (RLVR) is pivotal for enhancing LLM code generation, yet its efficacy is often hindered by insufficient test case coverage, leading to reward hacking and policy degradation. To address this, we propose RobustTests, a framework featuring a faulty-code-driven test case synthesis strategy. By leveraging "near-correct" faulty codes, RobustTests captures latent logical discrepancies and employs validator agents with behavioral feature clustering to filter invalid or redundant test cases. Additionally, a stepwise dense reward function based on pass rates is introduced to mitigate false negatives and enhance training robustness. Using this pipeline, we construct an augmented version of the CodeContests+ dataset with superior diagnostic utility. Experimental results show that RL fine-tuning of Qwen3-32B via RobustTests achieves a 3% absolute gain on LiveCodeBench, demonstrating its effectiveness in advancing LLM code generation proficiency. Codes and data are available at https://huggingface.co/datasets/sid6/RobustTests.
Figures & tables
Figure 1: Illustration of current methods for test cases synthesis by LLM.(a) Generate Test cases Directly , instructing the LLM to directly synthesize a suite of test cases compliant with the problem specifications.(b) Generate Test cases by Generator Program , where LLM is first instructed to generate generator programs designed to automate the synthesis of test case inputs.
Figure 2: Overview of the proposed RobustTests framework. The framework consists of three core components: (i) Automated Test case Generation in 3.1 , aims to generate high-quality test cases inputs capable of effectively distinguishing correct code from faulty codes; (ii) Test case Validation and Selection in 3.2 , ensures that the generated test cases are semantically sound and capable of diagnosing a broad spectrum of faulty code while simultaneously maximizing set parsimony; and (iii) Dense Reward Function Design in 3.3 , accounts for the potential fallibility of the synthesized test case suite.
Method
Livecodebench
Codeforces
Score
Score
Rating
Percentile
Naive LLM Generation
65.75
35.41
83.36
91.70
HardTests
65.25
35.65
83.58
91.35
CodeContests +
65.41
35.56
83.96
91.45
CodeContests-O
66.21
36.43
84.35
91.81
RobustTests(Ours)
68.39
38.50
85.99
94.67
Table 1: Performance of baselines and RobustTests on the LiveCodeBench and CodeForces benchmarks. The best result is bold and the second result is underline.
Figure 3: Correctness distribution of Qwen3-32B over ten trials across all CodeContests + problems, evaluated using the CodeContests + test case suite. The x-axis represents the distribution of pass rates over ten trials(pass@10), while the y-axis denotes the number of problems within each pass rate interval.
Figure 4: Relationship between TSP values and LiveCodeBench performance across different test cases.
Method
Livecodebench
Codeforces
Score
Score
Rating
Percentile
RobustTests*(Ours)
67.84
38.47
85.16
93.97
w/o Test case Synthesis
66.91
36.41
84.4
92.12
w/o Diversity-Driven Selection
66.82
36.23
84.12
91.96
w/o Both Module
65.41
35.56
83.96
91.45
Table 2: Ablation study on the test case synthesis strategy of RobustTests* on LiveCodeBench and CodeForces benchmarks. RobustTests* denotes a variant of the RobustTests framework that excludes the dense reward mechanism. We investigate the performance impact of without Test case Generation (the second row), without Diversity-Driven Selection (the third row) and without Both Module(the forth row). The best results are in bold.
Figure 5: The pass rate distribution of Qwen3-32B over ten trials across all CodeConteststrain+ problems, evaluated against the test cases provided by RobustTests and without Test case Generation and its configuration without Diversity-Driven Selection.
Figure 6: Correlation between the TSP values of test cases employed during training and the coding performance of LLM on the LiveCodeBench benchmark.
Dataset
Method
LiveCodeBench
Codeforces
Score
Score
Rating
Percentile
CodeConteststrain+
Sparse Reward
65.41
83.96
35.56
91.45
Dense Reward
66.30
84.24
37.74
92.05
RobustTests*
Sparse Reward
67.84
85.16
38.47
93.97
Dense Reward
68.39
85.99
38.50
94.67
RobustTests* w/o validator
Sparse Reward
66.02
83.82
35.91
91.28
Table 3: Ablation study on the Reward Module of RobustTests on LiveCodeBench and CodeForces benchmarks, using both RobustTests* and CodeContests + as the training datasets. The best results are in bold.
Figure 7: Ablation study on the reward scale in the dense reward function on LiveCodeBench and Codeforces benchmarks. The scale is varied from 0 to 0.20 using RobustTests ∗ as the training dataset.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Value
Optimizer
AdamW
Learning Rate ( η )
5×10−6
Adam β1,β2
0.9,0.95
Weight Decay
0.01
Gradient Clipping
1.0
Global Batch Size
256
Appendix
Table 4: Hyperparameters for Experiment
Figure 8: One of prompts on Faulty Code Generation Stage
Figure 9: One of prompts on Faulty Code Generation Stage
Figure 10: One of prompts on Faulty Code Generation Stage
Figure 11: Prompt on Faulty-Code-Driven Test Case Generation Stage
Figure 12: Prompt on Input Validator Generation Stage
Reinforcement learning for code generation trains models to write, repair, and optimize programs using execution-based and other task-specific feedback.
Reinforcement learning with verifiable rewards (RLVR) is a promising approach for improving code generation in large language models, but its effectiveness is limited by weak and static verification signals in existing coding RL datasets. In this paper, we propose a solution-conditioned and adversarial verification framework that iteratively refines test cases based on the execution behaviors of candidate solutions, with the goal of increasing difficulty, improving discriminative power, and reducing redundancy. Based on this framework, we introduce EvolveCoder-22k, a large-scale coding reinforcement learning dataset constructed through multiple rounds of adversarial test case evolution. Empirical analysis shows that iterative refinement substantially strengthens verification, with pass@1 decreasing from 43.80 to 31.22. Reinforcement learning on EvolveCoder-22k yields stable optimization and consistent performance gains, improving Qwen3-4B by an average of 4.2 points across four downstream benchmarks and outperforming strong 4B-scale baselines. Our results highlight the importance of adversarial, solution-conditioned verification for effective and scalable reinforcement learning in code generation.
Chi Ruan, Dongfu Jiang, Huaye Zeng +2
University of Waterloo · Vector Institute · Harvard University
Reinforcement learning with verifiable rewards (RLVR) trains language models using programmatically checkable signals such as unit-test outcomes, enabling direct optimization for functional correctness in code generation. We conduct an empirical study of RLVR for Python code generation on the MBPP benchmark using two small models (Qwen3-0.6B and Llama3.2-1B) with LoRA fine-tuning. Across multiple reward formulations such as: unit-test-only rewards, static-analysis-only shaping via the Ruff linter, and a combined reward, we compare group-based policy optimization variants (GRPO and GSPO) and evaluate both functional correctness and behavioral diagnostics. In our experimental setting, RLVR improves pass@1 on MBPP test by up to 13 percentage points under proposed combined reward configuration. However, we find that reward shaping can induce systematic behavioral shifts: using only static-analysis penalties may bias the policy toward shorter completions that reduce lint errors without reliably improving functional correctness. In contrast, combined rewards mitigate this degeneration and yield more stable trade-offs between correctness and style constraints. Overall, our results highlight that RLVR effectiveness for code generation is highly sensitive to reward design and optimization granularity, and that diagnostics beyond pass@1, including generation length, Ruff severity profiles, and execution error types are useful for identifying failure modes.
Large language models (LLMs) have demonstrated remarkable performance across various code-related tasks. However, unlike carefully curated datasets that are typically high-quality and error-free, real-world user instructions are often vague and error-prone, posing significant challenges to the robustness of code LLMs. Furthermore, robustness-oriented fine-tuning relies on paired clean-noisy samples, which are costly to curate and require sophisticated noisy simulation techniques. To address these challenges, we propose the Noisy Test-time Reinforcement Learning framework (NTRL-Code), which enables robust self-evolution of code LLMs using only unlabeled noisy data during the testing stage. Specifically, NTRL-Code uses conservative self-denoising to obtain a cleaner semantic anchor for target estimation, and employs an abstract-syntax-tree (AST)-based structural aggregation mechanism to estimate a proxy target from multiple candidate programs. The policy is then optimized on the original noisy prompts with a hybrid reward that combines format validity, code similarity, and anti-repetition signals. Extensive experiments on three benchmarks, each incorporating character-level, word-level, and paragraph-level perturbations, demonstrate that NTRL-Code yields robust and consistent improvements, stabilizing the predictions of various base models. Our code is available at https://github.com/Xikai97/NTRL-Code.
Xikai Yang, Hieu Trung Nguyen, Dunyuan Xu +4
The Chinese University of Hong Kong · Huazhong University of Science and Technology