ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning
Authors: Juyong Jiang, Jiasi Shen, Sunghun Kim, Kang Min Yoo, Jeonghoon Kim, Sungju Kim
Organizations: The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology · Amazon AGI · NAVER Cloud
While Large Language Models (LLMs) have revolutionized code generation, standard ``System 1'' approaches that generate solutions in a single forward pass often hit a performance ceiling on complex algorithmic tasks. Existing iterative refinement strategies attempt to bridge this gap at inference time, yet they predominantly rely on external oracles, execution feedback, or computationally expensive prompt-response cycles. In this work, we propose ReflexiCoder, a novel reinforcement learning (RL) framework that internalizes the structured reasoning trajectory, encompassing initial generation, bug and optimization aware reflection, and self-correction, directly into the model's weights. Unlike prior methods, ReflexiCoder shifts the paradigm from external-dependent refinement to an intrinsic, fully autonomous self-reflection and self-correction capabilities at inference time. We utilize an RL-only training paradigm with granular reward functions to optimize the entire reflection-correction trajectory, teaching the model how to debug without reliance on ground-truth feedback or execution engines at inference time. Extensive experiments across seven benchmarks demonstrate that our ReflexiCoder-8B establishes a new state-of-the-art (SOTA) among leading open-source models in the 1.5B to 14B range, achieving 94.51% (87.20%) on HumanEval (Plus), 81.80% (78.57%) on MBPP (Plus), 35.00% on BigCodeBench, 52.21% on LiveCodeBench, and 37.34% on CodeForces in a single-attempt setting, rivaling or surpassing proprietary models like GPT-5.1. Notably, our framework is significantly more token-efficient than base models, reducing inference-time compute overhead by approximately 40% through disciplined, efficient reasoning and reflection patterns. The source code and data are available at https://github.com/juyongjiang/ReflexiCoder.
Figures & tables
Figure 1: A comparative overview of iterative code refinement workflows at inference time. (a) Existing code LLMs often struggle to generate correct solutions for complex programming tasks on a single attempt. (b) Prior practices mitigate this by relying on external feedback (e.g., compilers, reflection or human oracles). (c) Our proposed ReflexiCoder fosters an intrinsic capability to self-reflect and self-correct via a structured reasoning trajectory, eliminating the need for external oracles and environmental interaction.
Figure 2: The architecture of ReflexiCoder formulates code generation as an RL-optimized intrinsic self-debugging trajectory. A carefully designed composite reward jointly incentivizes reflection quality and correction success.
Institution
Model
HE
HE+
MBPP
MBPP+
BCB
LCB
CF
OpenAI
GPT-5.1
95.12
87.20
84.00
79.10
39.56
48.03
34.70
GPT-4.1
96.34
78.88
85.20
79.10
41.32
42.77
31.37
Anthropic
Claude-Sonnet-4.5
98.17
77.44
76.80
75.40
45.00
50.78
53.79
Google
Gemini-2.5-Pro
97.56
92.07
94.20
84.39
41.32
62.01
52.40
Alibaba
Qwen3-8B
89.02
80.49
70.20
70.37
32.63
37.75
23.70
Qwen2.5-Coder-7B-Instruct
86.59
79.88
75.80
69.84
33.33
13.86
6.39
Table 1: Main results on seven code generation benchmarks, reporting pass@1 (%). We compare our ReflexiCoder against leading proprietary and open-source models. Our models establish a new state-of-the-art for open-source models in 1.5B-14B range and demonstrate competitive performance against much larger proprietary models. ∗ denotes results taken from ( Jiang et al., 2024 ) . HE(+) denotes HumanEval(+). BCB is BigCodeBench, LCB is LiveCodeBench, and CF is CodeForces. ReflexiCoder-8B (Single) is evaluated in single-attempt without the system prompt (see Figure 10 ), while ReflexiCoder-8B (Multiple) uses the full iterative reasoning-reflection setup. We show the score improvement ( ± ) of our model relative to its base model (Qwen3-8B). Bold indicates the best performance among open-source and proprietary models, respectively.
Method
HumanEval
BigCodeBench
LiveCodeBench
CodeForces
ReflexiCoder-8B (Full)
94.51
35.00
52.21
37.34
w/o Format Gating F(τ)
84.75
32.02
39.07
24.81
w/o Cycle Regulation P(n)
92.68
33.68
52.09
35.84
w/o Efficiency Reward E(n)
91.46
33.42
42.41
29.92
w/o Progressive Imp. mt
93.29
34.74
39.19
34.10
Table 2: Ablation study of reward components. Performance is reported as pass@1 (%) on four benchmarks. The results confirm that each component is vital for achieving optimal performance and trajectories.
Figure 3: Analysis of the impact of reasoning and reflection. We compare ReflexiCoder against baselines that lack a structured reasoning and reflection step. Performance is pass@1 (%). The significant performance gap highlights that the structured reasoning-reflection cycle is the key driver of improvement.
Figure 4: Scaling analysis of ReflexiCoder with model size. Performance is pass@1 (%) across four representative benchmarks. The performance grow with model scale, indicating a super-linear benefit.
Model
HumanEval (164 tasks)
BigCodeBench (1,140 tasks)
Min
Avg
Max
Reflection
Min
Avg
Max
Reflection
Qwen2.5-Coder-7B-Instruct
61
289.18
820
-
116
430.02
1104
-
Qwen2.5-Coder-14B-Instruct
52
376.15
749
-
89
329.63
770
-
Qwen3-8B (Full)
690
4,170.03
16,385
-
856
5,214.01
16,385
-
Qwen3-14B (Full)
402
3,558.22
15,956
-
608
4,152.02
14,572
-
ReflexiCoder (Single) (Full)
553
3,455.13
16,381
{0: 164}
860
3,477.28
16,384
{0: 1140}
Table 3: Token budget statistics and self-reflection count distributions across benchmarks of varying difficulty levels. The “Reflection” column details the frequency distribution of reflection cycles executed per task. The “Full” designation refers to the token count of the entire generated response, whereas “Reasoning” isolates the specific token footprint of the reasoning process.
Figure 5: Probability density distributions of token consumption for both full responses (left) and reasoning segments (right) across the HumanEval and BigCodeBench benchmarks. The reasoning segments correspond to the <think>... </think> generation phases. Note that the Qwen2.5-Coder-7B/14B-Instruct baselines are not reasoning models, and thus lack corresponding reasoning token budgets.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Source
Symbol
Value
Meaning
Eq. 4
α
0.1
polynomial decay strength
β
2.0
polynomial decay curvature
γ
0.05
exponential decay strength
δ
0.1
oscillation magnitude
n0
5
reflection depth with no penalty
Eq. 5
λ
0.2
weight concentration on later iterations
Appendix
Table 4: Reward-related hyperparameters. Unless otherwise stated, these values are kept fixed across all tasks and benchmarks for reproducibility.
Figure 6: Pass@1 sensitivity to reward weights Format ( ξ ), Iter. Quality ( φ ), Efficiency ( ψ ) on four benchmarks.
Figure 7: Training dynamics of key reward components and behaviors for different model sizes. Larger models learn to achieve higher progressive improvement and efficiency rewards more quickly. They also converge to a more optimal number of reflection cycles, whereas smaller models struggle to fully optimize the complex reward landscape.
Figure 8: Reward surface induced by shaping terms. For each model scale (0.6B-14B), we fit a smooth 2D surface z^=f(x,y) via RBF regression, where x is the cycle-count regulation reward P(n) , y is the iterative quality reward Rtrajectory , and z is final code-quality reward (reward on the last generated code). The surface height and colormap jointly encode the predicted z^ (higher/brighter indicates better code quality), while overlaid points denote the observed training samples (xt,yt,zt) colored by training step; grey contour projections highlight local gradients. Consistent high- z^ regions across scales indicate regimes where shaping terms synergistically improve code quality, whereas steep slopes/contour crowding reveal sensitivity to the corresponding reward component.
Figure 9: A qualitative example of ReflexiCoder’s iterative self-reflection and self-correction on a TACO task.
Large Language Models have shown remarkable capabilities in code generation. However, most existing evaluations focus only on single-attempt accuracy and overlook the iterative refinement process that is central to real-world programming. This study presents a systematic investigation of LLMs' ability to rectify their own code through execution feedback. Using real-world programming problems across four models and two major programming languages, this study evaluates performance using iterative refinement framework where LLMs receive compiler error messages and testcase feedback after each attempt. This study introduces metrics to evaluate code failures, analyze rectification patterns, and compare the effectiveness of reasoning and non-reasoning models, offering actionable insights into both the understanding and practical application of feedback loops in LLM-driven code generation systems. Results show that reasoning models consistently improve over iterations, substantially outperforming non-reasoning models in leveraging feedback, while syntactic and runtime errors are far more tractable than logical or algorithmic failures.
While large language models (LLMs) have demonstrated strong performance on complex reasoning tasks such as competitive programming (CP), existing methods predominantly focus on single-attempt settings, overlooking their capacity for iterative refinement. In this paper, we present RefineRL, a novel approach designed to unleash the self-refinement capabilities of LLMs for CP problem solving. RefineRL introduces two key innovations: (1) Skeptical-Agent, an iterative self-refinement agent equipped with local execution tools to validate generated solutions against public test cases of CP problems. This agent always remains skeptical of its own outputs and thereby enforces rigorous self-refinement even when validation suggests correctness. (2) A reinforcement learning (RL) solution to incentivize LLMs to self-refine with only standard RLVR data (i.e., problems paired with their verifiable answers). Extensive experiments on Qwen3-4B and Qwen3-4B-2507 demonstrate that our method yields substantial gains: after our RL training, these compact 4B models integrated with the Skeptical-Agent not only outperform much larger 32B models but also approach the single-attempt performance of 235B models. Notably, although our models are trained solely on CP coding data, RefineRL also substantially improves their performance on mathematical reasoning benchmarks, suggesting that the learned self-refinement behavior can transfer beyond coding-specific tasks. These findings highlight self-refinement as a promising direction for scaling LLMs' general reasoning ability.
Shaopeng Fu, Xingxing Zhang, Li Dong +1
Work done during an internship at Microsoft Research · KAUST · Microsoft Research
A promising research direction in enabling LLMs to generate consistently correct code involves addressing their inability to properly estimate program execution, particularly for code they generate. In this work, we demonstrate that Code LLMs can be trained to simulate program execution in a step-by-step manner and that this capability can be leveraged to improve competitive programming performance. Our approach combines supervised fine-tuning on natural language execution traces, textual explanations grounded in true execution, with reinforcement learning using verifiable rewards. We introduce two complementary objectives: output prediction given code and inputs, and solving competitive programming tasks with either ground-truth or self-predicted execution feedback. These objectives enable models to perform self-verification over multiple candidate solutions, and iterative self-fixing by simulating test execution. Across multiple competitive programming benchmarks, our method yields consistent improvements over standard reasoning approaches. We further present ablations and analysis to elucidate the role of execution simulation and its limitations.
Gallil Maimon, Ori Yoran, Felix Kreuk +4
FAIR team, Meta · Hebrew University of Jerusalem · Inria