A promising research direction in enabling LLMs to generate consistently correct code involves addressing their inability to properly estimate program execution, particularly for code they generate. In this work, we demonstrate that Code LLMs can be trained to simulate program execution in a step-by-step manner and that this capability can be leveraged to improve competitive programming performance. Our approach combines supervised fine-tuning on natural language execution traces, textual explanations grounded in true execution, with reinforcement learning using verifiable rewards. We introduce two complementary objectives: output prediction given code and inputs, and solving competitive programming tasks with either ground-truth or self-predicted execution feedback. These objectives enable models to perform self-verification over multiple candidate solutions, and iterative self-fixing by simulating test execution. Across multiple competitive programming benchmarks, our method yields consistent improvements over standard reasoning approaches. We further present ablations and analysis to elucidate the role of execution simulation and its limitations.
Figures & tables
Figure 1: A conceptual outline of how one can use self-execution simulation of a generated code solution (or solutions) on public or generated test cases to improve coding performance. The simulation can be used as feedback to select the best solution from a few candidates (best@k) or to iteratively fix the code as needed (self-RLEF). See Section 3 for details.
Figure 2: The two parts of our training pipeline. 1) Supervised fine tuning on natural language execution traces ( NLEX ), 2) Multi-task reinforcement learning on output prediction and competitive programming (optionally with multi-turn feedback and fixing).
Figure 3: CruxEval-O performance compared to model active parameters. Arrows demonstrate the benefit from training on NLEX data. We also compare to open models.
Figure 4: Best@k performance of self-verification with self-simulation . Solutions and output predictions are produced by the same model - based on Qwen2.5-7B or CWM, trained for both solving and output prediction. Even though the tests used for filtering are in the solve prompt, there is still room for notable gains from simulating them.
Base Model
LCB-IO -Out
DMC-Test-Out
pass@1
@5
@10
@1
@5
@10
Qwen-7B (ours)
79.7
89.4
91.1
77.3
89.3
91.6
w.o NLEX
75.7
87.8
89.7
71.8
87.0
89.9
Qwen-3B (ours)
66.4
80.6
83.9
59.4
78.2
82.8
w.o NLEX
57.1
74.2
78.3
45.9
66.2
72.4
Table 1: Output prediction performance of Qwen models trained with RLVR for output prediction, with and without NLEX data.
Model
RL obj.
DMC -Out
LCB-IO -Out
@1
@5
@1
@5
CWM
official
57.7
80.4
68.6
87.9
Joint
80.2
87.2
86.5
91.0
Out Pred
85.0
89.8
88.6
92.7
Qwen-7B
Joint
68.4
83.1
76.5
87.1
Out Pred
74.6
86.8
80.1
89.2
Table 2: Output prediction performance for models trained on standard code solving, jointly with output prediction (Joint), on their own solutions. We compare this to a model trained for output prediction only, models from Tab. 1 , (Out Pred), and official CWM.
DMC
LCB-IO
pass@1
pass@5
pass@10
public
pass@1
pass@5
pass@10
public
Execution RLEF (Oracle)
65.3
77.6
80.6
86.1
63.8
70.9
72.8
88.5
CWM (official)
49.0
63.7
67.9
60.8
57.4
67.3
70.1
71.4
CWM RL
60.8
72.8
76.0
74.7
61.0
67.6
69.2
82.9
Self-RLEF (Ours)
63.2
76.8
80.2
82.5
62.3
70.0
71.9
87.1
Table 3: Solve rates for training and evaluating with a standard reasoning approach vs using real or simulated execution feedback.
DMC
LCB-IO
Model \ Pass
@1
@5
@10
@1
@5
@10
Qwen3-32B
44.7
61.4
66.0
58.6
68.9
72.2
+ SRLEF ( Δ )
-10.6
-2.2
-1.4
-20.1
-1.0
-0.1
CWM
49.0
63.7
67.9
57.4
67.3
70.1
+ SRLEF ( Δ )
-4.8
+0.5
+0.9
-7.4
+0.1
+0.2
Table 4: Comparing performance of using the self-RLEF scaffold at inference only with open source reasoning models.
Figure 5: Comparing best@k when ranking Qwen3-32B solutions, using CWM post-trained only for output prediction as a verifier.
Table 5: Pass rates of the initial generated solution (Init), compared to the final submitted solution (Sub) in Self-RLEF inference on DMC considering both public and private tests.
DMC
LCB-IO
Model \ Public Pass
@1
@10
@1
@10
Qwen3-4B
42.5
65.4
64.4
80.9
Qwen3-32B
56.3
79.0
72.3
88.8
Qwen2.5-7B RL
55.1
76.5
68.0
84.7
CWM RL
73.4
87.6
81.7
90.6
Table 6: Public pass@1 and 10 of different models. The large gap of standard reasoning models can suggest that they under-utilise provided test information.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Model
CruxEval -In
MBPP
HumanEval+
LCBv5
GSM8k
Math 500
Qwen2.5-7B regular mix
0.469
0.634
0.652
0.414
0.842
0.518
+ NLEX
0.505
0.632
0.659
0.413
0.826
0.528
Qwen2.5-3B regular mix
0.361
0.522
0.543
0.195
0.748
0.398
+ NLEX
0.445
0.524
0.537
0.203
0.729
0.406
Appendix
Table 7: Investigating the impact of the NLEX data mix compared to a standard reasoning and instruction following mix on various standard coding and maths benchmarks. All models are trained with supervised fine tuning for the same budget only changing the data.
Base Model
LCB-IO -Out
DMC-Test-Out
pass@1
@5
@10
@1
@5
@10
CWM (Official)
60.4
79.3
82.0
68.9
85.8
87.8
CWM (wo. RL)
30.3
55.6
62.1
38.4
67.5
73.5
CWM (w. RL)
89.6
93.4
94.2
89.2
93.3
94.0
Appendix
Table 8: Output prediction performance of the CWM with and without RL. For reference we additionally report results for the official CWM model.
Figure 6: Comparing best@k when ranking solutions generated by CWM post-trained jointly for solving and output prediction, using the same model as a verifier. The model here was trained and evaluated without the public tests as part of the description.
DMC
LCB-IO
pass@1
pass@5
pass@10
public
pass@1
pass@5
pass@10
public
Self-RLEF (Ours)
61.5
75.6
79.6
79.0
61.5
69.2
71.1
84.2
Execution RLEF (Oracle)
62.7
75.8
78.8
81.5
63.3
70.3
72.2
86.3
Appendix
Table 9: Solve rates when using real or simulated execution feedback, but limiting to 3 turns. This extends Table 3 under a more compute constraint setup.
Figure 7: Comparing best@k when ranking Qwen3-4B solutions, using CWM post-trained only for output prediction as a verifier.
Figure 8: Comparing best@k when ranking solutions by CWM post-trained only for competitive programming solving (denoted SOLVE-RL in Table 3 ), using CWM post-trained only for output prediction as a verifier.
Figure 9: Comparing best@k when ranking solutions by Qwen-7B post-trained for competitive programming solving, using Qwen-7B post-trained only for output prediction as a verifier. This mirrors the results for Qwen in Figure 4 , but when each model has a dedicated role.
While large language models (LLMs) have demonstrated strong performance on complex reasoning tasks such as competitive programming (CP), existing methods predominantly focus on single-attempt settings, overlooking their capacity for iterative refinement. In this paper, we present RefineRL, a novel approach designed to unleash the self-refinement capabilities of LLMs for CP problem solving. RefineRL introduces two key innovations: (1) Skeptical-Agent, an iterative self-refinement agent equipped with local execution tools to validate generated solutions against public test cases of CP problems. This agent always remains skeptical of its own outputs and thereby enforces rigorous self-refinement even when validation suggests correctness. (2) A reinforcement learning (RL) solution to incentivize LLMs to self-refine with only standard RLVR data (i.e., problems paired with their verifiable answers). Extensive experiments on Qwen3-4B and Qwen3-4B-2507 demonstrate that our method yields substantial gains: after our RL training, these compact 4B models integrated with the Skeptical-Agent not only outperform much larger 32B models but also approach the single-attempt performance of 235B models. Notably, although our models are trained solely on CP coding data, RefineRL also substantially improves their performance on mathematical reasoning benchmarks, suggesting that the learned self-refinement behavior can transfer beyond coding-specific tasks. These findings highlight self-refinement as a promising direction for scaling LLMs' general reasoning ability.
Shaopeng Fu, Xingxing Zhang, Li Dong +1
Work done during an internship at Microsoft Research · KAUST · Microsoft Research
Competitive programming remains challenging for code LLMs. Despite recent progress, many training pipelines still depend on scarce real-world data, raising concerns about scalability and near-duplicate benchmark contamination. In this paper, we examine whether synthetic training artifacts can support the complete SFT-to-RL cycle for competitive programming: no real-world tasks, solutions, or test cases are directly used for post-training. To this end, we synthesize tasks, verified solutions, and reliable test cases that serve as reward signals for reinforcement learning. To improve reward reliability, we introduce a dual-verification strategy that reduces noise in both selected solutions and test outputs. Using this high-quality data, we train the X-Coder model series. X-Coder-14B achieves 67.5% avg@8 on LiveCodeBench v5 and 63.4% on v6, outperforming its base model by over 40 points. Further analysis provides practical insights into synthetic post-training, highlighting the value of diverse tasks, verified long-CoT supervision, and code-centric reinforcement. Our data and models are released at https://github.com/JieWu02/X-Coder
Large Language Models (LLMs) demonstrate strong potential for automated code generation, yet their ability to iteratively refine solutions using execution feedback remains underexplored. Competitive programming offers an ideal testbed for this investigation, as it demands end-to-end algorithmic reasoning, precise implementation under strict computational constraints, and complete functional correctness with rigorous evaluation. In this paper, we present A-ProS, an autonomous AI agent that solves competitive programming problems through a hybrid multi-model feedback framework separating solution generation from specialized debugging. A-ProS combines ChatGPT-based generators (GPT-4 and GPT-5) with three debugging critics: Codestral-2508, Llama-3.3-70B, and DeepSeek-R1, under a 2 x 3 factorial design. We evaluate six workflows on 367 problems from ICPC World Finals (2011-2024) and Codeforces (rated 1200-1800). The results show that GPT-5 workflows improve from 39 initial accepted solutions to 85-90 after three refinement rounds, while GPT-4 improves from 15 to 31-38. A controlled ablation on 47 problems shows that stateful refinement outperforms stateless approaches by 8.5-10.6 percentage points and reduces repeated failures by up to 3.5x. Compared to baseline agent loops, A-ProS achieves over 2x greater gains, highlighting the importance of persistent context and multi-model feedback for reliable autonomous program synthesis.
Anika Tabassum, Md Sifat Hossain, Md. Fahim Arefin +2
Dept. of Computer Science and Engineering, University of Dhaka, Bangladesh · Dept. of Information Systems, University of Maryland, Baltimore County, USA