CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

Bowen Ye; Lei Li; Shicheng Li; Zihao Yue; Linghao Zhang; Hanglong Lv; Yuanxin Liu; Wenhan Ma

cs.AISep 18, 2026

CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

Authors: Bowen Ye, Lei Li, Shicheng Li, Zihao Yue, Linghao Zhang, Hanglong Lv, Yuanxin Liu, Wenhan Ma,

Organizations: LLM Core, Xiaomi · Peking University · University of Hong Kong · Renmin University of China

Abstract

Training capable coding agents via reinforcement learning (RL) requires diverse tasks with reliable verifiers. Open-source codebases offer a rich source of such tasks, while existing methods typically rely on development artifacts such as issues and commits, limiting the range of tasks that can be extracted. To better scale RL environments, we present CodeMidas, an agentic pipeline that turns implemented functionality in existing codebases into executable RL environments using source code as its only task-specific input. CodeMidas allocates agentic compute to every stage of environment construction: agents explore implemented functionality to formulate behavioral specifications, construct tests grounded in execution of the original code, and validate and filter candidate tasks through execution checks and repeated solution rollouts. The resulting dataset has 5,545 training tasks from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains. Training MiMo-V2.5 on these tasks with GRPO improves performance on all five diverse benchmarks, covering issue repair (DeepSWE + 11.7%), whole-program construction (ProgramBench +17%), and terminal work (Terminal-Bench v2.1 +8.5%). Ablations show that increasing the number of high-quality training tasks improves performance. Trajectory analysis shows the RL-trained agent demonstrates better behaviors like increasing codebase exploration and more diverse self-verification. These results establish source code as a scalable foundation for constructing RL environments that improve coding agents across diverse software tasks.

Figures & tables

Method	w/o issue	w/o PR	w/o commit	w/o existing tests	w/o written description	Languages
SWE-rebench V2 ( Badertdinov et al., 2026 )	$\blacklozenge$	✗	✗	✗	✓	20
daVinci-Env ( Fu et al., 2026 )	✗	✗	✗	✗	✓	1
R2E-Gym ( Jain et al., 2025 )	✓	✓	✗	$\blacklozenge$	✓	1
SWE-smith ( Yang et al., 2025 )	✓	$\blacklozenge$	$\blacklozenge$	✗	✓	1
SWE-Flow ( Zhang et al., 2025 )	✓	✓	✓	✗	✓	1
SWE-Hub ( Zeng et al., 2026 )	✓	✓	✓	✗	$\blacklozenge$	11

Table 1: Task-specific input requirements and language coverage of representative pipelines for constructing coding environments. ✓ = not required;

\blacklozenge

= required by part of the pipeline; ✗ = required.

Figure 1: CodeMidas overview. Four modules cover task design, test construction, execution consistency, and post-rollout filtering. The pyramid shows retained task counts respectly.

Figure 4: Reference solution size. Bars show task percentages; the dashed line marks the median (142 lines). The piecewise log x-axis compresses the 1–10 interval to one fifth the width of subsequent decades.

Figure 5: Performance gains from RL on CodeMidas. Initial MiMo-V2.5 scores (gray) and CodeMidas RL scores (gold). ProgramBench reports Almost Solved; all other evaluations report pass rate. Labels on the right give absolute improvements over the initial policy in percentage points.

Figure 6: Learning curve on CodeMidas Val. Pass rate (green, left axis) and mean total length (gray-blue dashed, right axis) during RL on CodeMidas. Length is measured in thousands of tokens; the horizontal dotted line marks the pass rate achieved by the initial policy.

Figure 7: Learning curves on CodeMidas Val. Pass rates at five-step intervals from step 0 to 70, without smoothing. Solid lines with circles denote the high-quality 1k, 3k, and full (5k) pools; the dashed line with diamonds denotes the vanilla 8k sample drawn before filtering and cleaning.

Figure 8: Task scale and quality. Scores on SWE-bench Pro, DeepSWE, and CodeMidas Val. Connected circles denote high-quality 1k, 3k, and 5k pools; diamonds denote vanilla 8k. Panels use different y-axis ranges.

Figure 9: Codebase exploration, code drafting, and self-verification in one rollout. Columns pair reasoning with tool use from one CodeMidas training rollout: reading the caller, drafting hidden-file filtering and applying it via Edit , and testing both flag settings with .secret.csv . The diff marks additions; Passed indicates agreement with expected behavior.

Behavior	Measure	Early	Late	Change
Codebase exploration	Pre-edit read/search calls	27.2	40.1	+12.9
Code drafting	Drafting ratio	0.358	0.629	+0.271
Self-verification	Distinct post-edit commands	2.03	2.53	+0.50

Table 2: Changes during RL. Rollout means in early and late training, using defined measurements for each behavior. Changes are computed from unrounded means for each behavioral measure.

Evaluation	Codebase exploration (read/search calls)	Code drafting (drafting ratio)	Self-verification (distinct commands)	Interaction length (assistant turns)
SWE-bench Pro	23.1 $\rightarrow$ 35.5	0.304 $\rightarrow$ 0.653	0.80 $\rightarrow$ 0.96	37.3 $\rightarrow$ 50.1
ProgramBench	55.7 $\rightarrow$ 83.6	0.106 $\rightarrow$ 0.361	0.93 $\rightarrow$ 0.99	155.1 $\rightarrow$ 122.8
Terminal-Bench v2.1	11.9 $\rightarrow$ 16.8	N/A	2.01 $\rightarrow$ 2.61	59.2 $\rightarrow$ 69.5

Table 3: Behavior on held-out tasks. Means over the first

\rightarrow

last three observed checkpoints for each benchmark. The three behavioral measures follow Table 2 and include rollouts with defined measurements; interaction length includes all scored rollouts. Measured subsets can vary across checkpoints.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Initial policy	MiMo-V2.5
Training task pool	5,545 CodeMidas tasks
Algorithm	GRPO
Reward	Binary verifier outcome (0 or 1)
Advantage normalization by standard deviation	Disabled
Batch size	32

Appendix

Table A1: Training and evaluation configuration.

LLM Agent Training

Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.

See all 585 papers

Cross-Benchmark Transfer from RL on Agentic Coding Tasks
Oct 1, 2026Sushant Mehta, Logan Ritchie, Edwin ChenRL for Code Generation AI Agent Benchmarks
Surge AI
CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents
Mar 18, 2026Lintang Sutawika, Aditya Bharat Soni, Bharath Sriraam R R +8Code Localization Code Search
Carnegie Mellon University · Independent · OpenHands
Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments
Jul 30, 2026Haomin Qi, Xingliang Wang, Xuanqi Gao +9Code Generation Benchmarks Agent Benchmarks
Microsoft · University of California San Diego · Zhejiang University +2

Oct 1, 2026J/K move · Enter open · S save

Cross-Benchmark Transfer from RL on Agentic Coding Tasks

Sushant Mehta, Logan Ritchie, Edwin Chen

Surge AI

RL for Code Generation AI Agent Benchmarks Agentic Code Generation RL Post-Training

Open paper AskPDF arXiv

Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior that was supposed to stay intact, or validate against an unchecked assumption. We ask whether reinforcement learning (RL) on expert-built agentic coding tasks closes this gap, and whether what the agent learns transfers beyond the training distribution. We post-train Kimi K2.7 Code, a 1T-parameter (32B active) open-weight mixture-of-experts model, with RL alone on 1,700 tasks: 1,000 repository tasks graded by hidden fail-to-pass tests and by pass-to-pass tests of existing behavior, and 700 terminal tasks graded by expert-written hidden verifiers. The reward is the fraction of target checks passed and drops to zero if any pass-to-pass test fails. One epoch of GSPO on a rank-32 LoRA adapter improves pass@1 on each of the six external benchmarks we evaluated, across three agent harnesses: SWE-Bench Pro (60.1 to 64.8), DeepSWE (31.0 to 43.4), Terminal-Bench 2.1 (67.4 to 82.0), Terminal-Bench 3 (1.4 to 12.1), Terminal-Bench 4 (0.0 to 7.6), and SWE-Marathon (5.0 to 25.0). Pooled over the five independent task sets (Terminal-Bench 4 revises Terminal-Bench 3), the improvement is significant (p < 0.001), and it remains significant on the three sets released after the training data was collected (p = 0.004); the model also improves under both harnesses never used in training. Median trajectories on DeepSWE and Terminal-Bench 3 are 24-35% shorter in agent steps. The base model's failed DeepSWE runs are mostly near-misses, and on the tasks the trained model newly solves, paired trajectories show it avoiding each of the four failure modes above.

Abstract

LLM Agent Training

Explore similar work

Cross-Benchmark Transfer from RL on Agentic Coding Tasks

CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents

Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments