cs.SEMar 5, 2026
SaveRepoLaunch: Automating Build and Management of Code Repositories across Languages and Platforms
Organizations: Microsoft
Abstract
Language model (LM) agents have driven substantial progress in automated software engineering (SWE), yet building and testing software repositories at scale remains a largely manual and labor-intensive bottleneck. In this work, we introduce RepoLaunch, a novel agentic framework that automatically resolves dependencies, compiles source code, and extracts test results across diverse programming languages and operating systems. RepoLaunch achieves a 78% build success rate, outperforming the Python/Linux-only prior system by 18%. To demonstrate its application, we further present a fully automated pipeline for SWE dataset creation driven by RepoLaunch, which only requires human input at the task-design stage. RepoLaunch is open-sourced, and its automated task-generation pipeline has been adopted by several recent works on agentic benchmarking and training.
Figures & tables
Figure 1: RepoLaunch Pipeline.
Figure 2: RepoLaunch-Automated SWE Dataset Creation. Green blocks are steps automated by RepoLaunch. In the figure, Cross-Platform Code Migration Bench is our future work, also an SWE benchmark with task instances created by RepoLaunch.
| Language | Total | Verification | Build | Management | Validation |
| Python* | 1200 | 693 / 57.8% | 528 / 76.2% | – | 100 / 18.9% |
| JS/TS | 1200 | 460 / 38.3% | 410 / 89.1% | 378 / 92.2% | 204 / 54.0% |
| C/C++ | 1000 | 392 / 39.2% | 310 / 79.1% | 264 / 85.2% | 066 / 25.0% |
| C# | 800 | 251 / 31.4% | 231 / 92.0% | 220 / 95.2% | 078 / 35.5% |
| Java | 800 | 323 / 40.4% | 252 / 78.0% | 237 / 94.0% | 102 / 43.0% |
| Go | 500 | 213 / 42.6% | 154 / 72.3% | 146 / 94.8% | 098 / 67.1% |
Table 1: The number and ratio of instances retained at each stage from the last stage for each language on Linux and the overall result on Windows. Python* data were referred from an old version of RepoLaunch in prior works contributed to the SWE-bench-Live Python subset in July and August 2025, where test log parsing adopted the rule-based pytest-format parser of SWE-bench [ 3 ] , so it does not have the Management step.
Figure 3: The proportion of different failure patterns in the Build and the Management stage .
Table 2: Comparison of benchmarking results. (a) Success rate of agents averaged across the 6 LMs and 6 languages; (b) Success rate of LMs averaged across the 4 agents and 7 splits (6 splits of 6 languages on Linux and 1 Windows task split); (c) For each split, the best-performing combination of agents and LMs.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Language | OS | Docker Images / Versions |
|---|---|---|
| Python | Linux | python:3.{6,7,8…14} |
| Windows | python:3.{9,10,11}-windowsservercore-ltsc2022 | |
| python:3.{12,13,14}-windowsservercore-ltsc2025 | ||
| C/C++ | Linux | mcr.microsoft.com/devcontainers/cpp:1-ubuntu-{20,22,24}.04 |
| Windows | custom/windows_server:ltsc{2019,2022}_cmake_ninja_only | |
| custom/windows_server:ltsc2022_cmake_ninja_vsbuildtools_cl_msbuild |
| C/C++ | C# | Java | JS/TS | Go | Rust | Python | |
| Preparation Stage | |||||||
| Input tokens | 75.4k | 70.8k | 43.2k | 95.7k | 87.2k | 116.1k | 74.3k |
| Output tokens | 412 | 476 | 264 | 405 | 406 | 580 | 430 |
| USD cost | 0.073 | 0.075 | 0.043 | 0.073 | 0.075 | 0.103 | 0.071 |
| Build Stage | |||||||
| Input tokens | 2619.8k | 1090.0k | 561.0k | 1065.6k | 620.3k | 856.7k | 854.3k |
Table 4: Average token usage and cost of RepoLaunch per instance for each language and each stage. The experiments were done using GPT-5-Thinking-Medium. Python data were referred from an old version of RepoLaunch in prior works contributed to SWE-bench-Live Python tasks, which adopted a rule-based pytest-format parser to parse test logs, so it does not involve the Management stage .
Figure 4: Scalability of RepoLaunch when repo size grows. Each of the five repo size ranges contains 75 repos.
| Failure pattern | C/C++ | C# | Java | TS/JS | Go | Rust | Win |
| Build Stage | |||||||
| Repo not at production | 9.7% | 12.4% | 14.5% | 20.7% | 10.8% | 17.6% | 11.3% |
| Incomplete test framework | 7.8% | 6.2% | 4.8% | 4.6% | 4.3% | 9.9% | 1.4% |
| Require API key | 0.0% | 0.0% | 0.0% | 4.6% | 3.6% | 0.0% | 0.7% |
| Require database / docker backend | 0.0% | 9.9% | 1.2% | 5.1% | 8.6% | 3.3% | 1.4% |
| Command timeout / out of memory | 61.2% | 25.9% | 10.8% | 20.3% | 36.7% | 22.0% | 20.4% |
Table 5: Failure Pattern per Language and for Windows in Build and Management stages. The highest two values in each column for Build stage and the highest one value in each column for Management stage are bolded.
| instance_id | language | instance_id | language |
|---|---|---|---|
| jqlang__jq-2235 | C | apache__druid-13704 | Java |
| jqlang__jq-2681 | C | apache__druid-14136 | Java |
| jqlang__jq-2919 | C | apache__druid-16875 | Java |
| micropython__micropython-10095 | C | apache__lucene-11760 | Java |
| micropython__micropython-13039 | C | apache__lucene-12626 | Java |
| micropython__micropython-15898 | C | apache__lucene-13704 | Java |
| Language | Agent | Build | Management | Validation | Manage&Val |
| Python | RepoLaunch | 81.0% (34/42) | 94.1% (32/34) | 87.5% (28/32) | 82.4% (28/34) |
| Python | SWE-agent | 52.4% (22/42) | – | – | 63.6% (14/22) |
| C/C++ | RepoLaunch | 81.3% (13/16) | 100% (13/13) | 92.3% (12/13) | 92.3% (12/13) |
| C/C++ | SWE-agent | 62.5% (10/16) | – | – | 30.0% (3/10) |
| Java | RepoLaunch | 73.3% (11/15) | 81.8% (9/11) | 88.9% (8/9) | 72.7% (8/11) |
| Java | SWE-agent | 66.7% (10/15) | – | – | 40.0% (4/10) |
Table 7: Performance comparison of RepoLaunch against the baseline SWE-agent, a general-purpose SWE agent equipped with only bash and file-editor tools, on the repo build and management task to reconstruct task environments and extract test statuses for SWE-bench instances [ 3 , 5 , 20 ] . Here Manage&Val means Management&Validation combined success rate from the Build result. RepoLaunch effectively surpasses the baseline SWE-agent on both the Build and Management stages.
| Method | Type | Python | C/C++ | Java | JS/TS | Rust | Go |
| RepoLaunch | agentic | 81.0 % (34/42) | 81.3 % (13/16) | 73.3 % (11/15) | 84.4 % (27/32) | 70.0 % (14/20) | 81.5 % (22/27) |
| repo2run | agentic | 59.5% (25/42) | – | – | – | – | – |
| CXXCrafter | agentic | – | 68.8% (11/16) | – | – | – | – |
| SWE-agent | agentic | 52.4% (22/42) | 62.5% (10/16) | 66.7% (10/15) | 68.8% (22/32) | 65.0% (13/20) | 66.7% (18/27) |
| pipreq | rule-based | 40.5% (17/42) | – | – | – | – | |
| railpack | rule-based | 23.8% (10/42) | 31.3% (5/16) | 20.0% (3/15) | 34.4% (11/32) | 35.0% (7/20) | 44.4% (12/27) |
Table 8: Performance of different repo build methods on building repo-commit pairs of SWE-bench instances [ 3 , 5 , 20 ] . repo2run, CXXCrafter, pipreq and railpack do not have a test case extractor, so only the Build stage is evaluated and the Management stage performance cannot be compared. RepoLaunch achieves state-of-the-art performance on all languages.
| Method | Statistic Source | Cost in USD | Build Success Rate |
| RepoLaunch | from our experiment | 0.65 | 76% |
| repo2run | from our experiment | 0.70 | 60% |
| swe-rebench | [ 18 , 29 ] | 0.10 | 20~30% |
| OpenSWE | [ 19 ] | 19.1 | ~50% |
Table 9: Average LM API cost in USD per instance to build Python repos. RepoLaunch is cost-effective to build Python repos while achieving the highest success rate.
| Experiment | Build Success | Management Success | Verification Success | Avg. Build Steps | Avg. Manage. Steps |
| Linux | |||||
| search + prompt + image + str-replace-editor # | 84.9% (129/152) | 89.9% (116/129) | 92.2% (107/116) | 23.8 | 21.1 |
| Baseline (search + prompt + image) | 79.6% (121/152) | 88.2% (108/121) | 91.5% (99/108)) | 25.0 | 21.8 |
| w/o web search | 71.7% (109/152) | 85.3% (93/109) | 91.4% (85/93) | 27.7 | 25.1 |
| w/o language-specific prompt | 72.3% (86/152) | 80.2% (69/86) | 84.1% (58/69) | 26.1 | 24.7 |
| w/o language-specific image ∗ | 67.1% (102/152) | 88.2% (90/102) | 91.1% (82/90) | 31.1 | 22.5 |
Table 10: Ablation results on Linux and Windows. The baseline is the setting adopted in other experiments in the paper. # The string-replace-editor tool is the tool to edit file contents in the repos. As RepoLaunch is used to build environments for SWE tasks at base commits with issues, by default the file editor tool is disabled to prevent the agent from fixing the bugs originally in the repos. * When language-specific image is disabled, RepoLaunch builds all repos from a raw Ubuntu (for Linux) or Windows-Server (for Windows) base image with no other dependencies installed.
| Model | Agent | Build success | Management success | Validation Success | Manage&Val success |
| Claude-Sonnet-5 | RepoLaunch | 90.8 % (138/152) | 95.7% (132/138) | 96.2% (127/132) | 92.0 % (127/138) |
| Claude-Sonnet-5 | SWE-agent | 81.6% (124/152) | - | - | 64.5% (80/124) |
| Claude-Sonnet-5 | repo2run* | 69.0% (29/42) | - | - | - |
| Deepseek-v4-Pro | RepoLaunch | 84.9 % (129/152) | 92.2% (119/129) | 89.1% (106/119) | 82.1 % (106/129) |
| Deepseek-v4-Pro | SWE-agent | 77.6% (118/152) | - | - | 63.6% (75/118) |
| Deepseek-v4-Pro | repo2run | 71.4% (30/42) | - | - | - |
Table 11: RepoLaunch is robust when driven by different LM providers. RepoLaunch and SWE-agent are evaluated on the whole multilingual benchmark in Appendix F.1 on both Build and Manage&Val stages. * repo2run can only run on the Python subset of the benchmark on the Build stage.
| id | repo | language | id | repo | language |
|---|---|---|---|---|---|
| Android repos | Embedded system repos | ||||
| 1 | robolectric/robolectric | Java | 1 | raspberrypi/linux | C |
| 2 | skylot/jadx | Java | 2 | Koenkk/zigbee2mqtt | TypeScript |
| 3 | JesusFreke/smali | Java | 3 | esphome/esphome | C++ |
| 4 | zxing/zxing | Java | 4 | home-assistant/core | Python |
| 5 | google/bundletool | Java | 5 | node-red/node-red | JavaScript |
| Agent | Model | C/C++ | C# | Java | JS/TS | Go | Rust | Linux |
| SWE-agent | GPT-5.5 (Medium) | 34.8% | 44.9% | 50.0% | 47.5% | 34.7% | 28.1% | 40.0% |
| SWE-agent | Deepseek-v4-Pro | 24.2% | 44.9 % | 39.2% | 36.8% | 29.6% | 34.4% | 34.8% |
| SWE-agent | Claude-4.5-Sonnet | 43.8 % | 20.5% | 16.1% | 20.4% | 39.7% | 28.9% | 28.4% |
| SWE-agent | GPT-5.2 (Medium) | 41.7% | 21.8% | 22.6% | 16.1% | 25.0% | 28.9% | 26.0% |
| SWE-agent | Gemini-3-flash | 35.4% | 14.3% | 25.8% | 9.9% | 32.4% | 37.5% | 25.9% |
| SWE-agent | Deepseek-v3.1-terminus | 27.1% | 21.8% | 27.4% | 10.5% | 20.6% | 28.9% | 22.7% |
Table 13: The success rate of different model-agent combinations on SWE-bench-Live/MultiLang. The Linux column is the average values of previous columns so that languages with different number of instances have the same weight.
| LM | Claude-4.5 | GPT-5.5 | GPT-5.2 | Gemini-3 | Ds-4 | Ds-3.1 |
| Agent | Win-agent* | Win-agent | Win-agent | Win-agent | Win-agent | Win-agent |
| Resolved (%) | 30.0% | 26.7% | 20.0% | 16.7% | 21.7% | 20.0% |
Table 14: Benchmarking results of latest LMs on SWE-bench-Live/Windows. GPT models use medium reasoning effort. Claude-4.5: Claude-4.5-Sonnet, Gemini-3: Gemini-3-Flash, Ds-3.1: DeepSeek-v3.1-Terminus, Ds-4: Deepseek-v4-Pro. Win-agent* is the minimal agent implemented by our team with the same tool calls as SWE-agent and OpenHands. It is also the first coding agent that can run on Windows containers.
Figure 5: The proportion of different failure patterns of coding agents on SWE-bench-Live.
More in this topic
Automated Software Engineering
Automated Software Engineering uses AI, search, and verification methods to automate tasks across the software development lifecycle.
See all 80 papers