From SWE-ZERO to SWE-HERO: Execution-free to Execution-based Fine-tuning for Software Engineering Agents

Nikolai Ludwig; Wasi Uddin Ahmad; Somshubra Majumdar; Boris Ginsburg

cs.SEApr 2, 2026

From SWE-ZERO to SWE-HERO: Execution-free to Execution-based Fine-tuning for Software Engineering Agents

Authors: Nikolai Ludwig, Wasi Uddin Ahmad, Somshubra Majumdar, Boris Ginsburg

Organizations: NVIDIA Santa Clara, CA 95051, USA

Abstract

We introduce SWE-ZERO to SWE-HERO, a two-stage SFT recipe that achieves state-of-the-art results on SWE-bench by distilling open-weight frontier LLMs. Our pipeline replaces resource-heavy dependencies with an evolutionary refinement strategy: (1) SWE-ZERO utilizes large-scale, execution-free trajectories to master code semantics and repository-level reasoning, and (2) SWE-HERO applies targeted, execution-backed refinement to transition these semantic intuitions into rigorous engineering workflows. Our empirical results set a new benchmark for open-source models of comparable size. We release a dataset of 300k SWE-ZERO and 13k SWE-HERO trajectories distilled from Qwen3-Coder-480B, alongside a suite of agents based on the Qwen2.5-Coder series. Notably, SWE-HERO-32B achieves a 62.2% resolution rate on SWE-bench Verified. Furthermore, despite being trained exclusively on Python, our agents demonstrate robust zero-shot transferability on SWE-bench Multilingual, reaching 44.1% and confirming the paradigm's generalizability across diverse languages.

Figures & tables

Figure 1: Performance overview of various open-source foundational models and SWE agents on SWE-bench Verified. Our models, SWE-Hero agents establish a new frontier, outperforming same-scale competitors.

Model	Execution	SWE-bench (V)	SWE-bench (M)
MiniMax-M2.5	$\times$	69.5	57.2
MiniMax-M2.5	$\checkmark$	80.2	74.1
[0.5pt/2pt] Qwen3-Coder-Next	$\times$	56.9	50.7
[0.5pt/2pt] Qwen3-Coder-Next	$\checkmark$	71.3	64.3
[0.5pt/2pt] Qwen3-Coder-480B-A35B-Instruct	$\times$	59.4	44.3
[0.5pt/2pt] Qwen3-Coder-480B-A35B-Instruct	$\checkmark$	69.6	54.7

Table 1: Resolve rates (%) on SWE-bench Verified (V) and Multilingual (M). We compare the execution-free (

\times

) paradigm against the execution-backed setup (

\checkmark

) using OpenHands. Notably, our SWE-Hero -32B model is fine-tuned in two successive SFT stages using a collection of 300k execution-free and 13k execution-based Python trajectories.

Panel of Figure 2 — Figure 2: Comparative overview of Agentic Workflows in OpenHands. The bottom panel depicts the SWE-Zero setup, utilizing a restricted, execution-free environment to maximize data scalability. In contrast, the top panel illustrates the SWE-Hero configuration, which transitions to a standard, feedback-driven workflow grounded in physical execution.

Model	Scaffold	Training	Resolve Rate (%)
Parameters $\approx$ 7B
SWE-Gym-7B ( Pan et al., 2025 )	OpenHands	SFT	10.6
R2E-Gym-7B ( Jain et al., 2025 )	R2E-Gym	SFT	19.0
SWE-Mirror-LM-7B ( Wang et al., 2025a )	MOpenHands	SFT	22.8
SWE-Dev-7B ( Wang et al., 2025a )	OpenHands	RL	23.4
SERA-8B ( Shen et al., 2026 )	SWE-agent	SFT	31.7

Table 2: Performance comparison on the SWE-bench Verified. Results marked with † exclude Git hacking ; for all other scores, the use of Git hacking is unspecified.

Figure 3: Cumulative performance and resource efficiency on SWE-bench Verified. Tasks are ordered by SWE-Hero -32b turn-count (complexity proxy). The primary axis (thick lines) shows the natural decline in resolve rate as complexity grows, while the secondary axis (faint lines) highlights the significantly lower turn-costs of SWE-Zero models.

Figure 4: Scaling behavior of SWE-Zero -14B. The resolution rate on SWE-bench increases consistently as execution-free training samples scale from 4k to 150k.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Verifier	Pass@1 (avg of 32)	Pass@32	Best@32
SWE-Hero-7B	OpenHands-32B-Verifier	50.0	76.0	52.9
	R2EGym-Verifier			55.9
	SWE-Lego-Verifier-8B			57.9
	SWE-Lego-Verifier-30B-A3B			57.9
SWE-Hero-14B	OpenHands-32B-Verifier	57.4	78.4	62.0
	R2EGym-Verifier			62.6

Appendix

Table 3: Evaluation results of SWE-Hero LLMs under Test Time Scaling (TTS) setting using different open-sourced generative verifiers.

Figure 5: Increasing inference-time compute improves performance on SWE-Bench Verified using open-source SWE-Lego-Verifier-8B ( Tao et al., 2026 ) .

Figure 6: Cumulative performance and resource efficiency on SWE-bench Verified. Tasks are ordered by SWE-Hero -32b token-count (complexity proxy). The primary axis (thick lines) shows the natural decline in resolve rate as complexity grows, while the secondary axis (faint lines) highlights the significantly lower token-costs of SWE-Zero models.

Model	Scaffold	Training	Execution	Resolve Rate (%)
Proprietary Models
GPT-5.2	mini-SWE-agent	-	$\checkmark$	66.3
Claude 4.5 Opus	mini-SWE-agent	-	$\checkmark$	70.7
Gemini 3 Flash	mini-SWE-agent	-	$\checkmark$	72.7
Open-Source Foundation Models
DeepSeek-V3.2	mini-SWE-agent	-	$\checkmark$	62.3

Appendix

Table 4: Performance comparison on SWE-bench Multilingual, comparing execution-free (

\times

) and execution-backed (

\checkmark

) settings.

Model	Scaffold	Training	Resolve Rate (%)
Proprietary Models
GPT-5.2 ( OpenAI, 2025 )	mini-SWE-agent	-	72.8
Gemini 3 Flash ( Google, 2025 )	mini-SWE-agent	-	75.8
Claude 4.5 Opus ( Anthropic, 2025b )	mini-SWE-agent	-	76.8
Open-Source Foundation Models
DeepSeek-V3.2 ( Anthropic, 2025a )	mini-SWE-agent	-	70.0

Appendix

Table 5: Performance comparison on SWE-bench Verified. Results marked with † exclude Git hacking ; for all other scores, the use of Git hacking is unspecified.

Software Engineering Agents

Software Engineering Agents are AI systems that use code, tools, and feedback to perform software development tasks.

See all 323 papers

SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents
Feb 25, 2026Patrick Tser Jern Kon, Archana Pradeep, Ang Chen +5Software Engineering Agents Small Language Models
University of Michigan · Meta · Stanford University
Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents
Jun 14, 2026Wasi Uddin Ahmad, Nikolai Ludwig, Somshubra Majumdar +1Software Engineering Agents LLM Agent Training
NVIDIA
Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills
Jun 5, 2026Chuan Xiao, Zhengbo Jiao, Shaobo Wang +5Software Engineering Agents LLM Agent Skill Learning
2Shanghai Jiao Tong University · 1AI Data, Alibaba Group

Feb 25, 2026J/K move · Enter open · S save

SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents

Patrick Tser Jern Kon, Archana Pradeep, Ang Chen, Alexander P. Ellis+4

University of Michigan · Meta · Stanford University

Software Engineering Agents Small Language Models LLM Agents Automated Program Repair RL for Code Generation

Open paper AskPDF arXiv

Small language models (SLMs) offer compelling advantages in cost, latency, and adaptability, but have so far lagged behind larger models on long-horizon software engineering tasks such as SWE-bench, where they suffer from pervasive action looping and low resolution rates. We introduce SWE-Protégé, a post-training framework that reframes software repair as an expert-protégé collaboration problem. In SWE-Protégé, an SLM remains the sole decision-maker while learning to selectively seek guidance from a strong expert model, recognize stalled states, and follow through on expert feedback. Our approach combines supervised fine-tuning on expert-augmented trajectories with agentic reinforcement learning that explicitly discourages degenerative looping and unproductive expert collaboration. We lightly post-train Qwen2.5-Coder-7B-Instruct to achieve 42.4% Pass@1 on SWE-bench Verified, a +25.4% improvement over the prior SLM state of the art, while using expert assistance sparsely (~4 calls per task and 11% of total tokens).

Abstract

Software Engineering Agents

Explore similar work

SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents

Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents

Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills