SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks

Yucheng Zeng; Shupeng Li; Daxiang Dong; Ruijie Xu; Zimo Chen; Liwei Zheng; Yuxuan Li; Zhe Zhou

cs.AIFeb 28, 2026

SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks

Authors: Yucheng Zeng, Shupeng Li, Daxiang Dong, Ruijie Xu, Zimo Chen, Liwei Zheng, Yuxuan Li, Zhe Zhou,

Organizations: Qianfan Team, Baidu Inc.

Abstract

Progress in software-engineering agents is increasingly constrained by the scarcity of executable, scalable, and realistic data for training and evaluation. This scarcity stems from three fundamental challenges in existing pipelines: environments are brittle and difficult to reproduce across languages; synthesizing realistic, system-level bugs at scale is computationally expensive; and existing data predominantly consists of short-horizon repairs, failing to capture long-horizon competencies like architectural consistency. We introduce SWE-Hub, an end-to-end system that operationalizes the data factory abstraction by unifying environment automation, scalable synthesis, and diverse task generation into a coherent production stack. At its foundation, the Env Agent establishes a shared execution substrate by automatically converting raw repository snapshots into reproducible, multi-language container environments with standardized interfaces. Built upon this substrate, SWE-Scale engine addresses the need for high-throughput generation, combining cross-language code analysis with cluster-scale validation to synthesize massive volumes of localized bug-fix instances. Bug Agent generates high-fidelity repair tasks by synthesizing system-level regressions involving cross-module dependencies, paired with user-like issue reports that describe observable symptoms rather than root causes. Finally, SWE-Architect expands the task scope from repair to creation by translating natural-language requirements into repository-scale build-a-repo tasks. By integrating these components, SWE-Hub establishes a unified production pipeline capable of continuously delivering executable tasks across the entire software engineering lifecycle.

Figures & tables

Figure 1: SWE-Hub architecture. Starting from raw source repositories, the Env Agent provisions a deterministic execution substrate—pinned container images, a unified verification entrypoint, and deterministic artifacts—to make code runnable and testable. On top of this substrate, three task generators produce complementary SWE workloads: SWE-Scale for high-throughput local repair edits, Bug Agent for realism-oriented repo-level regressions with user-like symptoms and no root-cause hints, and SWE-Architect for building multi-file implementations from structured specifications.

Figure 2: SWE-Hub Environment Layer pipeline. Phase 1 (Env Agent) performs environment provisioning and build readiness, identifying toolchains and installing dependencies to achieve environment readiness without requiring tests to pass. Phase 2 (Test Agent) establishes a unified verification interface via entrypoint discovery and result standardization, employing dynamic adapters to handle heterogeneous ecosystems where native reporting is non-standard. Phase 3 (Solidification and Verification Gate) encapsulates the environment state and verification logic into a pinned container image, applying a strict execution check to ensure the substrate is deterministic and reusable for downstream task generation.

Figure 3: SWE-Scale architecture. The system transforms an Environment-Ready Substrate into executable bug-fix instances. The pipeline follows a structured flow: the Shared Parsing Backbone performs polyglot entity analysis using Tree-sitter ASTs to identify targets; the Candidate Generator synthesizes fault specifications via procedural or LLM-assisted strategies; and the Kubernetes-Scale Verification Cluster materializes patches in isolated sandbox pods, executing tests and performing behavioral validity checks to produce verified task records.

Category	Modifier	Action	Bug Type
Operational	Op. Change	Replace operator with group sibling (e.g., $+$ $\to$ $-$ ).	Operator Misuse
	Op. Flip	Invert boolean/comparison logic (e.g., $>$ $\to$ $\leq$ ).	Logic Inversion
	Operand Swap	Swap L/R operands ( $a,b\to b,a$ ).	Order Error
	Chain Break	Replace full expression with sub-operand ( $a+b\to a$ ).	Data Loss
	Const. Mod	Apply $\pm 1$ to integer literals.	Off-by-one
Control Flow	Branch Swap	Swap if body with else / elif body.	Logic Reversal

Table 1: Taxonomy of Procedural Modifiers in SWE-Scale.

Figure 4: Workflow of the Realism Layer in SWE-Hub. The pipeline initiates with Repo Analysis to extract call graphs and test baselines. The Bug Agent (center) then iteratively synthesizes system-level regressions that trigger non-local failures, validated by a Pass-to-Fail (P2F) signal. Finally, the Issue Agent (right) generates a user-like report from the verified regression, enforcing strict symptom-cause separation to prevent information leakage while adopting diverse personas.

Figure 5: SWE-Architect pipeline. (1) Mine a cohesive task scope from a real repository using coverage structure. (2) Create an initial repository state via code hollowing, and retain a golden reference patch. (3) Generate a structured requirement document via a two-agent reverse-engineering process (DOC Agent + API Agent). (4) Package the task with the pinned execution substrate and hidden tests in the unified task schema.

Synthetic Benchmark Generation

Synthetic benchmark generation creates controlled, verifiable evaluation tasks and datasets to test models across diverse settings.

See all 287 papers

SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale
Feb 27, 2026Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov +1Software Engineering Benchmarks Software Engineering Agents
Nebius, Amsterdam, Netherlands.
SWE-Next: Scalable Real-World Software Engineering Tasks for Agents
Mar 21, 2026Jiarong Liang, Zhiheng Lyu, Zijie Liu +4Automated Software Engineering Software Engineering Agents
University of Waterloo · University of North Carolina at Chapel Hill · Independent
SWE-Universe: Scale Real-World Verifiable Environments to Millions
Feb 2, 2026Mouxiang Chen, Lei Zhang, Yunlong Feng +16Software Engineering Benchmark Construction
Qwen Team, Alibaba Group · Zhejiang University

Feb 27, 2026J/K move · Enter open · S save

SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale

Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Alexander Golubev

Nebius, Amsterdam, Netherlands.

Software Engineering Benchmarks Software Engineering Agents LLM Agent Training RL for LLM Agents

Open paper AskPDF arXiv

Software engineering agents (SWE) are improving rapidly, with recent gains largely driven by reinforcement learning (RL). However, RL training is constrained by the scarcity of large-scale task collections with reproducible execution environments and reliable test suites. Although a growing number of benchmarks have emerged, datasets suitable for training remain limited in scale and diversity or often target a limited set of high-resource language ecosystems. We introduce SWE-rebench V2, a language-agnostic automated pipeline for harvesting executable real-world SWE tasks and constructing RL training environments at scale. The pipeline synthesizes repository-specific installation and test procedures via an interactive setup agent, and filters unsound instances using an ensemble of LLM judges, validated against human-verified SWE-bench annotations. Using this pipeline, we construct a dataset of 32,079 tasks spanning 20 languages and 3,617 repositories, with pre-built images for reproducible execution. To further scale training data, we additionally release 120,000+ tasks with installation instructions, fail-to-pass tests and rich metadata, where the problem statement is generated based on the original pull request description. We validate the collected instances through a diagnostic study that covers a subset of tasks in five programming languages across seven popular models, and provide instance-level metadata that flags common confounders such as overly restrictive tests and underspecified descriptions. We release the datasets, the collection and execution code, and associated artifacts to enable large-scale training of SWE agents across diverse languages and repositories.

Abstract

Synthetic Benchmark Generation

Explore similar work

SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale

SWE-Next: Scalable Real-World Software Engineering Tasks for Agents

SWE-Universe: Scale Real-World Verifiable Environments to Millions