Qwen3-Coder-Next Technical Report

Ruisheng Cao; Mouxiang Chen; Jiawei Chen; Zeyu Cui; Yunlong Feng; Binyuan Hui; Yuheng Jing; Kaixin Li

cs.CLFeb 28, 2026

Qwen3-Coder-Next Technical Report

Authors: Ruisheng Cao, Mouxiang Chen, Jiawei Chen, Zeyu Cui, Yunlong Feng, Binyuan Hui, Yuheng Jing, Kaixin Li,

Abstract

We present Qwen3-Coder-Next, an open-weight language model specialized for coding agents. Qwen3-Coder-Next is an 80-billion-parameter model that activates only 3 billion parameters during inference, enabling strong coding capability with efficient inference. In this work, we explore how far strong training recipes can push the capability limits of models with small parameter footprints. To achieve this, we perform agentic training through large-scale synthesis of verifiable coding tasks paired with executable environments, allowing learning directly from environment feedback via mid-training and reinforcement learning. Across agent-centric benchmarks including SWE-Bench and Terminal-Bench, Qwen3-Coder-Next achieves competitive performance relative to its active parameter count. We release both base and instruction-tuned open-weight versions to support research and real-world coding agent development.

Figures & tables

Figure 1: Comparison of Qwen3-Coder-Next to other open-weight models on SWE-Bench Verified, SWE-Bench Multilingual, SWE-Bench Pro, Terminal-Bench 2.0, and Aider.

Figure 2: Our pipeline for synthesizing bugs to scale up the number of software engineering tasks.

Model	Evalplus	MultiplE	CRUX-Eval
Baseline	54.38	36.02	57.13
Reformat	63.09	48.35	58.94

Table 1: Impact of web document reformatting during mid-training.

Figure 3: Scaling analysis of code agent pretraining on SWE tasks, SWE-Bench Verified and SWE-Bench Multilingual. Left: Within-scaffold scaling shows consistent improvement with the increase of mid-training tokens. Right: Cross-scaffold transfer reveals limited generalization across different agent frameworks.

Figure 4: Different tool chat templates, split by tool definition, tool calls, and tool response.

Figure 5: SWE-bench Verified performance vs. number of tool chat templates. Data volume and training configuration are kept identical.

Model	Scaffold 1	Scaffold 2	Scaffold 3	Scaffold 4	Scaffold 5	Avg.
GPT-5-2	84.0	14.0	41.8	29.8	77.0	49.3
Claude-sonnet-4-5	86.8	61.0	100.0	88.0	91.0	85.4
Gemini-3-pro	92.4	57.0	98.0	93.5	94.0	87.0
Deepseek-v3.2	98.0	87.0	100.0	92.5	91.0	93.7
GLM-4.6	85.8	86.0	100.0	82.9	24.0	75.7
GLM-4.7	91.0	64.0	100.0	94.7	0.0	69.9

Table 2: Template Following generalization accuracy across community-adopted IDE/CLI Environments.

Figure 6: Performance trends of various coding sub-capabilities throughout the single-turn RL steps, measured by our in-house benchmarks.

Figure 7: Left : SWE-bench Verified performance vs. RL steps with a reinforced reward-hacking blocker. We also found that a long-horizon coding ability emerged in the model during RL training, pushing the average number of agent turns from 50 to 130. Right : Performance without the blocker. Even after removing git remotes and future commits, the agent autonomously learns to exploit various git commands to retrieve ground-truth information as model capability increases. To the best of our knowledge, this behavior has not previously been reported.

Model	Size	SWE-Bench Verified
Model	Size	SWE-Agent	MiniSWE-Agent	OpenHands
Proprietary Models
Claude-Opus-4.5	?	78.2	77.8	79.0
Claude-Sonnet-4.5	?	76.0	68.4	74.6
Open-source Models
DeepSeek-V3.2	671A37	70.2	67.2	72.6

Table 3: SWE-Bench Verified. Dashes indicate we couldn’t reliably obtain test results in such settings. The maximum number of agent turns was set to 300.

Model	Size	SWE-Bench Multilingual			SWE-Bench-Pro
Model	Size	SWE-Agent	MiniSWE-Agent	OpenHands	SWE-Agent	MiniSWE-Agent
Proprietary Models
Claude-Opus-4.5	?	71.7	71.8	75.2	51.6	50.2
Claude-Sonnet-4.5	?	67.2	61.3	66.0	50.5	43.0
Open-source Models
DeepSeek-V3.2	671A37	62.3	55.5	61.8	46.0	32.4

Table 4: SWE-Bench Multilingual and SWE-Bench Pro. Dashes indicate we couldn’t reliably obtain test results in such settings. The maximum number of agent turns was set to 300.

Model	Size	TerminalBench 2.0
Model	Size	Terminus2-xml	Terminus2-json	ClaudeCode	QwenCode
Proprietary Models
Claude-Opus-4.5	?	58.4	57.3	53.9	51.7
Claude-Sonnet-4.5	?	51.7	51.7	41.6	37.1
Open-source Models
DeepSeek-V3.2	671A37	34.8	39.3	–	–

Table 5: Terminal-Bench 2.0 results. Dashes indicate we couldn’t reliably obtain test results in such settings.

Model	EvalPlus	MultiPL-E	CRUXEval	LiveCodeBench (v6)	OJBench	Codeforces
Qwen3-Coder-480B-A35B	86.66	88.00	92.13	44.93	14.98	1800
Qwen3-Next	89.00	89.00	94.81	51.79	20.04	1875
Qwen3-Coder-Next	86.56	88.23	95.88	58.93	23.01	2100

Table 6: Results on function-level coding and competitive-programming benchmarks.

Model	FullStackBench-en	FullStackBench-zh	Spider	BIRD-SQL	Aider-Polyglot
Qwen3-Coder-480B-A35B	62.54	63.07	85.98	64.15	60.40
Qwen3-Next	62.30	59.22	82.50	66.62	52.90
Qwen3-Coder-Next	60.58	57.38	83.66	63.56	66.20

Table 7: Results on full-stack development, text-to-SQL, and multilingual code editing benchmarks.

Model	MMLU	MMLU-Redux	MMLU-Pro	GPQA	SuperGPQA
Qwen3-Next	87.87	91.14	80.89	73.54	58.70
Qwen3-Coder-Next	87.73	91.18	80.52	74.49	57.45

Table 8: Results on general knowledge and reasoning benchmarks.

Model	HMMT25 Feb	HMMT25 Nov	AIME24	AIME25
Qwen3-Next	54.27	68.07	82.92	69.64
Qwen3-Coder-Next	70.21	75.57	89.01	83.07

Table 9: Results on competitive math benchmarks.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Language	Instances	Repos	Inst/Repo (Avg)	Avg Eval Lines
Python	202,302	13,098	15.45	25.01
Javascript / Typescript	175,660	11,604	15.14	27.41
Go	121,062	5,554	21.80	28.87
Java	86,105	4,700	18.32	24.75
Rust	74,180	4,445	16.69	19.31
C / C++	37,228	3,405	10.93	45.78

Appendix

Table 10: Data statistics for real-world repository instances.

Dataset	PL	# Repos (raw)	# Repos (cleaned)	# Repos (used)	Bug Sampling Strategy				# Total
Dataset	PL	# Repos (raw)	# Repos (cleaned)	# Repos (used)	lm_rewrite	lm_modify	func_pm	others	# Total
SWE-smith	Python	134	134	130	10,980	23,120	28,467	11,436	74,003
SWE-Flow	Python	2,203	2,203	1,987	81,963	119,892	182,686	-	384,541
SWE-rebench	Python	3,468	2,912	2,727	48,660	80,246	244,219	-	373,125
SWE-smith-multi	Multilingual	133	133	118	2,448	6,749	3,401	1,065	13,663
Multi-SWE-RL	Multilingual	74	74	57	1,399	3,362	1,805	-	6,566

Appendix

Table 11: Data statistics for generated task instances with method workflow-based pipelined bug synthesis.

Name	Model / Agent	Tool Definition Format	Tool Calls Format
qwen3_coder	Qwen-Coder-Next (this work)	XML	XML
qwen3_xml_mixed	Qwen-Coder-Next (this work)	JSON	XML
deepseekr1	DeepSeek-R1 ( Guo et al., 2025 )	JSON	Mixed XML+JSON
deepseekv3	DeepSeek-V3 ( Liu et al., 2024a )	JSON	Mixed XML+JSON
deepseekv31	DeepSeek-V3.1 ( Liu et al., 2024a )	Text	Mixed XML+JSON
deepseekv32	DeepSeek-V3.2 ( DeepSeek-AI, 2025 )	JSON	XML

Appendix

Table 12: The checklist of all used tool chat templates for scaling.

Figure 8: Two simple variants of the traditional concat-then-split sample packing strategy: restart last document (RLD) and pad last document (PLD).

Strategy	Tokens (B)	Fragm. Rate (%)	Padding Rate (%)	Model Loc (%)		GT Loc (%)		GT File (%)		AVG (%)
Strategy	Tokens (B)	Fragm. Rate (%)	Padding Rate (%)	sim $\uparrow$	empty $\downarrow$	sim $\uparrow$	empty $\downarrow$	sim $\uparrow$	empty $\downarrow$	sim $\uparrow$	empty $\downarrow$
concat-then-split	73	30.2	0.00	16.61	30.61	24.02	26.60	9.41	59.60	16.68	38.94
restart-last-document	73	17.8	0.00	17.92	25.92	23.51	27.00	10.28	54.60	17.24	35.84
pad-last-document	89	0.0	17.55	17.24	25.31	22.99	25.60	10.35	52.40	16.86	34.44
best-fit-packing	73	0.0	0.01	17.22	28.37	25.76	23.60	10.47	56.80	17.82	36.26
best-fit-packing w/ split	73	0.0	0.01	17.95	22.04	27.46	16.80	15.10	36.20	20.17	25.01

Appendix

Table 13: Ablation study on different sample packing strategies and how to tackle long documents exceeding model input length. Model Loc: use model predicted position for bug localization; GT Loc: use ground truth bug location; GT File: use the complete file containing bug or target solution.

\uparrow

means higher is better, while

\downarrow

means lower is better.

Model	CKT (Acc)	ATE (Acc)	RCM (Acc)	RMS (F1)	VSP (Acc)	TAA (Acc)
Claude-Opus-4-5	94.33	85.50	71.25	68.39	53.88	29.00
Claude-sonnet-4-5	93.22	78.67	64.00	61.69	45.17	18.67
Deepseek-V3.2	90.00	60.00	65.50	29.02	20.50	13.00
GLM-4.7	85.67	66.00	70.50	19.45	29.00	12.00
Ours	85.00	44.00	58.50	5.50	24.50	8.00

Appendix

Table 14: The ability of CTI analysis benchmarked by AthenaBench-Mini. We compute all the results with greedy decoding.

Model	Acc	Precision	Recall	F1	P-C $\downarrow$	P-V $\downarrow$	P-B $\downarrow$	P-R $\downarrow$
Claude-Opus-4-5	52.73	52.42	59.02	55.53	9.02	50.06	37.28	3.63
Claude-sonnet-4-5	52.57	51.44	91.80	65.93	8.52	83.38	4.93	3.17
Deepseek-V3.2	50.09	51.27	4.89	8.91	3.51	1.00	91.60	3.88
GLM-4.7	53.22	53.24	58.62	55.80	20.55	38.60	26.57	12.53
Ours	48.33	48.54	54.54	51.37	0.88	53.01	41.29	4.64

Appendix

Table 15: The function-level vulnerability detection capability evaluated by PrimeVul-Paired. We note that Pair-wise Correct Prediction (P-C), Pair-wise Vulnerable Prediction (P-V), Pair-wise Benign Prediction (P-B), and Pair-wise Reversed Prediction (P-R) are used to evaluate the model’s predictions on textually similar code pairs as single entities ( ) . We compute all the results with greedy decoding.

Model	SecCodeBench				CWEval
Model	Gen w/o Hint	Gen w/ Hint	Fix w/o Hint	Fix w/ Hint	func@1	func-sec@1
Claude-Opus-4-5	52.5	73.2	75.2	83.9	92.27	74.75
Claude-sonnet-4-5	43.4	57.6	62.2	76.4	91.55	71.72
Deepseek-V3.2	43.1	50.2	50.0	65.8	83.53	54.71
GLM-4.7	29.4	56.0	49.8	64.9	72.44	46.39
Ours	61.2	69.5	76.4	83.7	80.17	56.32

Appendix

Table 16: The secure coding capability measured by SecCodeBench and CWEval. We note that SecCodeBench scores are derived from pass@

k

computation with security severity weighting, with (and without) the hints that ask models to behave securely. In addition, func@1 and func-sec@1 are computed based on the definition of pass@

k

, with the random sampling of

n=10

and temperature of

0.8

Code Language Models

Code language models are AI models that generate, explain, edit, and reason about source code across programming tasks.

See all 113 papers

Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models
Jul 14, 2026Yubo Wang, Jiarong Liang, Yuxuan Zhang +5Masked Language Modeling Coding Agents
University of Waterloo · 5Vector Institute · University of British Columbia +2
KAT-Coder-V2 Technical Report
Mar 29, 2026Fengxiang Li, Han Zhang, Haoyang Huang +43AI Coding Agents Agentic Code Generation
FrogNano: Training a 4B Coding Agent via Online Task Synthesis
Sep 7, 2026Minseon Kim, Zhengyan Shi, Emiliano Penaloza +14RL for Code Generation Software Engineering Agents
Froggy Team – Microsoft Research Montréal

Jul 14, 2026J/K move · Enter open · S save

Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models

Yubo Wang, Jiarong Liang, Yuxuan Zhang, Xuye Liu+4

University of Waterloo · 5Vector Institute · University of British Columbia+2

Masked Language Modeling Coding Agents Code Language Models LLM Training

Open paper AskPDF arXiv

Coding agents must integrate external tool returns into ongoing reasoning - a capability that standard left-to-right pretraining on code exposes only in its forward direction. We observe that the action-observation-continuation loop of a coding agent is structurally isomorphic to a function call site, where a caller binds arguments, a callee returns a value computed elsewhere, and downstream code consumes that value. This conditioning structure exists at internet scale in ordinary code. We exploit it through function-aware fill-in-the-middle (FIM) mid-training: a self-supervised objective that masks functions selected via program dependency graph analysis and a complexity-inferability double criterion. We mid-train Qwen2.5-Coder-Instruct (7B/14B) and Qwen3-8B on a 2.6B-token decontaminated corpus drawn from 968 GitHub repositories, then apply existing agentic post-training pipelines. Mid-training improves SWE-Bench-Verified by +2.8/+3.0 at 7B/14B and by +3.2 on Qwen3-8B; SWE-Bench-Lite gains are +3.7/+4.0/+5.4 on the same models. The improvement holds across two post-training pipelines (R2E-Gym, SWE-Smith) and on a non-Qwen2.5 base (Qwen3-8B with SWE-Lego). Beyond in-domain gains, mid-training also mitigates the capability erosion that agentic post-training otherwise inflicts on non-agent coding (e.g., LiveCodeBench) and non-coding tool-use benchmarks (tau-bench, BFCL): although the mid-training corpus contains Python code only, the function-call inductive bias survives post-training and yields consistent gains.

Abstract

Code Language Models

Explore similar work

Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models

KAT-Coder-V2 Technical Report

FrogNano: Training a 4B Coding Agent via Online Task Synthesis