Code2Math: Can Your Code Agent Evolve Math Problems Through Exploration?

Dadi Guo; Yuejin Xie; Qingyu Liu; Weixian Huang; Jiayu Liu; Zhiyuan Fan; Qihan Ren; Shuai Shao

cs.CLMar 3, 2026

Code2Math: Can Your Code Agent Evolve Math Problems Through Exploration?

Authors: Dadi Guo, Yuejin Xie, Qingyu Liu, Weixian Huang, Jiayu Liu, Zhiyuan Fan, Qihan Ren, Shuai Shao, +6 more

Organizations: Hong Kong University of Science and Technology · Tsinghua University · Zhejiang University · Nanjing Tech University · Shanghai Jiao Tong University · University of Michigan · Independent Researcher

Abstract

As large language models (LLMs) advance their mathematical capabilities toward the IMO and research level, the scarcity of challenging, high-quality problems has become a significant bottleneck for training, evaluation and self-evolution of LLMs. Simultaneously, recent code agents have demonstrated sophisticated skills in agentic coding and reasoning, suggesting that code execution can serve as a scalable environment for mathematical experimentation. In this paper, we investigate the potential of code agents to autonomously evolve existing math problems into more complex variations. We introduce a multi-agent framework designed to perform problem evolution while validating the solvability and increased difficulty of the generated problems. Our experiments demonstrate that, given sufficient test-time exploration, code agents can synthesize new, solvable problems that are structurally distinct from and more challenging than the originals. This work provides empirical evidence that code-driven agents can serve as a viable mechanism for synthesizing high-difficulty mathematical reasoning problems within scalable computational environments. Code and data is available at https://github.com/TarferSoul/Code2Math.

Figures & tables

Figure 1: Example of code-driven problem evolution.

Figure 2: Overview of our multi-agent system. Our pipeline consists of three components: the Evolution Agent , the Solvability Verification Agent , and the Difficulty Verification Agent . It is equipped with code tools related to mathematics. The framework takes an original problem and its solution as input and outputs a validated new problem along with a solution for reference.

Solver model in evaluation phase		DeepSeek-Chat			DeepSeek-Reasoner			Gemini-3-Pro-Thinking			Kimi-K2-Thinking			Seed-2.0-Pro
Solver model in evaluation phase		AR	Origin-SR	Evolution-SR	AR	Origin-SR	Evolution-SR	AR	Origin-SR	Evolution-SR	AR	Origin-SR	Evolution-SR	AR	Origin-SR	Evolution-SR
Open- source	DeepSeek-Chat	83/94	16	19 $+3$	94/98	16	9 $-7$	98/98	16	7 $-9$	74/90	16	11 $-5$	83/97	16	9 $-7$
	DeepSeek-Reasoner	83/94	48	34 $-14$	94/98	48	36 $-12$	98/98	48	8 $-40$	74/90	48	39 $-9$	83/97	48	26 $-22$
	Qwen3-235B-A22B-Thinking	83/94	20	15 $-5$	94/98	20	19 $-1$	98/98	20	18 $-2$	74/90	20	14 $-6$	83/97	20	11 $-9$
Close- source	Gemini-3-Flash-Thinking	83/94	56	33 $-23$	94/98	56	35 $-21$	98/98	56	24 $-32$	74/90	56	28 $-28$	83/97	56	24 $-32$
	GPT-5.2-Medium	83/94	70	59 $-11$	94/98	70	55 $-15$	98/98	70	52 $-18$	74/90	70	58 $-12$	83/97	70	51 $-19$

Table 1: Cross-model evaluation of problem evolution effectiveness. Column groups indicate the LLM used in the evolutionary phase, while rows indicate the solver model used in the evaluation phase. Each AR cell lists the number of evolved problems certified by the external judge, followed by the number that passed internal verification. Origin-SR (%) and Evolution-SR (%) represent the solve rates on the original seed problems and the evolved problems, respectively. The difference ( Evolution-SR

-

Origin-SR ) measures the extent of problem evolution, where a smaller (more negative) value indicates a greater increase in difficulty introduced by the evolutionary process.

Figure 3: Distribution of Average Token Consumption ( ATC ) across original and agent-evolved problems. For each problem, we compute the average output tokens across all solver models. Timeout samples are assigned the maximum token limit.

	Qwen3-8B			Qwen3-14B
Benchmark	Original	Orig.+Evol.	$\Delta$	Original	Orig.+Evol.	$\Delta$
AIME24	42.3	48.0	+5.7	60.7	66.7	+6.0
AIME25	36.7	38.0	+1.3	48.0	51.3	+3.3
AIME26	40.7	47.3	+6.6	58.0	62.7	+4.7
Average	39.9	44.4	+4.5	55.6	60.2	+4.7

Table 2: Training utility of evolved problems. We report Avg@10 accuracy (%) at epoch 6 on three AIME-series benchmarks. Original denotes SFT on the original problems only, while Original + Evolved augments the same training set with problems generated by Code2Math.

Model	Total	Solvability	Difficulty
DeepSeek-Chat	4.11	2.65	1.46
DeepSeek-Reasoner	3.10	2.08	1.02
Gemini-3-Pro-Preview-Thinking	1.56	0.46	1.10
Seed-2.0-Pro	2.57	0.78	1.79
Kimi-K2-Thinking	6.55	5.71	0.84

Table 3: Average failure counts during problem evolution, decomposed into solvability- and difficulty-verification failures.

Figure 4: Efficiency Analysis of Agentic Problem Evolution. We visualize the distribution of failure counts encountered during the evolutionary process across three base models: DeepSeek-Chat, DeepSeek-Reasoner, and Gemini-3-Pro-Preview-Thinking.

Audit item	Metric	Result
Validity	Human-valid rate	126/127 = 99.2%
Solver correctness	Human–GPT agreement	1134/1226 = 92.5%
Difficulty score	Exact agreement	115/145 = 79.3%
Difficulty score	Within-one agreement	139/145 = 95.9%
IMO/CMO subset	Mean score, Human/GPT	4.03/3.80
IMO/CMO subset	Score $\geq 3$ , Human/GPT	34/35; 34/35

Table 4: Human audit summary for GPT-5.2-High judgments.

Solver	Rate	Evolver	Rate	Diff.
DS-Chat	74.2%	Gemini-Pro	61.8%	4.55
Qwen-235B	64.3%	Seed	55.7%	3.41
DS-Reasoner	55.2%	DS-Reasoner	50.7%	3.62
Gemini-Flash	55.2%	DS-Chat	41.4%	3.46
GPT-5.2-M	40.0%	Kimi-K2	33.9%	3.77
GPT-5.2-H	37.3%	Overall	49.1%	3.93

Table 5: Failure after evolution on human-valid problems.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Audit type	Scope	Metric	Result
Agreement with human judgments
Validity	GPT-valid evolved problems	Human-valid rate	126 / 127 = 99.2%
Solver correctness	Balanced solver responses	Human–GPT agreement	1134 / 1226 = 92.5%
Difficulty score	Clean evolved problems	Exact agreement	115 / 145 = 79.3%
Difficulty score	Clean evolved problems	Within-one agreement	139 / 145 = 95.9%
Difficulty score	Clean evolved problems	MAE / RMSE	0.262 / 0.632

Appendix

Table 7: Full human audit of GPT-5.2-High judgments. The upper block measures agreement between human annotators and GPT-5.2-High; the lower block focuses on difficulty improvement for IMO/CMO-source problems.

Explore similar work

ResearchMath-14K: Scaling Research-Level Mathematics via Agents
May 27, 2026Guijin Son, Seungyeop Yi, Minju Gwak +3Research-Level Mathematics Mathematics
Seoul National University · OneLineAI · Yonsei University
ReverseMath: Answer Inversion for Scalable and Verifiable Mathematical Problem Generation
May 26, 2026Raoyuan Zhao, Yihong Liu, Yupei Du +2Mathematical Reasoning Benchmarks Reverse
Center for Information and Language Processing, LMU Munich · 2Munich Center for Machine Learning (MCML) · 3Saarland University
ProofCouncil: An LLM Agent for Solving Open Mathematical Problems
Jul 10, 2026Johannes Schmitt, Tim Gehrunger, Jasper Dekoninck +4Open Problems Large Language Model Agents
ETH Zurich · Aarhus University · Independent Researcher +1

May 27, 2026, cs.CLJ/K move · Enter open · S save

ResearchMath-14K: Scaling Research-Level Mathematics via Agents

Guijin Son, Seungyeop Yi, Minju Gwak, Hyunwoo Ko+2

Seoul National University · OneLineAI · Yonsei University

Research-Level Mathematics Mathematics Scaling

Open paper AskPDF arXiv

The frontier of mathematics is defined by problems whose solutions are not yet known. However, whether language models can meaningfully engage with such problems without human intervention remains unclear. A major obstacle is the lack of large-scale research-level math datasets. To this end, we introduce ResearchMath-14k, a set of

14{,}056

problems curated from academic sources via a multi-agent pipeline. ResearchMath-14k spans 11 mathematical domains and ranks above existing math datasets on knowledge, novelty, and procedural difficulty. To our knowledge, it is the largest research-level mathematical problem set available for training. We additionally generate

220

K teacher trajectories through targeted prompting, followed by behavioral filtering. Notably, however, generating correct trajectories is nontrivial at this level, and two LLM judges label only

3.7\%

and

4.3\%

of sampled ResearchMath training trajectories as correct. Nevertheless, across three model families, full-parameter training on ResearchMath improves performance on graduate- and research-level mathematics benchmarks by

2.1

points over the starting checkpoints. In comparison, training on existing datasets such as DASD and Nemotron-SFT-Math-v4 changes performance by

0.0

and

-0.5

points, respectively. Notably, mixing DASD with ResearchMath yields higher scores than token-matched DASD alone on benchmarks covering olympiad short-form (

+2.0

), graduate- and research-level short-form (

+0.8

), graduate- and research-level symbolic (

+2.6

), and proof evaluation (

+5.7

). Further analysis suggests that research-level mathematical content and greater reasoning diversity may help explain why ResearchMath provides complementary supervision to contemporary datasets. We make ResearchMath-14k publicly available for future works on research-level mathematical reasoning.

Abstract

Explore similar work

ResearchMath-14K: Scaling Research-Level Mathematics via Agents

ReverseMath: Answer Inversion for Scalable and Verifiable Mathematical Problem Generation

ProofCouncil: An LLM Agent for Solving Open Mathematical Problems