Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesis

Yongxian Wei; Yilin Zhao; Li Shen; Zixuan Hu; Xinrui Chen; Runxi Cheng; Sinan Du; Hao Yu

cs.AINov 13, 2025

Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesis

Authors: Yongxian Wei, Yilin Zhao, Li Shen, Zixuan Hu, Xinrui Chen, Runxi Cheng, Sinan Du, Hao Yu,

Organizations: Tsinghua University · Tencent · Sun Yat-sen University · Nanyang Technological University

Abstract

Data synthesis for training large reasoning models offers a scalable alternative to limited, human-curated datasets, enabling the creation of high-quality data. However, existing approaches face several challenges: (i) indiscriminate generation that ignores the solver's ability and yields low-value problems, or reliance on complex data pipelines to balance problem difficulty; and (ii) a lack of reasoning in problem generation, leading to shallow problem variants. In this paper, we develop a problem generator that reasons explicitly to plan problem directions before synthesis and adapts difficulty to the solver's ability. We recover problem-design CoT from human-authored multi-part questions, using later subquestions as targets to infer the design rationale. These traces initialize the generator, which is then optimized with a seed-conditioned reward based on solver answer consistency. The reward guides difficulty adjustment relative to each seed while favoring problems near the solver's competence boundary. Across 10 mathematical and general reasoning benchmarks, our method outperforms seed-only training by 2.69 and 3.39 percentage points after one and three rounds, respectively. The framework also extends to vision-language models. Our code is publicly available at https://github.com/WalkerWorldPeace/Reasoning-Synthesis.

Figures & tables

Fig. 1: Training pipeline for our problem generator. A reasoning model recovers problem-design CoT from paired subquestions to initialize the generator. The generator synthesizes problems from seeds, and the solver answers each problem multiple times. Majority-answer consistency provides the feedback used to compute the seed-conditioned training reward.

Panel of Figure 2 — Fig. 2: Recover the latent problem-construction logic from paired sub-questions, using the earlier one to infer a rationale for designing the follow-up.

	MATH	GSM8K	AMC	Minerva	Olympiad	AIME24	AIME25	Avg.
Qwen3-4B-Base
Base Model	$66.40$	$70.49$	$44.68$	$17.71$	$31.01$	$9.38$	$7.22$	35.27
Seed Set	$79.60$	$92.79$	$56.35$	$31.73$	$39.76$	$14.52$	$12.02$	$46.68$
Self-Instruct [ 56 ]	$77.40$	$92.79$	$55.49$	$\bm{32.84}$	$37.69$	$12.17$	$7.40$	45.11
+ CoT-Self-Instruct [ 24 ]	$78.20$	$\bm{93.47}$	$56.00$	$\underline{32.47}$	$38.58$	$11.94$	$7.04$	45.39
+ RLMT [ 40 ]	$79.60$	$91.05$	$55.80$	$31.25$	$39.26$	$13.62$	$10.36$	$45.85$

TABLE I: Performance Comparison on Mathematical Reasoning Benchmarks. Pass@1 accuracy (%). AMC, AIME24, and AIME25 use the mean of 128 samples per problem (avg@128). Indented rows are variants of the method above (Sec. V-A3 ). Bold and underlined values are the best and second-best scores.

	Super GPQA	BBEH	MMLU- Pro	Avg.
Qwen3-4B-Base
Base Model	$20.88$	$7.57$	$37.38$	$21.94$
Seed Set	$29.74$	$11.22$	$52.88$	$31.28$
Self-Instruct	$30.07$	$11.13$	$56.79$	$32.66$
+ CoT-Self-Instruct	$30.35$	$11.50$	$57.86$	$33.24$
+ R-Zero	$31.27$	$11.19$	$58.79$	$33.75$

TABLE II: Performance Comparison on General-Domain Reasoning Benchmarks

	Math Vista	Math Vision	Math Verse	Avg.
Qwen2.5-VL-7B-Instruct
Base Model	$68.40$	$23.35$	$43.53$	$45.09$
Seed Set	$69.00$	$29.28$	$45.23$	$47.84$
Self-Instruct [ 56 ]	$69.80$	$29.60$	$46.40$	$48.60$
+ CoT-Self-Instruct [ 24 ]	$69.30$	$27.63$	$45.58$	$47.50$
+ R-Zero [ 38 ]	$69.30$	$26.64$	$44.72$	$46.89$

TABLE III: Performance Comparison on Multimodal Reasoning Benchmarks

	MATH	GSM8K	AMC	Minerva	Olympiad	AIME24	AIME25	Avg.
Qwen3-4B-Base
Base Model	$66.40$	$70.49$	$44.68$	$17.71$	$31.01$	$9.38$	$7.22$	35.27
Seed Set	$79.60$	$92.79$	$56.35$	$31.73$	$39.76$	$14.52$	$12.02$	$46.68$
Self-Instruct [ 56 ]	$79.60$	$92.57$	$55.16$	$31.99$	$38.07$	$14.53$	$13.96$	$46.55$
+ CoT-Self-Instruct [ 24 ]	$78.60$	$\bm{93.63}$	$55.53$	$\bm{33.82}$	$37.04$	$10.78$	$11.04$	$45.78$
+ R-Zero [ 38 ]	$80.80$	$92.80$	$60.00$	$30.51$	44.44	17.29	$14.79$	$48.66$

TABLE IV: Controlled Comparison with a Fixed Training-Set Size. Pass@1 accuracy (%), with avg@128 on AMC, AIME24, and AIME25. Each method uses the same 4,000 seeds plus 4,000 retained synthetic problems. Indented rows are variants of the method above (Sec. V-A3 ). Bold and underlined values are the best and second-best scores.

Method	Comparison setting	Avg. (%)
MetaMath	Seed-conditioned rewriting	46.80
ScaleDiff	Released-data utility	47.30
Ours	Seed-conditioned generation	49.90

TABLE V: Comparison with Mathematical Data Synthesis. Macro-average accuracy over seven mathematical benchmarks with the same solver and training-set size.

Reward variant	Avg.
Boundary only (R-Zero)	45.17
Inversion only	$47.61$
Full reward (Ours)	49.24

TABLE VI: Reward Ablation on Seven Mathematical Benchmarks

Fig. 3: Contour density plot of solver consistency and accuracy on MATH. The dashed red line shows the best-fit linear regression, and the yellow dotted line denotes the identity reference ( $y=x$ ). Pearson $r=0.887$ indicates a strong positive relationship.

Checkpoint	Pearson $r$	Spearman $\rho$	Confidently wrong (%)	Invalid / unsolvable (%)	Parseable answers (%)
Step 0	0.78	0.81	2.1	6.4	86
Step 5	0.74	0.79	2.6	7.1	85
Step 10	0.76	0.80	2.4	6.8	84
Step 20	0.73	0.77	2.9	7.5	83

TABLE VII: Reliability of Solver Consistency during Generator RL. Confidently wrong denotes consistency

\geq 0.8

with a majority answer that does not match the verifier’s reference answer. Parseable answers are responses with an extractable final answer.

Fig. 4: Training dynamics across iterations : the problem generator’s reward score, flip success rates for the original and new problems, and changes in difficulty for the original and new problems.

Panel of Figure 12 — Fig. 5: Case study under high solver feedback: increased difficulty. Responses to the line-intersection problem.

Method	Well-posed $\uparrow$	Solvable $\uparrow$	Ambiguous $\downarrow$	Seed near- duplicate $\downarrow$	Strategy changed $\uparrow$	Label–verifier agreement $\uparrow$
CoT-Self-Instruct	82	80	8	21	19	83
R-Zero	79	76	9	18	22	81
Ours	86	84	5	4	31	88

TABLE VIII: Quality Audit of Generated Problems. All values are percentages. Each method contributes 100 candidate problems sampled from the same seed pool.

Labeler pair	Retention agreement	Answer agreement
o3-mini vs. DeepSeek Pro	80.0	71.9
o3-mini vs. DeepSeek Flash	79.3	74.5
DeepSeek Pro vs. DeepSeek Flash	87.3	91.8

TABLE IX: Agreement between Labeling Models. All values are percentages on the same generated problems. Retention agreement compares decisions to retain or discard a problem; answer agreement compares normalized final answers.

Qwen3-4B-Base
	MATH	GSM8K	AMC	Minerva	Olympiad	AIME24	AIME25	Avg.
MATH Seed Set	$79.60$	$92.79$	$56.35$	$31.73$	$39.76$	$14.52$	$12.02$	46.68
MATH Augmentation	$82.00$	$92.87$	$60.52$	$32.84$	$44.36$	$17.44$	$14.65$	49.24 (+2.56 pp)
GSM8K Seed Set	$79.60$	$92.72$	$54.93$	$32.47$	$38.28$	$13.16$	$11.03$	46.03
GSM8K Augmentation	$78.40$	$93.55$	$59.38$	$32.10$	$41.84$	$13.14$	$13.71$	47.45 (+1.42 pp)
DAPO Seed Set	$76.00$	$91.88$	$53.82$	$29.52$	$36.94$	$9.78$	$6.80$	43.53

TABLE X: Generalization across Different Seed Sets. “Seed Set” trains the solver on the original sampled problems only. “Augmentation” expands the seed set with synthetic problems generated by our MATH-trained generator. We report pass@1 accuracy (%) on the seven mathematical reasoning benchmarks.

Fig. 6: Performance comparison across iterations on all ten benchmarks. The solver improves as the generator and solver are trained alternately.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Qwen3-4B-Base
	MATH	GSM8K	AMC	Minerva	Olympiad	AIME24	AIME25	SuperGPQA	BBEH	MMLU-Pro	Avg.
Iteration 1	$82.00$	$92.87$	$60.52$	$32.84$	$44.36$	$17.44$	$14.65$	$31.33$	$11.35$	$\bm{60.11}$	44.75
Iteration 2	$81.60$	$92.56$	$64.29$	$\bm{34.69}$	$\bm{45.25}$	$17.00$	$14.65$	$\bm{31.90}$	$11.81$	$59.52$	45.33
Iteration 3	$\bm{82.40}$	$\bm{93.25}$	$\bm{64.65}$	$32.10$	$44.51$	$\bm{18.67}$	$\bm{15.04}$	$31.86$	$\bm{12.27}$	$59.73$	45.45

Appendix

TABLE XI: Performance across Co-evolutionary Iterations on Ten Benchmarks. These scores correspond to the radar chart in Fig. 6 . The generator and solver are trained alternately in successive rounds.

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 609 papers

MathAgent: Adversarial Evolution of Constraint Graphs for Mathematical Reasoning Data Synthesis
Apr 13, 2026Zixiong Yu, Jun Rao, Guhan Chen +5Synthetic Data Generation Synthetic Data Generation for Language Models
Huawei Large Model Data Technology Lab · Tsinghua University · Harbin Institute of Technology, Shenzhen +1
MindLoom: Composing Thought Modes for Frontier-Level Reasoning Data Synthesis
May 20, 2026Haiyang Shen, Taian Guo, Xuanzhong Chen +11Synthetic Data Generation Structured Reasoning
1Peking University · 2Tsinghua University
Recursive Harness Self-Improvement for Frontier Reasoning Data Synthesis
Oct 2, 2026Wenlong Zhang, Zhengbo Jiao, Chenxu Zhang +5Synthetic Data Generation for Language Models Recursive Self-Improvement
EPIC Lab, Shanghai Jiao Tong University · University of Science and Technology of China · Shanghai University of Finance and Economics +5

Apr 13, 2026J/K move · Enter open · S save

MathAgent: Adversarial Evolution of Constraint Graphs for Mathematical Reasoning Data Synthesis

Zixiong Yu, Jun Rao, Guhan Chen, Songtao Tian+4

Huawei Large Model Data Technology Lab · Tsinghua University · Harbin Institute of Technology, Shenzhen+1

Synthetic Data Generation Synthetic Data Generation for Language Models LLM Mathematical Reasoning

Open paper AskPDF arXiv

Synthesizing high-quality mathematical reasoning data without human priors remains a significant challenge. Current approaches typically rely on seed data mutation or simple prompt engineering, often suffering from mode collapse and limited logical complexity. This paper proposes a hierarchical synthesis framework that formulates data synthesis as an unsupervised optimization problem over a constraint graph followed by semantic instantiation, rather than treating it as a direct text generation task. We introduce a Legislator-Executor paradigm: The Legislator adversarially evolves structured generation blueprints encoding the constraints of the problem, while the Executor instantiates these specifications into diverse natural language scenarios. This decoupling of skeleton design from linguistic realization enables a prioritized focus on constructing complex and diverse logical structures, thereby guiding high-quality data synthesis. Experiments conducted on a total of 10 models across the Qwen, Llama, Mistral, and Gemma series demonstrate that our method achieves notable results: models fine-tuned on 1K synthesized samples outperform widely-used datasets of comparable scale (LIMO, s1K) across eight mathematical benchmarks, exhibiting superior out-of-distribution generalization.

Abstract

Synthetic Data Generation

Explore similar work

MathAgent: Adversarial Evolution of Constraint Graphs for Mathematical Reasoning Data Synthesis

MindLoom: Composing Thought Modes for Frontier-Level Reasoning Data Synthesis

Recursive Harness Self-Improvement for Frontier Reasoning Data Synthesis