Adaptive Problem Generation via Symbolic Representations

Teresa Yeo; Myeongho Jeon; Dulaj Weerakoon; Rui Qiao; Alok Prakash; Armando Solar-Lezama; Archan Misra

cs.LGFeb 22, 2026

Adaptive Problem Generation via Symbolic Representations

Authors: Teresa Yeo, Myeongho Jeon, Dulaj Weerakoon, Rui Qiao, Alok Prakash, Armando Solar-Lezama, Archan Misra

Organizations: Singapore-MIT Alliance for Research and Technology · Korea Advanced Institute of Science and Technology · Massachusetts Institute of Technology · Singapore Management University

Abstract

We present a method for generating training data for reinforcement learning with verifiable rewards to improve small open-weights language models on mathematical tasks. Existing data generation approaches rely on open-loop pipelines and fixed modifications that do not adapt to the model's capabilities. Furthermore, they typically operate directly on word problems, limiting control over problem structure. To address this, we perform modifications in a symbolic problem space, representing each problem as a set of symbolic variables and constraints (e.g., via algebraic frameworks such as SymPy or SMT formulations). This representation enables precise control over problem structure, automatic generation of ground-truth solutions, and decouples mathematical reasoning from linguistic realization. We also show that this results in more diverse generations. To adapt the problem difficulty to the model, we introduce a closed-loop framework that learns modification strategies through prompt optimization in symbolic space. Experimental results demonstrate that both adaptive problem generation and symbolic representation modifications contribute to improving the model's math solving ability.

Figures & tables

Figure 1: Left: Symbolic Data Generation with Prompt Optimization. We generate training data for mathematical tasks by modifying problems in symbolic space rather than directly in natural language. Starting from an original word problem (top left), we convert it to a symbolic representation (top right). An optimized prompt (middle, italics)—learned through closed-loop feedback from the model’s performance—specifies how to modify the symbolic structure to generate appropriately challenging problems. We call these prompts Opt-Sym . The modified symbolic representation (bottom right) is then translated back to a new word problem (bottom left). The LLM responses guide prompt optimization (green loop) to produce problems that are challenging. This symbolic approach enables precise control over problem structure and automatic ground-truth generation. In addition to introducing coupled equations, nested substitutions, note how the new word problem also returns a different story line than the original, i.e., generations through symbolic representations returns in more diverse training data. See Section 4.3 for a more detailed analysis. Right: Training on data generated via Opt-Sym improves model performance. We show the performance from training on data from Opt-Sym (green) compared to training on the original seed data (grey). The y-axis shows the performance averaged over several benchmarks, and the x-axis shows the amount of seed data used. Opt-Sym achieves an 8% improvement in performance even with as little as 100 seed data points. This is compared to the baseline’s 4% improvement. This demonstrates both the effectiveness and data efficiency of our approach. See Tables 1 and 2 and Figure 3 for the full results and analysis.

Figure 2: Top: Data Generation Pipeline. We generate new math problems from seed examples using two approaches. Left (Natural Language (NL) representation): Given a seed problem $q\in\mathcal{D}_{\text{seed}}$ , we prompt LLM $M_{\text{gen}}$ to create a modified problem $q^{\prime}$ , then prompt it again to generate its solution. Right (Symbolic Representation): Given a seed problem $s\in\mathcal{D}_{\text{sym}}$ , prompt $M_{\text{gen}}$ to modify this representation to create $s^{\prime}$ , translate $s^{\prime}$ back to natural language to obtain $q^{\prime}$ , and use a solver to compute the answer. Both pipelines produce problem-answer pairs for training. Bottom: Examples of optimized prompts and generated problems . Each column shows a different optimized prompt $p_{\text{opt}}$ (top, truncated) applied to the same seed problem shown at the top. The resulting generated problem is show below. The blue text highlights the modifications requested by each prompt. These prompts are learned through closed-loop optimization (see Section 3.4 ) that takes student model performance as feedback, enabling them to target specific weaknesses. See Appendix D for complete optimized prompts across different settings. The optimized prompts for both natural language and symbolic representations suggest similar mathematical modifications (fractions, multi-step reasoning, additional variables), indicating some overlap in how problem difficulty should be increased for the student model. However, the actual generated problems exhibit distinct characteristics. The NL modifications produce problems with complex narratives that require careful interpretation, while symbolic modifications yield problems with explicit mathematical operations stated directly in the text. Thus, the space in which modifications are applied impacts the final problem structure and complexity.

Figure 3: Additional analysis. a. Data generation methods across subset sizes : The 1.5B model trained with Opt-Sym (green) outperforms the baseline approaches when trained on different data subsets. b. Data filtering impact : Filtering the generated training data improves performance on larger dataset sizes. This can be because for small datasets, filtering has a larger impact on data diversity, which can harm performance ( Setlur et al., 2024 ) . c. Prompt scaling : Performance increases monotonically with the number of prompts used during data generation. This indicates that having more diverse data helps improve performance (see Figure 4 for further analysis on data diversity). As we optimize for 4 prompts in total, there are

4\choose n

options for selecting each of the

n

prompts etc. This shows the averaged accuracy (%) and standard deviation over the different combinations. d. Representation sensitivity : The choice of mathematical problem representation significantly affects model performance.

Data Type	Iter. 0	Iter. 1	Iter. 2	Iter. 3
Base-Sym	51.40	52.54	52.58	54.30
Opt-Sym	51.40	53.90	54.71	56.26

Table 3: Averaged accuracy (%) across multiple iterations of generation and fine-tuning on the 1.5B model. Opt-Sym shows consistent improvement as iterations increase. Base-Sym results with the same number of prompts per iteration are shown for comparison.

Figure 4: Analysis of generated data . Left: Generation via symbolic representations produce more diverse data. For each seed problem, we generate 10 variants using baseline and optimized prompts ( $\mathcal{P}^{\text{NL}}$ and $\mathcal{P}^{\text{Sym}}$ ), compute their embeddings, and measure average pairwise cosine distance. Optimized prompts with symbolic representations (Sym) achieve substantially higher diversity than natural language (NL) variants. Right: Student models (1.5B and 3B) achieve lower accuracy on generated data compared to the seed data (dash-dot line), with optimized prompts producing the most challenging problems. Accuracy is similar across word-based and symbolic generation methods. The difficulty gaps from seed data suggests successful problem variations.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameters	GRPO	PPO
Training batch size	1024	1024
Samples per prompt	8	-
Max response length	1024	1024
Clip ratio	0.2	0.2
Scaling factor $\alpha$	1.0	-
Training temperature	1.0	1.0

Appendix

Table 4: Full hyperparameter setting.

Figure 5: Performance vs different subsets of data for different RL methods and student models.

Mathematical Problem Generation

See all 14 papers

Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesis
Nov 13, 2025Yongxian Wei, Yilin Zhao, Li Shen +7Synthetic Data Generation Synthetic Data Generation for Language Models
Tsinghua University · Tencent · Sun Yat-sen University +1
Verifier-Backed Hard Problem Generation for Mathematical Reasoning
May 7, 2026Yuhang Lai, Jiazhan Feng, Yee Whye Teh +1Mathematical Problem Generation Reinforcement Learning with Verifiable Rewards
Department of Data Science, City University of Hong Kong · School of Intelligence Science and Technology, Peking University · Work done during Jiazhan’s visiting at the Department of Statistics, University of Oxford. +2
Learn Hard Problems During RL with Reference Guided Fine-tuning
Mar 1, 2026Yangzhen Wu, Shanda Li, Zixin Wen +5LLM Mathematical Reasoning RL for Language Model Reasoning
UC Berkeley · Carnegie Mellon University · ByteDance Seed

Nov 13, 2025J/K move · Enter open · S save

Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesis

Yongxian Wei, Yilin Zhao, Li Shen, Zixuan Hu+6

Tsinghua University · Tencent · Sun Yat-sen University+1

Synthetic Data Generation Synthetic Data Generation for Language Models LLM Mathematical Reasoning

Open paper AskPDF arXiv

Data synthesis for training large reasoning models offers a scalable alternative to limited, human-curated datasets, enabling the creation of high-quality data. However, existing approaches face several challenges: (i) indiscriminate generation that ignores the solver's ability and yields low-value problems, or reliance on complex data pipelines to balance problem difficulty; and (ii) a lack of reasoning in problem generation, leading to shallow problem variants. In this paper, we develop a problem generator that reasons explicitly to plan problem directions before synthesis and adapts difficulty to the solver's ability. We recover problem-design CoT from human-authored multi-part questions, using later subquestions as targets to infer the design rationale. These traces initialize the generator, which is then optimized with a seed-conditioned reward based on solver answer consistency. The reward guides difficulty adjustment relative to each seed while favoring problems near the solver's competence boundary. Across 10 mathematical and general reasoning benchmarks, our method outperforms seed-only training by 2.69 and 3.39 percentage points after one and three rounds, respectively. The framework also extends to vision-language models. Our code is publicly available at https://github.com/WalkerWorldPeace/Reasoning-Synthesis.

Abstract

Mathematical Problem Generation

Explore similar work

Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesis

Verifier-Backed Hard Problem Generation for Mathematical Reasoning

Learn Hard Problems During RL with Reference Guided Fine-tuning