Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems

Zongqian Li; Tengchao Lv; Shaohan Huang; Yixuan Su; Qinzheng Sun; Qiufeng Yin; Ying Xin; Scarlett Li

cs.CLMar 8, 2026

Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems

Authors: Zongqian Li, Tengchao Lv, Shaohan Huang, Yixuan Su, Qinzheng Sun, Qiufeng Yin, Ying Xin, Scarlett Li,

Organizations: Microsoft Research · University of Cambridge

Abstract

Training next-generation code generation models requires high-quality datasets, yet existing datasets face difficulty imbalance, format inconsistency, and data quality problems. We address these challenges through systematic data processing and difficulty scaling. We introduce a four-stage Data Processing Framework encompassing collection, processing, filtering, and verification, incorporating Automatic Difficulty Filtering via an LLM-based predict-calibrate-select framework that leverages multi-dimensional difficulty metrics across five weighted dimensions to retain challenging problems while removing simplistic ones. The resulting MicroCoder dataset comprises tens of thousands of curated real competitive programming problems from diverse platforms, emphasizing recency and difficulty. Evaluations on strictly unseen LiveCodeBench demonstrate that MicroCoder achieves 3x larger performance gains within 300 training steps compared to widely-used baseline datasets of comparable size, with consistent advantages under both GRPO and its variant training algorithms. The MicroCoder dataset delivers obvious improvements on medium and hard problems across different model sizes, achieving up to 17.2% relative gains in overall performance where model capabilities are most stretched. These results validate that difficulty-aware data curation improves model performance on challenging tasks, providing multiple insights for dataset creation in code generation.

Figures & tables

Figure 1: Data Processing Framework Overview. Top: Four-stage pipeline transforming raw datasets into high-quality training data through data collection, standardization processing, quality filtering, correctness verification, and final dataset creation. Middle: Standardization steps including translation of foreign language problems to English, noise removal and refinement, format prompt unification, and LLM-based test case generation and selection. Bottom: Multi-stage filtering implementing hard requirements (text-only problems via content-type distinction, uniqueness via overlap identification, unseen status via test set comparison) and adaptive selection (difficulty-based filtering using multidimensional difficulty matrix with LLM to distinguish easy and difficult problems), producing the filtered dataset for verification.

Figure 2: Multi-dimensional Difficulty Metrics Framework. Top: Five-dimensional assessment system where each dimension is evaluated on a 1-5 scale with specific descriptors defining evaluation criteria. Bottom Left: Example of scoring a question with three independent GPT-4O assessments. Each assessment rates all dimensions (for example, Assessment 1: PCD=4, KBR=2, ATC=3, ID=3, OD=3) and applies weights (ATC 45%, ID 35%, OD 10%, PCD 5%, KBR 5%) to yield individual scores (3, 3, 3.1), which average to a final difficulty score of 3.03. Bottom Right: Distribution curve showing that the difficulty score of the example problem (3.03) positions it in the top 30% of the dataset, indicating a relatively challenging problem.

Figure 3: Difficulty Filtering Case Study on LiveCodeBench v6. Left: Actual difficulty distribution based on model pass rates. Middle Left: GPT-4O predicted difficulty calibrated against ground truth to determine boundaries at 2.5 (easy/medium) and 2.75 (medium/hard), achieving near-perfect distribution with actual distribution and enabling accurate difficulty-based filtering for subsequent data. Middle Right: Recall curves showing the proportion of each difficulty category removed, where high easy recall (more than 60% at 30% filtering) versus low difficult recall demonstrates selective removal of simple problems. Right: Difficulty composition of retained problems, where filtering 30% of data changes the distribution from 45% easy to under 25% easy while increasing difficult problem proportion to over 50%, confirming effective difficulty-based selection.

Figure 4: Dataset Composition and Processing Flow. Sankey diagram illustrating the complete data pipeline from diverse sources to final difficulty categorization. The dataset comprises exclusively real competitive programming problems without generated content. Open-source contributions undergo substantial filtering, with most problems abandoned and primarily difficult problems retained. Private collections contribute the majority of challenging and recent problems across platforms, resulting in a challenging and high-quality dataset.

Figure 5: Dataset Statistical Characteristics and Diversity Analysis. Left: Problem length distribution across platforms, with privately collected datasets (AIZU, AtCoder) showing comparable length profiles to open-source datasets (Kattis, CodeChef), where AIZU exhibits the longest average problem length. Middle Left: Test case distribution where privately collected datasets exhibit more test cases per problem, with all datasets capped at 15 longest test cases following DeepCoder to prevent training pause from problems with hundreds of lengthy test cases. Middle Right: t-SNE clustering visualization demonstrating clear platform separation and complementary coverage between privately collected and open-source datasets. Right: Cosine similarity table where training datasets (y-axis) show low similarity (0.04-0.14) against test benchmarks (x-axis: AtCoder, LeetCode, LiveCodeBench), confirming no overlap between training and test sets.

Figure 6: Training Dynamics Comparison Between DeepCoder and MicroCoder Datasets. Training curves under GRPO (left) and DAPO (right) algorithms, with accuracy on LiveCodeBench v6 test set and critic reward on training set. Darker lines represent smoothed curves over lighter raw training data. MicroCoder consistently achieves higher test accuracy, demonstrating stronger capability improvement for code generation models. DAPO, which removes KL loss and employs high clipping to encourage diverse solutions, yields better performance and more stable training than GRPO. MicroCoder exhibits lower training rewards than DeepCoder, indicating higher difficulty, yet achieves greater test improvement at comparable training progress, validating the effectiveness of challenging problems for model generalization.

Dataset Comparison
	AtCoder				LeetCode				LiveCodeBench
	Easy	Medium	Hard	All	Easy	Medium	Hard	All	Easy	Medium	Hard	All
GRPO, Qwen3 4B, 8K
APPS	96.2	36.5	11.3	36.8	79.4	14.4	1.3	27.8	89.5	25.5	8.7	33.6
DeepCoder	96.2	40.4	10.4	37.3	73.5	19.2	0.0	27.8	87.2	29.8	7.8	33.9
MicroCoder (Ours)	97.1	47.1	11.3	39.5	76.5	18.3	3.7	29.4	89.0	32.7	9.4	35.9

Table 1: Performance Evaluation Across Model Scales and Dataset Sources. Performance evaluation across different model scales using GRPO and its variants. Delta represents absolute improvement, Delta% represents relative improvement, with green indicating gains, red indicating decrease, and bold indicating the best performance. MicroCoder consistently outperforms DeepCoder across all benchmarks and difficulty levels, achieving improvements on medium and hard problems, such as +3.7 points on LeetCode Hard under GRPO and +9.7 points (+40.4%) on LeetCode Medium under DAPO, demonstrating that difficulty-based filtering achieves performance gains on challenging problems. Additional evaluations across model scales and data sources demonstrate that MicroCoder’s advantages increase with larger models, and its advantage stems from incorporating recent, challenging problems rather than merely filtering existing data.

Training Data Curation

Training data curation improves the quality, diversity, organization, and reliability of datasets used to train machine-learning models.

See all 155 papers

BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution
May 31, 2026Yangzhen Wu, Aaron J. Li, Wenjie Ma +10Synthetic Benchmark Generation Code Generation Evaluation
University of California, Berkeley · Institute for Interdisciplinary Information Sciences, Tsinghua University
CodeWalk: Generating Coding Benchmarks by Walking a Problem Graph
Oct 11, 2026Jiaxin Fang, Runyuan He, Arnav Gupta +3Synthetic Benchmark Generation Benchmark Design
University of California, Berkeley
Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
Mar 8, 2026Zongqian Li, Shaohan Huang, Zewen Chi +5RL for Code Generation LLM Code Generation
Microsoft Research · University of Cambridge · Princeton University

May 31, 2026J/K move · Enter open · S save

BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution

Yangzhen Wu, Aaron J. Li, Wenjie Ma, Li Cao+9

University of California, Berkeley · Institute for Interdisciplinary Information Sciences, Tsinghua University

Synthetic Benchmark Generation Code Generation Evaluation Code Generation Benchmarks

Open paper AskPDF arXiv

The rapid progress of frontier large language models has led to widespread benchmark saturation, limiting the ability of existing datasets to differentiate model capabilities or provide useful training signal. For instance, on LiveCodeBench, frontier models achieve over 99% Pass@1 on easy splits and exceed 90% Pass@1 on average across difficulty levels. Constructing new, challenging datasets typically requires substantial human effort, creating a bottleneck for progress. We introduce BenchEvolver, a solution-centric evolutionary framework that automatically transforms existing coding problems into harder variants. Rather than generating problems from scratch, BenchEvolver evolves reference solutions through structured transformations and derives corresponding statements and tests from the evolved solutions. This design grounds generation in executable semantics, enabling scalable construction of high-quality, diverse, and difficult tasks with verifiable correctness. Applying BenchEvolver to LiveCodeBench and SciCode, we obtain evolved tasks that are substantially harder while maintaining validity, reference correctness, and diversity. We further curate LiveCodeBench-Plus, a 91-problem benchmark combining evolved and difficult original LCB-v6 tasks, where frontier-model Pass@1 ranges from 27.5% to 62.6%, restoring clear discrimination among strong coding models. Importantly, evolved tasks remain challenging even for the model that generates them, enabling self-improvement. We further show that RL on evolved LCB tasks improves held-out coding performance: for gpt-oss-20b, seed+evolved training achieves +8.7 and +8.3 Pass@1 gains on LCB v6 Hard and LCB-Pro Easy, exceeding seed-only gains by 70.7% and 34.8%, respectively. Our results show that BenchEvolver can convert saturated benchmarks into frontier-level evaluation suites and reusable training signal.

Abstract

Training Data Curation

Explore similar work

BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution

CodeWalk: Generating Coding Benchmarks by Walking a Problem Graph

Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models