From Weeks to Hours: Fast and Principled SFT Data Generation for Reasoning LLMs
Organizations: University of California, Los Angeles · Optum AI
Abstract
Reasoning-oriented post-training enables large language models (LLMs) to solve complex tasks via multi-step inference. Supervised fine-tuning (SFT) on high-quality reasoning traces is a particularly efficient approach, but its effectiveness depends critically on careful data curation. Existing pipelines rely on a costly generate-then-filter paradigm: large pools of long reasoning traces are produced by multiple teacher models and subsequently filtered for difficulty and diversity using additional LLMs, often requiring weeks of computation. In this work, we propose SFTProbe, a fundamentally different approach that bypasses this pipeline by directly predicting the quality of reasoning data. We show that a brief LoRA-based adaptation, combined with evaluating loss on only 5% of each reasoning trace, suffices to estimate both difficulty and diversity. SFTProbe is grounded in a geometric view of post-training, where pretrained LLMs lie in a low-rank, anisotropic loss basin. By probing this basin via low-rank perturbations, we construct compact loss signatures whose clustering captures diversity, while loss at curvature inflection points provides a robust proxy for difficulty. We also provide a simple criterion for teacher selection. Empirically, our approach reduces data curation time from weeks to hours while maintaining or improving performance. Remarkably, on mathematical reasoning, 5k generated examples match the highly filtered 89k-example in OpenThoughts-Math dataset, and 7.5k generated examples outperforms it by 2.5% in accuracy. We further introduce a high-quality physics reasoning dataset, and use it to train a model that outperforms an IPhO 2025 silver-medal-winning model. The code and the dataset are released at https://bigml-cs-ucla.github.io/SFTProbe-page/.
Figures & tables
| Model | Average Score |
|---|---|
| Qwen2.5-7B-Instruct | 3.58 |
| Google/Gemma-4-26B-A4B-it | 9.65 |
| PRIME-RL/P1-30B-A3B [ 30 ] | 9.05 |
| SFTProbe (1k data generated) | 9.14 0.29 |
| SFTProbe (2k data generated) | 9.32 0.11 |
| SFTProbe (5k data generated) | 9.76 0.74 |
| Method | ID | OOD | Avg |
|---|---|---|---|
| (3 sets) | (7 sets) | (10 sets) | |
| [ 12 ] | 69.3% | 52.9% | 57.8% |
| SFTProbe | 68.7% | 53.1% | 57.8% |
| without Inflection search | 67.9% | 52.5% | 57.1% |
| without LoRA | 68.1% | 51.3% | 56.3% |
| Hyperparameter | ID | OOD | Avg |
|---|---|---|---|
| (3 sets) | (7 sets) | (10 sets) | |
| Lora Rank = 4, Train with 200 iters | 68.7% | 53.1% | 57.8% |
| Lora Rank = 2, Train with 200 iters | 68.5% | 52.6% | 57.3% |
| Lora Rank = 4, Train with 100 iters | 68.9% | 51.9% | 57.0% |
| Lora Rank = 4, Train with 400 iters | 69.0% | 52.0% | 57.1% |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Teacher | Avg. | AIME24 | AIME25 | AMC23 | MATH500 |
|---|---|---|---|---|---|
| Qwen/Qwen3.6-27B | |||||
| Openai/GPT-OSS-120B | |||||
| Google/Gemma-4-26B-A4B-IT | |||||
| Google/Gemma-4-31B-IT | |||||
| Microsoft/Phi-4-Reasoning-Plus | |||||
| Qwen/QwQ-32B |
| Teacher | Mean Loss | Avg. Finetune Acc. |
|---|---|---|
| Google/Gemma-4-26B-A4B-IT | ||
| Google/Gemma-4-31B-IT | ||
| Qwen/QwQ-32B | ||
| Qwen/Qwen3.6-27B | ||
| Openai/GPT-OSS-120B | ||
| Microsoft/Phi-4-Reasoning-Plus |
Training Data Selection
Training data selection studies how to choose, filter, weight, or schedule examples to improve learning while reducing data and compute costs.
See all 196 papers