cs.AIFeb 22, 2026
SaveAsking the Right Questions: Improving Reasoning with Generated Stepping Stones
Organizations: FAIR at Meta · Stanford University · University of Oxford
Abstract
Recent years have witnessed tremendous progress in enabling LLMs to solve complex reasoning tasks such as math and coding. As we start to apply LLMs to harder tasks that they may not be able to solve in one shot, it is worth paying attention to their ability to construct intermediate stepping stones that prepare them to better solve the tasks. Examples of stepping stones include simplifications, alternative framings, or subproblems. We study properties and benefits of stepping stones in the context of modern reasoning LLMs via ARQ (Asking the Right Questions), a simple framework that introduces a question generator to the default reasoning pipeline. We first show that good stepping stone questions exist and are transferrable, meaning that good questions can be generated, and they substantially help LLMs of various capabilities in solving the target tasks. We next frame stepping stone generation as a post-training task and show that we can fine-tune LLMs to generate more useful stepping stones by SFT and RL on synthetic data.
Figures & tables
Figure 1: Illustration of ARQ. Instead of directly solving a task that may be too hard for a given solver (top), ARQ (bottom) adds a question-asking step that prompts or trains LLMs to generate stepping stone questions which, once solved, provide guidance or inspiration for the original problem. The LLMs used to generate stepping stone questions are referred to as stepping stone generators . In this example, the stepping stone (generated by an LLM) focuses on a special case of the original problem.
| Solver only | Analogical | Least-to-Most | Plan-and-Solve | ARQ | MajVote | Self-Refine |
| 69.8% | 58.6% | 64.2% | 60.7% | 72.8% | 70.0% | 73.2% |
Table 1: Performance of running various test-time scaffold on BeyondAIME (100 questions) with GPT-OSS-120B (high reasoning effort). We run method 20 times on each problem. Solver only directly uses the LLM to solve the target question. The methods in the middle, including ARQ, share the similar idea of generating intermediate steps, either as analogical questions ( Analogical ), decomposition ( Least-to-Most ), or plan ( Plan-and-Solve ). On the right, we have two methods that directly improve the solver . We run MajVote with 3 parallel rollouts to match the number of rollouts of ARQ and also run Self-Refine , a well-established paradigm where the LLM can iteratively refine its previous solution. ARQ outperforms existing techniques focusing on introducing intermediate steps but slightly underperforms the mature Self-Refine method on existing LLMs.
Figure 2: Scores of individual stones on selected questions from AIME 2024. Each point represents the score of the target problem conditioned on a generated stone, and the short horizontal bar indicates the average score over all stepping stones. The horizontal dashed line is the average performance of the Solver only baseline. ARQ generates stepping stones of varying quality. The best ones are highly beneficial, but some are detrimental and reduce the average improvement. ARQ with the more capable reasoning LLM is able to generate better stones, both in terms of the best and the average.
Figure 5: Performance of ARQ with Qwen3-8B and Qwen2.5-32B before and after fine-tuning (FT). Our fine-tuning procedure noticeably improves their performance as stone generators for different solvers.
Figure 6: Performance of ARQ with multiple stones. We run ARQ with GPT-120B high (left), fine-tuned Qwen3-8B (middle), and fine-tuned Qwen2.5-32B (right) as stone generators. Results are averaged over the AIME 2024, AIME 2025, and BeyondAIME benchmarks. Each stone generator is asked to produce 20 sets of either 2 or 3 stepping stones, generated either sequentially or recursively . Best denotes performance conditioned on the best-performing set of stones, while Avg denotes performance averaged over the 20 sets. Sequentially generated stepping stones continuously improve both best-case and average-case performance. Results with 0 stepping stones correspond to the solver only in the previous sections.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Scale Reduction | Constraint Simplification | Generalization | Intermediate Step | Method Practice |
| ARQ high | 68 | 13 | 8 | 8 | 3 |
| ARQ low | 62 | 22 | 8 | 5 | 3 |
Table 2: Distribution of the best stepping stones for 100 target problems in BeyondAIME among five categories.
| No Maj | Maj@20 | Gain | |
| Solver only | 69.8 | 76.0 | +6.2 |
| ARQ high | 72.8 | 79.0 | +6.2 |
| ARQ low | 72.7 | 77.0 | +4.3 |
Table 3: The performance of ARQ and solver only baseline when combined with major voting.
Figure 7: Score of individual stones on all questions from AIME 2024, a complete version of Figure 2 . The solver is GPT-120B with low reasoning effort for all methods.
Figure 8: Score of individual stones on all questions from AIME 2025. The solver is GPT-120B with low reasoning effort for all methods.
| Dataset Size | Mean Prompt Length | Mean CoT Length | Mean Stepping Stone Length | |
| SFT | 918 | 184.25 | 4034.45 | 66.24 |
| DPO | 1063 | 193.93 | 4911.18 | 74.54 |
Table 4: The statistics of the SFT dataset and the DPO dataset in our experiments.
Figure 9: The distribution of the CoT length in our constructed SFT dataset (left) and DPO dataset (right).
| Hyper-parameter | Value |
| max_length | 16384 |
| batch_size | 16 |
| lr | 1e-5 |
| adam_betas | (0.9, 0.95) |
| grad_clip | 1.0 |
| weight_decay | 0.0001 |
Table 5: The value of the hyper-parameters in our reasoning-oriented training experiment (Section 5.2) for SFT (left) and RL (right).
Figure 10: Performance of sequential multiple stones and recursive multiple stones on AIME 2024 (left), AIME 2025 (middle), and BeyondAIME (right). Best is conditioned on the best generated stones for the target questions, and Avg is averaged over 20 stones.
Figure 11: Prompt used to generate stepping stone problems in ARQ. The {problem} is replaced by the target problem.
Figure 12: Prompt used to solve stepping stone problems in ARQ. The {question} is replaced by the stepping stone problem.
Figure 13: Prompt used to solve target problems in ARQ. The {stone problem} , {stone solution} , and {question} are replaced by the stepping stone problem, the solution to the stepping stone problem, and the target problem.
Figure 14: Prompt used to generate a random stepping stone in the Rand baseline.
Figure 15: Prompt used to generate the -th sequential problem in ARQ.
More in this topic
LLM Reasoning
Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.
See all 926 papers