Procedural Knowledge at Scale Improves Reasoning

Di Wu; Devendra Singh Sachan; Wen-tau Yih; Mingda Chen

cs.CLApr 1, 2026

Procedural Knowledge at Scale Improves Reasoning

Authors: Di Wu, Devendra Singh Sachan, Wen-tau Yih, Mingda Chen

Organizations: University of California, Los Angeles · Meta FAIR

Abstract

Test-time scaling has emerged as an effective way to improve language models on challenging reasoning tasks. However, most existing methods treat each problem in isolation and do not systematically reuse knowledge from prior reasoning trajectories. In particular, they underutilize procedural knowledge: how to reframe a problem, choose an approach, and verify or backtrack when needed. We introduce Reasoning Memory, a retrieval-augmented generation (RAG) framework for reasoning models that explicitly retrieves and reuses procedural knowledge at scale. Starting from existing corpora of step-by-step reasoning trajectories, we decompose each trajectory into self-contained subquestion-subroutine pairs, yielding a datastore of 32 million compact procedural knowledge entries. At inference time, a lightweight in-thought prompt lets the model verbalize the core subquestion, retrieve relevant subroutines within its reasoning trace, and reason under diverse retrieved subroutines as implicit procedural priors. Across six math, science, and coding benchmarks, Reasoning Memory consistently outperforms RAG with document, trajectory, and template knowledge, as well as a compute-matched test-time scaling baseline. With a higher inference budget, averaged across models and tasks, it improves over no retrieval by 19.5% and over the compute-matched baseline by 8.9%. Ablation studies show that these gains come from two key factors: the broad procedural coverage of the source trajectories and our decomposition and retrieval design, which together enable effective extraction and reuse of procedural knowledge. Our experiment code is available at https://github.com/facebookresearch/reasoning-memory.

Figures & tables

Figure 1: Illustration of the Reasoning Memory framework. (a) We extract self-contained procedural knowledge from diverse public reasoning trajectories to construct a datastore. (b) At inference time, we retrieve top- $k$ subquestion-subroutine pairs, generate separate groups of candidate trajectories under each subroutine, and filter and rank the candidates. The source trajectories are not inserted.

Figure 2: Standard document RAG benefits instruction-tuned models more than reasoning models. Under the CompactDS pipeline, instruction-tuned models obtain modest gains from retrieval, whereas the corresponding reasoning models often see limited gains or even degradation, despite much stronger no-retrieval performance.

DeepSeek-R1-Distill-Llama-8B
Method	Ret.	$m$	Math			Science	Code (LCB)
			AIME 24’	AIME 25’	MATH500	GPQA-D	V1-4	V5-6
No RAG	N/A	8	0.461	0.350	0.806	0.427	0.220	0.211
Trajectory RAG (Prefix)	Proc.	8	0.425	0.246	0.784	0.391	0.195	0.128
Trajectory RAG (Summary)	Proc.	8	0.408	0.321	0.822	0.431	0.184	0.121
Template RAG (ReasonFlux)	Proc.	8	0.450	0.342	0.828	0.417	0.174	0.210

Table 1: Main results of all methods in math, science, and coding domains. The column Ret. = type of retrieved knowledge, Proc. = procedural, Fact. = factual. Bold numbers mark the best performance under the same budget. ✣ = statistically significantly better than the second best via paired t-test (p

<

0.05). Reasoning Memory significantly outperforms baselines across LLMs and generally improves over Length Scaling across budgets.

Figure 3: Performance as a function of inference budget. We compare Length Scaling without retrieval with two Reasoning Memory variants on DeepSeek-R1-Distill-Llama-8B as the total sampling budget

m

increases.

Figure 4: Effect of datastore size and composition on Reasoning Memory. Performance of DeepSeek-R1-Distill-Llama-8B with budget

m=30

. Larger and more diverse datastores generally yield stronger performance.

Setting	AIME 2024	GPQA-D	LCB (V1-4)	Average
Removing Subroutine Decomposition	0.551	0.433	0.308	0.431
Removing Self-Generated Query	0.567	0.453	0.295	0.438
Full Reasoning Memory (Qwen3-8B)	0.550	0.474	0.320	0.448
Full Reasoning Memory (QwQ-32B)	0.575	0.461	0.310	0.449

Table 2: Ablations on datastore construction and query design. All results use DeepSeek-R1-Distill-Llama-8B with budget

m=30

. The full Reasoning Memory configuration uses QwQ-32B as the datastore generator, a decomposed datastore indexed by subquestions, and self-verbalized in-thought queries.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Utility of different types of synthesized knowledge. We report performance gains relative to the no-retrieval setting. Both factual and procedural knowledge help, but procedural knowledge yields larger gains across tasks and model families.

	Llama-3.1-8B (Inst.)		Llama-3.1-8B (Reas.)		Qwen2.5-7B (Inst.)		Qwen2.5-7B (Reas.)		Qwen2.5-32B (Inst.)		Qwen2.5-32B (Reas.)
	Fact.	Proc.	Fact.	Proc.	Fact.	Proc.	Fact.	Proc.	Fact.	Proc.	Fact.	Proc.
AIME	0.040	0.029	0.075	0.090	0.015	0.012	0.096	0.167	0.025	0.067	-0.013	-0.023
GPQA	0.223	0.254	0.150	0.151	0.173	0.204	0.076	0.084	0.159	0.160	0.065	0.076
LCB	0.027	0.059	0.041	0.077	0.029	0.033	0.033	0.053	0.178	0.159	0.004	0.032
Average	0.097	0.114	0.089	0.106	0.072	0.083	0.068	0.101	0.121	0.129	0.019	0.028

Appendix

Table 3: Per-task performance gains in the knowledge-synthesis pilot study. “Fact.” denotes factual knowledge and “Proc.” denotes procedural knowledge. Entries report the performance difference between prompting with synthesized knowledge and prompting without augmentation. Procedural knowledge generally produces larger gains.

Panel of Figure 6 — Figure 6: Prompt for synthesizing factual knowledge for the pilot study. GPT-5.2 is prompted to generate the knowledge.

Panel of Figure 7 — Figure 7: Prompt for synthesizing procedural knowledge for the pilot study. GPT-5.2 is prompted to generate the knowledge.

Panel of Figure 8 — Figure 8: Prompt for decomposing trajectory into the subquestions.

Panel of Figure 9 — Figure 9: Prompt for generating reusable subroutines for subquestions.

Panel of Figure 10 — Figure 10: An Example of the Reasoning Memory Datastore. We extract multiple self-contained subquestions from the original question and the long reasoning trajectory. Then, a corresponding subroutine is generated for each subquestion.

Model	Method	AIME 2024	GPQA-D
OpenThinker3-7B	Length Scaling	0.647	0.502
	Empty Context + Length Scaling	0.547	0.511
	Majority Vote	0.573	0.533
	Reasoning Memory	0.758	0.542
DeepSeek-R1-Distill-Llama-8B	Length Scaling	0.548	0.447
	Empty Context + Length Scaling	0.555	0.452

Appendix

Table 4: Empty-context and majority-vote controls. All results use

m=30

. Empty Context retains query verbalization but provides no retrieved subroutine before applying length-based selection. Majority voting is reported only for AIME 2024 and GPQA-D, where answer aggregation is well defined.

	DeepSeek-R1-Distill-Llama-8B			OpenThinker3-7B			Qwen3-32B
Method	AIME 24’	GPQA	LCB (V1–4)	AIME 24’	GPQA	LCB (V1–4)	AIME 24’	GPQA	LCB (V1–4)
No RAG	0.461	0.427	0.220	0.470	0.438	0.318	0.789	0.669	0.387
+ length scaling	0.508	0.426	0.255	0.612	0.484	0.333	0.792	0.699	0.399
Trajectory RAG (Summary)	0.408	0.431	0.184	0.438	0.451	0.242	0.629	0.595	0.267
+ length scaling	0.483	0.439	0.175	0.533	0.444	0.290	0.567	0.580	0.224
Template RAG (Self-Discover)	0.442	0.414	0.213	0.442	0.441	0.276	0.667	0.633	0.312

Appendix

Table 5: Effect of adding length-based uncertainty filtering to retrieval-based baselines. All results use

m=8

and

n=4

Method	AIME 24’	GPQA	LCB (V1–4)
No RAG	0.658	0.553	0.380
Length Scaling	0.692	0.548	0.396
Reasoning Memory	0.708	0.534	0.395

Appendix

Table 6: Results on AIME 24’, GPQA, and LCB (V1–4) using Llama-3.1-Nemotron-Nano-8B-v1. For both Length Scaling and Reasoning Memory we use the budget

m=30

and

n=8

Metric	AIME 24’	GPQA-D	LCB (V1–4)
Likelihood	0.571	0.429	0.273
Entropy	0.600	0.417	0.273
Contrast	0.563	0.419	0.269
Self-Eval	0.571	0.402	0.263
Length	0.575	0.461	0.310

Appendix

Table 7: Alternative metrics for step 1 filtering. Results for DeepSeek-R1-Distill-Llama-8B with budget m=30; stage 2 always uses length as the uncertainty proxy. Length achieves strong overall performance while being the simplest.

Question	Please answer the following math question. Find the sum of all integer bases $b>9$ for which $17_{b}$ is a divisor of $97_{b}.$
Query	17 in base $b$ is equal to what in decimal?
Retrieved Subquestion	How do you convert a number from an arbitrary base $b$ to its decimal (base 10) equivalent?
Associated Subroutine	For problems like this, I should first write the number as a sum of each digit multiplied by the base raised to the power of its position index. Here’s how: 1. Identify each digit of the number in the original base. 2. Assign positional indices starting from $0$ at the rightmost digit (the least significant digit). 3. Compute the weight of each digit by raising the base $b$ to the power of its position index. 4. Multiply each digit by its corresponding weight. 5. Sum all the products to obtain the decimal (base 10) equivalent. For example, a number $d_{n}d_{n-1}\ldots d_{1}d_{0}$ in base $b$ converts to decimal as: $d_{n}\cdot b^{n}+d_{n-1}\cdot b^{n-1}+\ldots+d_{1}\cdot b^{1}+d_{0}\cdot b^{0}.$ Ensure all digits are valid in the original base (i.e., $0\leq d_{i}<b$ ) before proceeding.
Continued Reasoning	Okay, so I have this math problem here: Find the sum of all integer bases $b>9$ for which $17_{b}$ is a divisor of $97_{b}$ . Hmm, okay. Let me try to figure this out step by step. First, I remember that when numbers are written with a subscript like $17_{b}$ , it means they’re in base $b$ . So I need to convert both $17_{b}$ and $97_{b}$ into decimal (base 10) to work with them more easily. Let’s start with $17_{b}$ . In base $b$ , the rightmost digit is the units place (which is $b^{0}$ ) and the next digit is the $b^{1}$ place. So $17_{b}$ would translate to: $1\times b^{1}+7\times b^{0}=b+7.$ Got that. So $17_{b}=b+7$ in decimal. Now, let’s do the same for $97_{b}$ …

Appendix

Table 8: An example from AIME 2025 where Reasoning Memory is effective.

Question	Two-thirds of the students at Baker Middle School take music. There are 834 students who take music. How many students are there at Baker Middle School?
Query	$\tfrac{2}{3}$ of something equals 834, what is that something?
Retrieved Subquestion	What is the result of dividing 3.6 by 1.6?
Associated Subroutine	For problems like this, I should first eliminate decimals by expressing the division as a fraction of whole numbers. For example, if dividing $a.b$ by $c.d$ , multiply both the numerator and denominator by 10 to convert them into whole numbers ( $\frac{ab}{cd}$ ). Then, simplify the fraction by dividing the numerator and denominator by their greatest common divisor or perform long division directly. If simplifying isn’t straightforward, break the division into parts: calculate how many times the divisor fits into the dividend, subtract that portion, and handle the remainder to find the exact value. This method works for any decimal division where precision is needed.
Continued Reasoning	Okay, so I’ve got this math problem here: Two-thirds of the students at Baker Middle School take music, and there are 834 students who take music. The question is asking how many students are there at Baker Middle School in total. Hmm, let’s think about this step by step. Alright, two-thirds of the school takes music, which means if I can figure out what number multiplied by two-thirds gives me 834, that number will be the total number of students at the school. So, essentially, I need to solve for the total number of students, which I’ll call $S$ . The equation would be: $\frac{2}{3}\cdot S=834.$ To find $S$ , I should isolate it on one side of the equation. To do that, I can multiply both sides by the reciprocal of two-thirds, which is three-halves $\left(\frac{3}{2}\right)$ . So, multiplying both sides by $\frac{3}{2}$ : $S=834\cdot\frac{3}{2}.$ Let me compute that. $834$ divided by $2$ is $417$ , and then $417$ multiplied by $3$ is: $400\times 3=1200$ , and $17\times 3=51$ , so adding those together gives me $1251$ . So, $S=1251$ .

Appendix

Table 9: An example from MATH500 where Reasoning Memory is effective.

Question	Generate a correct Python program that passes all tests for the given problem. Problem Title: $Ax+By<C$ Problem Statement: You are given three length- $N$ sequences of positive integers: $A=(A_{1},A_{2},\ldots,A_{N}),\quad B=(B_{1},B_{2},\ldots,B_{N}),\quad C=(C_{1},C_{2},\ldots,C_{N}).$ Find the number of pairs of positive integers $(x,y)$ that satisfy the following condition: $A_{i}\times x+B_{i}\times y<C_{i}\quad\text{for all }1\leq i\leq N.$ It can be proved that the number of such pairs of positive integers satisfying the condition is finite. You are given $T$ test cases, each of which should be solved.
Query	how to count the number of $(x,y)$ pairs that satisfy multiple inequalities $A_{i}x+B_{i}y<C_{i}$ for all $i$
Retrieved Subquestion	How do you count the number of integer pairs $(x,y)$ that satisfy a linear inequality with non-negative coefficients and bounds?
Associated Subroutine	For problems like this, I should consider the following steps: 1. Fix one variable and solve for the other. Choose one variable (e.g., $y$ ) and iterate through its possible values within the bounds. For each fixed $y$ , determine the minimum or maximum value of the other variable (e.g., $x$ ) required to satisfy the inequality. 2. Use complementary counting if feasible. If the inequality’s valid region is large, it may be easier to calculate the total number of candidate pairs (for example, $(N+1)^{2}$ for bounds $0\leq x,y\leq N$ ) and subtract the count of pairs that do not satisfy the inequality. This is especially useful when the invalid region is smaller or has a simpler structure. 3. Account for variable bounds. Ensure all computed ranges for $x$ or $y$ respect their given constraints (e.g., $0\leq x\leq N$ ). Adjust the ranges to stay within these bounds, even if the inequality suggests values outside them. 4. Sum valid counts systematically. For each fixed value of the first variable, compute the number of valid values for the second variable and accumulate these counts. Organize the calculations (for example, by grouping cases where the inequality trivially holds or fails) to avoid errors and redundant work. This approach balances direct enumeration with strategic shortcuts to efficiently count solutions while avoiding overcomplication.
Continued Reasoning	So I’ve got this programming problem to solve. Let’s read it carefully. The problem is about finding the number of pairs $(x,y)$ of positive integers that satisfy $A_{i}x+B_{i}y<C_{i}$ for all $i$ from $1$ to $N$ . So for each $i$ , the same $x$ and $y$ have to satisfy this inequality. In other words, we need to find all $(x,y)$ such that for every $i$ , $A_{i}x+B_{i}y<C_{i}$ holds simultaneously. We are given $T$ test cases, and for each test case we get $N$ lines of $A_{i},B_{i},C_{i}$ . The goal is: for each test case, compute the number of such $(x,y)$ pairs. Geometrically, each inequality $A_{i}x+B_{i}y<C_{i}$ describes a half-plane in the first quadrant (since $x$ and $y$ are positive integers). The feasible region is the intersection of all these half-planes restricted to $x\geq 1$ , $y\geq 1$ . The statement that the answer is finite means this intersection is a bounded polygonal region. One possible strategy is to treat $x$ as the main loop variable…

Appendix

Table 10: An example from LiveCodeBench where Reasoning Memory is effective.

Question

There are

n

values of

x

in the interval

0<x<2\pi

where

f(x)=\sin\bigl(7\pi\cdot\sin(5x)\bigr)=0.

For

t

of these

n

values of

x

, the graph of

y=f(x)

is tangent to the

x

-axis. Find

n+t

Query

How to find the number of solutions to

\sin(7\pi\cdot\sin(5x))=0

between

0

and

2\pi

, and how many of those have tangent to

x

-axis.

Top-3 Retrieved Subquestion

1. How do you analyze the behavior of a trigonometric function (e.g., using derivatives) to confirm the number of solutions in a given interval? 2. How do you systematically count and confirm the total number of valid solutions by combining results from both cases

(\sin(x)=0

and

2\cos(x)+\cos(2x)=0)

? 3. How do you analyze the behavior of a function (e.g.,

f(x)=\tan(2x)+\sin(x)

) in intervals between vertical asymptotes to determine the number of solutions?

Appendix

Table 11: A failure example from AIME 2025 caused by over-specific queries.

Question	[redacted, luminosities of two stars radiating as black bodies]
Query	astronomers luminosity star radius relation
Retrieved Subquestion	What is the relationship between a star’s radius and its luminosity?
Associated Subroutine	For problems asking about the relationship between a star’s radius and luminosity, I should start by recalling the Stefan–Boltzmann law: $L=4\pi R^{2}\sigma T^{4},$ where $L$ is luminosity, $R$ is radius, $T$ is temperature, and $\sigma$ is the Stefan–Boltzmann constant. Luminosity depends on both radius and temperature. To isolate the radius–luminosity relationship, consider how changes in radius affect $L$ while holding temperature constant. However, for main-sequence stars, radius and temperature are not independent—both increase with mass. Use the mass–radius and mass–luminosity relationships (e.g., $L\propto M^{3.5}$ , $R\propto M^{0.7}$ ) to deduce the overall correlation. Remember that temperature also contributes strongly to luminosity, so the radius’s role must be analyzed in concert with mass and temperature dependencies.
Continued Reasoning	Okay, so I have this astronomy problem here about the luminosities of two stars, Star 1 and Star 2. Let me try to work through it step by step. The question says that Star 1 has a radius $1.5$ times larger than Star 2, and the mass of Star 1 is also $1.5$ times larger. They also mention that the wavelengths at which the stars appear brightest are the same, and their radial velocities are $0$ for Star 1 and $700\ \mathrm{km/s}$ for Star 2. The question is about the ratio of their luminosities, so I need to compute how much brighter Star 1 is compared to Star 2. I remember that the luminosity of a star is related to its radius and temperature. The Stefan–Boltzmann law is $L=4\pi R^{2}\sigma T^{4},$ where $L$ is luminosity, $R$ is radius, $\sigma$ is the Stefan–Boltzmann constant, and $T$ is the effective temperature. At first I might wonder whether the Doppler effect due to different radial velocities matters here, but the problem states that the wavelengths at which they appear brightest are the same. That suggests their effective temperatures are the same, because the peak wavelength (or color) is determined by temperature via Wien’s law. So I can take $T_{1}=T_{2}$ . If the temperatures are equal, then the ratio of luminosities depends only on the ratio of radii: $\frac{L_{1}}{L_{2}}=\left(\frac{R_{1}}{R_{2}}\right)^{2}\left(\frac{T_{1}}{T_{2}}\right)^{4}=\left(\frac{R_{1}}{R_{2}}\right)^{2}.$ Since $R_{1}=1.5\,R_{2}$ , this gives $\frac{L_{1}}{L_{2}}=(1.5)^{2}=2.25,$ which corresponds to choice (D), approximately $2.25$ .

Appendix

Table 12: A failure example from GPQA-D caused by problem setup mismatch.

Explore similar work

Aug 11, 2026

ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling

Large Reasoning Models (LRMs) improve performance by allocating additional inference-time compute to generate extended chain-of-thought reasoning. However, recent studies reveal that sequential test-time scaling often yields diminishing or even negative returns, as longer traces exhibit increased uncertainty, error compounding, and drift from the original problem. We propose ThinkRetrieve, a test-time scaling framework that augments the reasoning traces of LRMs with dynamically retrieved solved examples at each reasoning step. Given an external corpus of problems paired with step-by-step solutions, ThinkRetrieve retrieves relevant exemplars at each intermediate step and injects them directly into the thinking trace, providing the model with guidance on how to reason rather than merely what facts are relevant. Experiments across five reasoning models (1.5B--8B parameters) on GSM-8K, MATH-500, AIME 2025, and SciQ demonstrate that ThinkRetrieve consistently improves accuracy over standard test-time scaling, with relative gains of up to

60\%

on AIME 2025.

Vaibhav Singh, Soumya Suvra Ghosal, Sarvesh Gharat +3

IIT Bombay · University of Maryland, College Park · Adobe Research

Numerical Reasoning in Language Models Test-Time Scaling Large Reasoning Models

Ask PDF

Jun 11, 2026

Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning

Retrieval-augmented generation (RAG) has become a standard mechanism for grounding language models in external knowledge, yet conventional retrieval based on lexical or semantic similarity is poorly suited for complex reasoning tasks: a semantically similar problem may demand an entirely different solution strategy, while a superficially different problem may share the same underlying reasoning pattern. We propose Retrieval-Augmented Reinforcement Fine-Tuning (RA-RFT), a post-training framework that teaches language models to reason by analogy. RA-RFT uses gold-relevance distillation to train a retriever that ranks contexts by expected reasoning benefit rather than semantic overlap, and then fine-tunes the policy model via reinforcement fine-tuning methods with retrieved analogous demonstrations, so the model learns to leverage reasoning traces under verifiable outcome rewards. We further analyze the diversity of retrieved contexts and find that reasoning-aware retrieval surfaces complementary solution strategies that provide distinct reasoning scaffolds for individual problems. Across challenging mathematical reasoning benchmarks, RA-RFT consistently outperforms standard reinforcement fine-tuning methods. For example, it improves AIME 2025 average@32 accuracy by 7.1 and 2.8 points over GRPO for Qwen3-1.7B and Qwen3-4B respectively -- suggesting that reasoning-aware retrieval is a complementary axis of improvement and orthogonal to advances in reward design or training curricula.

Zilin Xiao, Qi Ma, Chun-cheng Jason Chen +4

Meta Superintelligence Labs · Rice University · Work partially done during internship at Meta

Analogical Reasoning RL for Language Model Reasoning Mathematical Reasoning

Ask PDF

Apr 23, 2026

Thinking with Reasoning Skills: Fewer Tokens, More Accuracy

Reasoning LLMs often spend substantial tokens on long intermediate reasoning traces (e.g., chain-of-thought) when solving new problems. We propose to summarize and store reusable reasoning skills distilled from extensive deliberation and trial-and-error exploration, and to retrieve these skills at inference time to guide future reasoning. Unlike the prevailing reasoning from scratch paradigm, our approach first recalls relevant skills for each query, helping the model avoid redundant detours and focus on effective solution paths. We evaluate our method on coding and mathematical reasoning tasks, and find that it significantly reduces reasoning tokens while improving overall performance. The resulting lower per-request cost indicates strong practical and economic potential for real-world deployment.

Guangxiang Zhao, Qilong Shi, Xusen Xiao +3

Qiyuan Tech · Tsinghua University · The University of Hong Kong +1

Efficient Language Model Reasoning Efficient Language Model Inference LLM Mathematical Reasoning LLM Reasoning

Ask PDF