CPU-ARC: Agentic Reasoning Caching for on-CPU Reasoning Reuse

Peter Belcak; Greg Heinrich; Saurav Muralidharan; Shizhe Diao; Pavlo Molchanov

cs.AINov 6, 2026

CPU-ARC: Agentic Reasoning Caching for on-CPU Reasoning Reuse

Authors: Peter Belcak, Greg Heinrich, Saurav Muralidharan, Shizhe Diao, Pavlo Molchanov

Abstract

Large language models reinvent the wheel all the time. They conflate task-solving with the execution of the solutions, which forces them to solve every problem from scratch. We show that once a task has been reasoned through, the application of its solution to a new problem instance can be done very efficiently and with little involvement from language models. We introduce agentic reasoning caching, a method for constructing robust CPU-executable agentic reasoning caches (ARCs) from LLM reasoning traces. ARCs show on-par performance with frontier LLMs across a range of difficult mathematical and scientific tasks, are fully explainable, continually trainable, robust to input perturbations, and 16-291x cheaper to invoke than the source model.

Figures & tables

	Original	Rephrased	Reparameterized
Question	How many positive whole-number divisors does $\mathstrut 196$ have?	Find how many positive integers evenly divide $\mathstrut 196$ .	How many positive whole-number divisors does $\mathstrut 144$ have?
Reasoning	I need to find the prime factorisation of $\mathstrut 196$ first. ${\hbox{\pagecolor{hlparam}$ \mathstrut 196 $}}={\hbox{\pagecolor{hlderived}$ \mathstrut 4 $}}\times{\hbox{\pagecolor{hlderived}$ \mathstrut 49 $}}={\hbox{\pagecolor{hlderived}$ \mathstrut 2^{2} $}}\times{\hbox{\pagecolor{hlderived}$ \mathstrut 7^{2} $}}$ . For a number with prime factorisation $p_{1}^{a_{1}}\times p_{2}^{a_{2}}\times\cdots$ , the number of divisors is $(a_{1}+1)(a_{2}+1)\cdots$ . For ${\hbox{\pagecolor{hlparam}$ \mathstrut 196 $}}={\hbox{\pagecolor{hlderived}$ \mathstrut 2^{2} $}}\times{\hbox{\pagecolor{hlderived}$ \mathstrut 7^{2} $}}$ : ${\hbox{\pagecolor{hlderived}$ \mathstrut(2+1)(2+1) $}}={\hbox{\pagecolor{hlderived}$ \mathstrut 3\times 3 $}}={\hbox{\pagecolor{hlderived}$ \mathstrut\mathbf{9} $}}$ .		I need to find the prime factorisation of $\mathstrut 144$ first. ${\hbox{\pagecolor{hlparam}$ \mathstrut 144 $}}={\hbox{\pagecolor{hlderived}$ \mathstrut 16 $}}\times{\hbox{\pagecolor{hlderived}$ \mathstrut 9 $}}={\hbox{\pagecolor{hlderived}$ \mathstrut 2^{4} $}}\times{\hbox{\pagecolor{hlderived}$ \mathstrut 3^{2} $}}$ . For a number with prime factorisation $p_{1}^{a_{1}}\times p_{2}^{a_{2}}\times\cdots$ , the number of divisors is $(a_{1}+1)(a_{2}+1)\cdots$ . For ${\hbox{\pagecolor{hlparam}$ \mathstrut 144 $}}={\hbox{\pagecolor{hlderived}$ \mathstrut 2^{4} $}}\times{\hbox{\pagecolor{hlderived}$ \mathstrut 3^{2} $}}$ : ${\hbox{\pagecolor{hlderived}$ \mathstrut(4+1)(2+1) $}}={\hbox{\pagecolor{hlderived}$ \mathstrut 5\times 3 $}}={\hbox{\pagecolor{hlderived}$ \mathstrut\mathbf{15} $}}$ .

Table 1: A MATH-500 problem, its rephrasing while keeping all the parameters the same, its reparametrization while keeping the phrasing the same, and the corresponding reasoning traces. The rephrased variant changes only the wording; the reparameterized variant substitutes the numerical parameter and adjusts the arithmetic accordingly. Note how neither the rephrasing nor the reparametrization change even a single word in the reasoning steps leading to the final answer, not to mention their substance.

	Correctness (%)			Perturbation (%)			Avg. Cost ($/sample)			Multipliers
Benchmark	S Acc	C Acc	Fidelity	Phrasal	Param.	Combined	Source	Builder	Cache	Build $\times$	Savings $\times$
MATH-500	98.6	98.8	99.6	96.7	94.5	91.6	0.0388	1.1818	0.0004	30.5	88.7
AIME ’24	100.0	100.0	100.0	96.0	99.3	95.1	0.1101	1.4198	0.0004	12.9	262.4
AIME ’25	83.3	93.3	90.0	98.4	91.2	93.4	0.1203	1.4517	0.0004	12.1	291.1
AIME ’26	90.0	96.7	90.0	97.9	90.9	90.8	0.1197	1.5477	0.0004	12.9	281.4
GPQA Diamond	83.8	83.8	98.5	93.8	93.9	94.0	0.0708	1.7781	0.0019	25.1	38.1

Table 2: Results across benchmarks. S Acc : source model accuracy (transcript vs. golden label); C Acc : cache accuracy (cache vs. golden label); Fidelity : cache–source agreement (cache vs. transcript). Perturbation columns report pass rates for phrasal, parametric, and combined iso-substantive rewrites. Cost columns show average per-sample cost at retail API prices; Build

\times

= builder / source cost; Savings

\times

= source / cache cost. See Appendix C for per-role token usage.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Semantic functions constituting the agentic reasoning cache entry for the question in Figure 3 . Function (a) is a utility that converts temperature to Kelvin; it uses extract() to obtain the numeric value and lm() to classify the unit. Both of these require an LLM call at invocation time. Function (b) on the next page is the root function that orchestrates the full ideal-gas-law computation. A third function (not shown) converts pressure from kPa to MPa, bar, and atm via pure arithmetic using load() and narrate() . Runtime primitives load , do , narrate , extract , and lm are highlighted.

Figure 11: Semantic functions constituting the agentic reasoning cache entry for the question in Figure 9 . Function (a) is a utility that classifies the thermodynamic process; function (b) on the next page is the root function that orchestrates parameter loading, delegates to (a) via do() , and computes the result. Runtime primitives load , do , narrate , and extract are highlighted.

Role	Avg In	Avg Out	Cache Rd	Cache Wr	Cost ($)
Source	114	1,529	0	—	0.0388
Builder	355,140	27,591	324,207	41,587	1.1818
Target	546	393	0	—	0.0001
Extractor	5,271	369	0	—	0.0002
Converter	1,732	596	0	—	0.0002
Total					1.2210

Appendix

Table 3: MATH-500.

Role	Avg In	Avg Out	Cache Rd	Cache Wr	Cost ($)
Source	150	4,374	0	—	0.1101
Builder	451,054	32,829	414,183	49,435	1.4198
Target	226	76	0	—	0.0000
Extractor	5,851	383	0	—	0.0002
Converter	2,419	697	0	—	0.0002
Total					1.5303

Appendix

Table 4: AIME ’24.

Role	Avg In	Avg Out	Cache Rd	Cache Wr	Cost ($)
Source	225	4,766	0	—	0.1203
Builder	439,165	33,702	400,518	51,325	1.4517
Target	494	205	0	—	0.0001
Extractor	8,369	461	0	—	0.0003
Converter	1,231	268	0	—	0.0001
Total					1.5724

Appendix

Table 5: AIME ’25.

Role	Avg In	Avg Out	Cache Rd	Cache Wr	Cost ($)
Source	168	4,754	0	—	0.1197
Builder	462,836	36,763	424,685	53,344	1.5477
Target	289	116	0	—	0.0000
Extractor	7,232	393	0	—	0.0002
Converter	1,996	555	0	—	0.0002
Total					1.6678

Appendix

Table 6: AIME ’26.

Role	Avg In	Avg Out	Cache Rd	Cache Wr	Cost ($)
Source	268	2,779	0	—	0.0708
Builder	597,730	41,200	553,857	60,194	1.7781
Target	4,796	4,519	0	—	0.0010
Extractor	18,175	1,400	0	—	0.0007
Converter	1,729	729	0	—	0.0002
Total					1.8508

Appendix

Table 7: GPQA Diamond.

Role	Avg In	Avg Out	Cache Rd	Cache Wr	Cost ($)
Source	100	386	0	—	0.0101
Builder	164,598	21,110	140,887	31,599	0.8481
Target	317	96	0	—	0.0000
Extractor	6,458	415	0	—	0.0002
Converter	4,174	1,404	0	—	0.0004
Total					0.8589

Appendix

Table 8: GSM8K.

Role	Avg In	Avg Out	Cache Rd	Cache Wr	Cost ($)
Source	246	1,332	0	—	0.0345
Builder	477,206	32,045	439,344	48,698	1.4084
Target	3,707	3,035	0	—	0.0007
Extractor	14,786	1,449	0	—	0.0006
Converter	2,074	863	0	—	0.0002
Total					1.4445

Appendix

Table 9: MMLU-Pro.

Role	Avg In	Avg Out	Cache Rd	Cache Wr	Cost ($)
Source	130	1,709	0	—	0.0434
Builder	293,107	30,957	261,293	45,101	1.2572
Target	736	397	0	—	0.0001
Extractor	7,655	518	0	—	0.0003
Converter	1,513	607	0	—	0.0002
Total					1.3011

Appendix

Table 10: SciBench.

Parameter	Value
Base model	meta-llama/Llama-3.2-1B
Max training epochs	20
Learning rate	$2\times 10^{-5}$
Optimizer	AdamW
Per-device batch size	8 (auto-scaled)
Gradient accumulation	1

Appendix

Table 11: Fine-tuning hyperparameters for the 1B target model.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 388 papers

Parason: Revealing Subtask and Trial Parallelism in LLM Reasoning
Aug 25, 2026Zhengyang Zhang, Zijian Zhang, Jiaxuan Gao +4Parallel Reasoning in Language Models Efficient Language Model Reasoning
Tsinghua University · NVIDIA · Independent Researcher +1
Procedural Knowledge at Scale Improves Reasoning
Apr 1, 2026Di Wu, Devendra Singh Sachan, Wen-tau Yih +1Retrieval-Augmented Reasoning LLM Reasoning
University of California, Los Angeles · Meta FAIR
Thinking with Reasoning Skills: Fewer Tokens, More Accuracy
Apr 23, 2026Guangxiang Zhao, Qilong Shi, Xusen Xiao +3Efficient Language Model Reasoning Efficient Language Model Inference
Qiyuan Tech · Tsinghua University · The University of Hong Kong +1

Aug 25, 2026J/K move · Enter open · S save

Parason: Revealing Subtask and Trial Parallelism in LLM Reasoning

Zhengyang Zhang, Zijian Zhang, Jiaxuan Gao, Shusheng Xu+3

Tsinghua University · NVIDIA · Independent Researcher+1

Parallel Reasoning in Language Models Efficient Language Model Reasoning LLM Reasoning LLM Inference Acceleration Large Reasoning Models RL for Language Model Reasoning

Open paper AskPDF arXiv

Scaling test-time reasoning has substantially improved the problem-solving ability of large language models (LLMs), but standard autoregressive decoding still executes long reasoning traces sequentially, creating severe latency for difficult tasks (up to days and weeks). Parallel reasoning offers a natural remedy. However, prior systems primarily focus on Subtask Parallelism, where the model learns to decompose a high-level task into smaller chunks that can be solved independently. This approach overlooks another pervasive form of parallelism: Trial Parallelism, where multiple speculative attempts explore, verify, and aggregate competing hypotheses in parallel. In this paper, we introduce Parason, which reveals and learns both forms of parallelism in LLM reasoning. Our analysis identifies Trial Parallelism as the majority of parallelizable reasoning computation (65.5% in DeepSeek-V4's reasoning steps in HLE), and it becomes increasingly dominant on hard problems. Guided by this taxonomy, Parason converts sequential reasoning traces into structured parallel trajectories with a context-free grammar, then trains models with Parallelism-Aware Group Relative Policy Optimization (PA-GRPO), whose reward jointly balances accuracy, latency, and the two parallelism ratios. At inference time, Parason executes the learned parallel structure through tool calls, translating theoretical savings to real-world wall-clock acceleration. Experiments on mathematical reasoning benchmarks including AIME24 and AIME25 show that Parason achieves an average acceleration about 1.7

\times

while maintaining competitive accuracy.

Abstract

Efficient Language Model Reasoning

Explore similar work

Parason: Revealing Subtask and Trial Parallelism in LLM Reasoning

Procedural Knowledge at Scale Improves Reasoning

Thinking with Reasoning Skills: Fewer Tokens, More Accuracy