Large language models (LLMs) offer strong reasoning capabilities but are often costly to access through commercial APIs, while small language models (SLMs) are easier to deploy locally yet remain weaker in reasoning. This capability-deployment gap has motivated LLM-SLM collaboration, which aims to improve SLM reasoning using LLM capabilities while preserving the deployment advantages of SLMs. Existing approaches mainly follow two paradigms. Knowledge distillation uses LLM-generated answers and reasoning trajectories to train SLMs offline, but requires parameter updates and additional training. Alternatively, online collaboration routes difficult problems to an LLM or leverages LLM-generated guidance and corrections when an SLM encounters difficulties. Although effective, online collaboration requires repeated LLM access. Moreover, the guidance produced for a particular problem is discarded after inference and cannot benefit subsequent problems involving similar reasoning states. In the paper, we focus on a more constrained setting in which the LLM is accessed only offline, the SLM parameters remain fixed, and online inference is performed solely by the SLM. To this end, we propose Reusable Latent Correction (RLC), which converts one-off natural-language guidance from a black-box LLM into persistent corrective experiences in the hidden space of an SLM. RLC stores these experiences in an external bank and retrieves them according to the SLM's current reasoning state, enabling the SLM to reuse LLM-derived corrections during inference without any online LLM calls. Experiments across multiple reasoning benchmarks and SLM scales show that RLC consistently improves SLM reasoning without parameter updates or online LLM calls. Code is available at https://github.com/ZBH031/reusable-latent-correction.
Figures & tables
Figure 1: Comparison of LLM–SLM collaboration paradigms. (a) Offline knowledge distillation trains the SLM with LLM-generated CoT trajectories, requiring parameter updates. (b) Online collaboration avoids retraining but repeatedly queries the LLM during inference. (c) RLC uses the LLM only for offline experience construction and stores effective corrections in an external Bank. At inference time, the frozen SLM retrieves relevant corrections and applies hidden-state interventions without further LLM access.
Figure 2: Overall architecture of Reusable Latent Correction (RLC), which consists of offline experience construction and online experience reuse.
Method
MATH500
GSM8K
GPQA
AMC24
AIME25
Average
Acc.
Tok.
Acc.
Tok.
Acc.
Tok.
Acc.
Tok.
Acc.
Tok.
Acc.
Tok.
Qwen2.5-3B Native
67.00
607.84
84.69
299.69
30.30
740.65
4.44
967.53
3.33
888.30
37.95
700.80
Qwen2.5-7B Native
74.20
557.86
90.83
290.12
38.89
586.81
20.00
1030.87
10.00
931.47
46.78
679.43
ReBalance (3B)
70.60
619.74
86.73
290.96
39.45
724.48
22.22
764.89
3.33
870.37
44.47
654.09
PDS (3B)
71.80
636.74
86.58
295.21
33.84
700.39
22.22
963.78
6.67
1304.10
44.22
780.04
STEER (3B → 7B)
68.80
626.16
87.64
346.59
32.32
737.54
24.44
857.69
10.00
872.47
44.64
688.09
Table 1: Main results across five reasoning benchmarks. Among them, for STEER and GlimpRouter, A→B denotes that model A is assisted by model B during inference. Best and second-best results are highlighted in bold and underline, respectively.
Figure 3: Intervention behavior across tasks. Int. denotes the average number of interventions among Bank-routed problems, averaged across the RLC (3B, 7B, and 14B).
Figure 4: (a) Upward cross-scale Bank transfer from Qwen2.5-14B to Qwen2.5-32B. The Bank is constructed with the 14B model and applied to the 32B model for inference. (b) Comparison of online latent correction and direct LLM feedback in terms of accuracy and total generated output tokens, where the token count for direct feedback includes both local-model and API outputs.
Target
Configuration
MATH500
GSM8K
GPQA
AMC24
AIME
Average
3B Native
11.83
5.94
13.73
27.12
16.76
15.08
3B
3B RLC
11.33
5.44
12.92
29.10
19.70
15.70
7B Native
13.12
6.65
13.34
24.34
33.09
18.11
7B Native
13.12
6.65
13.34
24.34
33.09
18.11
7B
7B RLC
13.49
6.45
15.15
26.69
31.56
18.67
32B Native
16.73
7.34
15.60
29.26
27.76
19.34
Table 2: Inference efficiency comparison of RLC with same-scale and larger Native models. Results report the average inference time per problem, with lower values indicating better efficiency.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Variant
MATH500
GSM8K
GPQA
AMC24
AIME †
Average
Acc.
Tok.
Acc.
Tok.
Acc.
Tok.
Acc.
Tok.
Acc.
Tok.
Acc.
Tok.
3B
RLC
73.00
576.91
89.99
272.68
40.91
690.92
24.44
992.78
10.00
921.93
47.67
691.04
Mean Value
70.00
574.73
86.28
272.29
36.36
723.97
20.00
1003.78
10.00
963.77
44.53
707.71
7B
RLC
79.80
564.70
94.09
279.07
45.45
654.44
37.78
1100.38
20.00
1300.53
55.42
779.82
Mean Value
76.80
568.91
93.71
279.48
40.40
652.57
35.56
906.07
10.00
1335.27
51.29
748.46
14B
RLC
83.00
583.87
95.68
280.45
52.02
467.12
37.78
864.58
20.00
1120.07
57.70
663.22
Appendix
Table 3: Effect of state-conditioned correction retrieval. Mean Value replaces each retrieved correction with the global mean of all correction values in the Bank, removing the correspondence between the current reasoning state and its retrieved correction.
Injection layer
Acc.
−2
73.00
−4
75.00
−8
74.00
−12
74.00
−18
73.00
Appendix
Table 4: Sensitivity of RLC to the injection layer on Qwen2.5-3B.
Method
MATH500
GSM8K
GPQA
AMC24
AIME24
Average
Acc.
Tok.
Int.
Acc.
Tok.
Int.
Acc.
Tok.
Int.
Acc.
Tok.
Int.
Acc.
Tok.
Int.
Acc.
Tok.
32B Bank → 14B
Qwen2.5-14B Native
76.20
602.48
–
94.47
290.74
–
42.93
634.18
–
31.11
766.82
–
13.33
1122.87
–
51.61
683.42
32B Bank → 14B
78.40
561.40
3.16
95.45
280.93
0.37
42.93
647.59
0.32
33.33
1138.64
11.74
13.33
1363.53
16.30
52.69
798.42
14B Bank → 32B
Qwen2.5-32B Native
82.80
530.40
–
95.15
231.24
–
42.93
481.80
–
35.56
947.33
–
13.33
839.73
–
53.95
606.10
Appendix
Table 5: Full cross-scale Bank transfer results.
Route
MATH500
GSM8K
GPQA
AMC24
AIME25
Avg. Acc.
Acc.
Bank (%)
Acc.
Bank (%)
Acc.
Bank (%)
Acc.
Bank (%)
Acc.
Bank (%)
Qwen2.5-3B-Instruct
Native
67.00
0.00
84.69
0.00
30.30
0.00
4.44
0.00
3.33
0.00
37.95
Bank
65.00
100.00
83.70
100.00
31.82
100.00
22.22
100.00
6.67
100.00
41.88
Router
73.00
50.60
89.99
40.71
40.91
48.99
24.44
60.00
10.00
73.33
47.67
Appendix
Table 6: Effectiveness of the Router across different model scales and reasoning benchmarks.
Model
Method
MATH500
GSM8K
GPQA
AMC24
AIME
Acc.
Small
API
Total
Acc.
Small
API
Total
Acc.
Small
API
Total
Acc.
Small
API
Total
Acc.
Small
API
Total
3B
Qwen3-Max Direct
70.20
792.68
425.23
1217.91
87.82
297.15
126.13
423.28
36.90
896.59
308.09
1204.68
22.22
1529.93
1312.67
2842.60
10.00
2493.67
4033.10
6526.77
RLC
73.00
576.91
–
576.91
89.99
272.68
–
272.68
40.91
690.92
–
690.92
24.44
992.78
–
992.78
10.00
921.93
–
921.93
7B
Qwen3-Max Direct
79.60
757.36
556.52
1313.88
92.49
302.68
76.49
379.17
46.97
643.12
52.03
695.15
35.56
2329.78
3437.78
5767.56
16.67
2618.57
6114.10
8732.67
RLC
79.80
564.70
–
564.70
94.09
279.07
–
279.07
45.45
654.44
–
654.44
37.78
1100.38
–
1100.38
20.00
1300.53
–
1300.53
14B
Qwen3-Max Direct
81.60
668.33
452.76
1121.09
94.92
292.36
61.42
353.78
53.03
481.49
77.26
558.76
42.22
934.42
994.51
1928.93
23.33
1241.53
1100.77
2342.30
Appendix
Table 7: Comparison between online latent correction and direct LLM feedback using Qwen3-Max. Native is omitted because it does not involve any intervention. A dash indicates that RLC makes no online API calls. Small denotes the number of output tokens generated by the SLM, API denotes the number of output tokens generated by Qwen3-Max, and Total is the sum of the two.
Model
MATH500
GSM8K
GPQA
AMC24
AIME
Average
RLC (3B)
5.09
3.25
1.80
12.52
10.73
6.68
RLC (7B)
4.27
1.69
0.35
8.84
7.13
4.46
RLC (14B)
9.15
9.00
1.04
10.46
7.74
7.48
Average
6.17
4.65
1.06
10.61
8.53
6.20
Appendix
Table 8: Intervention behavior across tasks. We report the average number of interventions among Bank-routed problems. The rightmost column reports the mean across the five datasets for each model, whereas the bottom row reports the mean across the RLC 3B, 7B, and 14B for each dataset.
Figure 5: Case study of offline Bank construction in RLC. An incorrect reasoning step is identified, corrected with localized LLM guidance, and converted into a validated reusable latent experience.
Figure 6: Case study of online latent experience reuse on GSM8K. RLC retrieves relevant latent Bank entries to correct a local reasoning error and steer the frozen SLM toward the correct solution. The displayed Bank Construction Evidence is provided only to interpret the retrieved latent experiences and is not used during online inference.
Figure 7: Case study of online latent experience reuse on MATH500. RLC retrieves relevant latent Bank entries to correct a local reasoning error and steer the frozen SLM toward the correct solution. The displayed Bank Construction Evidence is provided only to interpret the retrieved latent experiences and is not used during online inference.
Figure 8: Case study of online latent experience reuse on AMC24. RLC retrieves relevant latent Bank entries to correct a local reasoning error and steer the frozen SLM toward the correct solution. The displayed Bank Construction Evidence is provided only to interpret the retrieved latent experiences and is not used during online inference.
Large language models (LLMs) demonstrate superior reasoning capabilities compared to small language models (SLMs), but incur substantially higher costs. We propose COllaborative REAsoner (COREA), a system that cascades an SLM with an LLM to achieve a balance between accuracy and cost in complex reasoning tasks. COREA first attempts to answer questions using the SLM, which outputs both an answer and a verbalized confidence score. Questions with confidence below a predefined threshold are deferred to the LLM for more accurate resolution. We introduce a reinforcement learning-based training algorithm that aligns the SLM's confidence through an additional confidence calibration reward. Extensive experiments demonstrate that our method jointly improves the SLM's reasoning ability and confidence calibration across diverse datasets and model backbones. Compared to using the LLM alone, COREA reduces cost by 21.5% and 16.8% on out-of-domain math and non-math datasets, respectively, with only an absolute pass@1 drop within 2%.
Collaboration between a small language model (SLM) and a large language model (LLM) offers an opportunity to combine the efficiency of smaller models with the strong reasoning capabilities of larger ones. Existing approaches primarily frame such collaboration as a computation allocation problem, determining which model should handle each portion of the reasoning process. In black-box API-based settings, however, this paradigm can be inefficient due to coarse-grained delegation or repeated transmission of context across model switches. In this work, we instead formulate SLM-LLM collaboration as an information acquisition problem, under an API budget constraint. The SLM remains the primary reasoner and selectively queries a black-box LLM advisor only when needed, issuing targeted queries rather than delegating the reasoning process itself. To realize this strategy, we develop a three-stage RLVR framework that learns whether to call the advisor, how to formulate useful queries, and how to integrate the collaboration into the reasoning process by jointly refining advisor invocation and information use. Across mathematical reasoning and coding tasks, our approach improves the performance--cost tradeoff over existing collaboration baselines and, in some settings, matches or exceeds oracle problem-level routing. Finally, we show that our strategy can transfer to other advisor model families, without further training.
Yongjun Kim, Xiaoxiao Li, Jaeho Lee
Pohang University of Science and Technology (POSTECH) · Department of Electrical and Computer Engineering, University of British Columbia · Vector Institute
Recently, language models have made rapid progress across various domains and applications. However, their capability for self-improvement, i.e., whether they are adept at recognising and correcting flaws in their own reasoning, remains dubious. In this study, we address this question by constructing a sufficiency test to rigorously examine the self-correction capabilities of small language models (SLMs). We propose a minimal three-step self-correction pipeline that collects initial SLM answers, prompts the same model to generate hints for its incorrect responses given the ground truth, and feeds the model the same question with its own feedback to refine the initial answer. We evaluate a variety of instruction-tuned and reasoning SLMs in this experimental setup on arithmetic and logical reasoning benchmarks. Our findings show that SLMs with injected hint sentences yield only a 4.4 percent gain over initial question-answering accuracy. Even though the correct answer was provided alongside the model's incorrect reasoning, the evaluated SLMs fail to understand what was missing in their reasoning and show minimal semantic difference between hints that lead to corrections and ones that do not. Furthermore, our experiments show that longer hints are positively correlated with incorrect final answers, suggesting that longer deliberation on problems can hinder the reasoning process, meaning that SLMs do not necessarily scale in performance with a larger compute budget.