HintMR: Eliciting Stronger Mathematical Reasoning in Small Language Models

Jawad Hossain; Xiangyu Guo; Jiawei Zhou; Chong Liu

cs.AIApr 14, 2026

HintMR: Eliciting Stronger Mathematical Reasoning in Small Language Models

Authors: Jawad Hossain, Xiangyu Guo, Jiawei Zhou, Chong Liu

Organizations: University at Albany · University at Buffalo · Stony Brook University

Abstract

Small language models (SLMs) often struggle with complex mathematical reasoning due to limited capacity to maintain long chains of intermediate steps and to recover from early errors. We address this challenge by introducing a hint-assisted reasoning framework that incrementally guides SLMs through multi-step mathematical problem solving. Our approach decomposes solutions into sequential reasoning steps and provides context-aware hints, where hints are generated by a separate SLM trained via distillation from a strong large language model. While the hint-generating SLM alone is not capable of solving the problems, its collaboration with a reasoning SLM enables effective guidance, forming a cooperative two-model system for reasoning. Each hint is generated conditionally on the problem statement and the accumulated reasoning history, providing stepwise, localized guidance without revealing full solutions. This reduces error propagation and allows the reasoning model to focus on manageable subproblems. Experiments across diverse mathematical benchmarks and models demonstrate that hint assistance consistently improves reasoning accuracy for SLMs, yielding substantial gains over standard prompting while preserving model efficiency. These results highlight that structured collaboration between SLMs-via hint generation and reasoning-offers an effective and lightweight mechanism for enhancing mathematical reasoning.

Figures & tables

Figure 1: Overview of the HintMR inference. A LLM generates step-wise oracle hints, which are distilled into a SLM for efficient hint generation. During inference, the SLM solver incrementally constructs its reasoning trajectory using either no hints, LLM-generated hints, or SLM-generated hints. While both SLM and LLM hints enable correct solutions, SLM hints achieve comparable performance at significantly lower token cost.

Dataset	Model	w/o Hints	w/ LLM Hints	w/ SLM Hints	w/ NFT SLM Hints
NuminaMath	Qwen2.5-Math-7B-Instruct	46.67% (35/75)	78.67% (59/75)	58.67% (44/75)	36.00% (27/75)
	DeepSeek-R1-Distill-Qwen-7B	44.00% (33/75)	69.33% (52/75)	60.00% (45/75)	42.67% (32/75)
	DeepSeek-R1-Distill-Llama-8B	42.67% (32/75)	68.00% (51/75)	45.33% (34/75)	26.67% (20/75)
	Phi-4-mini-reasoning	21.33% (16/75)	33.33% (25/75)	29.33% (22/75)	21.33% (16/75)
	FT DeepSeek-R1-Distill-Qwen-7B	38.67% (29/75)	65.33% (49/75)	37.33% (28/75)	45.33% (34/75)
MATH-500	Qwen2.5-Math-7B-Instruct	75.68% (56/74)	85.14% (63/74)	79.73% (59/74)	54.05% (40/74)

Table 1: Accuracy comparison (%) of SLMs with and without hint assistance across four mathematical reasoning datasets. Best with-hints performance per dataset is bolded.

Figure 2: Performance curves showing improvement in reasoning quality with more hints.

Dataset	Method	Avg Time (s)	Avg Tokens	Acc. (%)
MATH-500	No-Hint	61.35	920	64.86
	HintMR	165.34	1180	82.43
	SC (K=8)	689.46	7350	70.27
AIME-2024	No-Hint	88.84	929	20.69
	HintMR	205.43	781	68.97
	SC (K=8)	805.33	7670	20.69

Table 2: Accuracy–computing trade-offs for DeepSeek-R1-Distill-Qwen-7B across datasets. HintMR achieves significantly higher accuracy than no-hint inference with moderate computational overhead, while remaining substantially more efficient than SC decoding.

Dataset	LLM Tokens	SLM Tokens	Reduction (%)
NuminaMath	198.43	145.36	26.74%
MATH-500	212.58	149.82	29.53%
AIME-2024	236.79	152.70	35.52%
AIME-2025	229.66	148.91	35.18%

Table 3: Average tokens per problem for hint generation. SLM hints consistently require fewer tokens than LLM hints, reducing cost and latency.

Dataset	Setting	Mean Acc. (%)
AIME-2024	w/o Hints	18.10 $\pm$ 0.86
	w/ LLM Hints	46.55 $\pm$ 1.59
	w/ SLM Hints	29.31 $\pm$ 1.13
	w/ NFT SLM Hints	23.28 $\pm$ 0.86
AIME-2025	w/o Hints	9.17 $\pm$ 1.22
	w/ LLM Hints	8.34 $\pm$ 1.09

Table 4: Performance stability across 8 runs for FT DeepSeek-R1-Distill-Qwen-7B on AIME-2024 and AIME-2025 datasets.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
LoRA Rank ( $r$ )	16
LoRA Scaling ( $\alpha$ )	32
LoRA Dropout	0.1
Optimizer	8-bit Paged AdamW
Learning Rate	$1\times 10^{-4}$
Learning Rate Scheduler	Cosine

Appendix

Table 5: Hyperparameter settings for fine-tuning DeepSeek-R1-Distill-Qwen-7B.

Figure 3: Summarized reasoning trajectory of a predicted solution generated by DeepSeek-R1-Distill-Qwen-7B without hints. Without external guidance, the model struggles to maintain a coherent reasoning path and fails to reach the correct final answer, repeatedly producing phrases such as “getting stuck here” (highlighted in red), indicating stalled reasoning.

Figure 4: Summarized reasoning trajectory of a predicted solution generated by DeepSeek-R1-Distill-Qwen-7B using LLM-generated hints. With guidance from 7 hints, the model maintains a coherent reasoning path, progressively identifying the pattern in the game states and ultimately arriving at the correct final answer.

Figure 5: Summarized reasoning trajectory of a predicted solution generated by DeepSeek-R1-Distill-Qwen-7B using SLM-generated hints. Despite the first hint being suboptimal, the subsequent hints guide the model toward a coherent reasoning path, allowing it to reach the correct final answer using 7 hints, the same number required when using LLM-generated hints.

LLM Mathematical Reasoning

Large language model mathematical reasoning studies how AI systems solve, explain, verify, and generalize across mathematical problems and proofs.

See all 270 papers

Placing Puzzle Pieces Where They Matter: A Question Augmentation Framework for Reinforcement Learning
Apr 17, 2026Yangyi Fang, Jiaye Lin, Xiaoliang Fu +2RL for Language Model Reasoning LLM Mathematical Reasoning
Tsinghua University · Fudan University · Peking University
Cognitive Loop of Thought: Reversible Hierarchical Markov Chain for Efficient Mathematical Reasoning
Apr 8, 2026Jia-Chen Zhang, Yu-Jie Xiong, Zheng ZhouStructured CoT Reasoning LLM Mathematical Reasoning
School of Computer Science and Technology, East China Normal University 3663 North Zhongshan Road, Shanghai, China · School of Electronic and Electrical Engineering, Shanghai University of Engineering Science 333 Longteng Road, Shanghai, China
Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning
Apr 26, 2026Zichuan Fu, Xian Wu, Guojing Li +7Efficient Language Model Reasoning LLM Inference Acceleration
City University of Hong Kong · Tencent Jarvis Lab · Renmin University of China +1

Apr 17, 2026J/K move · Enter open · S save

Placing Puzzle Pieces Where They Matter: A Question Augmentation Framework for Reinforcement Learning

Yangyi Fang, Jiaye Lin, Xiaoliang Fu, Cong Qin+1

Tsinghua University · Fudan University · Peking University

RL for Language Model Reasoning LLM Mathematical Reasoning

Open paper AskPDF arXiv

Reinforcement learning has become a powerful approach for enhancing large language model reasoning, but faces a fundamental dilemma: training on easy problems can cause overfitting and pass@k degradation, while training on hard problems often results in sparse rewards. Recent question augmentation methods address this by prepending partial solutions as hints. However, uniform hint provision may introduce redundant information while missing critical reasoning bottlenecks, and excessive hints can reduce reasoning diversity, causing pass@k degradation. We propose PieceHint, a hint injection framework that strategically identifies and provides critical reasoning steps during training. By scoring the importance of different reasoning steps, selectively allocating hints based on problem difficulty, and progressively withdrawing scaffolding, PieceHint enables models to transition from guided learning to independent reasoning. Experiments on six mathematical reasoning benchmarks show that our 1.5B model achieves comparable average performance to 32B baselines while preserving pass@k diversity across all

k

values.

Abstract

LLM Mathematical Reasoning

Explore similar work

Placing Puzzle Pieces Where They Matter: A Question Augmentation Framework for Reinforcement Learning

Cognitive Loop of Thought: Reversible Hierarchical Markov Chain for Efficient Mathematical Reasoning

Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning