Organizations: China University of Petroleum (East China), Qingdao, China · College of Cryptology and Cyber Science, Nankai University, Tianjin, China · Goertek Inc., Qingdao, China
When facing complex problems, humans tend to try various ideas for different issues. Human thinking patterns exhibit remarkable flexibility in adapting to diverse scenarios. GPT-o1, GPT-o3, and DeepSeek-R1 adopt long chain-of-thought models to address complex problems by increasing reasoning depth, which default to a forward reasoning mode. We conducted statistical analysis on the accuracy of different mathematical problem datasets on models of different scales, and found five reasons for errors: Insufficient solution-space coverage, Computational mistakes, Unverified assumptions, Ignoring constraint conditions, Maximum response length limitation. To address the above issues, we proposed a backward reasoning pattern construction method aimed at enhancing the model's reverse thinking ability and dynamic adaptability. First, we constructed an easy-hard two-stage Math dataset for training large models and gradually improving their inference ability at different difficulty levels. The dataset contains forward reasoning paths as well as backward reasoning paths. And a two-stage supervised fine-tuning process is applied to progressively train the model's backward reasoning capability. Furthermore, a fine-grained reward mechanism is developed, employing smoothed reward signals to strengthen the model's ability to autonomously select thinking modes during the reasoning process, thereby avoiding reward hacking. A linear-decay balanced sampling strategy is designed to maintain a balance between forward and backward reasoning path samples during training, enabling the model to converge quickly and stably. Experimental results show that our method significantly improves reasoning efficiency and accuracy in tasks such as mathematical proofs, offering a flexible and efficient reasoning paradigm for solving complex problems.
Figures & tables
Figure 1: Error type distribution and average response length statistics.
Figure 2: Method framework diagram.
Figure 3: Comparison of model reasoning methods.
Method
GSM8K
MATH-500
AIME-2024
BQ
Avg
Acc
Length
Acc
Length
Acc
Length
Acc
Length
Acc
Length
Short-Cot
Qwen2.5-math
84.8
407
74.4
682
16.6
938
71.0
812
61.7
710
Qwen2.5-3b
85.0
316
62.0
608
10.0
1161
57.3
1056
53.6
785
Llama-3.2-1b
44.4
168
18.8
686
3.3
1004
19.3
1025
21.5
721
Llama-3.2-3b
77.7
205
29.0
657
1.6
1078
30.0
1023
34.6
741
Table 1: Performance comparison of various models across the four test datasets.
Method
GSM8K-B
MATH-500-B
AIME-B
Avg
Acc
Length
Acc
Length
Acc
Length
Acc
Length
Short-Cot
Qwen2.5-math
78.5
397
57.5
894
25.0
1018
53.7
770
Qwen2.5-3b
75.9
335
42.5
779
8.3
820
42.2
645
Llama-3.2-1b
11.4
189
12.5
798
0
1390
8.0
792
Llama-3.2-3b
59.5
223
15.0
1247
0
765
24.8
745
Table 2: Performance comparison of various models on the backward reasoning subsets across the four test datasets.
Method
GSM8K
MATH-500
AIME
BQ
Avg
Acc
Length
Acc
Length
Acc
Length
Acc
Length
Acc
Length
R1-Distill-1.5b
79.0
978
80.6
4887
23.3
11985
75.3
5124
64.6
5746
+SFT stage1+2
80.3
1064
81.4
4279
25.0
12717
76.7
4979
65.9
5851
+SFT stage1
81.0
1426
81.8
4909
25.0
12165
77.0
4953
66.2
5863
+SFT stage2
80.9
1674
83.0
4452
30.0
12974
79.0
4512
68.2
5903
+RL
83.7
1422
84.8
2938
33.3
9730
82.3
3248
71.0
4335
Table 3: Comparison of model performance across training stages.
Figure 4: Comparison of forward and backward reasoning ratios.
Multi-step Chain-of-Thought (CoT) has significantly advanced the mathematical reasoning capabilities of LLMs by leveraging explicit reasoning steps. However, the widespread adoption of Long CoT often results in sequence lengths that exceed manageable computational limits. While existing approaches attempt to alleviate this by reducing KV Cache redundancy via Markov chain-like structures, they introduce two critical limitations: inherent memorylessness (loss of context) and limited backward reasoning capability. To address these limitations, we propose a novel Chain-of-Thought framework based on Reversible Hierarchical Markov Chain, termed Cognitive Loop of Thought (CLoT), and a backward reasoning dataset CLoT-Instruct. In CLoT, problems are decomposed into sub-problems with hierarchical dependencies. Inspired by human cognitive processes, we introduce a backward verification mechanism at each hierarchical layer. Furthermore, we implement a pruning strategy: once higher-level sub-problems are verified, redundant lower-level sub-problems are pruned to maximize efficiency. This approach effectively mitigates error propagation and enhances reasoning robustness. Experiments on four mathematical benchmarks demonstrate the effectiveness of our method. Notably, on the AddSub dataset using GPT-4o-mini, CLoT achieves 99.0% accuracy, outperforming traditional CoT and CoT-SC by 4.1% and 2.9%, respectively.
Jia-Chen Zhang, Yu-Jie Xiong, Zheng Zhou
School of Computer Science and Technology, East China Normal University 3663 North Zhongshan Road, Shanghai, China · School of Electronic and Electrical Engineering, Shanghai University of Engineering Science 333 Longteng Road, Shanghai, China
Large reasoning models (LRMs) have attracted increasing attention for their ability to solve complex mathematical problems by generating extended reasoning chains. In this work, we focus on two critical yet underexplored aspects of the reasoning process: reasoning transitions capturing the distinct transitions between reasoning steps and answer candidates reflecting the variety of solution paths produced by the model. We collectively define these two aspects as thinking schemata. We observe a correlation between the diversity of thinking schemata and model performance, which motivates us to enhance diversity as a means to further improve reasoning potential. To this end, we propose Diverse Schemata Policy Optimization (DiScO), a framework that first endows the model with schemata awareness, then encourages diversity through reinforcement learning, and further promotes diverse reasoning at inference time. Experiments on multiple mathematical reasoning benchmarks demonstrate that DiScO consistently outperforms standard group relative policy optimization. Beyond accuracy, human-annotated analyses show that DiScO substantially improves the model's ability to recover from erroneous initial attempts. Overall, our work suggests the important role that diversity of the thinking schemata plays and points to scaling along the diversity dimension as a promising research direction.
Xinyue Liang, Yizhe Yang, Yu Bai +3
School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China
Mathematical reasoning benchmarks are vital for evaluating large language models (LLMs), but many are static and repeatedly exposed through public evaluation and training pipelines, making it difficult to separate genuine reasoning from memorization. Meanwhile, manually constructing new math problems with reliable answers remains costly. We introduce ReverseMath, a scalable method for generating new math problems through answer inversion. Given a problem and its answer, ReverseMath masks a numerical value in the original problem, treats the original answer as a known condition, and rewrites the problem so that the masked value becomes the new answer. The generated problem reverses the original input-output relation, making its answer known by construction. We study ReverseMath for both evaluation and training. For evaluation, paired original/reversed problems reveal substantial behavioral shifts: models sometimes fail on reversed problems and even incorrectly output the original answer, suggesting memorization-like behavior. For training, ReverseMath provides automatically labeled reversed problems as data augmentation for reinforcement learning (RL). Experiments show that including ReverseMath-generated data improves mathematical reasoning performance across multiple benchmarks, demonstrating its value as both an analysis tool and a scalable source of verifiable training data.
Raoyuan Zhao, Yihong Liu, Yupei Du +2
Center for Information and Language Processing, LMU Munich · 2Munich Center for Machine Learning (MCML) · 3Saarland University