Cognitive Loop of Thought: Reversible Hierarchical Markov Chain for Efficient Mathematical Reasoning
Authors: Jia-Chen Zhang, Yu-Jie Xiong, Zheng Zhou
Organizations: School of Computer Science and Technology, East China Normal University 3663 North Zhongshan Road, Shanghai, China · School of Electronic and Electrical Engineering, Shanghai University of Engineering Science 333 Longteng Road, Shanghai, China
Multi-step Chain-of-Thought (CoT) has significantly advanced the mathematical reasoning capabilities of LLMs by leveraging explicit reasoning steps. However, the widespread adoption of Long CoT often results in sequence lengths that exceed manageable computational limits. While existing approaches attempt to alleviate this by reducing KV Cache redundancy via Markov chain-like structures, they introduce two critical limitations: inherent memorylessness (loss of context) and limited backward reasoning capability. To address these limitations, we propose a novel Chain-of-Thought framework based on Reversible Hierarchical Markov Chain, termed Cognitive Loop of Thought (CLoT), and a backward reasoning dataset CLoT-Instruct. In CLoT, problems are decomposed into sub-problems with hierarchical dependencies. Inspired by human cognitive processes, we introduce a backward verification mechanism at each hierarchical layer. Furthermore, we implement a pruning strategy: once higher-level sub-problems are verified, redundant lower-level sub-problems are pruned to maximize efficiency. This approach effectively mitigates error propagation and enhances reasoning robustness. Experiments on four mathematical benchmarks demonstrate the effectiveness of our method. Notably, on the AddSub dataset using GPT-4o-mini, CLoT achieves 99.0% accuracy, outperforming traditional CoT and CoT-SC by 4.1% and 2.9%, respectively.
Figures & tables
Figure 1: Overview of the Reversible Hierarchical Markov Chain Framework.
Figure 2: Overview of the CLoT Framework.
Models
AddSub
GSM8K
SVAMP
MATH
AQuA
CSQA
Avg.
gpt-4o-mini
CoT
94.9
90.9
93.4
78.3
81.4
81.5
86.7
CoT-SC(n=5)
96.1
92.0
93.5
81.0
83.1
83.4
88.3
ToT
76.7
91.5
93.2
80.7
72.3
81.7
82.7
AR
82.3
91.7
92.4
73.1
77.5
78.7
81.0
C-CoT
97.2
92.4
93.4
76.9
83.3
81.0
87.4
Table 1: Accuracy of CLoT and baselines on six mainstream datasets. Higher is better for all metrics. The best score , and second best score are red, and orange, respectively.
Figure 3: The results of CLoT efficiency. The token consumption corresponds to the total number of tokens used to solve the same 100 gsm8k problem.
Figure 4: We present a visualization of the dataset analysis. Figure (a) shows the various accuracy metrics on the dataset, where V-A denotes Verification Accuracy. Figure (b) shows the maximum number of layers in the multi-layer Markov chains used in the dataset.
Model
Accuracy
Qwen-2.5-1.5B
75.33%
Qwen-2.5-1.5B-CLoT-Instruct
81.67%
Table 2: SFT Performance Comparison
Models
Token
AddSub
GSM8K
SVAMP
AQuA
Avg.
CoT (baseline)
60k
94.9
90.9
93.4
81.4
90.2
CoT + SV
98k
94.0
93.5
92.3
81.4
90.3
CoT + HMC
112k
94.9
90.9
93.4
81.0
90.1
CoT + RHMC
325k
94.4
93.5
93.9
84.6
91.6
CoT + RHMC + HP
136k
99.0
94.6
94.9
85.8
93.6
Table 3: Ablation study results of CLoT based on gpt-4o-mini. The experiment used four datasets and token consumption to evaluate the contributions of three components. The token consumption corresponds to the total number of tokens used to solve the same 100 GSM8K problems (idx0 – idx99).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Reasoning Task
Answer Type
Number
MATH
Mathematical
Number
900
GSM8K
Mathematical
Number
1,319
AddSub
Mathematical
Number
395
SVAMP
Mathematical
Number
1,000
AQuA
Mathematical
Multi-choice
254
CSQA
Commonsense
Multi-choice
1,220
Appendix
Table 4: Overview of datasets utilized in CLoT experiments.
Task
Answer-format Instructions
GSM8K, SVAMP, AddSub, math
You can freely reason in your response, but please enclose the final answer within < answer >< /answer > tags(pure number without units and explanations)
AQuA, CommonsenseQA
You can freely reason in your response, but please enclose the final option within < answer >< /answer > tags(pure uppercase option without explanations)
Appendix
Table 5: Answer-format instructions for different tasks.
SVAMP Problem
Q: Kylar went to the store to buy glasses for his new apartment. One glass costs $5, but every second glass costs only 60% of the price. Kylar wants to buy 16 glasses. How much does he need to pay?
Step 1 , First CoT
Q: Let’s think step by step.
Thought* < answer >64< /answer >
Step 2 , Backward Verification
Q: Please replace one numerical value in the original problem with X and incorporate the answer as a known condition.
Appendix
Table 6: Examples of Successful Verification by CLoT on Mathematical Problem Datasets.
Method
Accuracy (%)
CoT
26.67
CoT-SC
23.33
CLoT (Ours)
30.00
Appendix
Table 7: Performance comparison on the AIME dataset (gpt-4).
SVAMP Problem
Q: Kylar went to the store to buy glasses for his new apartment. One glass costs $5, but every second glass costs only 60% of the price. Kylar wants to buy 16 glasses. How much does he need to pay?
Step 1 , First CoT
Q: Let’s think step by step.
Thought* < answer >96< /answer >
Step 2 , Backward Verification
Q: Please replace one numerical value in the original problem with X and incorporate the answer as a known condition.
Appendix
Table 8: Examples of CLoT’s verification failures on mathematical problem datasets.
China University of Petroleum (East China), Qingdao, China · College of Cryptology and Cyber Science, Nankai University, Tianjin, China · Goertek Inc., Qingdao, China