PRISM-MCTS: Learning from Reasoning Trajectories with Metacognitive Reflection

Siyuan Cheng; Bozhong Tian; YanChao Hao; Zheng Wei

cs.AIApr 7, 2026

PRISM-MCTS: Learning from Reasoning Trajectories with Metacognitive Reflection

Authors: Siyuan Cheng, Bozhong Tian, YanChao Hao, Zheng Wei

Organizations: Tencent PCG

Abstract

PRISM-MCTS: Learning from Reasoning Trajectories with Metacognitive Reflection Siyuan Cheng, Bozhong Tian, Yanchao Hao, Zheng Wei Published: 06 Apr 2026, Last Modified: 06 Apr 2026 ACL 2026 Findings Conference, Area Chairs, Reviewers, Publication Chairs, Authors Revisions BibTeX CC BY 4.0 Keywords: Efficient/Low-Resource Methods for NLP, Generation, Question Answering Abstract: The emergence of reasoning models, exemplified by OpenAI o1, signifies a transition from intuitive to deliberative cognition, effectively reorienting the scaling laws from pre-training paradigms toward test-time computation. While Monte Carlo Tree Search (MCTS) has shown promise in this domain, existing approaches typically treat each rollout as an isolated trajectory. This lack of information sharing leads to severe inefficiency and substantial computational redundancy, as the search process fails to leverage insights from prior explorations. To address these limitations, we propose PRISM-MCTS, a novel reasoning framework that draws inspiration from human parallel thinking and reflective processes. PRISM-MCTS integrates a Process Reward Model (PRM) with a dynamic shared memory, capturing both "Heuristics" and "Fallacies". By reinforcing successful strategies and pruning error-prone branches, PRISM-MCTS effectively achieves refinement. Furthermore, we develop a data-efficient training strategy for the PRM, achieving high-fidelity evaluation under a few-shot regime. Empirical evaluations across diverse reasoning benchmarks substantiate the efficacy of PRISM-MCTS. Notably, it halves the trajectory requirements on GPQA while surpassing MCTS-RAG and Search-o1, demonstrating that it scales inference by reasoning judiciously rather than exhaustively.

Figures & tables

Figure 1: Standard MCTS (left) vs. PRISM-MCTS (right). PRISM-MCTS enables global information sharing to prune incorrect paths and reuse verified steps for higher efficiency.

Figure 2: Overview of the PRISM-MCTS framework. It integrates MCTS with a PRM (Continuous-PRM or Discrete-PRM) and a reflective memory system. Stored Heuristics Memory and Fallacies Memory facilitate global information sharing to guide generation and prune the search space efficiently.

Model	Method	GPQA-Diamond		FMT		MATH500	AIME25
Model	Method	EM $\uparrow$	F1 $\uparrow$	EM $\uparrow$	F1 $\uparrow$	Score $\uparrow$	Score $\uparrow$
GPT-4.1-mini	Zero-shot	47.55	46.55	68.34	68.75	79.25	48.67
	ReAct	55.05	60.75	65.00	67.61	79.10	50.00
	Search-O1	57.07	57.16	66.00	65.03	80.59	43.33
	ReST-MCTS ∗	60.61	60.53	69.00	68.91	70.90	20.00
	MCTS-RAG	64.65	64.80	69.50	68.98	80.60	50.00

Table 1: Main performance comparison of PRISM-MCTS against various baselines on scientific verification (GPQA-Diamond, FMT) and mathematical reasoning (MATH, AIME25) benchmarks. Best results for each backbone model are highlighted in bold .

Figure 3: Comparison of search efficiency between MCTS-RAG and PRISM-MCTS. The left panels display the average number of explored Trajectories (search breadth), while the right panels show the average reasoning Depth. PRISM-MCTS consistently reduces the search space while maintaining optimized reasoning paths.

	EM $\uparrow$	F1 $\uparrow$	LA $\uparrow$	Traj $\downarrow$	Depth $\downarrow$
GPT-4.1-mini
Oracle-PRM	70.50	70.07	76.00	10.60	2.24
Light-PRM	70.50	69.91	80.50	14.91	2.33
Qwen3-30B-A3B-2507
Oracle-PRM	68.00	67.41	82.50	5.95	2.59
Light-PRM	68.00	67.41	79.50	6.65	2.52

Table 2: Comparison of reasoning performance and search efficiency between the frontier Gemini-2.5-Pro and our proposed locally-trained dual-stage PRM.

Figure 4: Ablation study on MATH500 and AIME25 benchmarks evaluating the impact of Heuristics Memory ( $\mathcal{MEM}_{H}$ ) and Fallacies Memory ( $\mathcal{MEM}_{F}$ ) modules. The scatter plot correlates search efficiency (Trajectories) with reasoning performance (Score), where color indicates reasoning Depth. Removing memory components leads to performance degradation, while the full PRISM-MCTS maintains high accuracy with minimal search breadth, significantly outperforming the MCTS-RAG baseline in efficiency.

Explore similar work

Learning from Contrasts: Synthesizing Reasoning Paths from Diverse Search Trajectories
Apr 13, 2026Peiyang Liu, Zhirui Chen, Xi Wang +4LLM Reasoning Synthetic Data Generation for Language Models
National Engineering Research Center for Software Engineering, Peking University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · UCAS-Terminus AI Lab, University of Chinese Academy of Sciences, Shanghai, China +2
Thought-Level Beam Search for Reasoning
Aug 8, 2026Lijie Yang, Hongyin Luo, Jiawei Zhao +2Inference-Time Search Efficient Language Model Reasoning
Princeton University · MIT CSAIL · Meta AI
CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoning
Mar 30, 2026Siyuan Ma, Bo Gao, Zikai Xiao +6Efficient Language Model Reasoning Metacognition in Language Models
Nanyang Technological University · Carnegie Mellon University · Zhejiang University +5

Apr 13, 2026J/K move · Enter open · S save

Learning from Contrasts: Synthesizing Reasoning Paths from Diverse Search Trajectories

Peiyang Liu, Zhirui Chen, Xi Wang, Di Liang+3

National Engineering Research Center for Software Engineering, Peking University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · UCAS-Terminus AI Lab, University of Chinese Academy of Sciences, Shanghai, China+2

LLM Reasoning Synthetic Data Generation for Language Models Reasoning Trace Analysis

Open paper AskPDF arXiv

Monte Carlo Tree Search (MCTS) has been widely used for automated reasoning data exploration, but current supervision extraction methods remain inefficient. Standard approaches retain only the single highest-reward trajectory, discarding the comparative signals present in the many explored paths. Here we introduce Contrastive Reasoning Path Synthesis (CRPS), a framework that transforms supervision extraction from a filtering process into a synthesis procedure. CRPS uses a structured reflective process to analyze the differences between high- and low-quality search trajectories, extracting explicit information about strategic pivots and local failure modes. These insights guide the synthesis of reasoning chains that incorporate success patterns while avoiding identified pitfalls. We show empirically that models fine-tuned on just 60K CRPS-synthesized examples match or exceed the performance of baselines trained on 590K examples derived from standard rejection sampling, a 20

\times

reduction in dataset size. Furthermore, CRPS improves generalization on out-of-domain benchmarks, demonstrating that learning from the contrast between success and failure produces more transferable reasoning capabilities than learning from success alone.

Abstract

Explore similar work

Learning from Contrasts: Synthesizing Reasoning Paths from Diverse Search Trajectories

Thought-Level Beam Search for Reasoning

CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoning