PRISM-MCTS: Learning from Reasoning Trajectories with Metacognitive Reflection

Siyuan Cheng; Bozhong Tian; YanChao Hao; Zheng Wei

cs.AIApr 7, 2026

PRISM-MCTS: Learning from Reasoning Trajectories with Metacognitive Reflection

Authors: Siyuan Cheng, Bozhong Tian, YanChao Hao, Zheng Wei

Organizations: Tencent PCG

Abstract

PRISM-MCTS: Learning from Reasoning Trajectories with Metacognitive Reflection Siyuan Cheng, Bozhong Tian, Yanchao Hao, Zheng Wei Published: 06 Apr 2026, Last Modified: 06 Apr 2026 ACL 2026 Findings Conference, Area Chairs, Reviewers, Publication Chairs, Authors Revisions BibTeX CC BY 4.0 Keywords: Efficient/Low-Resource Methods for NLP, Generation, Question Answering Abstract: The emergence of reasoning models, exemplified by OpenAI o1, signifies a transition from intuitive to deliberative cognition, effectively reorienting the scaling laws from pre-training paradigms toward test-time computation. While Monte Carlo Tree Search (MCTS) has shown promise in this domain, existing approaches typically treat each rollout as an isolated trajectory. This lack of information sharing leads to severe inefficiency and substantial computational redundancy, as the search process fails to leverage insights from prior explorations. To address these limitations, we propose PRISM-MCTS, a novel reasoning framework that draws inspiration from human parallel thinking and reflective processes. PRISM-MCTS integrates a Process Reward Model (PRM) with a dynamic shared memory, capturing both "Heuristics" and "Fallacies". By reinforcing successful strategies and pruning error-prone branches, PRISM-MCTS effectively achieves refinement. Furthermore, we develop a data-efficient training strategy for the PRM, achieving high-fidelity evaluation under a few-shot regime. Empirical evaluations across diverse reasoning benchmarks substantiate the efficacy of PRISM-MCTS. Notably, it halves the trajectory requirements on GPQA while surpassing MCTS-RAG and Search-o1, demonstrating that it scales inference by reasoning judiciously rather than exhaustively.

Figures & tables

Figure 1: Standard MCTS (left) vs. PRISM-MCTS (right). PRISM-MCTS enables global information sharing to prune incorrect paths and reuse verified steps for higher efficiency.

Figure 2: Overview of the PRISM-MCTS framework. It integrates MCTS with a PRM (Continuous-PRM or Discrete-PRM) and a reflective memory system. Stored Heuristics Memory and Fallacies Memory facilitate global information sharing to guide generation and prune the search space efficiently.

Model	Method	GPQA-Diamond		FMT		MATH500	AIME25
Model	Method	EM $\uparrow$	F1 $\uparrow$	EM $\uparrow$	F1 $\uparrow$	Score $\uparrow$	Score $\uparrow$
GPT-4.1-mini	Zero-shot	47.55	46.55	68.34	68.75	79.25	48.67
	ReAct	55.05	60.75	65.00	67.61	79.10	50.00
	Search-O1	57.07	57.16	66.00	65.03	80.59	43.33
	ReST-MCTS ∗	60.61	60.53	69.00	68.91	70.90	20.00
	MCTS-RAG	64.65	64.80	69.50	68.98	80.60	50.00

Table 1: Main performance comparison of PRISM-MCTS against various baselines on scientific verification (GPQA-Diamond, FMT) and mathematical reasoning (MATH, AIME25) benchmarks. Best results for each backbone model are highlighted in bold .

Figure 3: Comparison of search efficiency between MCTS-RAG and PRISM-MCTS. The left panels display the average number of explored Trajectories (search breadth), while the right panels show the average reasoning Depth. PRISM-MCTS consistently reduces the search space while maintaining optimized reasoning paths.

	EM $\uparrow$	F1 $\uparrow$	LA $\uparrow$	Traj $\downarrow$	Depth $\downarrow$
GPT-4.1-mini
Oracle-PRM	70.50	70.07	76.00	10.60	2.24
Light-PRM	70.50	69.91	80.50	14.91	2.33
Qwen3-30B-A3B-2507
Oracle-PRM	68.00	67.41	82.50	5.95	2.59
Light-PRM	68.00	67.41	79.50	6.65	2.52

Table 2: Comparison of reasoning performance and search efficiency between the frontier Gemini-2.5-Pro and our proposed locally-trained dual-stage PRM.

Figure 4: Ablation study on MATH500 and AIME25 benchmarks evaluating the impact of Heuristics Memory ( $\mathcal{MEM}_{H}$ ) and Fallacies Memory ( $\mathcal{MEM}_{F}$ ) modules. The scatter plot correlates search efficiency (Trajectories) with reasoning performance (Score), where color indicates reasoning Depth. Removing memory components leads to performance degradation, while the full PRISM-MCTS maintains high accuracy with minimal search breadth, significantly outperforming the MCTS-RAG baseline in efficiency.

Explore similar work

Apr 13, 2026

Learning from Contrasts: Synthesizing Reasoning Paths from Diverse Search Trajectories

Monte Carlo Tree Search (MCTS) has been widely used for automated reasoning data exploration, but current supervision extraction methods remain inefficient. Standard approaches retain only the single highest-reward trajectory, discarding the comparative signals present in the many explored paths. Here we introduce Contrastive Reasoning Path Synthesis (CRPS), a framework that transforms supervision extraction from a filtering process into a synthesis procedure. CRPS uses a structured reflective process to analyze the differences between high- and low-quality search trajectories, extracting explicit information about strategic pivots and local failure modes. These insights guide the synthesis of reasoning chains that incorporate success patterns while avoiding identified pitfalls. We show empirically that models fine-tuned on just 60K CRPS-synthesized examples match or exceed the performance of baselines trained on 590K examples derived from standard rejection sampling, a 20

\times

reduction in dataset size. Furthermore, CRPS improves generalization on out-of-domain benchmarks, demonstrating that learning from the contrast between success and failure produces more transferable reasoning capabilities than learning from success alone.

Peiyang Liu, Zhirui Chen, Xi Wang +4

National Engineering Research Center for Software Engineering, Peking University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · UCAS-Terminus AI Lab, University of Chinese Academy of Sciences, Shanghai, China +2

LLM Reasoning Synthetic Data Generation for Language Models Reasoning Trace Analysis

Ask PDF

Aug 8, 2026

Thought-Level Beam Search for Reasoning

Test-time compute scaling is a primary driver of performance in large reasoning models (LRMs), but extreme inefficiency bounds current approaches, shifting the critical question from how much compute to spend, to where to allocate it. We formalize test-time reasoning as a constrained compute allocation problem over partial trajectories. Under a fixed hardware budget, existing paradigms fail to actively allocate the compute to the most promising partial progress: traditional parallel sampling treats traces independently and induces severe memory bottlenecks, while subtractive pruning starves hardware and fails to actively and sufficiently shift the output distribution. To overcome this dichotomy, we introduce Gambit, an inference algorithm that executes thought-level beam search. By periodically pruning unpromising trajectories and immediately branching from high-quality prefixes, Gambit dynamically concentrates compute onto the most promising reasoning traces via a light-weight scorer probing hidden states while maintaining continuous high hardware utilization. Extensive evaluations across multiple models and benchmarks demonstrate that Gambit strictly dominates existing baselines. Under identical hardware constraints, our method yields up to a +6.7% absolute accuracy gain on HMMT-24 and +3.3% on AIME-25 over pruning baselines, delivers

>2\times

higher throughput on trace completion, and reduces total token consumption by up to 68.5% relative to standard parallel sampling.

Lijie Yang, Hongyin Luo, Jiawei Zhao +2

Princeton University · MIT CSAIL · Meta AI

Inference-Time Search Efficient Language Model Reasoning Inference-Time Compute Allocation Efficient Language Model Inference Algorithmic Reasoning Large Reasoning Models

Ask PDF

Mar 30, 2026

CoT2-Meta: Budgeted Metacognitive Control for Test-Time Reasoning

Recent test-time reasoning methods improve performance by generating more candidate chains or searching over larger reasoning trees, but they typically lack explicit control over when to expand, what to prune, how to repair, and when to abstain. We introduce CoT2-Meta, a training-free metacognitive reasoning framework that combines object-level chain-of-thought generation with meta-level control over partial reasoning trajectories. The framework integrates four components: strategy-conditioned thought generation, tree-structured search, an online process oracle for step-level reasoning evaluation, and a meta-controller that allocates computation through expansion, pruning, repair, stopping, and fallback decisions. Under matched inference budgets, CoT2-Meta consistently outperforms strong single-path, sampling-based, and search-based baselines, including ReST-MCTS. On the default backbone, it achieves 92.8 EM on MATH, 90.4 accuracy on GPQA, 98.65 EM on GSM8K, 75.8 accuracy on BBEH, 85.6 accuracy on MMMU-Pro, and 48.8 accuracy on HLE, with gains over the strongest non-CoT2-Meta baseline of +3.6, +5.2, +1.15, +2.0, +4.3, and +4.3 points, respectively. Beyond these core results, the framework remains effective across a broader 15-benchmark suite spanning knowledge and QA, multi-hop reasoning, coding, and out-of-distribution evaluation. Additional analyses show better compute scaling, improved calibration, stronger selective prediction, targeted repair behavior, and consistent gains across backbone families. These results suggest that explicit metacognitive control is a practical design principle for reliable and compute-efficient test-time reasoning systems.

Siyuan Ma, Bo Gao, Zikai Xiao +6

Nanyang Technological University · Carnegie Mellon University · Zhejiang University +5

Efficient Language Model Reasoning Metacognition in Language Models Inference-Time Search Inference-Time Compute Allocation

Ask PDF