A Training-Free Regeneration Paradigm: Contrastive Reflection Memory Guided Self-Verification and Self-Improvement

Yuran Li; Di Wu; Benoit Boulet

cs.CLMar 20, 2026

A Training-Free Regeneration Paradigm: Contrastive Reflection Memory Guided Self-Verification and Self-Improvement

Authors: Yuran Li, Di Wu, Benoit Boulet

Organizations: Department of Electrical and Computer Engineering, McGill University · Department of Mechanical, Industrial and Aerospace Engineering, Concordia University

Abstract

Verification-guided self-improvement has recently emerged as a promising approach to improving the accuracy of large language model (LLM) outputs. However, existing approaches face a trade-off between inference efficiency and accuracy: iterative verification-rectification is computationally expensive and prone to being trapped in faulty reasoning, while best-of-N selection requires extensive sampling without addressing internal model flaws. We propose a training-free regeneration paradigm that leverages an offline-curated contrastive Reflection Memory (RM) to provide corrective guidance, while regenerating from scratch helps break out of faulty reasoning. At inference time, the method performs RM-guided self-verification followed by a single RM-guided regeneration, avoiding both iterative correction and multi-sample selection. We evaluated our method on nine benchmarks that span algorithmic, reasoning, symbolic, and domain-specific tasks in both small- and large-scale LLMs. Experiment results show that our method outperforms prior methods while maintaining low computational cost.

Figures & tables

Method	Learning	Verification	Improvement Method	Iterate
Prompt Engineering
Self-Refine Madaan et al. (2023)	ICL	self-critic	Explicit FB Rectify	✓
Reflexion Shinn et al. (2023)	ICL	Oracle	Explicit FB Rectify	✓
ProCo Wu et al. (2024)	ICL	Solve Verif. Q.	Implicit Rectify	✓
ST CoT Wu et al. (2025)	ICL	Uncertainty-guided	Implicit Rectify	✓
Ours	ICL	RM-guided	Regenerate	✗

Table 1: Comparison across major verification-guided improvement methods. ✓/✗ indicate presence/absence of a component; “Oracle” denotes access to ground-truth labels; “Rectify” vs. “Regenerate” indicate whether prior outputs are reused; “Explicit” vs. “Implicit” indicate whether textual feedback is produced.

Figure 1: Overview of our training-free regeneration framework. Left: Offline Reflection Curation. Middle: Online Retrieval. Right: Online Inference. (a) Direct Performance Boost: the LLM answers the query conditioned on the retrieved memory context. (b) RM-Guided Regeneration: the LLM first generates and verifies an answer; if incorrect, we retrieve reflections and regenerate a corrected answer conditioned on the memory context.

	Method	GSM-H	Bamb	Coin	LB	avg%
GPT-3.5	Few-shot CoT	41.25	55.00	74.25	44.21	61.15
	Contrastive CoT (w/o r)	39.40	49.00	59.00	35.75	57.01
	Contrastive CoT (w r)	38.00	50.00	69.25	55.79	60.33
	RM-Primed ( $\mathcal{R}^{+}$ )	42.50	59.00	78.00	64.21	64.99
	RM-Primed	44.60	62.00	79.75	67.37	66.01
Llama3-8B	Few-shot CoT	31.80	51.00	65.75	48.42	64.19

Table 2: Accuracy of different direct-boost methods on representative benchmarks and overall average accuracy. RM-Primed (

\mathcal{R}^{+}

) denotes that only correct entries are used. w/ r and w/o r indicate with and without reflection.

		Algorithmic			Reasoning		Symbolic		Domain		Overall
Model	Method	GSM8K	GSM-H	MATH	SQA	Bamb	Coin	Letter	LB	HL	avg%
GPT-3.5	Reflexion(3 iters)	85.60	45.20	62.25	71.00	66.00	88.25	78.67	85.26	72.00	70.65
	Best-of-N (N=3)	84.60	46.80	64.00	71.50	63.00	77.00	74.67	32.63	70.00	67.29
	RM-Regen	90.20	68.60	63.40	77.25	73.00	85.75	87.33	75.79	73.00	76.94
Llama3.1-8B	Reflexion(3 iters)	89.60	36.20	77.80	85.00	60.00	81.25	80.67	81.05	70.00	73.26
	ReflectEvo	85.40	32.00	73.60	71.25	53.00	67.75	68.67	50.53	62.00	64.74

Table 3: Accuracy under oracle verification (verification-and-rectification/selection baselines vs. ours).

Panel of Figure 2 — Figure 2: Performance robustness against verification noise. We compare our method with Reflexion and ReflectEvo on Llama-3.1-8B. The gray baseline denotes Few-shot CoT. The x-axis represents the noise rate, where $0$ denotes oracle verification and $1$ denotes verification is always incorrect.

Panel of Figure 3 — Figure 3: Verification F1 score of different verifiers on representative benchmarks using Llama-3.1-8B.

		Algorithmic			Reasoning		Symbolic		Domain		Overall
Model	Method	GSM8K	GSM-H	MATH	SQA	Bamb	Coin	Letter	LB	HL	avg%
GPT-3.5	Self-Refine (3 iters)	70.40	32.80	47.20	41.75	42.00	68.25	46.00	28.42	53.00	50.38
	ProCo (3 iters)	80.80	39.60	52.20	60.00	43.00	75.25	74.00	41.05	65.00	60.55
	ST CoT (3 iters)	80.20	39.80	53.60	62.00	59.00	75.00	72.67	23.16	69.00	61.02
	RM-Regen	83.60	64.00	56.80	69.75	59.00	78.50	77.33	60.00	71.00	69.87
Llama3.1-8B	Self-Refine (3 iters)	75.60	23.00	51.00	53.75	49.00	66.00	40.00	46.32	65.00	52.64

Table 4: Accuracy on nine benchmarks without oracle verification (verification-and-rectification baselines vs. ours).

Figure 4: Speed–accuracy trade-off of verification-guided improvement methods. Each bubble represents the average accuracy (y-axis) versus inference speed in items per second (x-axis, log-scale). Arrows connect configurations with different refinement iterations (2 iters - 4 iters) for the same method, and the attached labels report the resulting change in accuracy (

\Delta

Acc).

Teacher Model	GSM-Hard	Bamboogle	Coin
No Teacher (CoT)	31.80	51.00	65.75
GPT-5-mini	41.00	63.00	85.75
GPT-3.5	37.60	60.00	76.50
Llama 3.1-8B	39.00	60.00	74.50
Gemma 2-9B	39.60	58.00	77.75

Table 5: Teacher model dependency analysis. Performance evaluated with base model Llama 3.1-8B across different teachers.

	Algorithmic			Reasoning		Symbolic		Domain
Dataset	GSM8K 1 1 1 https://huggingface.co/datasets/gsm8k	GSM-Hard 2 2 2 https://huggingface.co/datasets/reasoning-machines/gsm-hard	MATH 3 3 3 https://huggingface.co/datasets/nlile/hendrycks-MATH-benchmark	StrategyQA 4 4 4 https://huggingface.co/datasets/tasksource/strategy-qa	Bamboogle 5 5 5 https://huggingface.co/datasets/chiayewken/bamboogle	Coin 6 6 6 https://huggingface.co/datasets/skrishna/coin_flip	Letter 7 7 7 https://huggingface.co/datasets/ChilleD/LastLetterConcat	LB 8 8 8 https://huggingface.co/datasets/nguha/legalbench	Headline 9 9 9 https://huggingface.co/datasets/SaguaroCapital/sentiment-analysis-in-commodity-market-gold
Train split	800	800	800	800	10	100	10	5	100
Test split	500	500	500	400	100	400	140	95	100
Answer type	Numeric	Numeric	Numeric	Boolean	Factoid	Boolean	Letter	Multiple-choice	Multiple-choice

Table 6: Dataset splits used in our experiments. For each benchmark, we report the number of train and test examples used for reflection-memory construction and evaluation. We randomly subsample train/test examples from each benchmark using a fixed random seed, ensuring the same subset across all methods.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

	Algorithmic			Reasoning		Symbolic		Domain		Overall
Method	GSM8K	GSM-Hard	MATH	StrategyQA	Bamboogle	Coin	Letter	LB	Headline	avg%
GPT-3.5
Few-shot CoT	79.25	41.25	51.60	61.50	55.00	74.25	72.00	44.21	70.00	61.15
Contrastive CoT (w/o r)	77.20	39.40	47.00	60.75	49.00	59.00	75.33	35.75	72.00	57.01
Contrastive CoT (w r)	78.40	38.00	52.20	63.00	50.00	69.25	73.33	55.79	71.00	60.33
RM-Primed ( $\mathcal{R}^{+}$ )	81.50	42.50	55.00	65.25	59.00	78.00	84.00	64.21	70.00	64.99

Appendix

Table 8: Full per-dataset accuracy of different direct-boost methods on nine reasoning benchmarks.

	Algorithmic			Reasoning		Symbolic		Domain		Overall
Method	GSM8K	GSM-Hard	MATH	StrategyQA	Bamboogle	Coin	Letter	LB	Headline	avg%
GPT-3.5
Self-Refine (2 iters)	73.40	33.80	48.20	44.00	40.00	67.25	40.00	53.68	59.00	52.17
Self-Refine (3 iters)	70.40	32.80	47.20	41.75	42.00	68.25	46.00	28.42	53.00	50.38
Self-Refine (4 iters)	70.60	34.60	48.80	44.75	43.00	69.75	41.33	22.11	62.00	51.58
ProCo (2 iters)	80.06	39.40	54.60	56.75	49.00	73.00	74.67	48.42	64.00	60.48

Appendix

Table 10: Accuracy on nine benchmarks without oracle verification. This table extends by additionally reporting results of 4 and 5 iterations.

Variant	Retrieval	Verification	Regeneration	Acc. (%)
Full (Ours)	✓	✓	✓	64.00
Fixed demos	✗	✓	✓	54.60
Direct boost	✓	✗	✗	44.60
Rectification	✓	✓	✗	53.60

Appendix

Table 11: GPT-3.5 on GSM_Hard: ablation of inference-stage configurations.

Variant	Refine (ICL)	Filter	Teacher	Acc. (%)
Full (Ours)	✓	✓	✓	68.60
w/o Refine	✗	✓	✓	63.20
w/o Filter	✓	✗	✓	66.40
w/o Teacher	✓	✓	✗	56.80

Appendix

Table 12: GPT-3.5 on GSM_Hard: ablation of RM curation components (Refine, Filter, Teacher) under oracle verification.

Variant	$\mathcal{R}^{+}$	$\mathcal{R}^{-}$	Principle	Acc. (%)
Full (Ours)	✓	✓	✓	68.60
w/o $\mathcal{R}^{+}$	✗	✓	✓	64.40
w/o $\mathcal{R}^{-}$	✓	✗	✗	64.20
w/o Principle	✓	✓	✗	67.80

Appendix

Table 13: GPT-3.5 on GSM_Hard: ablation of RM contents (

\mathcal{R}^{+}

\mathcal{R}^{-}

, principles) under oracle verification.

Figure 5: Effect of reflection-memory size on GPT-3.5 performance on GSM_Hard.

		RM-Regen		Reflexion (5 iters)
		Correct	Wrong	Correct	Wrong
Initial	Correct	100.00%	0.00%	100.00%	0.00%
Initial	Wrong	46.55%	53.45%	10.47%	89.53%

Appendix

Table 14: GPT-3.5 on GSM-Hard: Regeneration correctness flip (with oracle verification)

		RM-Regen		ST CoT (4 iters)
		Correct	Wrong	Correct	Wrong
Initial	Correct	95.52%	4.48%	86.05%	13.95%
Initial	Wrong	38.63%	61.37%	8.77%	91.23%

Appendix

Table 15: GPT-3.5 on GSM-Hard: Regeneration correctness flip (without oracle verification)

	GSM8K	GSM-Hard	MATH
GPT-3.5	90.00 /90.20	51.20 /68.60	66.00 /63.40
Llama 3.1-8B	91.60 /92.60	35.00 /41.00	80.60 /80.60
Gemma 2-9B	91.40 /92.00	52.20 /53.00	71.20 /72.20

Appendix

Table 16: Transferability of RM constructed for LLaMA-3.1-8B on MATH across models and datasets. Gray numbers denote native performance (the same model and dataset used for RM construction), while black numbers indicate cross-scenario performance under model or dataset shifts.

Language Model Self-Verification

See all 16 papers

Training-Free Inference-Time Self-Reflection and Cost-Bounded Early Stopping for Large Language Models
Aug 19, 2026Wei Yu, Suxing Liu, Minjie Yu +4Language Model Self-Verification LLM Self-Refinement
School of Digital Arts, Jiangxi Arts & Ceramics Technology Institute, Jingdezhen 33001, China · School of Computing, Universiti Sains Malaysia, 11800 USM Penang, Malaysia
Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline
May 20, 2026Tony Lee, Percy LiangLanguage Model Self-Verification Self-Training for Language Models
Stanford University
Reinforcing Step-level Reasoning for Effective Self-Correction in LLMs
Aug 12, 2026Vu Duc Anh, Nhat M. Hoang, Do Xuan Long +3LLM Self-Correction Direct Preference Optimization
Nanyang Technological University, Singapore · National University of Singapore · Institute for Infocomm Research (I2R), A*STAR +1

Aug 19, 2026J/K move · Enter open · S save

Training-Free Inference-Time Self-Reflection and Cost-Bounded Early Stopping for Large Language Models

Wei Yu, Suxing Liu, Minjie Yu, Jiahao Wang+3

School of Digital Arts, Jiangxi Arts & Ceramics Technology Institute, Jingdezhen 33001, China · School of Computing, Universiti Sains Malaysia, 11800 USM Penang, Malaysia

Language Model Self-Verification LLM Self-Refinement LLM Inference Efficiency

Open paper AskPDF arXiv

Reinforcement-learning training of reasoning LLMs (e.g., GRPO) is expensive and requires a controllable environment, committing every contribution to a full training pipeline. We present EvoResearcher, a training-free, inference-time protocol that adds cost-bounded self-reflection to a single frozen LLM backbone. The protocol iterates generate -> self-critique -> revise until a maximum depth D is reached or the critique returns the CONFIRMED sentinel, an implicit early stop that lets the backbone self-verify its answer under a strict compute budget. Four self-reflective meta-reward components (correctness, efficiency, reflection depth, tool-call diversity) act as design principles instantiated as prompt-level mechanisms, so their benefits accrue with zero gradient updates. We validate the protocol on Big-Bench Hard (100 questions) and establish cross-domain behavior on GSM8K (500) and MATH (500) on the same frozen backbone, with cross-model replication on Qwen2.5-72B. All experiments use pure-reasoning benchmarks; the tool-call diversity component is validated in prompt-level form, and the environment-level and multi-agent extensions are design blueprints left to future work. On clean BBH the protocol does not raise accuracy beyond the 95% Wilson interval; its value is cost-bounded self-verification, with the CONFIRMED early stop terminating 82-88% of items at equal accuracy (about 2.1 generations per question).

Abstract

Language Model Self-Verification

Explore similar work

Training-Free Inference-Time Self-Reflection and Cost-Bounded Early Stopping for Large Language Models

Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline

Reinforcing Step-level Reasoning for Effective Self-Correction in LLMs