Search-Guided Reasoning Trace Curation for Verilog Generation

Matthew DeLorenzo; Kevin Tieu; Prithwish Jana; Piyush Jha; Dileep Kalathil; Vijay Ganesh; Jeyavijayan Rajendran

cs.AROct 26, 2026

Search-Guided Reasoning Trace Curation for Verilog Generation

Authors: Matthew DeLorenzo, Kevin Tieu, Prithwish Jana, Piyush Jha, Dileep Kalathil, Vijay Ganesh, Jeyavijayan Rajendran

Organizations: Texas A&M University · Georgia Institute of Technology

Abstract

Recent advances in large language models (LLMs) have improved Verilog generation. However, most training pipelines distill natural-language reasoning traces whose intermediate design decisions are weakly structured and are primarily selected according to the correctness of the final generated RTL. We present MCTS-SFT, a verifier-guided framework for curating reasoning traces for supervised fine-tuning (SFT) of Verilog generation models. MCTS-SFT formulates trace construction as search over intermediate reasoning states and uses Monte Carlo Tree Search (MCTS) to propagate functional feedback from complete Verilog rollouts to earlier design decisions. Within this framework, we introduce Plan-MCTS, which searches over compact hardware-design plans that explicitly represent interfaces, timing, state, behavior, and update rules. We further study plan-preserving supervision, in which the searched structured state is retained as part of the downstream SFT target. Our experiments find that Plan-MCTS achieves the highest functional trace yield, with 84.2% correct traces compared with 81.9% for one-shot structured planning and 82.0% for unguided tree exploration. For identical Plan-MCTS trajectories and Verilog outputs, retaining the explicit searched plan during SFT raises pass@10 from 52.56% to 57.7%, demonstrating that the representation of intermediate design decisions materially affects downstream learning. In a separate 6,000-trace study, MCTS-curated supervision improves pass@10 over full-response chain-of-thought distillation by up to 3.7 points while reducing downstream generation length by approximately 50% on VerilogEval-v2. These results demonstrate that structured hardware planning, verifier-guided search, and explicit intermediate-state supervision provide complementary mechanisms for constructing more effective and token-efficient training data for Verilog generation.

Figures & tables

Figure 1: MCTS-SFT trace curation process, including state representations and MCTS stages.

Strategy	State representation	Exploration	Correct	Yield
CoT	Free-form	Independent	731 / 1,000	73.1%
Thought-MCTS	Free-form	Verifier-guided	836 / 1,000	83.6%
Plan-CoT	Structured plan	One-shot	819 / 1,000	81.9%
Plan-RandomTree	Structured plan	Random tree	820 / 1,000	82.0%
Plan-MCTS	Structured plan	Verifier-guided	842 / 1,000	84.2%

Table 1: RQ1: Functional trace discovery under the controlled 1k-trace setting. Each strategy generates 1,000 trajectories from the same 100 specifications using DeepSeek-R1 for all LLM calls. A correct trace passes all associated testbench cases.

Method / SFT target	pass@1	pass@5	pass@10
CoT	31.22	46.91	52.56
Thought-MCTS	27.37	46.71	51.28
Summary-MCTS	29.23	46.43	51.92
Plan-CoT	32.37	50.40	57.05
Plan-RandomTree	28.53	47.44	53.21
Plan-MCTS, rollout only	29.17	46.48	52.56

Table 2: RQ2: Effect of reasoning and SFT-target representation. All models are trained on 1,000 examples and evaluated on VerilogEval-v1-Human with

n=10

. The three Plan-MCTS target variants use identical search trajectories, terminal Verilog implementations, and verifier rewards.

Strategy / target	Mean reward	Mean tokens	Distinct-2
CoT	0.571	1,021	0.789
Thought-MCTS	0.754	3,121	0.607
Plan-CoT	0.755	1,649	0.747
Plan-RandomTree	0.739	1,556	0.740
Plan-MCTS, rollout only	0.780	1,561	0.743
Plan-MCTS + rendered NL plan	0.780	1,681	0.745

Table 3: Trace characteristics in the controlled study. Lower Distinct-2 indicates greater lexical repetition. The three Plan-MCTS target variants share the same terminal trajectories and therefore have the same mean verifier reward.

		VE-v1-H				VE-v2-CC				VE-v2-S2R
Training data	# SFT	p@1	p@5	p@10	Tokens	p@1	p@5	p@10	Tokens	p@1	p@5	p@10	Tokens
Baseline (no SFT)	0	21.4	40.9	46.6	$1132\pm 1640$	25.1	40.1	46.0	$456\pm 665$	24.1	38.6	43.7	$422\pm 649$
CoT (answer only)	6,000	36.1	48.1	52.6	$1888\pm 4217$	35.7	50.1	54.4	$442\pm 466$	34.0	49.0	55.2	$451\pm 627$
Best-of- $N$ (Plan-MCTS)	600	33.5	49.4	54.4	$3143\pm 3045$	35.3	50.7	55.4	$3556\pm 3178$	30.3	48.0	54.0	$3760\pm 3217$
CoT (full response)	6,000	36.9	52.5	57.2	$3162\pm 3159$	41.5	54.8	58.8	$7317\pm 1757$	41.2	55.6	60.3	$7313\pm 1715$
Thought-MCTS	6,000	35.9	53.5	57.8	$4521\pm 3057$	36.9	55.3	60.4	$4348\pm 3038$	41.3	55.8	60.3	$4483\pm 3017$

Table 4: RQ3: Downstream Verilog generation after SFT in the extended study. VE-v1-H denotes VerilogEval-v1-Human, VE-v2-CC denotes VerilogEval-v2 Code Completion, and VE-v2-S2R denotes VerilogEval-v2 Spec-to-RTL. Tokens report mean generated response length

\pm

standard deviation. Bold denotes the highest pass@

k

value for each benchmark.

Model	Overall	Easy	Medium
CoT-SFT	9.30% (16/172)	15.73%	2.41%
Thought-MCTS-SFT	11.63% (20/172)	20.22%	2.41%
Plan-MCTS-SFT	12.21% (21/172)	19.10%	4.82%

Table 5: Preliminary multi-module evaluation on 172 CVDP-002/003 tasks. Absolute success rates remain low, and we therefore interpret this experiment as evidence of transfer rather than a comprehensive evaluation of multi-module RTL generation.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Setting
Source specifications	600 VeriThoughts prompts
Trajectories per specification	10
Full dataset size	6,000 traces
Maximum MCTS iterations	10
Maximum child nodes per expansion	3
Intermediate-node token limit	2,048

Appendix

Table 6: Curation configuration reported for the extended 6k-trace study.

Parameter	Setting
Source specifications	100 VeriThoughts prompts
Trajectories per specification	10
Dataset size per strategy	1,000 traces
Teacher model	DeepSeek-R1
Teacher assignment	All LLM calls and stages
Downstream benchmark	VerilogEval-v1-Human

Appendix

Table 7: Controlled 1k-trace study configuration.

Parameter	Setting
Training framework	LlamaFactory
Learning rate	$5\times 10^{-5}$
Maximum sequence length	16,384 tokens
Training epochs	3
Adaptation method	LoRA
LoRA rank	64

Appendix

Table 8: Supervised fine-tuning configuration.

Parameter	Setting
Benchmarks	VerilogEval-v1-Human, VerilogEval-v2
VerilogEval-v2 tasks	Code Completion, Spec-to-RTL
Maximum generation length	8,192 tokens
Temperature	0.5
Top- $p$	0.9
Generations per problem	$n=20$

Appendix

Table 9: Downstream generation configuration reported for the extended study.

Method / representation	pass@1	pass@5	pass@10
Thought-MCTS, full free-form prefix	27.37	46.71	51.28
Summary-MCTS, compressed free-form prefix	29.23	46.43	51.92
Plan-MCTS, rollout only	29.17	46.48	52.56
Plan-MCTS + rendered NL plan	30.0	47.6	55.1
Plan-MCTS + explicit structured plan	31.5	50.8	57.7

Appendix

Table 10: Comparison of reasoning-state compression and hardware-specific structure in the controlled study.

Teacher strategy	pass@1	pass@5	pass@10
Thought-MCTS	33.21	45.90	50.64
Plan-MCTS	33.78	46.21	50.00

Appendix

Table 11: Generalization to GPT-5.4-mini as the curation teacher.

Training configuration	pass@1	pass@5	pass@10
Qwen2.5 baseline	25.6	38.9	42.6
SFT, 10k MCTS traces	38.2	51.8	56.6
SFT, 15k MCTS traces	41.7	54.9	59.6
SFT, 25k MCTS traces	43.7	54.1	58.0
SFT, 40k MCTS traces	45.6	56.0	58.9
SFT, 60k MCTS traces	49.0	60.0	63.3

Appendix

Table 12: Scaling standard Thought-MCTS supervision to larger training datasets.

Code Generation

Code generation studies methods for turning natural-language, visual, or formal specifications into executable programs and code edits.

See all 322 papers

StepPRM-RTL: Stepwise Process-Reward Guided LLM Fine-Tuning for Enhanced RTL Synthesis
Jun 2, 2026Prashanth Vijayaraghavan, Apoorva Nitsure, Luyao Shi +2RL for Code Generation Process Reward Models
IBM Research San Jose, CA, USA
Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation
Apr 16, 2026Mu-Chi Chen, Po-Hsuan Huang, Yu-Hung Kao +6LLM Fine-Tuning Code Generation
Academia Sinica, Taipei, Taiwan · National Taiwan University, Taipei, Taiwan · National Cheng Kung University, Tainan, Taiwan
Verilog-Evolve: Feedback-Driven and Skill-Evolving Verilog Generation
May 26, 2026Zehua Pei, Hui-Ling Zhen, Yu Zhang +3Large Language Model-Guided Optimization Execution-Guided Code Generation
The Chinese University of Hong Kong · Huawei Technologies Co., Ltd

Jun 2, 2026J/K move · Enter open · S save

StepPRM-RTL: Stepwise Process-Reward Guided LLM Fine-Tuning for Enhanced RTL Synthesis

Prashanth Vijayaraghavan, Apoorva Nitsure, Luyao Shi, Ehsan Degan+1

IBM Research San Jose, CA, USA

RL for Code Generation Process Reward Models Electronic Design Automation Code Generation LLM-Based Program Synthesis RTL Code Generation

Open paper AskPDF arXiv

Automatic generation of RTL code for digital hardware designs remains challenging due to long-horizon reasoning, multi-step dependencies, and strict correctness constraints in Verilog and VHDL. We present StepPRM-RTL, a novel framework that combines stepwise trajectory modeling, process-reward modeling (PRM), and retrieval-augmented fine-tuning (RAFT) to enhance both the functional correctness and reasoning fidelity of LLM-based RTL code generation. StepPRM-RTL constructs stepwise reasoning trajectories from canonical solutions, where each step contains a rationale and incremental code modification. A Process Reward Model (PRM) evaluates intermediate steps, providing dense feedback that guides reinforcement-style updates during RAFT fine-tuning. Monte Carlo Tree Search (MCTS) explores alternative reasoning paths, enriching the training dataset with high-quality trajectories. This integration of stepwise and outcome-aware rewards allows the model to learn both how and why to construct correct RTL, improving long-horizon reasoning beyond standard supervised or outcome-based training. Experimental evaluation on benchmark Verilog and VHDL datasets demonstrates that StepPRM-RTL outperforms the best prior methods by over 10% in functional correctness and reasoning fidelity metrics. Ablation studies confirm that the combination of PRM-guided rewards and stepwise trajectory exploration is key to its performance. StepPRM-RTL generalizes across RTL languages and provides a scalable framework for high-fidelity, interpretable code generation, establishing a new standard for LLM-assisted hardware design automation.

Abstract

Code Generation

Explore similar work

StepPRM-RTL: Stepwise Process-Reward Guided LLM Fine-Tuning for Enhanced RTL Synthesis

Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation

Verilog-Evolve: Feedback-Driven and Skill-Evolving Verilog Generation