Recent advances in large language models (LLMs) have improved Verilog generation. However, most training pipelines distill natural-language reasoning traces whose intermediate design decisions are weakly structured and are primarily selected according to the correctness of the final generated RTL. We present MCTS-SFT, a verifier-guided framework for curating reasoning traces for supervised fine-tuning (SFT) of Verilog generation models. MCTS-SFT formulates trace construction as search over intermediate reasoning states and uses Monte Carlo Tree Search (MCTS) to propagate functional feedback from complete Verilog rollouts to earlier design decisions. Within this framework, we introduce Plan-MCTS, which searches over compact hardware-design plans that explicitly represent interfaces, timing, state, behavior, and update rules. We further study plan-preserving supervision, in which the searched structured state is retained as part of the downstream SFT target. Our experiments find that Plan-MCTS achieves the highest functional trace yield, with 84.2% correct traces compared with 81.9% for one-shot structured planning and 82.0% for unguided tree exploration. For identical Plan-MCTS trajectories and Verilog outputs, retaining the explicit searched plan during SFT raises pass@10 from 52.56% to 57.7%, demonstrating that the representation of intermediate design decisions materially affects downstream learning. In a separate 6,000-trace study, MCTS-curated supervision improves pass@10 over full-response chain-of-thought distillation by up to 3.7 points while reducing downstream generation length by approximately 50% on VerilogEval-v2. These results demonstrate that structured hardware planning, verifier-guided search, and explicit intermediate-state supervision provide complementary mechanisms for constructing more effective and token-efficient training data for Verilog generation.
Figures & tables
Figure 1: MCTS-SFT trace curation process, including state representations and MCTS stages.
Strategy
State representation
Exploration
Correct
Yield
CoT
Free-form
Independent
731 / 1,000
73.1%
Thought-MCTS
Free-form
Verifier-guided
836 / 1,000
83.6%
Plan-CoT
Structured plan
One-shot
819 / 1,000
81.9%
Plan-RandomTree
Structured plan
Random tree
820 / 1,000
82.0%
Plan-MCTS
Structured plan
Verifier-guided
842 / 1,000
84.2%
Table 1: RQ1: Functional trace discovery under the controlled 1k-trace setting. Each strategy generates 1,000 trajectories from the same 100 specifications using DeepSeek-R1 for all LLM calls. A correct trace passes all associated testbench cases.
Method / SFT target
pass@1
pass@5
pass@10
CoT
31.22
46.91
52.56
Thought-MCTS
27.37
46.71
51.28
Summary-MCTS
29.23
46.43
51.92
Plan-CoT
32.37
50.40
57.05
Plan-RandomTree
28.53
47.44
53.21
Plan-MCTS, rollout only
29.17
46.48
52.56
Table 2: RQ2: Effect of reasoning and SFT-target representation. All models are trained on 1,000 examples and evaluated on VerilogEval-v1-Human with n=10 . The three Plan-MCTS target variants use identical search trajectories, terminal Verilog implementations, and verifier rewards.
Strategy / target
Mean reward
Mean tokens
Distinct-2
CoT
0.571
1,021
0.789
Thought-MCTS
0.754
3,121
0.607
Plan-CoT
0.755
1,649
0.747
Plan-RandomTree
0.739
1,556
0.740
Plan-MCTS, rollout only
0.780
1,561
0.743
Plan-MCTS + rendered NL plan
0.780
1,681
0.745
Table 3: Trace characteristics in the controlled study. Lower Distinct-2 indicates greater lexical repetition. The three Plan-MCTS target variants share the same terminal trajectories and therefore have the same mean verifier reward.
VE-v1-H
VE-v2-CC
VE-v2-S2R
Training data
# SFT
p@1
p@5
p@10
Tokens
p@1
p@5
p@10
Tokens
p@1
p@5
p@10
Tokens
Baseline (no SFT)
0
21.4
40.9
46.6
1132±1640
25.1
40.1
46.0
456±665
24.1
38.6
43.7
422±649
CoT (answer only)
6,000
36.1
48.1
52.6
1888±4217
35.7
50.1
54.4
442±466
34.0
49.0
55.2
451±627
Best-of- N (Plan-MCTS)
600
33.5
49.4
54.4
3143±3045
35.3
50.7
55.4
3556±3178
30.3
48.0
54.0
3760±3217
CoT (full response)
6,000
36.9
52.5
57.2
3162±3159
41.5
54.8
58.8
7317±1757
41.2
55.6
60.3
7313±1715
Thought-MCTS
6,000
35.9
53.5
57.8
4521±3057
36.9
55.3
60.4
4348±3038
41.3
55.8
60.3
4483±3017
Table 4: RQ3: Downstream Verilog generation after SFT in the extended study. VE-v1-H denotes VerilogEval-v1-Human, VE-v2-CC denotes VerilogEval-v2 Code Completion, and VE-v2-S2R denotes VerilogEval-v2 Spec-to-RTL. Tokens report mean generated response length ± standard deviation. Bold denotes the highest pass@ k value for each benchmark.
Model
Overall
Easy
Medium
CoT-SFT
9.30% (16/172)
15.73%
2.41%
Thought-MCTS-SFT
11.63% (20/172)
20.22%
2.41%
Plan-MCTS-SFT
12.21% (21/172)
19.10%
4.82%
Table 5: Preliminary multi-module evaluation on 172 CVDP-002/003 tasks. Absolute success rates remain low, and we therefore interpret this experiment as evidence of transfer rather than a comprehensive evaluation of multi-module RTL generation.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Setting
Source specifications
600 VeriThoughts prompts
Trajectories per specification
10
Full dataset size
6,000 traces
Maximum MCTS iterations
10
Maximum child nodes per expansion
3
Intermediate-node token limit
2,048
Appendix
Table 6: Curation configuration reported for the extended 6k-trace study.
Parameter
Setting
Source specifications
100 VeriThoughts prompts
Trajectories per specification
10
Dataset size per strategy
1,000 traces
Teacher model
DeepSeek-R1
Teacher assignment
All LLM calls and stages
Downstream benchmark
VerilogEval-v1-Human
Appendix
Table 7: Controlled 1k-trace study configuration.
Parameter
Setting
Training framework
LlamaFactory
Learning rate
5×10−5
Maximum sequence length
16,384 tokens
Training epochs
3
Adaptation method
LoRA
LoRA rank
64
Appendix
Table 8: Supervised fine-tuning configuration.
Parameter
Setting
Benchmarks
VerilogEval-v1-Human, VerilogEval-v2
VerilogEval-v2 tasks
Code Completion, Spec-to-RTL
Maximum generation length
8,192 tokens
Temperature
0.5
Top- p
0.9
Generations per problem
n=20
Appendix
Table 9: Downstream generation configuration reported for the extended study.
Method / representation
pass@1
pass@5
pass@10
Thought-MCTS, full free-form prefix
27.37
46.71
51.28
Summary-MCTS, compressed free-form prefix
29.23
46.43
51.92
Plan-MCTS, rollout only
29.17
46.48
52.56
Plan-MCTS + rendered NL plan
30.0
47.6
55.1
Plan-MCTS + explicit structured plan
31.5
50.8
57.7
Appendix
Table 10: Comparison of reasoning-state compression and hardware-specific structure in the controlled study.
Teacher strategy
pass@1
pass@5
pass@10
Thought-MCTS
33.21
45.90
50.64
Plan-MCTS
33.78
46.21
50.00
Appendix
Table 11: Generalization to GPT-5.4-mini as the curation teacher.
Training configuration
pass@1
pass@5
pass@10
Qwen2.5 baseline
25.6
38.9
42.6
SFT, 10k MCTS traces
38.2
51.8
56.6
SFT, 15k MCTS traces
41.7
54.9
59.6
SFT, 25k MCTS traces
43.7
54.1
58.0
SFT, 40k MCTS traces
45.6
56.0
58.9
SFT, 60k MCTS traces
49.0
60.0
63.3
Appendix
Table 12: Scaling standard Thought-MCTS supervision to larger training datasets.
Automatic generation of RTL code for digital hardware designs remains challenging due to long-horizon reasoning, multi-step dependencies, and strict correctness constraints in Verilog and VHDL. We present StepPRM-RTL, a novel framework that combines stepwise trajectory modeling, process-reward modeling (PRM), and retrieval-augmented fine-tuning (RAFT) to enhance both the functional correctness and reasoning fidelity of LLM-based RTL code generation. StepPRM-RTL constructs stepwise reasoning trajectories from canonical solutions, where each step contains a rationale and incremental code modification. A Process Reward Model (PRM) evaluates intermediate steps, providing dense feedback that guides reinforcement-style updates during RAFT fine-tuning. Monte Carlo Tree Search (MCTS) explores alternative reasoning paths, enriching the training dataset with high-quality trajectories. This integration of stepwise and outcome-aware rewards allows the model to learn both how and why to construct correct RTL, improving long-horizon reasoning beyond standard supervised or outcome-based training. Experimental evaluation on benchmark Verilog and VHDL datasets demonstrates that StepPRM-RTL outperforms the best prior methods by over 10% in functional correctness and reasoning fidelity metrics. Ablation studies confirm that the combination of PRM-guided rewards and stepwise trajectory exploration is key to its performance. StepPRM-RTL generalizes across RTL languages and provides a scalable framework for high-fidelity, interpretable code generation, establishing a new standard for LLM-assisted hardware design automation.
Prashanth Vijayaraghavan, Apoorva Nitsure, Luyao Shi +2
Recent advances in large language models have improved code generation, but their use in hardware description languages is still limited. Moreover, training data and testbenches for these models are often scarce. This paper presents a workflow that uses multi-agent models to generate testbenches for high-quality fine-tuning data. By automating testbench creation, the fine-tuned model for the specification-to-Verilog task achieves performance comparable to state-of-the-art methods on the refined VerilogEval v2 benchmark while using less training data. This study provides a basis for future work on LLM-based HDL generation and automated verification.
Mu-Chi Chen, Po-Hsuan Huang, Yu-Hung Kao +6
Academia Sinica, Taipei, Taiwan · National Taiwan University, Taipei, Taiwan · National Cheng Kung University, Tainan, Taiwan
Large language models (LLMs) have improved Verilog generation from natural-language specifications, but most pipelines still treat generation as isolated sampling followed by functional checking. This is insufficient for practical RTL design, where useful Verilog must be correct, synthesizable, timing-conscious, and friendly to downstream hardware objectives. We present Verilog-Evolve, a feedback-driven framework for versioned Verilog refinement and cross-session skill evolution. For each task, Verilog-Evolve generates diverse minor candidates, evaluates them with executable feedback from functional simulation, Yosys synthesis, ABC timing proxy, and optional GEMM metrics, then promotes the best candidate into a major version under configurable scoring. To improve across tasks, the system maintains modular skill guidance, retrieves skills according to task and feedback context, and evolves candidate skills from logged histories through create/improve/skip decisions and verifier reports. Experiments on VerilogEval and mixed-precision GEMM tasks show that Verilog-Evolve improves final functional success and promotion stability while producing more downstream-friendly RTL under open-source synthesis, timing-proxy, and netlist-level GEMM objectives. Validation-gated skill evolution further improves GEMM downstream quality and achieves the best downstream score and GEMM held-out pass rate among the evaluated skill modes.
Zehua Pei, Hui-Ling Zhen, Yu Zhang +3
The Chinese University of Hong Kong · Huawei Technologies Co., Ltd