RTLSeek: Boosting the LLM-Based RTL Generation with Multi-Stage Diversity-Oriented Reinforcement Learning
Organizations: State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences, China · University of Chinese Academy of Sciences, China · Peking University, China · CASTEST Co., Ltd., China
Abstract
Register Transfer Level (RTL) design translates high-level specifications into hardware using HDLs such as Verilog. Although LLM-based RTL generation is promising, the scarcity of functionally verifiable high-quality data limits both accuracy and diversity. Existing post-training typically produces a single HDL implementation per specification, lacking awareness of RTL variations needed for different design goals. We propose RTLSeek, a post-training paradigm that applies rule-based Diversity-Oriented Reinforcement Learning to improve RTL correctness and diversity. Our Diversity-Centric Multi-Objective Reward Scheduling integrates expert knowledge with EDA feedback, and a three-stage framework maximizes the utility of limited data. Experiments on the RTLLM benchmark show that RTLSeek surpasses prior methods, with ablation results confirming that encouraging broader design-space exploration improves RTL quality and achieves the principle of "the more generated, the better results." Implementation framework, including the dataset, source code, and model weights, is shown at https://anonymous.4open.science/r/DAC2026ID71-ACB4/.
Figures & tables
| Ablation | Gen. Num | Syn. Num | Fun. Num | Syn.OPMO _pass@1 | Syn.OPMO _pass@5 | Fun.OPMO _pass@1 | Fun.OPMO _pass@5 | Success Rate |
|---|---|---|---|---|---|---|---|---|
| RTLSeek | 3.22 | 2.23 | 1.27 | 0.86 | 0.96 | 0.76 | 0.86 | 0.40 |
| RTLSeek.w/o DR | 1.70 | 0.85 | 0.54 | 0.64 | 0.73 | 0.55 | 0.59 | 0.35 |
| RTLSeek.w/o S3 | 4.14 | 1.31 | 0.61 | 0.41 | 0.73 | 0.32 | 0.55 | 0.15 |
| RTLSeek.w/o S2 | 2.42 | 1.33 | 0.67 | 0.55 | 0.77 | 0.45 | 0.59 | 0.28 |
| RTLSeek.only S1 | 1.15 | 0.32 | 0.17 | 0.32 | 0.59 | 0.23 | 0.31 | 0.15 |
| Model | w/o DR Pass | w/o DR NO Pass |
|---|---|---|
| RTLSeek | 0.56 | 0.131 |
| RTLSeek.w/o DR | 0.47 | 0 |
RTL Code Generation
Register-transfer-level (RTL) code generation turns hardware specifications into synthesizable circuits, balancing functional correctness with area, timing, and power.
See all 31 papers