Recent studies have proposed reinforcement learning with verifiable rewards (RLVR) methods that enable LLMs to generate high-quality RTL circuits from natural-language specifications. Existing approaches optimize circuit PPA (Power, Performance, Area) by aggregating multiple objectives into a single scalar reward, making it difficult to simultaneously generate diverse Pareto-optimal solutions across different trade-offs such as low power, low latency, and small area. However, in practical design, the relative importance of each objective varies depending on product requirements and design constraints; thus, obtaining a diverse and high-quality candidate set is beneficial in that designers can choose a preferred trade-off a posteriori. In this work, we are the first to propose a learning method for LLM-based circuit generation that optimizes, not the quality of a single circuit, but the quality and diversity of the entire circuit set obtained through multiple generations in the PPA objective space. The proposed method consists of (1) a generation-based training pipeline that reformulates coverage improvement of the entire Pareto solution set into a sequential circuit generation problem, and (2) a reward function based on Hypervolume Contribution (HVC) that measures how much a generated circuit contributes to the Pareto front. Evaluation on the RTLLM v2 benchmark shows that the proposed method achieves +13.3 percentage points in relative Hypervolume over a baseline without PPA optimization and +14.4 percentage points over a single-objective reinforcement learning method, while also demonstrating superiority in circuit diversity.
Figures & tables
Figure 1: Overview of the proposed method. By alternating between reinforcement learning with HVC reward (top) and dataset reconstruction (bottom), the model learns Pareto solution exploration capabilities while progressively extending circuit history.
Figure 2: Example of Hypervolume Contribution (HVC). In the objective function space, the HVC is the Hypervolume increment with origin as the reference point when adding candidate i . A larger HVC means the candidate contributes more to the Pareto front.
Base Model
Method
rHV (iter=1)
rHV (iter=5)
rHV (iter=10)
VeriReason-3B
Base
13.1%
17.5%
19.1%
Single-Objective PPA
10.4%
15.4%
18.0%
Sequential TB
15.5%
18.5%
19.2%
Sequential TB+PPA
17.2%
20.6%
21.6%
Sequential HVC (Ours)
22.9%
30.7%
32.4%
CodeV-R1-Distil-7B
Base
23.1%
34.1%
38.4%
Table 1: Relative Hypervolume (rHV) for each model. Each column shows the rHV measured after a given number of sequential generation iterations ( iter ), where the model generates a new circuit conditioned on the circuit history accumulated so far.
Figure 3: Win Rate progression per generation count.
Figure 4: Circuit diversity comparison.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
VeriReason-3B
CodeV-R1-Distil-7B
Num. GPUs
4
4
GPU
H200
H200
Batch Size (per device)
2
2
Gradient Accumulation Steps
4
4
Learning Rate
1.0×10−6
1.0×10−6
Max Steps
15,000
1,500
Appendix
Table 2: Training hyperparameters. Common GRPO settings are shared across all methods on both base models; Max Steps differs by base model.
Base Model
Comparison
rHV Diff (95% CI)
VeriReason-3B
Ours − Base
+13.3[+2.1,+24.8]
Ours − Single-Objective PPA
+14.5[+2.5,+27.0]
Ours − Sequential TB
+13.2[+4.7,+22.7]
Ours − Sequential TB+PPA
+10.8[+1.2,+20.9]
CodeV-R1-Distil-7B
Ours − Base
+4.8[+0.6,+9.0]
Ours − Single-Objective PPA
+5.4[+1.4,+9.9]
Appendix
Table 3: Bootstrap confidence intervals (95%) of the rHV difference between Sequential HVC (Ours) and each baseline at iter=10.
Base Model
Method
rHV (iter=1)
rHV (iter=5)
rHV (iter=10)
VeriReason-3B
Base
2.2%
6.4%
10.3%
Single-Objective PPA
0.9%
5.3%
10.3%
Sequential TB
0.0%
0.2%
0.3%
Sequential TB+PPA
0.1%
0.9%
3.4%
Sequential HVC (Ours)
3.0%
15.1%
19.4%
CodeV-R1-Distil-7B
Base
11.5%
23.9%
32.1%
Appendix
Table 4: Median relative Hypervolume (rHV) computed with respect to the baseline-only archive. Each column shows the median rHV after a given number of sequential generation iterations ( iter ).
Figure 5: Scatter plots of the generated solutions in the objective space for each base model. Visualizing the solutions in the objective space clarifies how each method covers different trade-off regions of the Pareto front. Sequential HVC (Ours) attains a broader coverage of the objective space than the baselines.
Figure 6: Effect of adding an explicit repetition penalty to the sequential baseline, evaluated in terms of relative Hypervolume (top), Win Rate (bottom left), and circuit diversity (bottom right). Although the repetition penalty reduces duplicate generations, it does not match the improvement of Sequential HVC, indicating that the gains of our method arise from directly optimizing the Pareto-front contribution rather than merely avoiding duplicates.
Figure 7: Relative Hypervolume (rHV) evaluated on the union of solutions generated across multiple trials for each method. Aggregating the solutions from independent trials measures the overall Pareto-front coverage attainable by each method. Sequential HVC (Ours) retains its advantage over the baselines for smaller numbers of unioned trials. The results for CodeV-R1-Distil-7B are shown on the following page.
Figure 8: (continued) Relative Hypervolume (rHV) evaluated on the union of solutions generated across multiple trials for each method.
LLM-based RTL code generation methods increasingly target both functional correctness and PPA quality, yet existing approaches universally decouple the two objectives, optimizing PPA only after correctness is fully achieved. Whether through sequential multi-agent pipelines, evolutionary search with binary correctness gates, or hierarchical reward dependencies, partially correct but architecturally promising candidates are systematically discarded. Moreover, existing methods reduce the multi-objective PPA space to a single scalar fitness, obscuring the trade-offs among area, delay, and power. To address these limitations, we propose COEVO, a co-evolutionary framework that unifies correctness and PPA optimization within a single evolutionary loop. COEVO formulates correctness as a continuous co-optimization dimension alongside area, delay, and power, enabled by an enhanced testbench that provides fine-grained scoring and detailed diagnostic feedback. An adaptive correctness gate with annealing allows PPA-promising but partially correct candidates to guide the search toward jointly optimal solutions. To preserve the full PPA trade-off structure, COEVO employs four-dimensional Pareto-based non-dominated sorting with configurable intra-level sorting, replacing scalar fitness without manual weight tuning. Evaluated on VerilogEval 2.0 and RTLLM 2.0, COEVO achieves 97.5% and 94.5% Pass@1 with GPT-5.4-mini, surpassing all agentic baselines across four LLM backbones, while attaining the best PPA on 43 out of 49 synthesizable RTLLM designs.
Applying large language models (LLMs) to RTL code optimization for improved power, performance, and area (PPA) faces two key challenges: ensuring functional correctness of optimized designs despite LLM hallucination, and systematically prioritizing power reduction within the multi-objective PPA trade-off space. We propose POET (Power-Oriented Evolutionary Tuning), a framework that addresses both challenges. For functional correctness, POET introduces a differential-testing-based testbench generation pipeline that treats the original design as a functional oracle, using deterministic simulation to produce golden references and eliminating LLM hallucination from the verification process. For PPA optimization, POET employs an LLM-driven evolutionary mechanism with non-dominated sorting, power-first intra-level ranking, and proportional survivor selection to steer the search toward the low-power region of the Pareto front without manual weight tuning. Evaluated on the RTL-OPT benchmark across 40 diverse RTL designs, POET achieves 100% functional correctness, the best power on all 40 designs, and competitive area and delay improvements.
Register Transfer Level (RTL) design translates high-level specifications into hardware using HDLs such as Verilog. Although LLM-based RTL generation is promising, the scarcity of functionally verifiable high-quality data limits both accuracy and diversity. Existing post-training typically produces a single HDL implementation per specification, lacking awareness of RTL variations needed for different design goals. We propose RTLSeek, a post-training paradigm that applies rule-based Diversity-Oriented Reinforcement Learning to improve RTL correctness and diversity. Our Diversity-Centric Multi-Objective Reward Scheduling integrates expert knowledge with EDA feedback, and a three-stage framework maximizes the utility of limited data. Experiments on the RTLLM benchmark show that RTLSeek surpasses prior methods, with ablation results confirming that encouraging broader design-space exploration improves RTL quality and achieves the principle of "the more generated, the better results." Implementation framework, including the dataset, source code, and model weights, is shown at https://anonymous.4open.science/r/DAC2026ID71-ACB4/.
Xinyu Zhang, Zhiteng Chao, Yonghao Wang +6
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences, China · University of Chinese Academy of Sciences, China · Peking University, China +1