VeriGRAG: Enhancing LLM-Based Verilog Code Generation with Structure-Aware Soft Prompts

Jiayu Zhao; Zehua Pei; Zetao Guo; Bei Yu; Song Chen

cs.ARSep 27, 2025

VeriGRAG: Enhancing LLM-Based Verilog Code Generation with Structure-Aware Soft Prompts

Authors: Jiayu Zhao, Zehua Pei, Zetao Guo, Bei Yu, Song Chen

Organizations: University of Science and Technology of China

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities in generating Verilog code from natural language descriptions. However, Verilog code inherently encodes the structural characteristics of hardware circuits, including data path organization and interconnections among components. Effectively leveraging these structural characteristics to enhance the functional and syntactic correctness of LLM-based Verilog code generation remains a significant challenge. To address this challenge, we propose VeriGRAG, a novel framework that converts Verilog code into data path graphs and employs graph neural networks (GNNs) to encode their structural information. A multimodal retriever, trained through knowledge distillation from a cross-attention encoder to a dual encoder, then selects the graph embeddings most relevant to the input hardware description. The VeriFormer module uses learnable query tokens and two-stage training to extract code-relevant structural features and align them with the LLM's input embedding space, producing structure-aware soft prompts. Experiments on VerilogEval and RTLLM show improvements in both syntactic and functional correctness. With Qwen2.5-Coder-32B as its backbone, VeriGRAG achieves the highest functional $pass@1$ and $pass@5$ scores among the compared methods on both benchmarks.

Figures & tables

Figure 1. Performance comparison of different base models and VeriGRAG on the VerilogEval-Machine ( Liu et al., 2023 ) benchmark. Comparison of pass@1 and pass@5 scores on VerilogEval-Machine. VeriGRAG improves both metrics over the corresponding base models across different LLM backbones.

Figure 2. Overview of the Structure-Aware VeriGRAG Framework. (a) We first extract data path graphs from Verilog code using Yosys and encode them with a graph neural network (GNN). (b) Given a user-provided hardware description (SPEC), we retrieve the most relevant graph embedding from the graph database. (c) VeriFormer aligns the graph embedding with the LLM’s input embedding space and provides structure-aware soft prompts to enhance the LLM’s capability in generating accurate Verilog code. Verilog code is converted into data path graphs using Yosys and encoded by a graph neural network. A hardware description is used to retrieve the most relevant graph embedding from the graph database. VeriFormer aligns the retrieved embedding with the LLM's input embedding space to provide structure-aware soft prompts for Verilog code generation.

Figure 3. An example of data path graph from Verilog code. (a) Verilog code of a flip-flop; (b) the corresponding data path. The left panel shows the Verilog code of a flip-flop, and the right panel shows its corresponding data path graph. Graph nodes represent hardware modules or signals, and edges represent their interconnections.

Figure 4. The knowledge distillation training process of our multimodal retriever A cross-attention teacher encoder is first trained using the InfoNCE loss. With the teacher's parameters held fixed, a dual-encoder student is trained using InfoNCE and mean squared error losses to learn the teacher's embedding behavior. The student's independent encoders allow graph embeddings to be computed in advance and cached for efficient retrieval during inference.

Figure 5. The overall framework of VeriFormer aligns the GNN and the LLM through a two-stage training paradigm to optimize Verilog code generation. The lightweight VeriFormer module is trained using this two-stage paradigm. VeriFormer is trained in two stages. In the first stage, the GNN is frozen, while VeriFormer and learnable query tokens are trained with graph-code contrastive learning, generation, and matching objectives. In the second stage, the code transformer is discarded, and graph-transformer query outputs are projected into the LLM's input embedding space as soft prompts. Training combines the LLM generation loss with a KL-divergence loss for distribution alignment.

Type	Model	Params	VerilogEval-Machine		VerilogEval-Human		VerilogEval v2
			Function		Function		Syntax		Function
			$pass@1$	$pass@5$	$pass@1$	$pass@5$	$pass@1$	$pass@5$	$pass@1$	$pass@5$
General LLMs	GPT-4o ( Hurst et al., 2024 )	N/A	65.9	71.4	57.1	63.9	90.7	95.2	56.5	65.1
	Llama3.1 ( Grattafiori et al., 2024 )	405B	67.3	75.1	53.8	61.0	91.2	93.5	50.2	57.1
	CodeLlama ( Roziere et al., 2023 )	7B	43.1	47.1	18.2	22.7	37.1	56.1	12.1	20.2
	CodeQwen ( Bai et al., 2023 )	7B	46.5	54.9	22.5	26.1	52.4	57.0	19.8	24.3

Table 1. Comparison of Model Performance On VerilogEval-Machine, VerilogEval-Human and VerilogEval v2.

Type	Model	Params	RTLLM v1.1				RTLLM v2
			Syntax		Function		Syntax		Function
			$pass@1$	$pass@5$	$pass@1$	$pass@5$	$pass@1$	$pass@5$	$pass@1$	$pass@5$
General LLMs	GPT-4o ( Hurst et al., 2024 )	N/A	82.4	86.2	47.9	58.0	80.0	89.5	47.9	58.0
	Llama3.1 ( Grattafiori et al., 2024 )	405B	73.2	81.8	38.9	45.8	73.0	81.6	38.8	45.6
	CodeLlama ( Roziere et al., 2023 )	7B	46.6	62.6	17.9	29.9	49.5	76.7	21.2	31.9
	CodeQwen ( Bai et al., 2023 )	7B	45.8	65.8	24.1	34.0	47.1	66.4	25.8	29.0

Table 2. Comparison of Model Performance On RTLLM v1.1 and RTLLM v2.

Panel of Figure 6 — Figure 6. Positive and negative transfers at different temperatures, measured relative to No-RAG. NTR denotes the negative transfer rate, defined as the proportion of No-RAG passing cases that fail after retrieval. Two panels compare retrieval-induced transfer for VeriGRAG and Text-RAG at temperatures 0.2, 0.5, and 0.8 using Qwen2.5-Coder-7B. Panel (a) evaluates 299 VerilogEval-Machine and VerilogEval-Human samples: VeriGRAG yields larger net gains and fewer negative transfers at all three temperatures. Panel (b) evaluates 50 RTL designs with localized modifications and conflicting retrieval context: VeriGRAG reduces negative transfers at all three temperatures. Positive transfer corrects a No-RAG failure; negative transfer causes a No-RAG passing case to fail.

Method	Machine		Human
Method	$pass@1$	$pass@5$	$pass@1$	$pass@5$
Text-RAG	77.2	86.6	53.8	62.5
VeriGRAG w/o KD	78.4	86.3	56.0	63.8
VeriGRAG w/ KD	79.4	87.3	56.4	65.6

Table 3. Functional correctness results on VerilogEval v1.

Method	$pass@1$	$pass@5$
Text-RAG	58.2	65.3
VeriGRAG w/o KD	60.4	65.5
VeriGRAG w/ KD	60.4	66.8

Table 4. Functional correctness results on VerilogEval v2.

RTL Code Generation

Register-transfer-level (RTL) code generation turns hardware specifications into synthesizable circuits, balancing functional correctness with area, timing, and power.

See all 30 papers

Exploring the Agentic Frontier of Verilog Code Generation
Mar 19, 2026Patrick Yubeaton, Siddharth Garg, Chinmay HegdeCode Generation Agentic Code Generation
New York University, Tandon School of Engineering, Brooklyn, NY
Benchmarking LLMs for Verilog Design Flows
Jul 23, 2026Angshuman Chakravertty, Rahul Koshti, Buddhi Prakash Sharma +1Code Generation Code Generation Evaluation
School of Technology Management and Engineering, NMIMS, Hyderabad, India · Department of Electronics and Communication Engineering, Indian Institute of Technology, Roorkee, India · Department of Electrical and Electronics Engineering and the Anuradha and Prashanth Palakurthi Centre for Artificial Intelligence Research (APPCAIR), Birla Institute of Technology and Science, Pilani (BITS Pilani), Pilani 333031, India
Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation
Apr 16, 2026Mu-Chi Chen, Po-Hsuan Huang, Yu-Hung Kao +6LLM Fine-Tuning Code Generation
Academia Sinica, Taipei, Taiwan · National Taiwan University, Taipei, Taiwan · National Cheng Kung University, Tainan, Taiwan

Mar 19, 2026J/K move · Enter open · S save

Exploring the Agentic Frontier of Verilog Code Generation

Patrick Yubeaton, Siddharth Garg, Chinmay Hegde

New York University, Tandon School of Engineering, Brooklyn, NY

Code Generation Agentic Code Generation LLM Agent Harnesses Electronic Design Automation

Open paper AskPDF arXiv

Large language models (LLMs) have made rapid advancements in code generation for popular languages such as Python and C++. Many of these recent gains can be attributed to the use of ``agents'' that wrap domain-relevant tools alongside LLMs. Hardware design languages such as Verilog have also seen improved code generation in recent years, but the impact of agentic frameworks on Verilog code generation tasks remains unclear. In this work, we present the first systematic evaluation of agentic LLMs for Verilog generation, using the recently introduced CVDP benchmark. We also introduce several open-source hardware design agent harnesses, providing a model-agnostic baseline for future work. Through controlled experiments across frontier models, we study how structured prompting and tool design affect performance, analyze agent failure modes and tool usage patterns, compare open-source and closed-source models, and provide qualitative examples of successful and failed agent runs. Our results show that naive agentic wrapping around frontier models can degrade performance (relative to standard forward passes with optimized prompts), but that structured harnesses meaningfully match and in some cases exceed non-agentic baselines. We find that the performance gap between open and closed source models is driven by both higher crash rates and weaker tool output interpretation. Our exploration illuminates the path towards designing special-purpose agents for verilog generation in the future.

Abstract

RTL Code Generation

Explore similar work

Exploring the Agentic Frontier of Verilog Code Generation

Benchmarking LLMs for Verilog Design Flows

Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation