cs.AINov 9, 2026

Causal Signal Credit Assignment for Verilog Code Generation via Counterfactual Hardware Verification

Authors: Yali Du, Ming Li

Organizations: National Key Laboratory for Novel Software Technology, and the School of Artificial Intelligence, Nanjing University, China

Abstract

Large language models have improved Verilog generation, yet functional correctness remains difficult to optimize because simulation commonly provides a coarse module-level pass/fail outcome. Such feedback discards useful evidence from partially correct RTL modules and cannot determine whether a source statement is functionally responsible for an observed output behavior. We present a new method named Causal Signal Credit Assignment (CSCA), which is a counterfactual preference-optimization framework for Verilog generation. CSCA first performs continuous signal-level behavioral verification on sampled candidate modules. It then locates output-driving assignment spans and applies five interventions while holding the remaining module and testbench fixed. The resulting changes in signal-level errors are combined with behavioral agreement to form a continuous statement credit, which is mapped to response-token weights for strict weighted direct preference optimization. This enables CSCA to learn from locally supported statements without uniformly reinforcing an otherwise incorrect response. Built on Icarus Verilog and a CodeV-derived corpus, CSCA is evaluated on VerilogEval2.0 and RTLLMv2.0, including both code completion and specification-to-RTL tasks. With a Qwen2.5-Coder-7B-Instruct backbone, CSCA leads the evaluated 7B-scale models and remains comparable to the much larger GPT-5.4-mini on the reported Verilog-generation benchmarks.

Figures & tables

Explore similar work

Mar 19, 2026

Exploring the Agentic Frontier of Verilog Code Generation

Large language models (LLMs) have made rapid advancements in code generation for popular languages such as Python and C++. Many of these recent gains can be attributed to the use of ``agents'' that wrap domain-relevant tools alongside LLMs. Hardware design languages such as Verilog have also seen improved code generation in recent years, but the impact of agentic frameworks on Verilog code generation tasks remains unclear. In this work, we present the first systematic evaluation of agentic LLMs for Verilog generation, using the recently introduced CVDP benchmark. We also introduce several open-source hardware design agent harnesses, providing a model-agnostic baseline for future work. Through controlled experiments across frontier models, we study how structured prompting and tool design affect performance, analyze agent failure modes and tool usage patterns, compare open-source and closed-source models, and provide qualitative examples of successful and failed agent runs. Our results show that naive agentic wrapping around frontier models can degrade performance (relative to standard forward passes with optimized prompts), but that structured harnesses meaningfully match and in some cases exceed non-agentic baselines. We find that the performance gap between open and closed source models is driven by both higher crash rates and weaker tool output interpretation. Our exploration illuminates the path towards designing special-purpose agents for verilog generation in the future.
Sep 27, 2025

VeriGRAG: Enhancing LLM-Based Verilog Code Generation with Structure-Aware Soft Prompts

Large language models (LLMs) have demonstrated remarkable capabilities in generating Verilog code from natural language descriptions. However, Verilog code inherently encodes the structural characteristics of hardware circuits, including data path organization and interconnections among components. Effectively leveraging these structural characteristics to enhance the functional and syntactic correctness of LLM-based Verilog code generation remains a significant challenge. To address this challenge, we propose VeriGRAG, a novel framework that converts Verilog code into data path graphs and employs graph neural networks (GNNs) to encode their structural information. A multimodal retriever, trained through knowledge distillation from a cross-attention encoder to a dual encoder, then selects the graph embeddings most relevant to the input hardware description. The VeriFormer module uses learnable query tokens and two-stage training to extract code-relevant structural features and align them with the LLM's input embedding space, producing structure-aware soft prompts. Experiments on VerilogEval and RTLLM show improvements in both syntactic and functional correctness. With Qwen2.5-Coder-32B as its backbone, VeriGRAG achieves the highest functional pass@1pass@1 and pass@5pass@5 scores among the compared methods on both benchmarks.
Apr 16, 2026

Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation

Recent advances in large language models have improved code generation, but their use in hardware description languages is still limited. Moreover, training data and testbenches for these models are often scarce. This paper presents a workflow that uses multi-agent models to generate testbenches for high-quality fine-tuning data. By automating testbench creation, the fine-tuned model for the specification-to-Verilog task achieves performance comparable to state-of-the-art methods on the refined VerilogEval v2 benchmark while using less training data. This study provides a basis for future work on LLM-based HDL generation and automated verification.