Large language models have improved Verilog generation, yet functional correctness remains difficult to optimize because simulation commonly provides a coarse module-level pass/fail outcome. Such feedback discards useful evidence from partially correct RTL modules and cannot determine whether a source statement is functionally responsible for an observed output behavior. We present a new method named Causal Signal Credit Assignment (CSCA), which is a counterfactual preference-optimization framework for Verilog generation. CSCA first performs continuous signal-level behavioral verification on sampled candidate modules. It then locates output-driving assignment spans and applies five interventions while holding the remaining module and testbench fixed. The resulting changes in signal-level errors are combined with behavioral agreement to form a continuous statement credit, which is mapped to response-token weights for strict weighted direct preference optimization. This enables CSCA to learn from locally supported statements without uniformly reinforcing an otherwise incorrect response. Built on Icarus Verilog and a CodeV-derived corpus, CSCA is evaluated on VerilogEval2.0 and RTLLMv2.0, including both code completion and specification-to-RTL tasks. With a Qwen2.5-Coder-7B-Instruct backbone, CSCA leads the evaluated 7B-scale models and remains comparable to the much larger GPT-5.4-mini on the reported Verilog-generation benchmarks.
Figures & tables
Figure 1. Overview of CSCA. A naive code model is first fine-tuned on prompt–reference-code pairs to obtain an SFT model. The SFT model samples candidate RTL implementations, which are assessed by continuous behavioral verification and counterfactual causal attribution. The resulting statement-level credits are used to localize weighted DPO updates.
Figure 2. Examples of CSCA’s five local counterfactual interventions. Starting from the same output-driving statement, CSCA creates Zero, One, Hold, Invert, and Delete variants by modifying only the target assignment while preserving the remaining statements in the local begin...end block. Each variant is subsequently executed using the original testbench to measure its signal-level behavioral impact.
Type
Model
Size
VerilogEval (Completion)
VerilogEval (Specification)
RTLLM v2.0
T=0
T=0.8
T=0
T=0.8
T=0
T=0.8
Pass@1
Pass@1
Pass@5
Pass@1
Pass@1
Pass@5
Pass@1
Pass@1
Pass@5
Foundation General Model
Llama3.1
8B
2.6
2.1
6.4
19.1
16.8
32.1
6.0
6.0
16.0
Qwen2.5-7B
7B
21.8
12.8
35.3
24.4
21.4
42.3
24.0
19.6
42.0
Qwen3-30B-A3B
30B
43.3
43.5
51.9
44.2
46.2
57.1
38.0
39.6
52.0
DeepSeek-v3
671B
68.0
66.1
71.8
68.8
66.9
72.4
56.0
59.1
71.5
Table 1. Main Results on VerilogEval2.0 and RTLLM v2.0 Benchmarks (%). T=0 denotes greedy decoding, for which only Pass@1 can be evaluated since a single sample is generated. For T=0.8 , we report both Pass@1 and Pass@5. Bold values indicate the best performance among Verilog-specific models.
Configuration
VerilogEval 2.0
RTLLM v2.0
Code Completion
Spec.-to-RTL
Syntax Pass@1
Syntax Pass@5
Pass@1
Pass@5
Pass@1
Pass@5
Pass@1
Pass@5
Base Model (Qwen2.5-Coder)
27.9
43.0
22.0
50.6
74.0
88.0
33.2
52.5
+ SFT
32.7
44.2
31.0
53.8
75.2
96.2
35.5
50.0
+ SFT + Counterfactual Effect
49.7
65.6
43.2
62.0
84.5
94.0
41.0
55.3
+ SFT + Behavioral Correctness
49.3
65.2
42.8
61.3
84.8
94.0
41.2
56.0
Table 2. Credit component ablation study of CSCA ( T=0.8 ). All variants share the same base model, prompt set, and simulation budget. VerilogEval 2.0 is split into Code Completion and Specification-to-RTL subtasks. The largest improvement among all ablations for each metric is marked in bold.
Figure 5
Temperature
VerilogEval 2.0
RTLLM v2.0
Code Completion
Spec.-to-RTL
Syntax Pass@1
Syntax Pass@5
Pass@1
Pass@5
Pass@1
Pass@5
Pass@1
Pass@5
T=0.0†
56.4
—
49.3
—
88.0
—
50.0
—
T=0.1
53.7
57.7
46.3
52.6
87.2
90.0
45.2
54.0
T=0.2
54.1
59.6
45.1
51.9
87.2
92.0
45.6
56.0
T=0.3
55.1
60.9
45.4
56.4
86.8
94.0
47.2
60.0
Table 3. Functional and syntax pass rates (%) over temperature T . T=0 denotes greedy decoding ( N=1 ), where Pass@5 metrics are not applicable. For T≥0 , N=5 samples are used.
Figure 5. Counterfactual gain statistics from 893225 RTL intervention samples. (a) Distribution of intervention outcomes. (b) Histogram of nonzero counterfactual gains for informative records.
Task
Verilog generated by SFT
Verilog generated by CSCA
Prob030_popcount255
assign count_8bit = in[7:0] + in[15:8] + ⋯ + in[255:248]; Adds byte values rather than their constituent bits.
always @(*) begin pop_count = 0; for (integer i=0; i<255; i=i+1) pop_count = pop_count + in[i]; end
Testbench
0/215
215/215
Pass rate
0.0%
100.0%
Prob037_ review2015_count1k
always @(posedge clk or posedge reset) if (reset) counter <= 0; else counter <= counter + 1; Uses asynchronous reset and never wraps at 1000.
always @(posedge clk) begin if (reset) q <= 0; else if (q == 10’d999) q <= 0; else q <= q + 1; end
Testbench
4896/8027
8027/8027
Pass rate
61.0%
100.0%
Table 4. Representative functional improvements over the SFT baseline. Both candidates compile. The benchmark testbench reports passing samples and pass rate.