cs.AR · 2604.15388 Copy arXiv ID · Apr 16, 2026 Save Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation Authors: Mu-Chi Chen , Po-Hsuan Huang , Yu-Hung Kao , Yen-Fu Liu , Yu-Kai Hung , Cheng Liang , Shao-Chun Ho , Chia-Heng Tu , +1 more
Organizations: Academia Sinica, Taipei, Taiwan · National Taiwan University, Taipei, Taiwan · National Cheng Kung University, Tainan, Taiwan
Abstract Recent advances in large language models have improved code generation, but their use in hardware description languages is still limited. Moreover, training data and testbenches for these models are often scarce. This paper presents a workflow that uses multi-agent models to generate testbenches for high-quality fine-tuning data. By automating testbench creation, the fine-tuned model for the specification-to-Verilog task achieves performance comparable to state-of-the-art methods on the refined VerilogEval v2 benchmark while using less training data. This study provides a basis for future work on LLM-based HDL generation and automated verification.
Explore similar work Jul 23, 2026 · Angshuman Chakravertty, Rahul Koshti, Buddhi Prakash Sharma +1 Vhdl High-Level Synthesis
Jun 11, 2026 · Yijun Shen, Minghao Shao, Yichen Zhao +4 Vhdl Hardware Description Languages
Jul 23, 2026 · cs.AR J/K move · Enter open · S save
Angshuman Chakravertty, Rahul Koshti, Buddhi Prakash Sharma, Vinay Chamola
Large language models (LLMs) show promise in code generation, but their capabilities to produce correct, synthesizable hardware description language (HDL) code still remain to be properly benchmarked. Existing evaluations are primarily relying on pass@k metrics and lack proper end-to-end toolchain validation. This paper presents a reproducible benchmarking platform that evaluates open-source LLMs on Verilog RTL generation across 50 curated tasks consisting of combinational, sequential, finite state machine (FSM), and mixed designs. The pipeline consisting of constrained prompting, post-processing, and semantic-aware iterative refinement with waveform analysis, formal equivalence verification, and Abstract Syntax Tree (AST)-based repair validates the generated code via Verilator compilation and Icarus Verilog simulation. Across the 12 benchmarks and the 1,610 total runs evaluating three models of different sizes (Llama-3-8B, StarCoder2-7B, and TinyLlama-1.1B), the pipeline improved syntax validity from 0% to a 70.43% average and simulation pass rate to 51.8% across three open-source models. Most notably TinyLlama (1.1B parameters) achieved the highest individual syntax validity at 80.0%, with functional correctness comparable to the 8B model. The platform and dataset are open-source, enabling reproducible evaluation of generative AI for hardware design workflows.