cs.ARSep 9, 2026

SyntheticHLS: Building Diverse Synthetic High-Level Synthesis Datasets using LLMs

Authors: Stefan Abi-Karam, Miaoyan Zhou, Callie Hao

Organizations: Georgia Institute of Technology · Georgia Tech Research Institute

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Deep learning and large language models (LLMs) are rapidly gaining adoption in semiconductor design, driving demand for training datasets. Most efforts focus on hardware description languages (HDLs) while designs for high-level synthesis (HLS), a popular approach to domain-specific accelerators, remain scarce. HLS dataset efforts emphasize manual curation or design parameterization, seldom addressing high-quality LLM-based generation or diversity in code length, hierarchy, design-space size, latency, resource utilization, and application domain, potentially limiting model generalization. We propose SyntheticHLS, a framework for generating large-scale, complex, diverse synthetic HLS datasets using LLMs. Its two key ideas are: 1) an iterative feedback-guided mutation loop that uses paired HLS source code and design-space specifications to incrementally transform seed designs into more complex, scalable designs; and 2) quantitative metrics of HLS design complexity and design-space scalability that serve as measurable objectives for LLM-guided mutation. We systematically cross-validate an HLS Quality-of-Results (QoR) deep learning model trained and tested across common HLS benchmarks, zero-shot synthetic designs, and iteratively mutated synthetic designs. Synthetic designs transfer well to common benchmark test sets while the reverse does not hold. SyntheticHLS's iteratively mutated designs provide the most generalizable training corpus among the datasets studied. Analysis of the mutation process and dataset shows that metric-guided trajectories consistently improve targeted complexity and scalability objectives without regressing non-target metrics. Mutated designs span a substantially broader, more diverse design space than zero-shot generated designs. Our framework, dataset, and evaluation are open-source: https://github.com/sharc-lab/synthetic-hls.

Figures & tables

Fig. 1 : Overview of SyntheticHLS highlighting our domain-driven seed design generation as well as our iterative metric-guided design mutation to create a final set of diverse, complex, and well-scalable HLS designs for generalizable deep learning training.
Fig. 2 : Selection strategy for picking the final design at each mutation stage. For passing designs, select the candidate with the best target metric, breaking ties with avg. func. LoC. If all candidates fail, attempt LLM-based repair; if none are fixable, the trajectory is terminated.
Fig. 4 : Cross-dataset generalization results for HLS QoR modeling. All bars report root-mean-squared error (RMSE) of the predicted value (lower is better). Each pair of bars denotes a [train]→[test] configuration, where training and test sets are drawn from: Poly (PolyBench), Mach (MachSuite), Poly+Mach (their union), Base (zero-shot synthetic), and Final (feedback-mutated synthetic). Synthetic training data generalizes broadly to benchmark test sets (green region), while the reverse does not hold (red region).
Fig. 5 : Average pass@1 and pass@8 rates of valid, synthesizable designs across the seed generation stage and nine feedback-driven mutation stages, broken down by HLS application domain. Each group of mutation stages targets a different design metric (number of functions, max call depth, average function LoC, and Pareto score). Pass@8 remains near 100%100\% for most domains and is 100%100\% at the final stage, while pass@1 ranges from roughly 30%30\% to 80%80\% .
Fig. 6 : Mutation trajectories for all synthetic HLS designs from SyntheticHLS. Thin lines track individual designs; bold shows the mean. Highlighted regions mark each metric’s targeted optimization phase. Non-targeted metrics remain stable or improve passively alongside targeted ones.
Fig. 7 : Distributions of nine design metrics for synthetic seed designs versus final mutated designs generated by SyntheticHLS, covering post-synthesis QoR (latency and resource usage) and program structure (e.g., number of functions, call depth, function LoC). Final mutated designs exhibit notably more diverse program structure and slightly more diverse design metrics than seed designs.
Fig. 8 : 2D PCA projection of seed and final mutated designs, where each point represents one design described by its nine-metric feature vector (the same metrics as in Fig. 7 ). The seed design space is narrow, restricted largely along a single direction, while the final mutated design space is broader and more uniform, indicating greater overall design diversity.
More in this topic

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 620 papers

Explore similar work

CardsList
Apr 10, 2026

DiffHLS: Differential Learning for High-Level Synthesis QoR Prediction with GNNs and LLM Code Embeddings

High-Level Synthesis (HLS) compiles C/C++ into RTL, but exploring pragma-driven optimization choices remains expensive because each design point requires time-consuming synthesis. We propose \DiffHLS, a differential learning framework for HLS Quality-of-Result (QoR) prediction that learns from kernel--design pairs: a kernel baseline and a pragma-inserted design variant. \DiffHLSencodes kernel and design intermediate-representation graphs with dedicated graph neural network (GNN) branches, and augments the delta pathway with code embeddings from a pretrained code large language model (LLM). Instead of regressing absolute targets directly, we jointly predict the kernel baseline and the design-induced delta, and compose them to obtain the design prediction. On PolyBench, \DiffHLSattains lower average MAPE than GNN baselines under four GNN backbones, and LLM code embeddings consistently improve over a GNN-only ablation. We further validate scalability on the ForgeHLS dataset.

Zedong Peng, Zeju Li, Qiang Xu +1

Shanghai Jiao Tong University, Shanghai, China · The Chinese University of Hong Kong, Hong Kong, China

Electronic Design Automation
AskPDF
May 13, 2026

HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning

High-Level Synthesis (HLS) compiles algorithmic C/C++ descriptions into hardware, with Quality of Results (QoR) -- latency and resource utilization -- critically governed by pragma configurations and code structure. Existing LLM-based HLS approaches train for functional correctness but ignore QoR entirely. We observe that reinforcement learning (RL) for HLS does not require absolute synthesis results -- only relative comparisons between candidates. Based on this insight, we propose HLS-Seek, a QoR-aware NL-to-HLS framework that replaces expensive synthesis-in-the-loop RL with a comparative proxy reward model achieving 99.53% Pareto-dominance accuracy. To prevent reward hacking, we introduce uncertainty-aware Monte Carlo (MC) dropout switching that selectively invokes real Vitis HLS synthesis for low-confidence candidates and online updates the proxy, creating a self-improving reward system. HLS-Seek achieves 81.5% syntax correctness pass@1 and 81.4% Func@5 on HLS-eval with only 7B parameters, surpassing GPT-5.1 and other frontier models while achieving 8.5×\times faster training than real-reward RL. On QoR evaluation, HLS-Seek achieves the lowest latency on 16/30 kernels and Pareto-dominates HLS-specific baselines on 9 kernels.

Qingyun Zou, Feng Yu, Hongshi Tan +3

National University of Singapore

RL for Code GenerationReward ModelingElectronic Design AutomationCode GenerationLLM-Based Program Synthesis
AskPDF
Jul 23, 2026

Benchmarking LLMs for Verilog Design Flows

Large language models (LLMs) show promise in code generation, but their capabilities to produce correct, synthesizable hardware description language (HDL) code still remain to be properly benchmarked. Existing evaluations are primarily relying on pass@k metrics and lack proper end-to-end toolchain validation. This paper presents a reproducible benchmarking platform that evaluates open-source LLMs on Verilog RTL generation across 50 curated tasks consisting of combinational, sequential, finite state machine (FSM), and mixed designs. The pipeline consisting of constrained prompting, post-processing, and semantic-aware iterative refinement with waveform analysis, formal equivalence verification, and Abstract Syntax Tree (AST)-based repair validates the generated code via Verilator compilation and Icarus Verilog simulation. Across the 12 benchmarks and the 1,610 total runs evaluating three models of different sizes (Llama-3-8B, StarCoder2-7B, and TinyLlama-1.1B), the pipeline improved syntax validity from 0% to a 70.43% average and simulation pass rate to 51.8% across three open-source models. Most notably TinyLlama (1.1B parameters) achieved the highest individual syntax validity at 80.0%, with functional correctness comparable to the 8B model. The platform and dataset are open-source, enabling reproducible evaluation of generative AI for hardware design workflows.

Angshuman Chakravertty, Rahul Koshti, Buddhi Prakash Sharma +1

School of Technology Management and Engineering, NMIMS, Hyderabad, India · Department of Electronics and Communication Engineering, Indian Institute of Technology, Roorkee, India · Department of Electrical and Electronics Engineering and the Anuradha and Prashanth Palakurthi Centre for Artificial Intelligence Research (APPCAIR), Birla Institute of Technology and Science, Pilani (BITS Pilani), Pilani 333031, India

Code GenerationCode Generation EvaluationCode Generation BenchmarksLLM Code Generation
AskPDF