DiffHLS: Differential Learning for High-Level Synthesis QoR Prediction with GNNs and LLM Code Embeddings

Zedong Peng; Zeju Li; Qiang Xu; Jieru Zhao

cs.LGApr 10, 2026

DiffHLS: Differential Learning for High-Level Synthesis QoR Prediction with GNNs and LLM Code Embeddings

Authors: Zedong Peng, Zeju Li, Qiang Xu, Jieru Zhao

Organizations: Shanghai Jiao Tong University, Shanghai, China · The Chinese University of Hong Kong, Hong Kong, China

Abstract

High-Level Synthesis (HLS) compiles C/C++ into RTL, but exploring pragma-driven optimization choices remains expensive because each design point requires time-consuming synthesis. We propose \DiffHLS, a differential learning framework for HLS Quality-of-Result (QoR) prediction that learns from kernel--design pairs: a kernel baseline and a pragma-inserted design variant. \DiffHLSencodes kernel and design intermediate-representation graphs with dedicated graph neural network (GNN) branches, and augments the delta pathway with code embeddings from a pretrained code large language model (LLM). Instead of regressing absolute targets directly, we jointly predict the kernel baseline and the design-induced delta, and compose them to obtain the design prediction. On PolyBench, \DiffHLSattains lower average MAPE than GNN baselines under four GNN backbones, and LLM code embeddings consistently improve over a GNN-only ablation. We further validate scalability on the ForgeHLS dataset.

Figures & tables

Fig. 1: Overview of DiffHLS . We encode kernel and design IR graphs, inject LLM code embeddings into the delta pathway, and predict the kernel baseline and delta; finally the design prediction is obtained by composition.

Panel of Figure 2 — Fig. 2: Example kernel code (top) and design code (bottom) illustrating HLS pragma insertion.

[min, max]	Kernel	Delta	Design
DSP	[3, 29]	[ $-$ 7, 75]	[5, 94]
FF	[495, 3994]	[ $-$ 174, 25414]	[1296, 28967]
LUT	[1214, 5192]	[151, 35865]	[1842, 39464]
CP	[5.31, 7.21]	[ $-$ 0.10, 0.78]	[5.55, 7.75]

TABLE I: Target value ranges (kernel, delta, design).

Backbone	Method	DSP	FF	LUT	CP
PNA	GNN [ 1 ]	14.71	26.47	22.86	8.87
	HGNN [ 2 ]	7.84	9.65	10.55	-
	DiffHLS	3.63	8.18	5.52	1.34
GraphSAGE	GNN [ 1 ]	17.01	39.11	28.09	8.25
	HGNN [ 2 ]	6.94	9.99	9.86	-
	DiffHLS	3.31	7.34	5.12	1.06

TABLE II: QoR prediction accuracy under different GNN backbones.

Backbone	Method	DSP			FF			LUT			CP
Backbone	Method	MAPE $\downarrow$	MAE $\downarrow$	$R^{2}$ $\uparrow$	MAPE $\downarrow$	MAE $\downarrow$	$R^{2}$ $\uparrow$	MAPE $\downarrow$	MAE $\downarrow$	$R^{2}$ $\uparrow$	MAPE $\downarrow$	MAE $\downarrow$	$R^{2}$ $\uparrow$
PNA	DiffHLS (w/o diff)	3.86	1.16	0.980	10.75	725.94	0.972	8.50	891.66	0.969	1.22	0.090	0.711
	DiffHLS (w/o code emb.)	4.18	1.39	0.976	8.51	677.34	0.963	6.30	641.35	0.967	1.22	0.090	0.735
	DiffHLS	3.63	1.21	0.899	8.18	629.84	0.973	5.52	627.03	0.989	1.34	0.099	0.710
GraphSAGE	DiffHLS (w/o diff)	4.19	1.25	0.971	10.60	680.01	0.926	7.76	707.05	0.935	1.31	0.096	0.679
	DiffHLS (w/o code emb.)	5.11	1.81	0.945	8.64	698.86	0.974	5.78	576.78	0.991	1.24	0.092	0.721

TABLE III: Ablation study: differential vs. direct prediction and effect of code embeddings.

Head	Backbone	Code Emb.	DSP			FF			LUT			CP
Head	Backbone	Code Emb.	MAPE	MAE	$R^{2}$	MAPE	MAE	$R^{2}$	MAPE	MAE	$R^{2}$	MAPE	MAE	$R^{2}$
Kernel $\hat{y}_{k}$	PNA	No	0.43	0.08	1.000	0.74	15.53	0.999	0.55	13.36	1.000	0.06	0.004	0.993
	PNA	Yes	0.84	0.12	0.999	0.66	15.08	1.000	0.38	8.81	1.000	0.12	0.008	0.933
	GraphSAGE	No	0.89	0.14	0.999	0.94	20.13	1.000	0.58	13.54	1.000	0.11	0.008	0.955
	GraphSAGE	Yes	0.51	0.09	1.000	0.86	20.48	1.000	0.61	13.83	1.000	0.07	0.005	0.998
	GCN	No	1.20	0.15	0.999	0.67	13.44	1.000	0.82	19.48	0.999	0.06	0.004	0.996

TABLE IV: Ablation study: prediction head accuracy.

Fig. 3: Average MAPE (%) across DSP, FF, LUT, and CP for the three method variants under four GNN backbones.

Backbone	Model	DSP			FF			LUT			CP
Backbone	Model	MAPE $\downarrow$	MAE $\downarrow$	$R^{2}$ $\uparrow$	MAPE $\downarrow$	MAE $\downarrow$	$R^{2}$ $\uparrow$	MAPE $\downarrow$	MAE $\downarrow$	$R^{2}$ $\uparrow$	MAPE $\downarrow$	MAE $\downarrow$	$R^{2}$ $\uparrow$
PNA	Qwen2.5-Coder-1.5B	3.63	1.21	0.899	8.18	629.84	0.973	5.52	627.03	0.989	1.34	0.099	0.710
PNA	Llama-3.2-1B	3.96	1.40	0.979	8.09	524.57	0.967	5.60	457.07	0.984	1.13	0.084	0.776
GraphSAGE	Qwen2.5-Coder-1.5B	3.31	1.28	0.964	7.34	647.85	0.897	5.12	510.83	0.991	1.06	0.078	0.847
GraphSAGE	Llama-3.2-1B	4.52	1.71	0.926	8.14	528.27	0.963	5.56	453.39	0.989	1.17	0.086	0.738
GCN	Qwen2.5-Coder-1.5B	8.10	2.84	0.950	9.31	1001.49	0.413	5.46	621.47	0.985	1.37	0.101	0.664

TABLE V: Ablation study: code LLM backbone.

Backbone	DSP	FF	LUT	CP
PNA	5.70	25.75	10.05	4.21
GraphSAGE	5.69	17.30	7.77	4.93
GCN	9.11	25.18	11.94	5.83
GAT	10.75	20.16	9.27	4.41

TABLE VI: ForgeHLS large-scale evaluation.

Electronic Design Automation

Electronic Design Automation uses algorithms and software to design, verify, optimize, and manufacture electronic circuits and systems.

See all 212 papers

HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning
May 13, 2026Qingyun Zou, Feng Yu, Hongshi Tan +3RL for Code Generation Reward Modeling
National University of Singapore
SyntheticHLS: Building Diverse Synthetic High-Level Synthesis Datasets using LLMs
Sep 9, 2026Stefan Abi-Karam, Miaoyan Zhou, Callie HaoSynthetic Data Generation Electronic Design Automation
Georgia Institute of Technology · Georgia Tech Research Institute
HLSmith: An Expert-Guided Agentic Framework for C/C++-to-HLS Translation
Aug 7, 2026Yuebo Luo, Ahmad Sedigh Baroughi, Philip Stachura +4Code Translation Large Language Model-Guided Optimization
University of Minnesota Twin Cities · Simon Fraser University · Argonne National Laboratory

May 13, 2026J/K move · Enter open · S save

HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning

Qingyun Zou, Feng Yu, Hongshi Tan, Yao Chen+2

National University of Singapore

RL for Code Generation Reward Modeling Electronic Design Automation Code Generation LLM-Based Program Synthesis

Open paper AskPDF arXiv

High-Level Synthesis (HLS) compiles algorithmic C/C++ descriptions into hardware, with Quality of Results (QoR) -- latency and resource utilization -- critically governed by pragma configurations and code structure. Existing LLM-based HLS approaches train for functional correctness but ignore QoR entirely. We observe that reinforcement learning (RL) for HLS does not require absolute synthesis results -- only relative comparisons between candidates. Based on this insight, we propose HLS-Seek, a QoR-aware NL-to-HLS framework that replaces expensive synthesis-in-the-loop RL with a comparative proxy reward model achieving 99.53% Pareto-dominance accuracy. To prevent reward hacking, we introduce uncertainty-aware Monte Carlo (MC) dropout switching that selectively invokes real Vitis HLS synthesis for low-confidence candidates and online updates the proxy, creating a self-improving reward system. HLS-Seek achieves 81.5% syntax correctness pass@1 and 81.4% Func@5 on HLS-eval with only 7B parameters, surpassing GPT-5.1 and other frontier models while achieving 8.5

\times

faster training than real-reward RL. On QoR evaluation, HLS-Seek achieves the lowest latency on 16/30 kernels and Pareto-dominates HLS-specific baselines on 9 kernels.

Abstract

Electronic Design Automation

Explore similar work

HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning

SyntheticHLS: Building Diverse Synthetic High-Level Synthesis Datasets using LLMs

HLSmith: An Expert-Guided Agentic Framework for C/C++-to-HLS Translation