Climbing the Design Ladder: Sequential Knowledge Distillation for Early-Stage Circuit Timing Prediction

Reza Moravej; Fahad Rahman Amik; Zhanguang Zhang; Didier Chételat; Yingxue Zhang

cs.LGOct 5, 2026

Climbing the Design Ladder: Sequential Knowledge Distillation for Early-Stage Circuit Timing Prediction

Authors: Reza Moravej, Fahad Rahman Amik, Zhanguang Zhang, Didier Chételat, Yingxue Zhang

Organizations: Huawei Noah’s Ark Lab · McGill University

Abstract

Integrated circuit design involves multiple design stages: logic synthesis, floorplanning, placement, and routing, with each stage taking hours to weeks to complete. Discovering timing violations late in this flow forces costly iterations back to earlier stages, wasting computational resources and delaying product launches. While predicting post-routing timing from early-stage data could prevent these failures, existing machine learning approaches struggle with the massive abstraction gap between post-synthesis logical descriptions and post-routing physical layouts. We propose STEP-KD (Sequential Timing Evaluation via Progressive Knowledge Distillation), which leverages intermediate design stages as ``stepping stones'' for progressive knowledge transfer rather than attempting direct prediction. STEP-KD trains teacher models at the post-routing, post-placement, and post-floorplan stages, then sequentially distills their knowledge to a post-synthesis student model through representation alignment. Experiments on diverse circuits demonstrate that STEP-KD reduces timing prediction error compared to direct distillation and supervised baselines, and in most settings compared to the industry-standard Static Timing Analysis (STA) tool. STEP-KD reduces the weighted mean absolute percentage error of Total Negative Slack prediction to 19.78%, compared with 74.84% for STA. Our proposed method is step forward to identify timing problems earlier, avoiding expensive late-stage redesigns.

Figures & tables

Panel of Figure 1 — Figure 1: Training flow (left) : Models are trained one at a time down the design ladder, in the order PR, PP, PF, PS. Each model is trained to predict post-routing slack and, except for PR, is regularized by knowledge distillation (KD) losses that align its representations, passed through its own cascade of learned projections, with those of the frozen models at all later stages, which have access to physical information not yet available at its own stage. Only the post-synthesis model is used at inference. Model architecture (right) : The architecture of our machine learning model. It takes as input a graphical representation of a circuit at the post-synthesis stage (netlist graph) and outputs the slack for every sequential element in the circuit. From these predictions, circuit-wide metrics such as the total negative slack (TNS) and worst negative slack (WNS) can be computed.

Held-out test set
	TNS			WNS
Method	MedAE $\downarrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MedAE $\downarrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$
STA	$512.61\pm 20.42$	$74.84\pm 0.60$	$145.49\pm 3.11$	$23.69\pm 0.56$	$35.59\pm 1.53$	$46.52\pm 14.72$
No KD	$740.05\pm 52.44$	$38.75\pm 2.20$	$16.83\pm 4.06$	$25.82\pm 1.73$	$44.49\pm 2.23$	$62.14\pm 14.24$
Direct KD (PR teacher)	$399.07\pm 27.79$	$24.05\pm 1.51$	$11.24\pm 3.13$	$15.45\pm 0.99$	$28.64\pm 2.30$	$45.16\pm 18.84$
Direct KD (PP teacher)	$396.84\pm 31.08$	$24.49\pm 1.84$	$10.95\pm 3.71$	$16.56\pm 0.83$	$29.14\pm 2.12$	$45.33\pm 19.40$

Table 1: Performance of methods taking post-synthesis inputs and predicting post-routing timing on the held-out test set (top) and the OpenLLM-RTL benchmark (bottom). Direct KD distills from a single teacher (stage in parentheses). STEP-KD w/o sequential KD aligns the student to independently trained PF, PP and PR teachers; STEP-KD uses teachers that were themselves distilled along the design ladder (PR

\rightarrow

\rightarrow

PF). MedAE (median absolute error) is in the units of the slack labels. Best per dataset in bold.

Panel of Figure 2 — Figure 2: Predicted versus post-routing TNS and WNS (magnitudes) for post-synthesis STA and STEP-KD on the held-out test set (top) and the OpenLLM-RTL benchmark (bottom). The diagonal marks perfect prediction. Each color is one of the 20 evaluation runs, each of which samples one implementation per design.

		TNS			WNS
Stage	Predictor	MedAE $\downarrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MedAE $\downarrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$
PR	STA	$0.00\pm 0.00$	$0.00\pm 0.00$	$0.00\pm 0.00$	$0.00\pm 0.00$	$0.00\pm 0.00$	$0.00\pm 0.00$
PR	Teacher †	$32.61\pm 2.88$	$1.74\pm 0.15$	$0.06\pm 0.03$	$1.12\pm 0.08$	$1.88\pm 0.22$	$0.23\pm 0.21$
PP	STA	$378.28\pm 16.03$	$63.57\pm 0.91$	$105.06\pm 3.82$	$18.80\pm 0.61$	$29.71\pm 1.06$	$22.64\pm 5.65$
	Independent (no KD)	$355.77\pm 24.70$	$19.61\pm 1.52$	$6.01\pm 2.13$	$\mathbf{12.11\pm 0.73}$	$\mathbf{21.22\pm 1.53}$	$20.09\pm 9.96$
	KD from PR	$\mathbf{306.37\pm 19.85}$	$\mathbf{17.29\pm 1.02}$	$\mathbf{4.56\pm 1.19}$	$14.46\pm 0.60$	$22.78\pm 1.08$	$\mathbf{17.06\pm 5.26}$

Table 2: STA, teacher models and the effect of sequential knowledge distillation (held-out test set). At each intermediate stage we compare STA, a model trained without distillation from later stages, and a model distilled along the design ladder. At PS both STEP-KD variants align to PF, PP and PR teachers one at a time and differ only in whether those teachers were themselves distilled; post-synthesis STA is reported in Table 1 . MedAE (median absolute error) is in the units of the slack labels. Best within each stage in bold. † The PR teacher receives post-routing slack as an input feature, so its near-perfect score is expected and shown only for completeness.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Histogram of Dataset Attributes

	TNS			WNS
Node-level representation	MedAE $\downarrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MedAE $\downarrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$
(i) GNN output $\mathbf{z}_{r}$	$2201.40\pm 132.98$	$78.78\pm 2.45$	$60.46\pm 5.65$	$55.85\pm 1.66$	$75.76\pm 1.90$	$103.59\pm 5.26$
(ii) Slack embedding $p(\text{slack}(r))$	$495.26\pm 33.28$	$24.97\pm 2.01$	$11.48\pm 3.52$	$15.83\pm 0.94$	$29.38\pm 2.16$	$\mathbf{43.00\pm 18.45}$
(iii) $\mathbf{z}_{r}+p(\text{slack}(r))$ (ours)	$\mathbf{399.07\pm 27.79}$	$\mathbf{24.05\pm 1.51}$	$\mathbf{11.24\pm 3.13}$	$\mathbf{15.45\pm 0.99}$	$\mathbf{28.64\pm 2.30}$	$45.16\pm 18.84$

Appendix

Table 3: Node-level input of the timing predictor for the post-synthesis model trained with Direct KD from the PR teacher (held-out test set). Row (iii) is the configuration used throughout the paper and corresponds to Direct KD (PR teacher) in Table 1 . MedAE is in the units of the slack labels. Best in bold.

Method	Input stage	Prediction target	Training paradigm
RTL-Timer [ Fang et al., 2024b ]	RTL (pre-synthesis)	Post-synthesis timing	Supervised
NetTAG [ Fang et al., 2025b ]	Post-synthesis netlist	Slack and other tasks	Self-supervised pre-training
CircuitFusion [ Fang et al., 2025a ]	RTL (multimodal)	TNS/WNS and other tasks	Self-supervised pre-training
STEP-KD (ours)	Post-synthesis netlist	Post-routing TNS/WNS	Supervised sequential KD

Appendix

Table 4: Differences between STEP-KD and prior timing prediction methods.

Panel of Figure 7 — Figure 7: Post-routing TNS and WNS (magnitudes, vertical axis) versus the STA estimate at the post-floorplan (PF) and post-placement (PP) stages (horizontal axis) on the held-out test set (top) and the OpenLLM-RTL benchmark (bottom). The diagonal marks agreement between the intermediate-stage and post-routing STA.

Panel of Figure 8 — Figure 8: Predicted versus post-routing TNS and WNS (magnitudes) for the No KD baseline, in the same format as Figure 2 .

Held-out test set
	TNS		WNS
Method	R 2 (%) $\uparrow$	MAE $\downarrow$	R 2 (%) $\uparrow$	MAE $\downarrow$
STA	$-45.49\pm 3.11$	$5072.45\pm 106.68$	$53.48\pm 14.72$	$32.25\pm 2.32$
No KD	$83.17\pm 4.06$	$2604.43\pm 180.34$	$37.86\pm 14.24$	$39.81\pm 3.06$
Direct KD (PR teacher)	$88.76\pm 3.13$	$1619.63\pm 126.85$	$54.84\pm 18.84$	$25.74\pm 2.80$
Direct KD (PP teacher)	$89.05\pm 3.71$	$1649.69\pm 151.67$	$54.67\pm 19.40$	$26.17\pm 2.64$

Appendix

Table 5: Results of Table 1 reported with

R^{2}

and the mean absolute error (MAE, in the units of the slack labels) on the held-out test set (top) and the OpenLLM-RTL benchmark (bottom). Best per dataset in bold.

	TNS				WNS
KD granularity	R 2 (%) $\uparrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MAE $\downarrow$	R 2 (%) $\uparrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MAE $\downarrow$
No KD (reference)	$83.17\pm 4.06$	$38.75\pm 2.20$	$16.83\pm 4.06$	$2604.43\pm 180.34$	$37.86\pm 14.24$	$44.49\pm 2.23$	$62.14\pm 14.24$	$39.81\pm 3.06$
Graph only	$90.74\pm 2.78$	$24.07\pm 1.27$	$9.26\pm 2.78$	$1617.55\pm 110.02$	$56.93\pm 16.72$	$26.53\pm 1.82$	$43.07\pm 16.72$	$23.70\pm 2.21$
Subgraph only	$49.72\pm 3.98$	$56.36\pm 1.58$	$50.28\pm 3.98$	$3786.66\pm 155.30$	$52.70\pm 14.19$	$38.69\pm 1.54$	$47.30\pm 14.19$	$34.54\pm 2.22$
Node only	$89.84\pm 3.22$	$26.95\pm 1.92$	$10.16\pm 3.22$	$1812.13\pm 158.15$	$49.84\pm 16.68$	$30.62\pm 1.82$	$50.16\pm 16.68$	$27.35\pm 2.29$
All (STEP-KD)	$\mathbf{92.15\pm 2.95}$	$\mathbf{19.78\pm 1.54}$	$\mathbf{7.85\pm 2.95}$	$\mathbf{1330.00\pm 125.66}$	$\mathbf{58.91\pm 17.81}$	$\mathbf{21.39\pm 1.97}$	$\mathbf{41.09\pm 17.81}$	$\mathbf{19.12\pm 2.24}$

Appendix

Table 6: Complete results for the multi-granularity ablation on the held-out test set (Figure 4 ). Each variant distills from the same PF, PP and PR teachers as STEP-KD but aligns only the listed granularity; All is the full STEP-KD. No KD is included for reference. MAE is in the units of the slack labels. Best in bold.

	TNS				WNS
Projection layers	R 2 (%) $\uparrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MAE $\downarrow$	R 2 (%) $\uparrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MAE $\downarrow$
4	$88.45\pm 2.89$	$24.76\pm 1.22$	$11.55\pm 2.89$	$1673.53\pm 108.72$	$51.62\pm 16.35$	$26.74\pm 2.01$	$48.38\pm 16.35$	$24.19\pm 2.45$
6	$88.46\pm 4.16$	$25.47\pm 1.78$	$11.54\pm 4.16$	$1730.35\pm 159.76$	$54.50\pm 22.18$	$27.34\pm 2.76$	$45.50\pm 22.18$	$24.82\pm 3.45$
8	$88.10\pm 3.84$	$23.81\pm 1.86$	$11.90\pm 3.84$	$1625.50\pm 158.01$	$43.20\pm 18.49$	$31.93\pm 2.42$	$56.80\pm 18.49$	$29.32\pm 3.18$
10	$90.42\pm 2.91$	$20.96\pm 1.20$	$9.58\pm 2.91$	$1423.49\pm 99.74$	$48.40\pm 15.69$	$25.91\pm 1.81$	$51.60\pm 15.69$	$23.61\pm 2.18$
STEP-KD ( $3\times 3$ layers)	$\mathbf{92.15\pm 2.95}$	$\mathbf{19.78\pm 1.54}$	$\mathbf{7.85\pm 2.95}$	$\mathbf{1330.00\pm 125.66}$	$\mathbf{58.91\pm 17.81}$	$\mathbf{21.39\pm 1.97}$	$\mathbf{41.09\pm 17.81}$	$\mathbf{19.12\pm 2.24}$

Appendix

Table 7: Complete results for the projection-depth ablation of Direct KD from the PR teacher on the held-out test set (Figure 4 ). The default Direct KD (PR teacher) in Table 1 uses 3-layer projections. STEP-KD, whose PS model composes three 3-layer projections, is included for reference. Best in bold.

		TNS			WNS
Stage	Predictor	MedAE $\downarrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MedAE $\downarrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$
PR	STA	$0.00\pm 0.00$	$0.04\pm 0.00$	$0.00\pm 0.00$	$0.00\pm 0.00$	$0.39\pm 0.02$	$0.01\pm 0.00$
PR	Teacher †	$7.82\pm 0.76$	$4.11\pm 0.32$	$0.16\pm 0.04$	$1.30\pm 0.15$	$4.61\pm 0.20$	$0.56\pm 0.04$
PP	STA	$84.52\pm 7.03$	$61.25\pm 0.53$	$47.19\pm 0.67$	$18.89\pm 0.17$	$48.28\pm 0.45$	$34.23\pm 1.27$
	Independent (no KD)	$\mathbf{46.65\pm 2.91}$	$\mathbf{29.55\pm 1.17}$	$\mathbf{9.86\pm 0.84}$	$\mathbf{13.00\pm 0.77}$	$\mathbf{32.59\pm 0.85}$	$\mathbf{14.13\pm 0.89}$
	KD from PR	$91.96\pm 4.35$	$40.77\pm 1.32$	$16.27\pm 1.60$	$18.68\pm 0.65$	$44.54\pm 1.08$	$23.47\pm 1.08$

Appendix

Table 8: STA, teacher models and the effect of sequential knowledge distillation (OpenLLM-RTL benchmark). At each intermediate stage we compare STA, a model trained without distillation from later stages, and a model distilled along the design ladder. At PS both STEP-KD variants align to PF, PP and PR teachers one at a time and differ only in whether those teachers were themselves distilled; post-synthesis STA is reported in Table 1 . MedAE (median absolute error) is in the units of the slack labels. Best within each stage in bold. † The PR teacher receives post-routing slack as an input feature, so its near-perfect score is expected and shown only for completeness.

	TNS				WNS
KD granularity	R 2 (%) $\uparrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MAE $\downarrow$	R 2 (%) $\uparrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MAE $\downarrow$
No KD (reference)	$47.93\pm 9.63$	$79.37\pm 4.03$	$52.07\pm 9.63$	$420.68\pm 21.49$	$-58.13\pm 7.75$	$99.34\pm 3.75$	$158.13\pm 7.75$	$51.03\pm 1.91$
Graph only	$87.69\pm 1.72$	$37.60\pm 1.76$	$12.31\pm 1.72$	$199.62\pm 9.28$	$59.42\pm 1.88$	$51.50\pm 1.32$	$40.58\pm 1.88$	$26.47\pm 0.66$
Subgraph only	$72.15\pm 2.78$	$65.89\pm 1.94$	$27.85\pm 2.78$	$349.78\pm 9.78$	$42.28\pm 2.67$	$74.73\pm 1.53$	$57.72\pm 2.67$	$38.41\pm 0.73$
Node only	$87.43\pm 1.09$	$46.17\pm 1.77$	$12.57\pm 1.09$	$245.13\pm 9.72$	$37.91\pm 2.50$	$60.66\pm 1.30$	$62.09\pm 2.50$	$31.18\pm 0.69$
All (STEP-KD)	$\mathbf{93.51\pm 1.23}$	$\mathbf{27.43\pm 1.41}$	$\mathbf{6.49\pm 1.23}$	$\mathbf{145.60\pm 7.50}$	$\mathbf{83.48\pm 0.64}$	$\mathbf{36.55\pm 0.72}$	$\mathbf{16.52\pm 0.64}$	$\mathbf{18.78\pm 0.35}$

Appendix

Table 9: Complete results for the multi-granularity ablation on the OpenLLM-RTL benchmark (Figure 4 ). Each variant distills from the same PF, PP and PR teachers as STEP-KD but aligns only the listed granularity; All is the full STEP-KD. No KD is included for reference. MAE is in the units of the slack labels. Best in bold.

	TNS				WNS
Projection layers	R 2 (%) $\uparrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MAE $\downarrow$	R 2 (%) $\uparrow$	WMAPE (%) $\downarrow$	RSE (%) $\downarrow$	MAE $\downarrow$
4	$88.02\pm 2.49$	$34.08\pm 2.42$	$11.98\pm 2.49$	$180.95\pm 12.68$	$54.56\pm 2.49$	$53.96\pm 1.61$	$45.44\pm 2.49$	$27.72\pm 0.83$
6	$52.43\pm 5.18$	$51.66\pm 1.56$	$47.57\pm 5.18$	$274.15\pm 8.13$	$57.36\pm 1.42$	$53.70\pm 1.16$	$42.64\pm 1.42$	$27.61\pm 0.60$
8	$38.41\pm 8.33$	$60.24\pm 2.58$	$61.59\pm 8.33$	$320.05\pm 13.70$	$39.30\pm 3.16$	$67.33\pm 1.63$	$60.70\pm 3.16$	$34.62\pm 0.87$
10	$91.42\pm 0.52$	$32.64\pm 0.64$	$8.58\pm 0.52$	$173.11\pm 3.57$	$68.97\pm 1.03$	$41.95\pm 0.72$	$31.03\pm 1.03$	$21.56\pm 0.33$
STEP-KD ( $3\times 3$ layers)	$\mathbf{93.51\pm 1.23}$	$\mathbf{27.43\pm 1.41}$	$\mathbf{6.49\pm 1.23}$	$\mathbf{145.60\pm 7.50}$	$\mathbf{83.48\pm 0.64}$	$\mathbf{36.55\pm 0.72}$	$\mathbf{16.52\pm 0.64}$	$\mathbf{18.78\pm 0.35}$

Appendix

Table 10: Complete results for the projection-depth ablation of Direct KD from the PR teacher on the OpenLLM-RTL benchmark (Figure 4 ). The default Direct KD (PR teacher) in Table 1 uses 3-layer projections. STEP-KD, whose PS model composes three 3-layer projections, is included for reference. Best in bold.

Explore similar work

TimingLLM: A Two-Stage Retrieval-Augmented Framework for Pre-Synthesis Timing Prediction from Verilog
Apr 26, 2026Armin Abdollahi, Negin Ashrafi, Mehdi Kamal +1High-Level Synthesis Register Transfer Level
University of Southern California Los Angeles, CA, USA
SwiftCTS: Fast Cross-Design Prediction and Pareto Optimization of Clock Tree Metrics via Few-Shot Calibration
Jun 9, 2026Barsat Khadka, Kawsher Roxy, Md Rubel AhmedClocks Invariant Synthesis
The University of Southern Mississippi Hattiesburg, MS, USA · Intel Corporation Hillsboro, OR, USA · Louisiana Tech University Ruston, LA, USA
RTL-Sequencer: Towards Scalable RTL Timing Prediction with the Sequence-based Paradigm
Jul 17, 2026Ziyan Guo, Wenji Fang, Wenkai Li +3Sequence Modeling Register Transfer Level
Hong Kong University of Science and Technology (HKUST)

Apr 26, 2026, cs.ARJ/K move · Enter open · S save

TimingLLM: A Two-Stage Retrieval-Augmented Framework for Pre-Synthesis Timing Prediction from Verilog

Armin Abdollahi, Negin Ashrafi, Mehdi Kamal, Massoud Pedram

University of Southern California Los Angeles, CA, USA

High-Level Synthesis Register Transfer Level Large Language Models(Llms Synthesis

Open paper AskPDF arXiv

Early, tool-free prediction of post-synthesis timing remains a key obstacle to rapid RTL iteration. We introduce TimingLLM, a two-stage retrieval-augmented LLM pipeline that estimates worst negative slack (WNS) and total negative slack (TNS) directly from Verilog. Stage 1 is a fine-tuned LLM that acts as a compact post-synthesis timing oracle, producing path-level arrivals/required times that are summarized into lightweight structural-timing cues (e.g., bag-of-gates counts, critical-path depth, gate-type patterns). Stage 2 is an LLM-based regressor that predicts WNS/TNS and applies a learned diagonal steering vector at the last transformer block, computed from the k nearest timing-labeled modules in a disjoint retrieval bank. On VerilogEval, TimingLLM attains R_WNS = 0.91 (MAPE 12%) and R_TNS=0.97 (MAPE 16%) while running 1.3-1.6 times faster than prior methods. Training uses a new 60k-module Verilog corpus with synthesis reports, which we will release. After training once, TimingLLM can be adapted to new technology libraries and PVT corners by refitting only a small regression head on 1000 labeled modules per setting, consistently outperforming state-of-the-art baselines.

Abstract

Explore similar work

TimingLLM: A Two-Stage Retrieval-Augmented Framework for Pre-Synthesis Timing Prediction from Verilog

SwiftCTS: Fast Cross-Design Prediction and Pareto Optimization of Clock Tree Metrics via Few-Shot Calibration

RTL-Sequencer: Towards Scalable RTL Timing Prediction with the Sequence-based Paradigm