cs.ARMar 11, 2025

ResBench: Benchmarking LLM-Generated FPGA Designs with Resource Awareness

Authors: Ce Guo, Tong Zhao

Organizations: Imperial College London United Kingdom

Abstract

Field-Programmable Gate Arrays (FPGAs) are widely used in modern hardware design, yet writing Hardware Description Language (HDL) code for FPGA implementation remains a complex and time-consuming task. Large Language Models (LLMs) have emerged as a promising tool for HDL generation, but existing benchmarks for LLM-based code generation primarily focus on functional correctness while overlooking hardware resource usage. Furthermore, current benchmarks offer limited diversity and do not fully represent the wide range of real-world FPGA applications. To address these shortcomings, we introduce ResBench, the first resource-focused benchmark explicitly designed to distinguish between resource-optimized and inefficient LLM-generated HDL code. ResBench consists of 56 problems across 12 categories, covering applications from finite state machines to financial computing. Our open-source evaluation framework automatically tests LLMs by generating Verilog code, verifying correctness, and measuring resource usage. The experiments, which primarily analyze Lookup Table (LUT) usage, reveal significant differences among LLMs, demonstrating ResBench's capability to identify models that generate more resource-optimized FPGA designs.

Figures & tables

Explore similar work

CardsList
  1. VHDL-REPOBENCH: A Repository-Level Benchmark for Evaluating Large Language Models on VHDL Design Generation

    Oct 4, 2026Prashanth Vijayaraghavan, Akul Malhotra, Ashutosh Jadhav +2LLM EvaluationCode Generation

  2. Benchmarking LLMs for Verilog Design Flows

    Jul 23, 2026Angshuman Chakravertty, Rahul Koshti, Buddhi Prakash Sharma +1Code GenerationCode Generation Evaluation

  3. FinHardBench: Can LLMs Generate Latency-Aware Hardware for Financial Computing?

    Aug 2, 2026Weimin Fu, Hejia Zhang, Minghao Shao +6Large Language Model-Guided OptimizationElectronic Design Automation