cs.ARFeb 25, 2026
SaveArchitectural Design and Performance Analysis of FPGA based AI Accelerators: A Comprehensive Review
Organizations: Indian Institute of Engineering Science and Technology, Shibpur, India
Abstract
Deep learning (DL) has emerged as a rapidly developing advanced technology, enabling the performance of complex tasks involving image recognition, natural language processing, and autonomous decision-making with high levels of accuracy. However, as these technologies evolve and strive to meet the growing demands of real-life applications, the complexity of DL models continues to increase. These models require processing of massive volumes of data, demanding substantial computational power and memory bandwidth. This gives rise to the critical need for hardware accelerators that can deliver both high performance and energy efficiency. Accelerator types include ASIC based solutions, GPU accelerators, and FPGA based implementations. The limitations of ASIC and GPU accelerators have led to FPGAs becoming one of the prominent solutions, offering distinct advantages for DL workloads. FPGAs provide a flexible and reconfigurable platform, allowing model specific customization while maintaining high efficiency. This article explores various hardware level optimizations for DL. These optimizations include techniques such as loop pipelining, parallelism, quantization, and various memory hierarchy enhancements. In addition, it provides an overview of state-of-the-art FPGA-based neural network accelerators. Through the study and analysis of these accelerators, several challenges have been identified, paving the way for future optimizations and innovations in the design of FPGA-based hardware accelerators.
Figures & tables
| Accelerator Type | Architecture | Operation | Merits | Demerits |
|---|---|---|---|---|
| GPU ( Kahng et al., 2024 ) , ( Xie et al., 2011 ) , ( Li et al., 2016 ) , ( Xie et al., 2023 ) | Massively parallel SIMD/SIMT architecture with thousands of cores; optimized for throughput-based computing. | Executes large numbers of parallel threads; ideal for matrix multiplications and convolution operations. | • Mature and well-supported software ecosystem (CUDA, ROCm) • High computational throughput • Flexible for both training and inference • Widely available and cost-effective | • High power consumption • Lower energy efficiency compared to ASICs • Limited memory bandwidth for certain workloads • General-purpose, not fully optimized for specific networks |
| ASIC (NPUs, TPUs) ( Chen et al., 2014a ) , ( Chen et al., 2014b ) , ( Liu et al., 2015 ) , ( Wang et al., 2025 ) , ( Yuan et al., 2019 ) | Custom dataflow architectures optimized for neural network operations; fixed-function hardware tailored for AI kernels. | Executes highly optimized tensor and matrix operations using specialized systolic arrays or dataflow engines. | • Highest performance and energy efficiency • Predictable latency • Optimized for specific DNN workloads • Suitable for large-scale datacenter deployments | • Very high development cost and long design cycle • Inflexible—cannot be reconfigured after fabrication • Limited adaptability to evolving AI models |
| FPGA ( Geng et al., 2020 ) , ( Chang and Culurciello, 2017 ) , ( AbdElbaky et al., 2025 ) , ( Chen et al., 2024 ) | Reconfigurable logic fabric with LUTs, DSP blocks, BRAM, and interconnects; adaptable hardware architecture. | Implements custom data paths and operators through reconfiguration; supports pipelined and parallel execution. | • Highly flexible and reprogrammable • Low latency and energy-efficient for edge workloads • Customizable precision (e.g., INT8, binary networks) • Suitable for prototyping custom accelerators | • Lower peak performance than GPUs/ASICs • Programming requires hardware expertise (HDL, HLS) • Long compilation and bitstream generation times • Limited on-chip memory |
Table 1. Comparison of GPU, ASIC, and FPGA-based AI Accelerators
Figure 1. Performance Comparison Metrics Across CPU, GPU, ASIC and FPGA. Motivation chart.
Figure 2. Key features, existing optimization techniques, limitations & need for further optimization of FPGA-based hardware accelerators. Motivation chart.
Figure 3. Architecture of AccUDNN showing the process flow between the memory optimizer and hyperparameter tuner modules. ( Guo et al., 2019 ) diagram for hardware accelerator
Figure 4. Architecture of an analog in-memory computing AIMC core with integrated crossbar arrays & memory-based unit cells. ( Le Gallo et al., 2023 ) AIMC Core Architectural Diagram
Figure 5. Architecture of TPU v1 showing dataflow between the systolic array, unified buffer & high-bandwidth memory interface. ( Jouppi et al., 2018 ) TPU Architectural Diagram
Figure 6. Complete Architecture of SoC based Accelerator depicting the sub-modules- PS, PL & and shared DDR memory. DrawIO Diagram for FPGA Architecture
Figure 7. Overview of CNN, SNN, RNN, and GNN with principles, key advantages & disadvantages. Architectural Details of Various Neural Networks
Figure 8. Block Diagram for CNN Architectures
Figure 9. Block Diagram for SNN Accelerator Architectures SNN_2_diagram
Figure 10. Block Diagram for RNN Accelerator Architectures
Figure 11. Block diagram for GNN accelerator architectures.
Figure 12. Summary of the different types of Hardware-Level Optimizations- computation, memory and multi-level optimizations. Chart for Optimizations
Figure 13. Design flow for a HLS-based dynamic memory optimized framework with offline pattern analysis & Monte-Carlo-guided heap optimization. ( Kokkinis et al., 2022 ) Diagram for memory optimization technique
Figure 14. Summary of optimization methodologies and roles of acceleration of various neural models A conceptual diagram summarizing FPGA optimization methodologies.
| Model | Methodology | Platform | Throughput | Precision | Frequency | Resources (Utilization %) | Speedup | Baseline | |||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CNP (2009) ( Farabet et al., 2009 ) | LeNex-5 | Virtex-4 SX35 | 5.25 GOPS | fixed 16 bit | 200 MHz | 90.00% | 28.00% | – | 90.00% | – | – |
| MAPLE (2010) ( Cadambi et al., 2010 ) | 4 CONV layers | Virtex-5 SX240T | 7 GOPS | – | 125 MHz | – | – | 92.00% | – | 0.5 | C870 GPU |
| DLA (2017) ( Aydonat et al., 2017 ) | AlexNet | Arria 10 GX 1150 | 1,382 GOPS | fixed 16 bit | 214 MHz | 258.00% | 97.00% | 92.00% | 40.00% | – | – |
| OpenCL CNN Accelerator (2017) ( Zhang and Li, 2017 ) | VGG-16 | Arria 10 GX 1150 | 1790 GOPS | floating 16 bit | 385 MHz | – | 87.00% | 53.40% | – | – | – |
| OpenCL CNN Accelerator (2017) ( Zhang and Li, 2017 ) | VGG-16 | Arria 10 GX 1150 | 866 GOPS | floating 32 bit | 370 MHz | – | 90.80% | 46.10% | – | – | – |
| Automatic RTL Generator (2017) ( Ma et al., 2017 ) | NiN | Stratix-V GXA7 | 282.67 GOPS | fixed 16 bit | 150 MHz | 96.00% | 100.00% | 59.00% | – | – | – |
Table 2. Performance Summary of FPGA-based CNN Accelerators
| Model | Methodology | Platform | Throughput | Precision | Frequency | Resources (Utilization %) | Speedup | Baseline | |||
|---|---|---|---|---|---|---|---|---|---|---|---|
| LUT | DSP | BRAM | FF | ||||||||
| SyncNN (2021) ( Panchapakesan et al., 2021 ) | SCNN | Xilinx ZCU102 | N/A | fixed 4 bit | 200 MHz | 37.45% | N/A | N/A | N/A | N/A | N/A |
| BP-STDP SNN Accelerator (2022) ( Zhang et al., 2022 ) | SFNN | Xilinx XCVU9P | 5980 | floating 16 bit | 100 MHz | 15.64% | 0.00% | N/A | 12.77% | 49.42 | CPU with 3.2GHz CPU and 16GB memory |
| Cerebron (2022) ( Chen et al., 2022b ) | LIF | Xilinx XC7Z100 | 43.07 GSOP/s | N/A | 200 MHz | 30.97% | 0.00% | 37.48% | 12.71% | N/A | N/A |
| Spiker (2022) ( Carpegna et al., 2022 ) | SFNN | Artix 7 | N/A | fixed 16 bit | 100 MHz | 55.00% | N/A | 32.00% | 25.00% | N/A | N/A |
| Skydiver (2022) ( Chen et al., 2022a ) | SCNN | Xilinx XA7Z020 | 22,600 FPS | N/A | 200 MHz | 21.04% | 0.00% | 48.07% | 4.70% | N/A | N/A |
Table 3. Performance Summary of FPGA-based SNN Accelerators
| Model | Methodology | Platform | Throughput | Precision | Frequency | Resources (Utilization %) | Speedup | Baseline | |||
|---|---|---|---|---|---|---|---|---|---|---|---|
| LUT | DSP | BRAM | FF | ||||||||
| ESE ( Han et al., 2017 ) (2017) | Std. LSTM | Xilinx XCKU060 | 282 GOPS | integer 16 bit | 200 MHz | 88.60% | 54.50% | 87.70% | 68.30% | 43.00 | Corei 75930k CPU |
| DeepRNN (2017) ( Chang and Culurciello, 2017 ) | 2 LSTM Layers | Xilinx ZC706 | 400.98 Mops/s/W | integer 16 bit | 142 MHz | – | – | – | – | 45.67 | Exynos 54224 Cortex-A15 |
| LSTM-RNN (2017) ( Guan et al., 2017 ) | Std. LSTM | Xilinx VC707 | 7.26 GFLOP/s | floating 32 bit | 150 MHz | 65.31% | 42.00% | 52.04% | 30.08% | 20.18 | Intel Xeon CPU E5-2430 |
| FINN-L (2018) ( Rybalkin et al., 2018 ) | 2 layer Std. LSTM | Xilinx ZCU104 | 746@8bit | 8 bit | 266 MHz | 26.08% | – | 28.84% | – | – | – |
| IndRNN Accelerator (2018) ( Gao and Zhang, 2018 ) | 2 layers IndRNN | Xilinx ZC706 | 3.15 GOPS/s/W | fixed 16 bit | 125 MHz | 2.54% | 3.44% | 10.27% | 2.89% | Intel CORE i5 CPU | – |
Table 4. Performance Summary of FPGA-based RNN Accelerators
| Model | Methodology | Platform | Throughput | Precision | Frequency | Resources (Utilization %) | Speedup | Baseline | |||
|---|---|---|---|---|---|---|---|---|---|---|---|
| LUT | DSP | BRAM | FF | ||||||||
| AWB-GCN (2020) ( Geng et al., 2020 ) | GCN | Stratix 10SX | 873.1 | floating 32 bit | 330 MHz | 25% | 71.11% | N/A | 13.7% | 3255 | Intel Xeon E5-2680 |
| DeepBurning-GL (2020) ( Liang et al., 2020 ) | GCN, GS-Pool, R-GCN, EdgeConv | Xilinx ZC706 | 8.6 GOPS/W | fixed 16 bit | 100 MHz | 52.72% | 85.57% | 76.21% | 18.36% | 7.43 | Intel Xeon E5-2680 |
| DeepBurning-GL ( Liang et al., 2020 ) (2020) | GCN, GS-Pool, R-GCN, EdgeConv | Xilinx Alveo U50 | 53.5 GOPS/W | fixed 16 bit | 200 MHz | N/A | 93.57% | 45.14% | 32.93% | 346.98 | Intel Xeon (Skylake) 6151 |
| BoostGCN (2021) ( Zhang et al., 2021 ) | GCN | Stratix 10SX | 512 or 1280 | floating 32 bit | 250 MHz | 10.5% | 33.33% | N/A | N/A | 45 | Intel Xeon Gold 5120 CPU |
| FP-GNN (2022) ( Tian et al., 2022 ) | GCN, Graphsage, GAT | Xilinx VCU128 | 922 | floating 32 bit | 225 MHz | 25.16% | 90.78% | 88.88% | 19.83% | 665.33 | Intel Xeon Platinum 8260 |
Table 5. Performance Summary of FPGA-based GNN Accelerators
More in this topic
FPGA Acceleration
FPGA acceleration uses reconfigurable logic to speed compute-intensive workloads with hardware tailored to their needs.
See all 32 papers