cs.LGOct 20, 2026
SaveReinforcement learning for hardware acceleration and system-level optimization on FPGA and integrated-circuit platforms: A survey
Organizations: LSEA Laboratory, Department of Electrical Engineering, University of Medea, 26000, Algeria
Abstract
Reinforcement learning (RL) enables adaptive and autonomous decision-making in embedded, edge, and heterogeneous computing systems. Field-programmable gate arrays (FPGAs) and integrated circuits (ICs) offer low latency, energy efficiency, and architectural flexibility for RL deployment, while introducing challenges in resource constraints, implementation complexity, and hardware-algorithm co-design. This review surveys more than 100 studies on RL implementations across FPGA and IC platforms, emphasizing architectures, implementation strategies, performance evaluation, and applications. The literature is organized by RL family, including model-based Markov decision processes, classical value-based methods, deep RL, actor-critic, policy-gradient, and multi-agent approaches. We further classify studies by abstraction level, hardware realization strategy, and application domain, examining how RL algorithms map onto reconfigurable and fixed hardware. Unlike surveys restricted to particular algorithms or platforms, this work provides a unified analysis of FPGA and IC implementations, covering acceleration, reconfigurable architectures, energy-aware control, and system-level optimization. Comparative assessments consider resource utilization, latency, throughput, power consumption, and reported performance gains. Finally, we identify recurring architectural patterns, practical limitations, open challenges, and research directions for efficient RL-driven hardware systems. The survey offers a consolidated reference for researchers developing and evaluating RL solutions on specialized hardware.
Figures & tables
| Ref. | Paper | NoPs | Related | IC | FPGA | BGD | RL types | Research | RL | Case | Limitations | ||||||||
| type | RL | work | MDP | QL | SARSA | DQN | DDPG | AC | MC | PPO | areas | Apps. | study | Perspectives | |||||
| [ 118 ] | Survey | 14 | ○ | ○ | ● | ◐ | ○ | ● | ○ | ● | ● | ○ | ○ | ● | ○ | ○ | ○ | ◐ | |
| [ 44 ] | Survey | 9 | ● | ○ | ● | ● | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ◐ | ○ | ◐ | |
| [ 162 ] | Review | 1 | ○ | ◐ | ● | ◐ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ◐ | ○ | ◐ | |
| [ 99 ] | Review | 1 | ○ | ● | ● | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ○ | ◐ | |
| Our | Review | 120 | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | |
Table 1: Comparison of the proposed review with existing FPGA-related surveys and reviews. A mark (●) signifies that a specific area has been covered, whereas a mark (○) denotes that it has not been addressed. The symbol (◐) indicates that key aspects of the field remain largely unaddressed.
Figure 1: Survey roadmap outlining the organization of RL -based FPGA and IC platforms.
Figure 2: Overview of two hardware architecture representations that could be employed for RL workloads. (a) A simplified FPGA floorplan showing the distribution of on-chip resources—CLB logic tiles, BRAM columns, global routing spines, configuration/control logic, global buffers, clock management blocks, and I/O banks—with a CLB slice highlighting LUT, FF, LUTRAM, and carry logic elements. (b) A conceptual IC / SoC architecture illustrating functional subsystems relevant to RL execution, including the CPU core, hardware accelerator, on-chip memory, interconnect (Bus/NoC), power management, clock management, and I/O interfaces and pads.
| Design Level | FPGA Workflow | IC / ASIC Workflow |
| SLD | Algorithm modeling, dataflow simulation, partitioning (MATLAB/Simulink, SystemC, Python, PYNQ) | Same high-level modeling and system exploration (MATLAB/Simulink, SystemC, C/C++) |
| DSC | RL graph-level optimization, operator fusion, quantization, memory tiling, dataflow scheduling, accelerator mapping; possible generation of HLS / RTL or accelerator intermediate representation | Similar graph/dataflow-level compilation targeting ASIC / SoC accelerators; supports early exploration of data reuse, memory hierarchy, and hardware mapping |
| HLS | C/C++/SystemC RTL (Vitis HLS, Intel HLS, CatapultC); directives for unrolling, pipelining, memory partitioning | C/C++/SystemC RTL (Cadence Stratus, Mentor Catapult, Synopsys HLS); early power/performance design-space exploration |
| RTL | Verilog/VHDL/SystemVerilog RTL for datapaths, FSM ( FSM )s, controllers; synthesizable design for LUT , FFs, DSP s, BRAMs | Verilog/SystemVerilog RTL for logic synthesis, optimization, timing-driven transformations |
| Gate-Level | FPGA-specific placement, routing, timing closure (Vivado, Quartus); bitstream generation for predefined fabric | ASIC physical design including floorplanning, placement, routing, CTS, STA, IR drop analysis, DRC/LVS; GDSII tape-out |
Table 2: Unified abstraction levels for FPGA and IC design flows
| Term | Meaning |
| Current action at time | |
| Present state at time | |
| Next state observed after taking in state | |
| Next action taken in state following the policy being used. | |
| The discount factor, which scales the importance of future rewards. ( ) | |
| The learning rate, determining how much the new information should influence the existing Q-value. |
Table 3: Notation
| Model | Problem scenario suitable for | Advantages | Limitations | Typical applications | Real-time capability |
| MDP | Model-based control with known or identifiable dynamics; small/medium discrete | Interpretable, analyzable; formal guarantees on optimal policy and safety | Requires accurate transition model; poor scalability, mainly offline planning | Test-configuration generation, interconnect coverage, coarse DVFS or power-state selection | Low–moderate (offline planning, light online lookup) |
| QL | Discrete, low-dimensional state/action spaces; no explicit model available | Simple tabular updates; easy to implement in fixed-point hardware | Explodes with large ; slow exploration and convergence | Mode selection for DVFS , cache / memory policy, reconfiguration of small regions | Moderate (for compact tables in BRAM) |
| SARSA | Streaming control tasks requiring smooth policy updates; tasks sensitive to abrupt action changes | Stable incremental learning; updates follow executed behavior; well-suited for hardware-friendly TD ( TD ) updates | Still tabular with limited scalability; performance depends strongly on exploration strategy | Thermal/power throttling, adaptive subsystem tuning, sequential reconfiguration tasks | Moderate (lightweight updates suitable for real-time streams) |
| DQN | High-dimensional state, discrete actions; rich sensor/monitor signals on FPGA | Handles large with deep nets; good for complex policies | Training cost, replay memory, stability and quantization issues on-chip | Placement / routing guidance, task scheduling, NoC routing, adaptive accelerator tuning | High for inference (training mostly off-chip) |
| DDQN | Same as DQN where value overestimation hurts reliability | Reduces Q-overestimation; more stable value estimates | Still heavy training; needs two networks and memory resources | Latency–power trade-off control, adaptive mapping of kernels, thermal-aware control | High for inference (training offline, batched) |
| AC | Mixed continuous/discrete control; need both policy and value estimates | Supports continuous actions; good sample efficiency vs. pure policy gradient | Coupled AC training; sensitive to hyperparameters | Fine-grained DVFS , analog front-end tuning, throughput–latency control | Moderate–high (inference light, training heavier) |
Table 4: Refined comparison of RL - based FPGA algorithms covering key applications
| Ref | RL | Hardware | Resource consumption (%) | N | A | CLK | PW | IL | Best result | Limitations | ||||
| Used | BRAM | LUTS | LUTRAM | FF | DSP | |||||||||
| [ 82 ] | MDP | Zynq-7000 | – | 15.35 | – | 1.85 | – | 8 | – | 100 | – | HLS | 7.8 acceleration | HLS MDP generator; limited scope; no BRAM/DSP data |
| [ 89 ] | QL | Zynq UltraScale+ MPSoC | – | 0.056 | 0.010 | 0.012 | 0.231 | 8 | 4 | 240 | 0.038 | RTL | Usage: -50% in resource | Limited to small action spaces, offline training, no large-scale benchmarks. |
| [ 132 ] | QL | Zynq UltraScale+ MPSoC | – | 0.08 | 0.03 | 0.03 | 0 | 8 | 4 | 222 | 0.037 | RTL | – | Uses small Q-tables, limited exploration strategies, simplified multiplier logic, and no dynamic environment testing. |
| [ 135 ] | QL | PYNQ-Z1 | 24.29 | 9.77 | 1.90 | 6.34 | 0 | 32 | 4 | 100 | 1.75 | RTL | 1400 speedup | Limited state modeling, no multi-intersection support, and high BRAM usage for large environments. |
Table 5: Overview of representative RL - based FPGA / IC proposed schemes, highlighting hardware platforms, resource consumption, implementation level, performance results, and limitations.
Figure 3: High-level architecture of the QL agent mapped to FPGA / IC hardware. The QL accelerator executes the tabular update rule using DSP -based MAC units and fixed-point arithmetic, with Q-values stored in BRAM/URAM. The learning rate governs the magnitude of the parameter update, while the discount factor scales the contribution of future reward terms. The PG , implemented with LUT logic and a lightweight pseudo-random module, produces the next action to support stochastic exploration. Temporal elements act as register-based pipeline delays to synchronize , , and in hardware. The architecture highlights a practical mapping of QL operators to FPGA resources ( DSP , LUT , BRAM/URAM) without enforcing a device-specific implementation [ 132 ] .
Figure 4: High-level architecture of a DQN agent on FPGA / IC . The Main Q-network is implemented using DSP arrays and fixed-point MAC units, with parameters stored in BRAM/URAM. Experience replay is mapped to BRAM/URAM or external DDR depending on capacity. The Target Q-network is maintained as a BRAM mirror and updated via a soft-update unit ( DSP + FSM ). The loss is computed using the TD target .
Figure 5: High-level functional architecture of a DDPG agent. The Main Actor generates continuous control actions, with exploration noise injected during interaction with the environment. The Main Critic evaluates state–action pairs and computes the TD target using the Target Actor and Target Critic, whose parameters are updated through soft (Polyak) averaging. A replay buffer enables off-policy learning by storing and supplying past transitions for both actor and critic updates. State–action concatenation indicates where state and action vectors are combined prior to critic evaluation. Solid lines denote forward dataflow paths used during online interaction and gradient-based learning, whereas dashed lines indicate auxiliary or delayed pathways used exclusively for target value computation and parameter synchronization. Target networks are passive, auxiliary models not used for online control, and are invoked only during TD-target evaluation and soft updates. Learning hyperparameters (e.g., , , ) are applied during loss computation and update rules and are not explicit network inputs.
| Metric | Expression | Interpretation | Preferred trend |
| Throughput-based energy efficiency | Measures the amount of useful RL computation achieved per watt. The throughput may represent inferences/s, updates/s, samples/s, or full iterations/s, depending on the reported workload. | Higher | |
| FIPS/W | Measures full RL iterations processed per watt. This metric is particularly suitable for accelerators that execute both decision-making and learning-update stages. | Higher | |
| Energy per operation | or | Estimates the energy consumed by one inference, update, sample, or full RL iteration. It is useful when comparing low-power and real-time accelerators. | Lower |
| Average resource footprint | Represents the mean utilization of the reported FPGA resources, where may include BRAM , LUT , LUTRAM , FF , and DSP usage. | Lower | |
| Bottleneck resource utilization | Identifies the most heavily used hardware resource. This metric helps reveal scalability limits even when the average resource usage appears moderate. | Lower | |
| Resource efficiency | Relates useful RL throughput to average resource utilization. It helps compare designs that achieve similar performance with different hardware costs. | Higher |
Table 6: Comprehensive normalized metrics for cross-design comparison of RL -based FPGA / IC implementations.
| Research Area | Primary Focus | Typical Objectives | Key Challenges | |
| RL as the Workload | RL-based acceleration | Mapping RL computation (value updates, NN inference, gradients) onto FPGA pipelines | Reduce latency; increase throughput; exploit parallelism; support on-device training | Resource constraints; precision tuning; balancing compute pipelines |
| Multi-Agent RL | FPGA -based architectures supporting distributed or local MARL agents | Scalable learning; cooperative control; reduced centralized bottlenecks | Non-stationarity; coordination overhead; gradient synchronization | |
| Multi-FPGA RL platforms | Distributed RL execution across multiple interconnected FPGA s | Large-scale RL; higher throughput; extended memory capacity | Communication latency; consistency of updates; partitioning strategies | |
| Energy-Efficient RL Acceleration | Energy-efficient RL execution on FPGA using neuromorphic, mixed-precision, and hierarchical designs | Reduce energy consumption of RL learning/inference; improve GOP/s/W; enable low-power on-device adaptation | Task specificity; limited online learning flexibility; memory and precision trade-offs | |
| HW–SW co-exploration | Joint exploration of algorithmic and hardware design parameters using RL -based search | Automated design-space exploration; precision selection; mapping and scheduling optimization | Large search spaces; long exploration time; generalization to unseen workloads | |
| RL as the Optimizer | Reconfig. RL approaches | Runtime reconfiguration controlled by RL agents (DPR, adaptive tuning) | Adaptive performance; robustness under workload/environment variations; real-time control | Reconfiguration cost; policy stability; ensuring safe reconfiguration |
Table 7: Overview of the two principal roles of RL in FPGA / IC research and their associated sub-domains, highlighting the primary focus, typical objectives, and key challenges of each area.
Figure 6: Unified RL -assisted synthesis workflow covering both HLS and logic-synthesis stages. RL agents interact with several points in the design flow: Task (1) optimizing HLS transformations; Task (2) incorporating feedback from specifications and HLS reports; Task (3) guiding logic optimization; and Task (4) assisting FPGA and ASIC technology mapping. This illustrates how RL provides an adaptive optimization layer across the full IC/ FPGA synthesis pipeline.
Figure 7: Application domains of RL - based FPGA and IC platforms.
| App. domain | Typical RL algo. | Hardware implementation strategy | Main objective | Key constraints |
| Networking | QL, DQN, dueling DQN, TD3, MARL | FPGA-accelerated routing, CNN/RL inference, distributed Q-tables, and pipelined decision engines | Low-latency routing, spectrum access, waveform adaptation, and traffic control | Dynamic traffic, scalability, communication overhead, and real-time response |
| Hardware and embedded security | QL, DQN, PPO, AC, RL-RNN, HDC-based RL | RTL-level verification engines, FPGA-based detection pipelines, HDC/RL accelerators, and adaptive test engines | Vulnerability detection, intrusion detection, side-channel analysis, and security verification | Large state spaces, adversarial behavior, verification complexity, and limited generalization |
| Robotics | QL, DQN, AC, DDPG | Real-time FPGA control pipelines, fixed-point policy inference, and hardware-in-the-loop control | Fast feedback control, adaptive behavior, and RL training acceleration | Strict timing, limited resources, control stability, and sensor uncertainty |
| Autonomous vehicles | PPO, AC, DQN, MARL | CPU–GPU–FPGA task placement, FPGA-accelerated policy inference, and hardware-in-the-loop validation | Self-driving control, UAV tracking, trajectory optimization, and energy-aware operation | Safety, latency, energy consumption, and multi-objective optimization |
| Intelligent sensing and imaging | Deep RL, PDS-RL, SAC, PG methods | Lightweight accelerators, stochastic computing, FPGA–CPU hybrid control, and adaptive sensing pipelines | Feature selection, signal reconstruction, adaptive acquisition, and sensing optimization | Sensor noise, low-power operation, real-time imaging, and data uncertainty |
| Other applications | QL, DQN, model-free RL | FPGA-based inference/control cores, edge-training pipelines, and safety-filtered decision engines | Motor torque control, DNA sequence alignment, power converter control, and embedded optimization | Safety constraints, domain specificity, limited benchmarks, and deployment reliability |
Table 8: Cross-domain comparison of RL algorithm selection and FPGA/IC implementation strategies.
| Apps. | Ref. | FPGA used | RL | Dedicated task and limitations | Compared to | Result | Improvement |
| [ 24 ] | Nexys A7 / PYNQ-Z2 | QL | Proposes a QL-based fault- and congestion-aware NoC routing framework with a compact Q-table; however, it relies on offline training, synthetic evaluations, and lacks formal deadlock guarantees. | HARAQ, RELAR | 0.95 delivery rate at 20% faults | – | |
| [ 76 ] | PXIe-7975 | REINFORCE | Employs RL to develop a flexible duplex system, targeting maximization of area spectral efficiency. However, the system’s reliance on the OP-map for transmission decisions limits it in dynamic environments. | Full duplex OP-map | Better spectral efficiency | – | |
| Networking | [ 83 ] | Zynq Z7020 | Model-free RL | The work targeted accelerated RL - based routing in delay-tolerant networks using FPGA, but faced scalability constraints, and communication overhead. | No RL (CPU) | Response Time 2.5s Throughput 41 bundles/s | 32 faster |
| [ 32 ] | Xilinx xc7z045ffg900-2 | TD3 | Real-time waveform synthesis improves resilient wireless signal classification, but faces complexity, FPGA usage, training instability, and sensitivity to channel noise. | No-FIR DeepRadioID | Acc. (High SNR) =100% | 4.1 (No-FIR) 1.9 (DeepRadioID) | |
| [ 75 ] | PYNQ-Z2 | Dueling- DQN | Enabled real-time Dueling- DQN routing on LEO satellites using FPGA-accelerated CNNs. But faced higher design complexity, resource limits, and scalability constraints. | CPU - only | Inf. time =0.3 s | 3.10 faster | |
| [ 60 ] | Kintex-7 FPGA KC705 | QL-MARL | It learned adaptive fault-tolerant routing paths in mesh NoCs, but required per-router Q-tables and added hardware overhead with higher resource usage. | QL–based routing | Packet delivery +7.1% | 2.7% fewer hardware resources and 0.89% less power |
Table 9: Summary of well-investigated FPGA-based RL applications, including FPGA boards used, dedicated tasks, limitations, and performance improvements over baseline methods.
Figure 8: Operational flow of AdaTest, an RL -based hardware Trojan ( HT ) detection framework [ 19 ] . Adaptive test patterns generated by the agent are applied to the CUT , and the resulting responses are compared with expected outputs: mismatches indicate a Trojaned circuit, while matching responses classify the design as Benign .
Figure 9: RL - based Internet of vehicles model. The vehicle primarily consisting of a CPU, GPU, and FPGA placement model, is trained on a cloud server with the objective of reducing both execution latency and energy consumption using RL [ 164 ] .
Figure 10: RL-based FPGA EV charging management system: (a) overall FPGA architecture; (b) RL engine; (c) policy engine with LFSR-based exploration; (d) state discretizer; (e) action generator; (f) reward generator [ 27 ] .