Despite the data-rich environment in which memory systems of modern computing platforms operate, many state-of-the-art architectural policies employed in the memory system rely on static, human-designed heuristics that fail to truly adapt to the workload and system behavior via principled learning methodologies. In this article, we propose a fundamentally different design approach: using lightweight and practical machine learning (ML) methods to enable adaptive, data-driven control throughout the memory hierarchy. We present three ML-guided architectural policies: (1) Pythia, a reinforcement learning-based data prefetcher for on-chip caches, (2) Hermes, a perceptron learning-based off-chip predictor for multi-level cache hierarchies, and (3) Sibyl, a reinforcement learning-based data placement policy for hybrid storage systems. Our evaluation shows that Pythia, Hermes, and Sibyl significantly outperform the best-prior human-designed policies, while incurring modest hardware overheads. Collectively, this article demonstrates that integrating adaptive learning into memory subsystems can lead to intelligent, self-optimizing architectures that unlock performance and efficiency gains beyond what is possible with traditional human-designed approaches.
Figures & tables
Figure 1: (a) Coverage, overprediction, and (b) performance comparison of two recently-proposed prefetchers: SPP and Bingo. (c) Percentage of loads that miss the LLC and go off-chip (on the left y-axis) and the LLC MPKI (on the right y-axis) in the baseline system with a state-of-the-art prefetcher. (d) Accuracy of two off-chip predictors, HMP and TTP. Average request latency of CDE and RNN-HSS on (e) performance-oriented and (f) cost-oriented HSS. The average request latency is normalized to Fast-Only policy. Figures adapted from our MICRO 2021 [ 27 ] , MICRO 2022 [ 26 ] , and ISCA 2022 [ 175 ] papers.
Figure 2: Overview of a reinforcement learning system.
Figure 3: Overview of a single-layer perceptron model.
Figure 4: (a) Formulating prefetcher as an RL agent. (b) Overview of Pythia. Figures adapted from our MICRO 2021 paper [ 27 ] .
Figure 5: Average performance improvement of prior prefetchers and Pythia in systems with varying (a) number of cores and (b) DRAM million transfers per second (MTPS). Figures adapted from our MICRO 2021 paper [ 27 ] .
Figure 6: (a) Overview of Hermes. (b) Hardware stages to make a prediction by POPET. Figures adapted from our MICRO 2022 paper [ 26 ] .
Figure 7: Performance comparison by varying (a) off-chip predictors and (b) the underlying prefetcher. Figures adapted from our MICRO 2022 paper [ 26 ] .
Figure 8: (a) Formulation of data placement as an RL Problem. (b) Design of Sibyl. Figures adapted from our ISCA 2022 paper [ 175 ] .
Feature
Description
# of bins
Encoding (bits)
sizet
Size of the requested page (in pages)
8
8
typet
Type of the current request (read/write)
2
4
intrt
Access interval of the requested page
64
8
cntt
Access count of the requested page
64
8
capt
Remaining capacity in the fast storage device
8
8
currt
Current placement of the requested page (fast/slow)
2
4
Table 1: State features used by Sibyl
Figure 9: Average request latency under two different hybrid storage configurations (normalized to Fast-Only). Figures adapted from our ISCA 2022 paper [ 175 ] .
Mar 18, 2026·Panuganti Chirag Sai, Gandholi Sarat, R. Raghunatha Sarma +2RL Control
Department of Mathematics and Computer Science, Sri Sathya Sai Institute of Higher Learning · Department of Computer Science and Engineering, Indian Institute of Technology Ropar · AI Performance Engineer, Red Hat