cs.ARMar 8, 2026

Mitigating the Memory Bottleneck with Machine Learning-Driven and Data-Aware Microarchitectural Techniques

Authors: Rahul Bera

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Modern applications process massive data volumes that overwhelm the storage and retrieval capabilities of memory systems, making memory the primary performance and energy-efficiency bottleneck of computing systems. Although many microarchitectural techniques attempt to hide or tolerate long memory access latency, rapidly growing data footprints continue to outpace technology scaling, requiring more effective solutions. This dissertation shows that modern processors observe large amounts of application and system data during execution, yet many microarchitectural mechanisms make decisions largely independent of this information. Through four case studies, we demonstrate that such data-agnostic design leads to substantial missed opportunities for improving performance and energy efficiency. To address this limitation, this dissertation advocates shifting microarchitecture design from data-agnostic to data-informed. We propose mechanisms that (1) learn policies from observed execution behavior (data-driven design) and (2) exploit semantic characteristics of application data (data-aware design). We apply lightweight machine learning techniques and previously underexplored data characteristics across four processor components: a reinforcement learning-based hardware data prefetcher that learns memory access patterns online; a perceptron predictor that identifies memory requests likely to access off-chip memory; a reinforcement learning mechanism that coordinates data prefetching and off-chip prediction; and a mechanism that exploits repeatability in memory addresses and loaded values to eliminate predictable load instructions. Our extensive evaluation shows that the proposed techniques significantly improve performance and energy efficiency compared to prior state-of-the-art approaches.

Figures & tables

Figure 1.1 : Performance comparison of three contemporary prefetchers - stride [ 302 ] , SPP [ 465 ] , and Bingo [ 120 ] - across diverse applications. No prefetcher provides the highest performance across all applications.
Figure 1.2 : Fraction of dynamic loads that fetch the same data value from the same load address throughout the entire execution of an application (on the primary y-axis), and the performance improvement potential by ideally value-predicting and eliminating the execution of such loads (on the secondary y-axis).
Figure 3.1 : Interaction between an agent and the environment in a reinforcement learning system.
Figure 3.2 : Overview of a single-layer perceptron model. Each blue circle denotes an input and the green circle denotes the output of the perceptron.
 
 Workload type# traces
 
Single-coreAI/ML applications (AIML)81
Graph Mining Suite (GMS)43
Google datacenter workloads (Google)359
 Total483
Table 4.1: DPC4 workload traces used for evaluation.
Figure 5.1 : Comparison of (a) coverage, overprediction, and (b) performance of two recently-proposed prefetchers, SPP [ 465 ] and Bingo [ 120 ] , and our new proposal, Pythia.
Figure 5.2 : Formulating the prefetcher as an RL-agent.
FeatureControl-flowData-flow
 Info.HistoryInfo.History
Last 3-PCsPClast 3✖✖
Last 4-deltas✖✖Cacheline deltalast 4
PC+DeltaPCcurrentCacheline deltacurrent
Last 4-PCs+Page no.PClast 4Page no.current
Table 5.1: Example program features.
Figure 5.3 : Overview of Pythia.
Algorithm 1 Pythia’s reinforcement learning based prefetching algorithm
Figure 5.4 : (a) The QVStore is comprised of multiple vaults. (b) Each vault is comprised of multiple planes. (c) Index generation from feature value.
Figure 5.5 : Pipelined organization of QVStore search operation. The illustration depicts three program features, each having three planes.
 
FeaturesPC+Delta , Sequence of last-4 deltas
Prefetch Action List{-6,-3,-1,0,1,3,4,5,10,11,12,16,22,23,30,32}
Reward Level ValuesRAT\mathcal{R}_{AT} = 2020 , RAL\mathcal{R}_{AL} = 1212 , RCL\mathcal{R}_{CL} = −12-12 , RINH\mathcal{R}_{IN}^{H} = −14-14 , RINL\mathcal{R}_{IN}^{L} = −8-8 , RNPH\mathcal{R}_{NP}^{H} = −2-2 , RNPL\mathcal{R}_{NP}^{L} = −4-4
Hyperparametersα=0.0065\alpha=0.0065 , γ=0.556\gamma=0.556 , ϵ=0.002\epsilon=0.002
 
Table 5.2: Basic Pythia configuration derived from our automated design-space exploration.
 
Control-flow ComponentsData-flow Components
1. PC of load request 2. PC-path (XOR-ed last-3 PCs) 3. PC XOR-ed branch-PC 4. None1. Load cacheline address 2. Page number 3. Page offset 4. Load address delta 5. Sequence of last-4 offsets 6. Sequence of last-4 deltas 7. Offset XOR-ed with delta 8. None
 
Table 5.3: List of program control-flow and data-flow components used to derive the list of features for exploration.
StructureDescriptionSize
QVStore• # vaults = 22 • # planes/vault = 33 • # entries/plane = feature dimension ( 128128 ) ×\times action dimension ( 1616 ) • Entry size = Q-value width ( 1616 b)24 KB
EQ• # entries = 256 • Entry size = state ( 2121 b) + action index ( 55 b) + reward ( 55 b) + filled-bit ( 11 b) + address ( 1616 b)1.5 KB
Total 25.5 KB
Table 5.4: Storage overhead of Pythia.
 
Core1-12 cores, 4-wide OoO, 256-entry ROB, 72/56-entry LQ/SQ, Perceptron-based branch predictor [ 411 ] , 20-cycle misprediction penalty
L1/L2 CachesPrivate, 32KB/256KB, 64B line, 8 way, LRU, 16/32 MSHRs, 4-cycle/14-cycle round-trip latency
LLC2MB/core, 64B line, 16 way, SHiP [ 876 ] , 64 MSHRs per LLC Bank, 34-cycle round-trip latency
Main Memory1C: Single channel, 1 rank/channel; 4C: Dual channel, 2 ranks/channel; 8C: Quad channel, 2 ranks/channel; 8 banks/rank, 2400 MTPS, 64b data-bus per channel, 2KB row buffer/bank, tRCD = tRP = 15ns, tCAS = 12.5ns
 
Table 5.5: Simulated system parameters.
Suite# Workloads# TracesExample Workloads
SPEC061628gcc, mcf, cactusADM, lbm, …
SPEC171218gcc, mcf, pop2, fotonik3d, …
PARSEC511canneal, facesim, raytrace, …
Ligra1340BFS, PageRank, Bellman-ford, …
Cloudsuite453cassandra, cloud9, nutch, …
Table 5.6: Workloads used for evaluation.
 
SPP [ 465 ]256-entry ST, 512-entry 4-way PT, 8-entry GHR6.2 KB
Bingo [ 120 ]2KB region, 64/128/4K-entry FT/AT/PHT46 KB
MLOP [ 761 ]128-entry AMT, 500-update, 16-degree8 KB
DSPatch [ 141 ]Same configuration as in [ 141 ]3.6 KB
PPF [ 148 ]Same configuration as in [ 148 ]39.3 KB
 
Table 5.7: Configuration of evaluated prefetchers.
Figure 5.6 : Coverage and overprediction with respect to the baseline LLC misses in the single-core system.
Figure 5.7 : Performance improvement in single-core workloads. St=Stride, S=SPP, B=Bingo, D=DSPatch, and M=MLOP.
Figure 5.8 : Performance line graph of 150 single-core traces.
Figure 5.9 : Performance in the four-core system.
Figure 5.10 : Performance line graph of 272 four-core trace mixes.
Figure 5.11 : Performance on unseen traces.
Figure 5.12 : Performance of memory bandwidth-oblivious Pythia versus the basic Pythia.
Figure 5.13 : Geomean performance improvement of prefetchers in systems with varying (a) number of cores, (b) DRAM million transfers per second (MTPS), and (c) LLC size. Each DRAM bandwidth configuration roughly matches MTPS/core of various commercial processors [ 29 , 5 , 4 ] . The baseline bandwidth/LLC configuration is marked in red.
Figure 5.14 : Performance sensitivity of Pythia towards (a) the exploration rate ( ϵ\epsilon ), and (b) the learning rate ( α\alpha ) hyperparameter values. The values in basic Pythia configuration are marked in red.
Figure 5.15 : Performance, coverage, and overprediction of Pythia with different feature combinations. The x-axis shows experiments with different feature combinations.
Figure 5.16 : Performance sensitivity of all prefetchers to number of warmup instructions.
Figure 5.17 : Performance of multi-level prefetching schemes (a) in the baseline 2400-MTPS configuration, and (b) with varying main memory bandwidth.
Figure 5.18 : Performance of Pythia vs. the context prefetcher [ 660 ] using hardware contexts.
Figure 5.19 : Performance comparison against IBM POWER7 prefetcher [ 416 ] .
Figure 5.20 : Q-value curves of PC+Delta feature values (a) 0x436a81+0 and (b) 0x4377c5+0 in 459.GemsFDTD-1320B .
Figure 5.21 : Performance and main memory bandwidth usage of prefetchers in Ligra-CC .
Figure 5.22 : Performance of the basic and strict Pythia configurations in the Ligra workloads.
Figure 5.23 : Performance of the basic and feature-optimized Pythia on the SPEC CPU2006 suite.
Figure 5.24 : Performance improvement in 483 single-core DPC4 traces.
Figure 5.25 : Performance improvement in 966 four-core DPC4 traces.
Pythia’s area : 0.330.33 mm2/core; Pythia’s power : 55.1155.11 mW/core
Overhead compared to real systemsAreaPower
4-core Skylake D-2123IT, 60W TDP [ 27 ]1.03%0.37%
18-core Skylake 6150, 165W TDP [ 28 ]1.24%0.60%
28-core Skylake 8180M, 205W TDP [ 30 ]1.33%0.75%
Table 5.8: Area and power overhead of Pythia.
Figure 6.1 : The distribution of ROB-blocking and non-blocking load requests (on the left y-axis), and LLC misses per kilo instructions (on the right y-axis) in the absence and presence of a state-of-the-art hardware data prefetcher [ 136 ] .
Figure 6.2 : The average number of cycles a core stalls due to an off-chip load blocking any instruction from retiring from the ROB across all workload categories. The dark portion in each bar shows the cycles that can be completely eliminated by removing the on-chip cache access latency from an off-chip load’s critical path.
Figure 6.3 : Comparison of the execution timeline of an off-chip load request in a conventional processor and in Hermes.
Figure 6.4 : (a) Speedup of Ideal Hermes by itself and when combined with Pythia in single-core workloads. (b) Speedup of Ideal Hermes when combined with four recently-proposed prefetchers: Bingo [ 120 ] , SPP [ 465 , 148 ] , MLOP [ 761 ] , and SMS [ 792 ] .
Figure 6.5 : Percentage of loads that miss the LLC and goes off-chip (on the left y-axis) and the LLC MPKI (on the right y-axis) in the baseline system with Pythia.
Figure 6.6 : Overview of Hermes.
Figure 6.7 : Stages to make a prediction by POPET.
 
Features without control-flow informationFeatures with control-flow information
1. Load virtual address 2. Virtual page number 3. Cacheline offset in page 4. First access 5. Cacheline offset + first access 6. Byte offset in cacheline 7. Word offset in cacheline8. Load PC 9. PC ⊕\oplus load virtual address 10. PC ⊕\oplus virtual page number 11. PC ⊕\oplus cacheline offset 12. PC + first access 13. PC ⊕\oplus byte offset 14. PC ⊕\oplus word offset 15. Last- 44 load PCs 16. Last- 44 PCs
 
Table 6.1: The initial set of program features used for automated feature selection. ⊕\oplus represents a bitwise XOR operation.
 
Selected features• PC ⊕\oplus cacheline offset • PC ⊕\oplus byte offset • PC + first access • Cacheline offset + first access • Last- 44 load PCs
Threshold valuesτact=−18\tau_{act}=-18 , TN=−35T_{N}=-35 , TP=40T_{P}=40
 
Table 6.2: POPET configuration parameters.
 
StructureDescriptionSize
POPET• Perceptron weight tables – PC ⊕\oplus cacheline offset: 1024×5b1024\times 5b – PC ⊕\oplus byte offset: 1024×5b1024\times 5b – PC + first access: 1024×5b1024\times 5b – Cacheline offset + first access: 128×5b128\times 5b – Last- 44 load PCs: 1024×5b1024\times 5b • Page buffer: 64×80b64\times 80b3.2 KB
LQ MetadataHashed PC: 128×32b128\times 32b ; Last-4 PC: 128×10b128\times 10b ; First access: 128×1b128\times 1b ; perceptron weight: 128×5b128\times 5b ; prediction: 128×1b128\times 1b0.8 KB
Total 4.0 KB
Table 6.3: Storage overhead of Hermes.
 
Core1 and 8 cores, 6-wide fetch/execute/commit, 512-entry ROB, 128/72-entry LQ/SQ, Perceptron branch predictor [ 411 ] with 17-cycle misprediction penalty
L1/L2 CachesPrivate, 48KB/1.25MB, 64B line, 12/20-way, 16/48 MSHRs, LRU, 5/15-cycle round-trip latency [ 60 ]
LLC3MB/core, 64B line, 12 way, 64 MSHRs/slice, SHiP [ 876 ] , 55-cycle round-trip latency [ 56 , 60 ] , Pythia prefetcher [ 136 ]
Main Memory1C: 1 channel, 1 rank per channel; 8C: 4 channels, 2 ranks per channel; 8 banks per rank, DDR4-3200 MTPS, 64b data-bus per channel, 2KB row buffer per bank, tRCD=12.5ns, tRP=12.5ns, tCAS=12.5ns
HermesHermes-O/P : 66 / 1818 -cycle Hermes request issue latency
 
Table 6.4: Simulated system parameters.
Suite# Workloads# TracesExample Workloads
SPEC061422gcc, mcf, cactusADM, lbm, …
SPEC171123gcc, mcf, pop2, fotonik3d, …
PARSEC412canneal, facesim, raytrace, …
Ligra1120BFS, PageRank, Radii, …
CVP3333integer, floating-point, server, …
Table 6.5: Workloads used for evaluation.
 
HMP [ 905 ] with local, gshare, and gskew predictors11 KB
TTP with a metadata budget similar to the L2 cache1536 KB
Pythia [ 136 ] with the same configuration in [ 136 ]25.5 KB
Bingo [ 120 ] with the same configuration in [ 120 ]46 KB
SPP [ 465 ] with perceptron-based prefetch filter [ 148 ]39.3 KB
MLOP [ 761 ] with the same configuration in [ 761 ]8 KB
Table 6.6: Storage overhead of all evaluated mechanisms.
Figure 6.8 : Comparison of (a) accuracy and (b) coverage of POPET against those of HMP [ 905 ] and TTP.
Figure 6.9 : The accuracy and coverage of POPET using each program feature individually and in various combinations.
Figure 6.10 : Line graph of POPET’s (a) accuracy and (b) coverage using each of the five program features individually across all 110 single-core workloads. No single feature can provide the best accuracy or coverage across all workloads.
Figure 6.11 : Speedup in single-core workloads.
Figure 6.12 : Single-core performance of all 110 workloads.
Figure 6.13 : Speedup of Hermes with three off-chip load predictors (HMP, TTP, and POPET) and the Ideal Hermes.
Figure 6.14 : (a) Reduction in stall cycles caused by off-chip loads. (b) Overhead in the main memory requests.
Figure 6.15 : Speedup in eight-core workloads.
Figure 6.16 : Performance sensitivity to (a) main memory bandwidth and (b) baseline prefetcher. The baseline system configuration is highlighted in green. Other highlighted configurations closely match with various commercial processors [ 29 , 5 , 4 ] .
Figure 6.17 : Performance sensitivity to (a) Hermes request issue latency and (b) on-chip cache hierarchy access latency. The baseline system configuration is highlighted in green.
Figure 6.18 : Performance sensitivity to (a) reorder buffer size and (b) LLC size. The baseline system configuration is highlighted in green.
Figure 6.19 : Effect of the activation threshold on POPET’s accuracy and coverage (on the left y-axis) and Hermes’s speedup (on the right y-axis) for all single-core workloads.
Figure 6.20 : (a) Variation of off-chip load prediction accuracy and coverage and (b) the increase in the main memory requests with different data prefetchers.
Figure 6.21 : Performance of Hermes and Pythia+Hermes while varying Hermes request issue latency.
Figure 6.22 : Speedup in 483 single-core DPC4 workload traces.
Figure 6.23 : (a) Speedup of Hermes with three different off-chip predictors, HMP, TTP, and POPET in single-core DPC4 workloads. Comparison of (b) accuracy and (c) coverage of POPET against those of HMP and TTP in single-core DPC4 workloads.
Figure 6.24 : Performance sensitivity to baseline prefetcher in single-core DPC4 workloads.
Figure 6.25 : Processor power consumption of Hermes, Pythia, and Hermes combined with Pythia.
Figure 7.1 : Performance line graph of a state-of-the-art off-chip predictor (OCP), POPET [ 133 ] , and a state-of-the-art prefetcher, Pythia [ 136 ] , across 100 workloads.
Figure 7.2 : Geomean speedup of POPET, Pythia, Naive, and StaticBest combinations across all workloads.
Figure 7.3 : Fraction of prefetch fills from off-chip main memory that are inaccurate.
Figure 7.4 : Geomean speedup of Naive, HPAC, MAB, and StaticBest combinations across all workloads.
Figure 7.5 : High-level overview of Athena as an RL agent.
 
FeatureMeasurementRationale
 
Prefetcher accuracy# demand hits# prefetches issued\dfrac{\text{\# demand hits}}{\text{\# prefetches issued}}Effectiveness of prefetching
OCP accuracy# off-chip demand hits# off-chip predictions\dfrac{\text{\# off-chip demand hits}}{\text{\# off-chip predictions}}Effectiveness of OCP
Bandwidth usagecurrent DRAM bandwidthmax DRAM bandwidth\dfrac{\text{current DRAM bandwidth}}{\text{max DRAM bandwidth}}Memory bus pressure
Cache pollution# prefetch-evicted demand misses # total demand misses\dfrac{\text{\# prefetch-evicted demand misses }}{\text{\# total demand misses}}Interference caused by prefetches
Table 7.1: Candidate features considered for Athena’s state representation.
Algorithm 2 Q-value-driven prefetcher aggressiveness control
 
Reward ComponentConstituent MetricWeight
 
RtcorrR_{t}^{\text{corr}}# Cyclesλcycle\lambda_{\text{cycle}}
 # LLC missesλLLCm\lambda_{\text{LLC}_{\text{m}}}
 LLC miss latencyλLLCt\lambda_{\text{LLC}_{\text{t}}}
RtuncorrR_{t}^{\text{uncorr}}# Load instructionsλload\lambda_{\text{load}}
Table 7.2: Constituent metrics of Athena’s reward.
Figure 7.6 : Organization of QVStore.
 
CategoryFinal Values
 
Selected Features(1) prefetcher accuracy, (2) OCP accuracy, (3) bandwidth usage, (4) prefetch-induced cache pollution
Reward Weightsλcycle=1.6\lambda_{\text{cycle}}=1.6 , λLLCm=0.0\lambda_{\text{LLC}_{\text{m}}}=0.0 , λLLCt=0.0\lambda_{\text{LLC}_{\text{t}}}=0.0 , λload=0.6\lambda_{\text{load}}=0.6 , λMBr=1.0\lambda_{\text{MBr}}=1.0
Hyperparametersα=0.6\alpha=0.6 , γ=0.6\gamma=0.6 , ϵ=0.0\epsilon=0.0 , τ=0.12\tau=0.12 , Epoch length ( NN ) = 2K2K instructions
 
Table 7.3: Final Athena configuration derived through automated design-space exploration.
 
StructureDescriptionSize
 
QVStore# planes = 88 , # rows = 64642 KB
# columns = 44 , entry size = 88 bits
Accuracy Tracker40964096 -bit Bloom filter, 22 hashes0.5 KB
Pollution Tracker40964096 -bit Bloom filter, 22 hashes0.5 KB
Table 7.4: Storage overhead of Athena.
 
Core6-wide fetch/issue/commit, 512-entry ROB, 128-entry LQ, 72-entry SQ, perceptron branch predictor [ 411 ] , 17-cycle misprediction penalty
L1I/DPrivate, 48KB, 64B line, 12-way, 16 MSHRs, LRU, 4/5-cycle round-trip latency
L2CPrivate, 1.25MB, 64B line, 20-way, 48 MSHRs, LRU, 15-cycle round-trip latency [ 60 ]
LLCShared, 3MB/core, 64B line, 12-way, 64 MSHRs/slice, SHiP [ 876 ] , 55-cycle round-trip latency [ 56 , 60 ]
Main Memory1 rank per channel, 8 banks per rank, 64-bit data bus, 2KB row buffer, tRCDt_{\text{RCD}} = tRPt_{\text{RP}} = tCASt_{\text{CAS}} = 12.5ns; DDR4 with 3.2 GB/s per core
 
Table 7.5: Simulated system parameters.
Suite# WorkloadsExample Workloads
SPEC CPU0629astar, leslie3d, libquantum, milc, omnetpp, sphinx3, soplex …
SPEC CPU1720bwaves, cactuBSSN, fotonik3d, gcc, lbm, mcf, xalancbmk …
Ligra13BC, BFS, BFSCC, PageRankDelta, Radii, …
PARSEC13canneal, facesim, fluidanimate, raytrace, streamcluster …
CVP25integer, floating point, …
Table 7.6: Workloads used for evaluation.
Cache DesignDescriptionComparison Points
CD1OCP + 1 L2C prefetcherHPAC, MAB
CD2OCP + 1 L1D prefetcherHPAC, MAB, TLP
CD3OCP + 2 L2C prefetchersHPAC, MAB
CD4OCP + 1 L1D + 1 L2C prefetcherHPAC, MAB, TLP
Table 7.7: Evaluated cache designs (CD) and corresponding comparison points.
PrefetchersL1DIPCP [ 628 ] , as an L1D-only prefetcher0.7 KB
  Berti , as configured in [ 604 ]2.55 KB
 L2CPythia , as configured in [ 136 ]25.5 KB
  SPP+PPF , as configured in [ 465 , 148 ]39.3 KB
  MLOP , as configured in [ 761 ]8 KB
  SMS , as configured in [ 792 ]20 KB
OCPsPOPET [ 133 ] , with 5 features4 KB
Table 7.8: Storage overhead of all evaluated mechanisms.
Figure 7.7 : Speedup in cache design 1 (CD1).
Figure 7.8 : (a) Workload category-wise performance analysis in CD1. (b) Performance comparison with StaticBest in CD1.
Figure 7.9 : Comparison of (a) the number of main memory requests and (b) average last-level cache (LLC) load miss latency.
Figure 7.10 : Speedup in cache design 2 (CD2).
Figure 7.11 : Speedup in cache design 3 (CD3).
Figure 7.12 : Speedup in cache design 4 (CD4).
Figure 7.13 : Performance sensitivity to underlying prefetching mechanism at L2C in CD1.
Figure 7.14 : Performance sensitivity to off-chip prediction mechanisms in CD1.
Figure 7.15 : Performance sensitivity off-chip predicted request issue latency in CD1.
Figure 7.16 : Performance sensitivity to prefetching mechanism at L1D in CD4.
Figure 7.17 : Performance sensitivity to main memory bandwidth in CD4.
Figure 7.18 : Speedup in four-core workloads.
Figure 7.19 : Speedup in eight-core workloads.
Figure 7.20 : Distribution of Athena’s action in coordinating Pythia and POPET and speedup of different Pythia-POPET combinations in compute_fp_78 workload from CVP suite, while varying memory bandwidth: 3.2 GB/s and 25.6 GB/s.
Figure 7.21 : Contribution of individual state features and reward components to Athena’s geomean performance across all 100 workloads.
Figure 7.22 : Geomean performance of Athena coordinating two L2C prefetchers without OCP.
Figure 7.23 : Speedup in (a) CD1 and (b) CD2 across 483 DPC4 traces.
Figure 7.24 : Speedup in (a) CD3 and (b) CD4 across 483 DPC4 traces.
Figure 8.1 : Execution timeline of a code example in a processor (a) without a load value predictor (LVP), (b) with LVP, and (c) with LVP and load elimination.
Figure 8.2 : (a) Fraction of dynamic loads that are global-stable. Distribution of global-stable loads by their (b) addressing mode and (c) inter-occurrence distance. (d) Distribution of inter-occurrence distance of global-stable loads from each addressing mode.
Figure 8.3 : Code example and disassembly from 541.leela_r of SPEC CPU 2017 suite. The highlighted load instructions are global-stable.
Figure 8.4 : Code example and disassembly from 557.xz_r of SPEC CPU 2017 suite. The highlighted load instructions are global-stable.
Figure 8.5 : Fraction of all dynamic loads that are global-stable in workloads compiled without and with APX (on the left y-axis) and the reduction in dynamic loads by APX (on the right y-axis).
Figure 8.6 : Distribution of global-stable loads by their addressing modes in workloads without and with APX. Each number on the x-axis corresponds to the respective workload from SPEC CPU 2017 suite.
Figure 8.7 : (a) Fraction of total execution cycles where at least one load port is utilized (we call such cycles load-utilized ). (b) Categorization of load-utilized cycles based on whether or not a global-stable load utilizes a load port.
Figure 8.8 : Speedup of Ideal Constable against Ideal Stable LVP and a processor with 2×2\times load execution width of the baseline.
Figure 8.9 : Overview of Constable.
Figure 8.10 : (a) Average number of SLD updates per cycle during rename stage. (b) Change in performance when Constable’s structures are updated only by correct path instructions vs. all instructions without updating Constable’s structures on branch misprediction recovery.
Figure 8.11 : An illustrative example of Constable’s operation.
 
StructureDescriptionSize
 
SLD• # entries: 512512 ( 3232 sets ×\times 1616 ways) • Entry size: tag ( 24b24b ) + addr ( 32b32b ) + val ( 64b64b ) + confidence level ( 5b5b ) + can_eliminate flag ( 1b1b )7.9 KB
RMT• 1616 load PCs for each stack registers (RSP and RBP) • 88 load PCs for each remaining 1414 architectural registers in x86-640.4 KB
AMT• # entries: 256 ( 3232 sets ×\times 88 ways) • Entry size: physical address tag ( 32b32b ) + # hashed load PCs ( 4×24b4\times 24b )4.0 KB
 
Table 8.1: Storage overhead of Constable.
 
Basicx86-64 core clocked at 3.2 GHz with 2-way SMT support
Fetch & Decode8-wide fetch, TAGE/ITTAGE branch predictors [ 757 ] , 20-cycle misprediction penalty, 32KB 8-way L1-I cache, 4K-entry 8-way micro-op cache, 6-wide decode, 144-entry IDQ, loop-stream detector [ 150 ]
Rename6-wide, 288 integer, 220 512-bit and 320 256-bit physical registers, Memory Renaming [ 844 ] , zero elimination [ 300 ] , move elimination [ 300 , 278 ] , constant folding [ 300 , 278 ] , branch folding [ 251 ]
Allocate512-entry ROB, 240-entry LB, 112-entry SB, 248-entry RS
Issue & Retire6-wide issue to 12 execution ports; 5, 3, 2, and 2 ports for ALU, load, store-address, and store-data execution. Port 0, 1, and 5 are used for vector instructions. Aggressive out-of-order load scheduling with memory dependence prediction [ 220 , 569 ] , 6-wide retire
CachesL1-D : 48KB, 12-way, 5-cycle latency, LRU, PC-based stride prefetcher [ 302 ] ; L2 : 2MB, 16-way, 12-cycle round-trip latency, LRU, stride + streamer [ 201 ] + SPP [ 465 ] ; LLC : 3MB, 12-way, 50-cycle data round trip latency [ 44 , 133 ] , dead-block-aware replacement policy [ 452 ] , streamer, MESIF [ 564 ] protocol
Table 8.2: Simulation parameters. IDQ: Instruction Decode Queue, SB: Store Buffer.
 
ComponentPort countRead access energy (pJ)Write access energy (pJ)Leakage power (mW)Area (mm2)
 
SLD33 R/ 22 W10.7610.7616.7016.701.021.020.2110.211
RMT22 R/ 66 W0.150.150.200.200.310.310.0040.004
AMT11 R/ 11 W1.581.584.224.220.740.740.0170.017
 
Table 8.3: Access energy, leakage power, and area estimates of Constable’s structures in 14nm technology. R: read port, W: write port.
 
Suite#Workloads#TracesExample Workloads
 
Client1622DaCapo [ 12 ] , SYSmark [ 58 ] , TabletMark [ 59 ] , JetStream2 [ 34 ]
Enterprise914SPECjEnterprise [ 55 ] , SPECjbb [ 54 ] , LAMMPS [ 36 ]
FSPEC171329All from SPECrate FP 2017 [ 53 ]
ISPEC171011All from SPECrate Integer 2017 [ 53 ]
Table 8.4: Workloads used for evaluation.
 
MechanismOverhead
 
EVES [ 754 ] with the same configuration as the winner of CVP-1 3232 KB storage budget track [ 15 ]32 KB
ELAR [ 131 ] with an additional adder in decode stage and a new copy of the ESP registerAdder + 64-b register
RFP [ 770 ] with 2K-entry prefetch table, 64-entry page address table, and 128-entry RFP-Inflight table12.4 KB
Constable (this work)15.4 KB
Table 8.5: Overhead of all evaluated mechanisms.
Figure 8.12 : Speedup over the baseline (noSMT).
Figure 8.13 : Speedup of all workloads (noSMT).
Figure 8.14 : Speedup of Constable by eliminating execution of only PC-relative, stack-relative, and register-relative loads.
Figure 8.15 : Speedup over the baseline (SMT2).
Figure 8.16 : Speedup of Constable over ELAR and RFP.
Figure 8.17 : Load coverage of Constable versus EVES.
Figure 8.18 : Breakdown of eliminated and non-eliminated loads as fractions of global-stable loads.
Figure 8.19 : Reduction in (a) RS allocations and (b) L1-D accesses.
Figure 8.20 : (a) Overall core power consumption normalized to baseline. Expanded view of (b) OOO power and (c) MEU power.
Figure 8.21 : Performance sensitivity to (a) load execution width, and (b) pipeline depth.
Figure 8.22 : (a) Fraction of loads eliminated by Constable that violate memory ordering. (b) Increase in instructions allocated to ROB in presence of Constable.
Figure 8.23 : (a) Speedup and (b) coverage of Constable with AMT invalidation on L1D eviction compared to a vanilla Constable.

Explore similar work

CardsList
Mar 15, 2026

Machine Learning-Driven Intelligent Memory System Design: From On-Chip Caches to Storage

Despite the data-rich environment in which memory systems of modern computing platforms operate, many state-of-the-art architectural policies employed in the memory system rely on static, human-designed heuristics that fail to truly adapt to the workload and system behavior via principled learning methodologies. In this article, we propose a fundamentally different design approach: using lightweight and practical machine learning (ML) methods to enable adaptive, data-driven control throughout the memory hierarchy. We present three ML-guided architectural policies: (1) Pythia, a reinforcement learning-based data prefetcher for on-chip caches, (2) Hermes, a perceptron learning-based off-chip predictor for multi-level cache hierarchies, and (3) Sibyl, a reinforcement learning-based data placement policy for hybrid storage systems. Our evaluation shows that Pythia, Hermes, and Sibyl significantly outperform the best-prior human-designed policies, while incurring modest hardware overheads. Collectively, this article demonstrates that integrating adaptive learning into memory subsystems can lead to intelligent, self-optimizing architectures that unlock performance and efficiency gains beyond what is possible with traditional human-designed approaches.

Rahul Bera, Rakesh Nadig, Onur Mutlu

SAFARI Research Group, ETH Zürich, Switzerland

Adaptive Learning
AskPDF
Aug 10, 2026

ArchAgent v2: A Case Study with the Data Prefetching Championship

Agentic artificial intelligence has shown great promise in automating algorithm design, but scaling similar techniques to computer microarchitecture discovery remains challenging due to vast search spaces, strict hardware budgets, and long simulation times. In this work, we present ArchAgent v2, a framework which scales automated microarchitecture search to multi-level data prefetching. While the original ArchAgent successfully discovered single-level cache replacement policies in competition settings, it does not scale to multi-level prefetching where the design space and degrees of freedom are larger. To overcome this, we introduce two new additions to ArchAgent: a cascaded evolutionary search that subdivides the design space by sequentially evolving and freezing prefetchers at individual cache levels, and a hardware-realizability feedback loop that embeds real-time size-estimation directly into the evolution process. Evaluated under identical rules of the 4th Data Prefetching Championship (DPC4), ArchAgent v2 automatically designs a three-level prefetcher that outperforms the winning hand-designed solution, further demonstrating automated agentic discovery as a useful tool for computer architects. Our discovered policy achieves a 3.8% geometric mean IPC speedup over the baseline overall and a 0.3% improvement over the prior champion, BertiGO. On low-bandwidth single-core configurations, our policy yields a 4.6% performance speedup compared to only 2.6% for BertiGO. However, multi-core evolution still remains a significant challenge due to simulation latency impeding evolution speed. Finally, our profiling of an ArchAgent evolution of over 12,000 candidate designs provides key insights into how automated evolutionary agents explore and synthesize complex microarchitectural logic.

Abraham Gonzalez, Raghav Gupta, Akanksha Jain +12

Google · University of California, Berkeley · Google DeepMind

Automated Algorithm DiscoveryAutomated Heuristic DesignEvolutionary Optimization
AskPDF
Aug 2, 2026

On the Limits of Machine-Learned Ranking for Modern Microarchitectural Policies

Machine-learning predictors estimate processor performance far faster than cycle-level simulation. For design-space exploration, however, the valuable test is not merely reproducing the usual hardware ordering, but identifying how different hardware configurations rank on individual program phases. We evaluate four ML-predictors in two design regimes: Structural Parameters (SP), varying hardware resources such as issue width, ROB size, and cache capacity; and Behavioral Policies (BP), varying prefetching and replacement algorithms. In the SP regime, aggregate ranking is strong, yet counter-intuitive windows(CIW)---where the configuration expected to be slower is faster---constitute 22.4%22.4\% of non-tied windows across five pairs with a clear architectural prior. CIW match across these pairs is only 23.323.3--39.9%39.9\%; every point estimate is below the 50%50\% random strict-ordering reference. The BP regime presents a different failure: ground-truth ties cover 37.8%37.8\% of pair-windows, most strict pairs have margins of only a few cycles, and no model family reliably beats a feature-free majority baseline. NeuroScalar and SimNet fall below that baseline, Concorde is statistically tied with it, and the best selected OneDSE head improves by only 2.12.1 percentage points. Accuracy rises mainly at large margins. We further show that this failure is not a matter of model capacity: an information-theoretic analysis reveals that when ranking outcomes depend on hidden microarchitectural state absent from the instruction stream, no trace-based predictor can exceed the Bayes accuracy determined by observable inputs alone. Thus high cycle or aggregate ranking accuracy can reflect mastery of easy, high-margin cases while missing the local reversals that carry the most architectural insight and for which cycle-level simulation remains indispensable.

Yanxin Zhang, Shayne Wadle, Yuxuan Xiong +3

NVIDIA · University of Wisconsin–Madison

Learning to RankElectronic Design AutomationPairwise Ranking
AskPDF