Joint Hardware-Workload Co-Optimization for In-Memory Computing Accelerators

Olga Krestinskaya; Mohammed E. Fouda; Ahmed Eltawil; Khaled N. Salama

cs.ARMar 4, 2026

Joint Hardware-Workload Co-Optimization for In-Memory Computing Accelerators

Authors: Olga Krestinskaya, Mohammed E. Fouda, Ahmed Eltawil, Khaled N. Salama

Organizations: King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Compumacy for Artificial Intelligence Solutions, Cairo, Egypt

Abstract

Software-hardware co-design is essential for optimizing in-memory computing (IMC) hardware accelerators for neural networks. However, most existing optimization frameworks target a single workload, leading to highly specialized hardware designs that do not generalize well across models and applications. In contrast, practical deployment scenarios require a single IMC platform that can efficiently support multiple neural network workloads. This work presents a joint hardware-workload co-optimization framework based on an optimized evolutionary algorithm for designing generalized IMC accelerator architectures. By explicitly capturing cross-workload trade-offs rather than optimizing for a single model, the proposed approach significantly reduces the performance gap between workload-specific and generalized IMC designs. The framework is evaluated on both RRAM- and SRAM-based IMC architectures, demonstrating strong robustness and adaptability across diverse design scenarios. Compared to baseline methods, the optimized designs achieve energy-delay-area product (EDAP) reductions of up to 76.2% and 95.5% when optimizing across a small set (4 workloads) and a large set (9 workloads), respectively. The source code of the framework is available at https://github.com/OlgaKrestinskaya/JointHardwareWorkloadOptimizationIMC.

Figures & tables

Fig. 1: Research gap and representative state-of-the-art frameworks (including [ 26 , 27 , 28 , 25 , 20 , 35 , 22 , 23 , 24 , 18 , 21 , 19 , 29 , 30 , 31 , 32 , 33 , 34 , 17 ] , which are discussed in detail in Section II ).

Frame- work	Algorithm ∗1	Models tested	Hardware search space size ∗2	Optimization parameters				Work- load	Generalized hardware solution ∗7
Frame- work	Algorithm ∗1	Models tested	Hardware search space size ∗2	D ∗3	C ∗4	A ∗5	S ∗6	Work- load	Generalized hardware solution ∗7
NAX ∗8 [ 26 ]	DS	ResNet	$4.2\times 10^{6}$ (layer-wise)	-	Crossbar size	-	-	single	-
Yang X. et al. [ 25 ]	CF- MESMO	ResNet, VGG	$1.35\times 10^{2}$	Bits per cell	Crossbar size	-	Frequency, temperature	single	-
Gibbon [ 27 ]	EA	ResNet (modified)	$6.4\times 10^{1}$	Bits per cell	Crossbar size, ADC/DAC resolution	-	-	single	-
XPert ∗8 [ 28 ]	DS	VGG	$4.5\times 10^{15}$ (layer-wise)	-	Column sharing, ADC precision/type	-	-	single	-
CoMN [ 35 ]	BO	ResNet, VGG, MobileNet	$2.2\times 10^{4}$	-	Crossbar size, number of ADCs, ADC resolution /power/frequency	Crossbars per tile, buffer size /bandwidth, router flit bandwidth	Mapping	single	-

TABLE I: Comparison of the proposed framework with state-of-the-art approaches.

Fig. 2: Proposed joint hardware-workload co-optimization framework for in-memory computing hardware.

Aspect	EA	RL	BO	DS
Discrete and categorical variables	$\checkmark$	$\triangle$	$\triangle$	$\triangle$
Large combinatorial space ( $10^{6}$ – $10^{7}$ )	$\checkmark$	$\triangle$	$\times$	$\times$
Hard and conditional constraints	$\checkmark$	$\triangle$	$\triangle$	$\times$
Non-smooth objectives	$\checkmark$	$\triangle$	$\times$	$\times$
Expensive hardware evaluation	$\checkmark$	$\times$	$\times$	$\times$
Extra modeling or training required	$\times$	$\checkmark$	$\checkmark$	$\checkmark$

TABLE II: Comparison of optimization methods for discrete hardware design-space exploration.

\checkmark

: well suited,

\triangle

: possible with overhead,

\times

: poorly suited.

Algorithm	Global minima	Relative speed
GA	$\checkmark$	Fastest
ES/ERES	$\checkmark$	Moderate
PSO	$\times$ (local minima)	Fast
G3PCX	$\times$ (local minima)	Moderate
CMA-ES	$\times$ (no convergence)	Slow

TABLE III: Comparison of evolutionary and genetic-inspired optimization algorithms on a small search space.

Phase	Crossover $\mathbb{P}_{c}$	Crossover $\eta_{c}$	Mutation $\mathbb{P}_{m}$	Mutation $\eta_{m}$
Exploration	1.0	3.0	1.0	3.0
Transition	0.9	7.0	0.5	7.0
Convergence	1.0	15.0	0.2	15.0
Fine-tuning	1.0	25.0	0.05	25.0

TABLE IV: Parameters for different search phases.

Fig. 3: EDAP comparison of optimized designs for RRAM-based and SRAM-based IMC hardware, obtained using separate optimization for the largest workload and joint optimization across multiple workloads.

Fig. 4: Improved convergence and EDAP scores of optimized designs achieved using the proposed 4-phase genetic algorithm (GA) with enhanced sampling, compared to the traditional, non-modified GA approach (six independent experiments with RRAM-based hardware for each case).

RRAM
Case	EDAP ( $mJ\times ms\times mm^{2}$ )				Search time
	ResNet18	VGG16	AlexNet	MobileNetV3	( $hours$ )
All	0.02051	0.88869	0.00324	0.03040	22.79
Max	0.02024	0.96181	0.00320	0.02974	19.89
Mean	0.02040	0.90902	0.00322	0.02753	20.25
SRAM

TABLE V: EDAP per optimized designs and search time for RRAM and SRAM across different aggregation strategies.

Fig. 5: Comparison of optimization strategies for generating generalized architectures across RRAM-based (a-d) and SRAM-based (e-h) experiments using different objective functions. These include separate search, maximum-workload-based optimization, joint search with a non-modified genetic algorithm (GA) [ 44 ] , joint search with enhanced sampling, and joint search using the proposed four-phase GA with optimized sampling (top-5 designs are shown in each experiment with top-1 marked with a star). The goal is to achieve performance that is as close as possible to the individually workload-optimized (separate search) designs, indicating minimal loss in hardware efficiency when transitioning to generalized hardware. The results demonstrate that the proposed algorithm is the most effective among all approaches in the majority of the cases, supporting the selection of optimized designs where the transition to generalized architectures results in the least compromise in hardware performance.

Method	Search / Sampling Time	Total Time
Separate	Up to 0.2 h (RRAM)	8.8 h–14.9 h
Joint (non-modified)	Up to 0.5 h (RRAM)	12.4 h–12.6 h
Joint (proposed)	5.6 h–6.4 h	18 h–19 h

TABLE VI: Runtime comparison on 64 CPU cores for equivalent population size and number of generations.

Fig. 6: Comparison of optimized RRAM- and SRAM-based hardware designs across various objective functions, with energy and latency results shown for the largest workload (VGG16).

Fig. 7: Ablation studies comparing the proposed joint hardware-workload optimization to a sequential hardware stack optimization.

Fig. 8: Experiments considering RRAM hardware non-idealities.

Technology	Average wafer cost (USD)	Yield	Average cost $\alpha$ per mm 2 (normalized to 32nm)	Voltage range for simulation
90nm	1651.5	90-95%	0.413	0.95-1.3V
65nm	1939.0	90-95%	0.477	0.85-1.2V
45nm	2237.5	80-90%	0.606	0.75-1.1V
32nm	3500.0	70-90%	1	0.65-1.0V
22nm	4338.5	70-90%	1.282	0.65-1.0V
14nm	4492.0	60-80%	1.498	0.55-0.9V

TABLE VII: Cost calculation parameters and voltage range for different technologies.

Fig. 9: Trade-offs between EDAP and fabrication cost and corresponding Pareto front for SRAM-based hardware-workload-technology design optimization.

Fig. 10: Scalability and generalization of the proposed joint hardware-workload co-optimization method across multiple workloads and network types, with a comparison to optimization targeting the maximum workload (i.e., the workload with the largest layer, in the context of weight-swapping SRAM-based IMC).

Compute-in-Memory

Compute-in-memory (CIM) performs computation within or beside memory arrays to reduce data movement and accelerate AI workloads.

See all 56 papers

Optimizing ML Workload Partitioning between CPUs and CIM Accelerators for Heterogeneous Computing
Jul 6, 2026Joel Klein, Rebecca Pelke, Roberto Laudani +2Compute-in-Memory Heterogeneous Computing
Institute for Communication Technologies and Embedded Systems, RWTH Aachen University, Germany
ChatNeuroSim: An LLM Agent Framework for Automated Compute-in-Memory Accelerator Deployment and Optimization
Mar 4, 2026Ming-Yen Lee, Shimeng YuCompute-in-Memory Electronic Design Automation
Georgia Institute of Technology, Atlanta, Georgia, USA
A3C3: AI Algorithm and Accelerator Co-design, Co-search, and Co-generation
Jun 18, 2026Selin Yildirim, Yingbing Huang, Deming ChenHardware-Aware NAS Hardware-Software Co-Design
Siebel School of Computing and Data Science · Department of Electrical and Computer Engineering University of Illinois Urbana-Champaign

Jul 6, 2026J/K move · Enter open · S save

Optimizing ML Workload Partitioning between CPUs and CIM Accelerators for Heterogeneous Computing

Joel Klein, Rebecca Pelke, Roberto Laudani, Jan Moritz Joseph+1

Institute for Communication Technologies and Embedded Systems, RWTH Aachen University, Germany

Compute-in-Memory Heterogeneous Computing Integer Linear Programming

Open paper AskPDF arXiv

Computing-in-Memory (CIM) accelerators execute Matrix-Vector Multiplications (MVMs) in memory, making them a compelling solution for Machine Learning (ML) workloads. However, existing ML workload partitioning approaches for CIM accelerators do not fully account for Resistive Random Access Memory (RRAM) constraints such as limited memory, high write latency, and limited endurance. They also neglect parallelism, low-level architectural effects, or the Central Processing Unit (CPU) as a complementary compute resource. To address these limitations, we propose an Integer Linear Programming (ILP)-based workload partitioning framework for heterogeneous CPU-CIM systems. It minimizes end-to-end inference latency under RRAM constraints, captures parallelism, and combines empirical profiling with analytical models. Using our framework, heterogeneous CPU-CIM execution achieves speedups of up to 30.9x over CPU-only execution on an edge CPU and 7.3x over a high-performance CPU. A Design Space Exploration (DSE) yields further design insights for future CIM accelerators.

Abstract

Compute-in-Memory

Explore similar work

Optimizing ML Workload Partitioning between CPUs and CIM Accelerators for Heterogeneous Computing

ChatNeuroSim: An LLM Agent Framework for Automated Compute-in-Memory Accelerator Deployment and Optimization

A3C3: AI Algorithm and Accelerator Co-design, Co-search, and Co-generation