ChatNeuroSim: An LLM Agent Framework for Automated Compute-in-Memory Accelerator Deployment and Optimization

The growing demand for deploying Small Language Models (SLMs) on edge devices, including laptops, smartphones, and embedded platforms, has exposed fundamental inefficiencies in existing accelerators. While GPUs handle prefill workloads efficiently, the autoregressive decoding phase is dominated by GEMV operations that are inherently memory-bound, resulting in poor utilization and prohibitive energy costs at the edge. In this work, we present EdgeCIM, a hardware-software co-design framework that rethinks accelerator design for end-to-end decoder-only inference. At its core is a CIM macro, implemented in 65nm, coupled with a tile-based mapping strategy that balances pipeline stages, maximizing parallelism while alleviating DRAM bandwidth bottlenecks. Our simulator enables design space exploration of SLMs up to 4B parameters, identifying Pareto-optimal configurations in terms of latency and energy. Compared to an NVIDIA Orin Nano, EdgeCIM achieves up to 7.3x higher throughput and 49.59x better energy efficiency on LLaMA3.2-1B, and delivers 9.95x higher throughput than Qualcomm SA8255P on LLaMA3.2-3B. Extensive benchmarks on TinyLLaMA-1.1B, LLaMA3.2 (1B, 3B), Phi-3.5-mini-3.8B, Qwen2.5 (0.5B, 1.5B, 3B), SmolLM2-1.7B, SmolLM3-3B, and Qwen3 (0.6B, 1.7B, 4B) reveal that our accelerator, under INT4 precision, achieves on average 336.42 tokens/s and 173.02 tokens/J. These results establish EdgeCIM as a compelling solution towards real-time, energy-efficient edge-scale SLM inference.

Computer Architecture research has traditionally required the exploration of vast, highly complex design spaces through manual human effort. Large Language Models (LLMs) offer an opportunity to accelerate this process by enabling fast and automated implementation and testing of different architectural mechanisms and their combinations. We introduce Agentic Architect, a framework for agentic CPU microarchitectural design-space exploration and optimization. The Agentic Architect combines LLM-driven code evolution with cycle-accurate simulation and hardware cost modeling to explore performance, area, and power tradeoffs. Starting from a seed design, the LLM is prompted to explore the design space by implementing changes, guided by prior evaluation feedback. Each new design is compiled, simulated on training workloads, evaluated against the optimization objective, and constrained by area and power budgets. Its score and diagnostics update the candidate population and guide subsequent iterations. We evaluate the Agentic Architect framework across prefetching, cache replacement, and branch prediction. On 213 traces spanning five benchmark suites, its evolved prefetchers lead the evaluated L1D and L2 designs, delivering up to 6.7% and 2.6% additional speedup, respectively, over the strongest state-of-the-art design, while the best evolved cache replacement design improves over Mockingjay by up to 1.7%. Moreover, across 191 branch-prediction traces, the evolved branch predictor achieves 97.62% of TAGE-SC-L's geometric mean IPC at 10.58% of its area. Our analysis highlights the importance of seed selection, hardware constraints, and training workloads in shaping search outcomes. Overall, Agentic Architect establishes the first end-to-end framework for agentic CPU microarchitectural design optimization, opening a new direction for architecture research.

Software-hardware co-design is essential for optimizing in-memory computing (IMC) hardware accelerators for neural networks. However, most existing optimization frameworks target a single workload, leading to highly specialized hardware designs that do not generalize well across models and applications. In contrast, practical deployment scenarios require a single IMC platform that can efficiently support multiple neural network workloads. This work presents a joint hardware-workload co-optimization framework based on an optimized evolutionary algorithm for designing generalized IMC accelerator architectures. By explicitly capturing cross-workload trade-offs rather than optimizing for a single model, the proposed approach significantly reduces the performance gap between workload-specific and generalized IMC designs. The framework is evaluated on both RRAM- and SRAM-based IMC architectures, demonstrating strong robustness and adaptability across diverse design scenarios. Compared to baseline methods, the optimized designs achieve energy-delay-area product (EDAP) reductions of up to 76.2% and 95.5% when optimizing across a small set (4 workloads) and a large set (9 workloads), respectively. The source code of the framework is available at https://github.com/OlgaKrestinskaya/JointHardwareWorkloadOptimizationIMC.