Hardware-Aware

Follow this topic Ask about this topic

Momentum

1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 6Week of Sep 21

Latest papers 24

Sep 28, 2026cs.LG

Hardware-Aware Features for CUTLASS Kernel Selection

GPU libraries such as CUTLASS expose tens of thousands of semantically equivalent kernels for a single operation, making exhaustive autotuning expensive and execution-free selection difficult. Existing analytical selectors require hand-designed performance rules, while learned selectors operate on raw configuration parameters and must infer hardware consequences from data. We introduce a hardware-aware representation for CUTLASS kernel selection that augments candidate configurations with statically computable estimates of induced hardware behavior. We construct a dataset of 4.9 million CUTLASS kernels and train gradient-boosted and neural learning-to-rank models to rank candidates within each problem. On held-out exhaustive evaluation problems, hardware-aware representations reduce selection regret by up to 40% relative to structural baselines and 64.2% relative to NVIDIA's matrix-multiply heuristics. We further evaluate data-efficient cross-precision and epilogue-fusion transfer within CUTLASS GEMM, showing that explicitly representing candidate-induced hardware behavior provides a useful inductive bias for learned kernel selection.

Shriram Chandran, Dominic Rinderer, Yakup Budanaz +3

ETH Zurich Zurich, Switzerland

Cuda Hardware-Aware

Sep 14, 2026cs.LG

Beyond Noise: Understanding and Overcoming Temperature Effects in Analog DNN Inference

The energy efficiency of analog computing makes it one of the most promising candidates for deploying resource-intensive machine learning workloads on constrained platforms such as mobile and embedded devices. However, analog accelerators are inherently susceptible to noise and non-idealities arising from physical component variations, whose behavior is further sensitive to environmental factors. These effects can significantly degrade inference accuracy. In this work, we conduct a comprehensive experimental study on a representative example of analog hardware to investigate the impact of temperature. We first characterize the behavior of stochastic and systematic non-idealities across a range of operating temperatures. Following this, we compare a set of simulation-based and hardware-based mitigation strategies aimed at improving robustness against temperature-induced performance degradation. Our results suggest that temperature-induced degradation is driven primarily by systematic non-idealities rather than stochastic noise alone. Noise-aware training improves robustness, while hardware-in-the-loop training and temperature-aware calibration provide the strongest accuracy retention across varying thermal conditions.

Niklas Summ, Xiao Wang, Hendrik Borras +2

Hardware and Artificial Intelligence Lab, Institute of Computer Engineering, Heidelberg University, Germany

Neural Network Inference Hardware-Aware Noise-Aware Multiplicative Gamma Noise

Aug 13, 2026cs.AI

NAS-Driven Hardware Accelerator Exploration for Edge AI and Quantization Effects on the Pareto Space

Edge AI deployment demands neural architectures that are simultaneously accurate, computationally efficient, and hardware-deployable - a challenge addressed by hardware-aware Neural Architecture Search (NAS). While recent works incorporate quantization directly into the NAS loop, these approaches expand search complexity and tightly couple architecture and quantization design. The simpler post-search quantization strategy has received little analytical attention: the effects of Post-Training Quantization (PTQ) on the NAS-discovered Pareto structure remain uncharacterised, and no framework combines quantized architecture mapping onto reconfigurable accelerators with automated hardware exploration. This paper addresses both gaps. First, a three-stage pipeline is proposed: a hardware-agnostic Pareto rank surrogate frontend on NAS-Bench-201, a quantization bridge with Pareto-aware filtering and feedback control, and an evolutionary Domain Space Exploration (DSE) backend on CGRA4ML for optimal hardware mapping. Second, an empirical study characterises how INT4 PTQ perturbs the NAS-Bench-201 Pareto space through formal stability metrics on ground-truth data for all 15,625 architectures, and demonstrates that an FP32 zero-shot surrogate outperforms a dedicated INT4-trained surrogate in Pareto space coverage across two standard search strategies.

Eleftherios Mylonas, Angelos Kouprizas, Michael Birbas +1

Dept. of ECE University of Patras Patras, Greece

Neural Architecture Search Hardware-Aware Pareto Frontier Machine Learning Surrogates Artificial Intelligence Systems

Aug 11, 2026eess.IV

Hardware-Aware Deployment of Joint SAR Compression and Despeckling on FPGA

Next-generation Synthetic Aperture Radar (SAR) missions will generate data far faster than they can downlink, making onboard data reduction essential for near-real-time Earth observation. Learned Image Compression (LIC) offers better rate-distortion performance than handcrafted codecs used operationally today, and recent work shows that simultaneously despeckling and compressing SAR imagery enables better representation capacity while unlocking higher compression rates. These methods, however, have yet to be confronted with the strict power, compute, and operational constraints of spaceborne systems. In this work, we bridge this gap by deploying a joint SAR Despeckling and Data Compression (DDC) framework on an embedded ZCU102 FPGA-based platform, introducing model adaptations that respect the accelerator's fixed-point arithmetic and limited set of supported operations. We evaluate four model topologies across precision levels and across CPU, GPU, and FPGA platforms, revealing several findings with direct design implications. We find that replacing conventional GDN activation functions with plain ReLU improves quality on SAR, suggesting that design principles established for compression of natural images do not necessarily transfer to SAR imagery. In addition, we demonstrate that residual blocks offer little representational benefit for ten times the compute, and show that the FPGA is the most energy-efficient of the platforms tested. Together, these results set a functioning edge deployment workflow and an evidence-based starting point for onboard SAR compression. The code is available at https://github.com/CedricLeon/SAR_DDC_FPGA.

Cédric Léonard, Francescopaolo Sica, Martin Schulz

Technical University of Munich, Munich, Germany · Remote Sensing Technology Institute, German Aerospace Center (DLR), Weßling, Germany · Department of Aerospace Engineering, University of the Bundeswehr Munich, Neubiberg, Germany

Synthetic Aperture Radar Resource-Constrained Edge Devices Rate-Distortion Theory Field-Programmable Gate Arrays Speckle Hardware-Aware

Jul 28, 2026cs.LG

Recursive transformers for semiconductor thermo-mechanical reliability

Transformer-based surrogate models are increasingly used to replace expensive first-principles simulation in engineering design. But conventional transformer architectures are often over parameterized for the small, low-dimensional datasets typical of engineering design spaces, where large simulation data is expensive to generate. Under these conditions, excess parameter capacity leads to overfitting rather than improved accuracy, while also incurring unnecessary memory and compute overhead. This motivates a shift towards architectures that focus on additional compute rather than additional learnable parameters. This paper presents a hardware-aware evaluation of three recursive transformer paradigms for surrogate thermo-mechanical analysis of advanced packages: a)Tiny Recursive Model, b) our proposed Depth Recursive transformer, c) and a simple recursive transformer. We systematically compare their predictive performance (Recall, Mean Reciprocal Rank), parameter count, computational complexity (FLOPs), providing practical design guidelines for selecting recursive transformer architectures under resource-constrained scenarios. We validate this principle on two low-dimensional engineering prediction tasks: 1) thermo-mechanical reliability analysis of advanced semiconductor packages, where stress and warpage from thermal cycling must be evaluated repeatedly across a design-of-experiments sweep under costly finite element analysis (FEA). 2) Laplace PDE iterative numerical solver for capacitance field. Overall, recursive weight-sharing transformers provide an effective and generalizable trade-off between prediction accuracy, parameter efficiency, and computational cost for small data engineering surrogate modeling.

Kart-leong Lim

Institute of Microelectronics (IME), Agency for Science, Technology and Research (A*STAR), Singapore

Energy-Based Transformer Surrogate Models Transformer Architectures Semiconductor Manufacturing First-Principles Calculation Hardware-Aware

Jul 22, 2026cs.LG

Self-organizing Architecture of Receptron Units: a Hardware-Aware Framework for Edge Intelligence

The growing demand for intelligent processing at the edge of IoT networks is constrained by the severe computational and memory limitations of microcontroller units, which render impractical conventional deep learning approaches. We propose a neuromorphicinspired classifier based on the Receptron model, a single-unit architecture capable of implementing non-linearly separable decision boundaries, without resorting to multi-layer networks. The model is designed for direct deployment on mid-range MCUs, while supporting continuous on-device adaptation. Experimental evaluation on basic dataset benchmarks yields cross-validated accuracies compatible with standard machine learning method baselines. These results position the Receptron as a viable and interpretable alternative for resource-constrained neuromorphic edge systems operating in dynamic, non-stationary environments.

Stefano Radice, Ludovico Casaccia, Riccaro Emanuele Beccalli +2

Department of Physics “Aldo Pontremoli” and CIMaINa University of Milano Milano, Italy

Neuromorphic Computing Hardware-Aware Multilayer Perceptrons Internet Of Thing

Jul 13, 2026cs.LG

HiFi-LLP: High-Fidelity, Low-Cost Latency Predictors with Confidence for Robust HW-NAS

With deep neural networks (DNNs) increasingly deployed on edge devices, hardware (HW)-aware optimization techniques--such as HW-aware compression and HW-aware neural architecture search (HW-NAS)--have become essential. These methods rely on real feedback from the target hardware to tailor DNN architectures for efficient deployment. While the search can be parallelized, latency measurements via hardware-in-the-loop (HIL) remain a bottleneck due to their sequential nature. Recent approaches use latency predictors to replace costly HIL feedback, but challenges persist: (1) platform-specific predictors often require tens of thousands of samples, and (2) inaccurate predictions can mislead the NAS process. To address this, we introduce HiFi-LLP, a high-fidelity, low-cost latency predictor based on graph attention networks, augmented with a confidence metric. HiFi-LLP outperforms prior platform-specific predictors by up to 9 percentage points (p.p.) in the 10% accuracy bound and achieves a Spearman's rank correlation of up to 0.996 across six devices in the LatBench dataset. We further propose a hybrid NAS framework that routes low-confidence predictions to HIL, achieving up to 8.6

\times

speedup compared to typical NAS while maintaining a competitive Pareto front.

Shambhavi Balamuthu Sampath, Behzad Shomali, Nael Fasfous +7

Technical University of Munich · BMW Group · University of Bonn

Hardware-Aware Neural Architecture Search High-Fidelity Human-In-The-Loop Data Compression Methods

Jul 7, 2026eess.SP

A Hardware-Aware Open-Source Framework for Design Space Exploration of Mixed-Signal Spiking Neural Networks

Energy-efficient neuromorphic computing at the edge requires simulation tools that can capture the non-ideal behavior of mixed-signal spiking neural network (SNN) hardware while supporting system-level design exploration. This work presents an open-source hardware-aware simulation framework for mixed-signal SNNs that enables comparative analysis across neuron, synapse and architecture choices. The framework supports multiple neuron models, including Leaky Integrate-and-Fire (LIF), Hodgkin-Huxley (HH) and Axon-Hillock (AH), together with non-volatile analog synapses based on floating-gate transistors and ReRAM devices. By incorporating device-level nonlinearities directly into PyTorch-based training and inference, the tool enables optimization of physical synaptic parameters rather than idealized abstract weights. The framework is evaluated on standard neuromorphic benchmarks, including N-MNIST, DVS Gesture and Spiking Heidelberg Digits (SHD). For each model dataset configuration, it reports classification accuracy together with hardware-oriented metrics such as silicon area, power consumption and quantization sensitivity. These capabilities enable cross-layer design space exploration and help identify neuron-synapse configurations that best satisfy application-specific constraints on accuracy, energy efficiency, area and hardware fidelity.

Sayma Nowshin Chowdhury, Vineeta Nair, Taseen Forhad +3

Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA · Hewlett Packard Labs, USA · Lockheed Martin, USA

Spiking Neural Networks Neuromorphic Computing Synaptic Plasticity Hardware-Aware

Jul 2, 2026cs.AI

Hawk: Harnessing Hardware-Aware Knowledge for High-Performance NPU Kernel Generation

Developing high-performance kernels for Neural Processing Units (NPUs) is a critical industry bottleneck, requiring developers to manually navigate implicit hardware constraints and strict memory hierarchies. While large language models offer immense automation potential, they fail catastrophically on NPUs due to a fundamental lack of hardware-specific priors. Naively transplanting code snippets from similar NPU kernels may pass the compiler, but it consistently triggers runtime crashes and performance degradation by blindly violating underlying hardware constraints. To overcome this, we introduce Hawk, a training-free framework that harnesses hardware-aware knowledge through three core modules: (1) Run-Time Knowledge Synthesis Module, which employs a Triple-Part Executable Knowledge Representation to inherently couple the error context with executable semantics; (2) Bottleneck-Aware Knowledge Retrieval Module, which implements a 2D-Retrieval paradigm to project queries into orthogonal syntactic and hardware-aligned semantic spaces; and (3) Effect-Driven Knowledge Distillation Module, which leverages LLM-driven semantic arbitration to continuously distill the knowledge by pruning errors and consolidating redundancies based on the empirical execution feedback. Extensive evaluations on real-world NPU workloads demonstrate that Hawk elevates generation accuracy from 49.4% to 80.0%, while achieving up to a 2.2x execution speedup over state-of-the-art baselines.

Junyi Wen, Ruiyan Zhuang, Yongjia Xu +7

Sun Yat-sen University, Zhuhai, China · Greater Bay Area National Technology Innovation Center, Guangzhou, China · Peng Cheng Laboratory, Shenzhen, China

Graphics Processing Unit Kernels Neural Processing Units Hardware-Aware Knowledge Distillation

Jun 22, 2026cs.LG

Efficient Network Inference via Hardware-Aware Architecture Search, Model Pruning & Quantization

Embedded global navigation satellite system (GNSS) interference monitoring requires fast and memory-efficient inference to process large volumes of raw in-phase and quadrature (IQ) samples in real time. At the same time, increasingly expressive deep neural networks (DNNs) are needed for robust interference classification and characterization across diverse signal conditions. This creates a fundamental tension between predictive performance and deployability on resource-constrained hardware. In this paper, we investigate efficient network inference for GNSS interference characterization using iterative structured pruning, post-training static quantization, and hardware-aware zero-shot neural architecture search (NAS). Starting from MCUNet as a compact baseline, we analyze how model compression and automated architecture optimization affect model size, computational complexity, and memory usage while maintaining task performance. Experiments on a GNSS interference dataset, covering both classification and generalized characterization, show the benefits of combining compression and hardware-aware design for embedded deployment. Our results provide practical guidance for developing compact machine learning (ML) models for real-time GNSS interference monitoring on embedded platforms (iMXRT1062 MCU, Raspberry Pi Zero 2W, and Raspberry Pi 5).

Lucas Heublein, Mark Deutel, Axel Plinge +1

Fraunhofer Institute for Integrated Circuits IIS, 90411 Nürnberg, Germany

Neural Network Inference Hardware-Aware Mixed-Precision Quantization Global Navigation Satellite System Interference

Jun 15, 2026cs.LG

An affordable hardware-aware neural architecture search for deploying convolutional neural networks on ultra-low-power computing platforms

Hardware-aware neural architecture search (HW-NAS) allows the integration of Convolutional Neural Networks (CNNs) in microcontrollers devices by automatically designing neural architectures that can fit prearranged hardware constraints. However, state-of-the-art HW-NAS target high-performance microcontrollers, whose power consumption does not meet sensing nodes requirements. This work presents a HW-NAS generating tiny CNNs that can run on ultra-low-power microcontrollers, featuring a lightweight search procedure enabling its execution even on embedded devices. Empirical results on three well-known benchmarks for tiny computer vision proved that the proposed HW-NAS was able to generate tiny CNNs while preserving state-of-the-art classification accuracy.

Andrea Mattia Garavagno, Edoardo Ragusa, Antonio Frisoli +1

Dept. of Electrical, Electronic, Telecommunications Engineering and Naval Architecture (DITEN), University of Genoa, Genoa, Italy · Department of Excellence in Robotics & AI and Institute of Mechanical Intelligence, Scuola Superiore Sant’Anna, 56127 Pisa, Italy

Neural Architecture Search Hardware-Aware Cortex Convolutional Neural Networks

Jun 12, 2026cs.AR

Running hardware-aware neural architecture search on embedded devices under 512MB of RAM

This document proposes a novel approach to hardware-aware neural architecture search (HW NAS) that considers the resources available on the computing platform running it, enabling its execution on various embedded devices. The presented HW NAS produces tiny convolutional neural networks (CNNs) targeting low-end microcontroller units (MCUs), typically involved in the Internet of Things (IoT) or wearable robotics, opening new use cases. A gateway could run it to tailor CNNs' architecture on the acquired data without using external servers, ensuring privacy. The proposed technique achieves state-of-the-art results in the human-recognition tasks on the Visual Wake Word dataset, a standard TinyML benchmark, on several embedded devices.

Andrea Mattia Garavagno, Edoardo Ragusa, Paolo Gastaldo +1

Department of Electrical, Electronic, Telecommunication Engineering and Naval Architecture, DITEN University of Genoa, Genoa 16145, Italy · Department of Excellence in Robotics & AI, Institute of Mechanical Intelligence, Scuola Superiore Sant’Anna, Pisa 56127, Italy

Neural Architecture Search Hardware-Aware Neural Network Inference Device

Jun 8, 2026cs.CL

AGENTSERVESIM: A Hardware-aware Simulator for Multi-Turn LLM Agent Serving

Multi-turn LLM agents interleave model calls with external tool invocations, shifting serving from stateless request processing to stateful program execution. Serving these workloads requires scheduling, KV-cache management, and routing policies that use program-level context, including turn dependencies, tool-induced gaps, and reusable KV state. Evaluating such policies directly on real systems is costly, since each design point may require dedicated accelerator time across arrival rates, model scales, serving-instance counts, and memory hierarchies. Simulation offers a scalable alternative, but existing LLM serving simulators target stateless request-level workloads and therefore omit the core dynamics of agent serving: multi-turn program execution, cross-turn cache locality, and KV-cache residency during tool gaps. We present AGENTSERVESIM, a hardware-aware simulator for multi-turn LLM agent serving. AGENTSERVESIM evaluates serving policies at program granularity through composable modules: a Program Orchestrator preserves program identity and turn order, a Tool Simulator materializes tool-induced gaps, a Session-Aware Router maintains program-to-instance affinity for cache-aware dispatch, and a KV Residency Model tracks policy-defined KV placement across HBM, host DRAM/CXL, and eviction. Across real serving deployments and hardware configurations, AGENTSERVESIM reproduces real-system behavior within 6% error across key performance metrics while running entirely on commodity CPUs. These results show that AGENTSERVESIM enables controlled, repeatable exploration of agent-serving policies without requiring exhaustive deployment on costly accelerators.

Rakibul Hasan Rajib, Mengxin Zheng, Qian Lou

University of Central Florida

Large Language Model Agents Hardware-Aware

Jun 4, 2026quant-ph

Hardware-aware Low-latency Quantum Compilation with Data-driven Lightweight Error Detection for Early Fault-Tolerant Systems

Noisy intermediate-scale quantum (NISQ) processors are entering an early fault-tolerance regime where full quantum error correction carries prohibitive resource costs, yet lightweight error detection can meaningfully improve algorithmic success rates. Existing compilation and error-detection toolchains treat these concerns in isolation, with no principled way to balance detection overhead against success probability under latency constraints. We present an integrated hardware-aware compilation and data-driven quantum error-detection (QED) framework that jointly optimises qubit mapping, SWAP insertion, and syndrome-schedule placement via a noise-weighted cost function and a learned multi-objective scheduler. Simulation experiments on an HPC cluster using GPU-accelerated density-matrix simulation (NVIDIA cuQuantum SDK) across VQE, phase-estimation, and Grover benchmarks, three noise profiles, and circuit sizes of 6-20 qubits (depths 10-160), show that joint co-design raises algorithmic success probability by up to 68 percent (95 percent CI: 60 percent to 76 percent) over SABRE on an 8-qubit VQE instance with post-selection.

Sumit Chongder

Indian Institute of Technology Jodhpur · Inter-Disciplinary Research Platform, Quantum Information and Computation, Indian Institute of Technology Jodhpur, Jodhpur, Rajasthan 342037, India

Quantum Error Correction Hardware-Aware Fault Tolerance

Jun 2, 2026quant-ph

Scalable On-Hardware Training of Quantum Neural Networks and Application to Clinical Data Imputation

Training quantum neural networks (QNNs) on quantum hardware is currently bottlenecked by the cost of gradient estimation: standard parameter-shift methods require a number of circuit evaluations that grows quadratically with the number of trainable parameters, making hardware-based optimisation impractical beyond small system sizes. In this work, we introduce a training framework that reduces this cost to logarithmic in the number of qubits, making gradient-based QNN optimisation feasible on near-term hardware at increasing scales. Our framework combines three co-designed ingredients: (i) a structured, subspace-preserving Butterfly circuit architecture with

O(n \log n)

parameters and logarithmic depth; (ii) a layer-wise training strategy that confines on-hardware optimisation to one small, well-structured layer at a time; and (iii) a parallelised parameter-shift rule that exploits the commuting structure within each Butterfly layer to extract all gradients in a constant number of circuit executions. Together these reduce the number of distinct circuit evaluations per optimisation step from

O(n^2)

to

O(\log n)

. We validate the framework on clinical data imputation using the MIMIC-III electronic health record dataset, a demanding benchmark sensitive to optimisation instability and model variance. Hybrid classical-quantum models are trained directly on IonQ Forte Enterprise trapped-ion hardware at 16 qubits without performance degradation relative to ideal or noisy simulation and via tensor-network simulation at 32 qubits, with 32-qubit inference executed on hardware. The resulting models match or exceed strong classical neural baselines in downstream patient survival prediction while exhibiting reduced variance across runs, demonstrating that the proposed framework enables practical, scalable QNN training under realistic hardware constraints.

Natansh Mathur, Panagiotis Kl. Barkoutsos, Masako Yamada +2

IRIF, CNRS and Université Paris Cité and QC Ware, France · QC Ware, France · IonQ +2

Quantum Neural Networks Hardware-Aware Imputation

May 30, 2026cs.DC

ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Serving

In distributed Mixture-of-Experts (MoE) inference, input-dependent token routing interacts with GPU performance variability to create persistent stragglers under synchronized execution, where the slowest GPU determines layer latency. This performance variability is inherent to modern accelerators: manufacturing variation, power limits, and thermal conditions introduce measurable execution-time differences across nominally identical GPUs. The core challenge is that MoE execution-time imbalance arises from the interaction of workload skew and hardware asymmetry. Token routing produces uneven and layer-varying expert loads, while GPU throughput depends on device-specific operating characteristics and workload intensity. Prior work mitigates routing skew but assumes homogeneous hardware, optimizing token balance rather than execution latency. As a result, even balanced token assignments can leave hardware-induced stragglers unaddressed. Thus, we propose Variability-Informed Binning of Experts (ViBE), a hardware-aware expert placement framework that minimizes execution-time imbalance across GPUs. ViBE combines per-GPU performance modeling with expert activation profiling to assign high-load experts to faster devices and low-load experts to slower ones, reducing layer-level stragglers without modifying model semantics or hardware. Because both workload characteristics and effective GPU throughput can shift across serving conditions, ViBE supports lightweight recalibration under workload/performance drift to refresh its routing and performance estimates when needed. Results show that ViBE consistently reduces execution-time imbalance and improves SLO attainment by 14%, while lowering P90 TTFT by up to 45%. We further show that the impact of hardware variability increases at scale, making variability-aware placement important for efficient, high-utilization LLM serving.

Seokjin Go, Marko Scrbak, Ephrem Wu +2

Georgia Institute of Technology Atlanta, GA, USA · Advanced Micro Devices, Inc. Austin, TX, USA · Advanced Micro Devices, Inc. Santa Clara, CA, USA +1

Mixture-Of-Expert Inference Inference Workloads Mixture-Of-Experts Workload Graphics Processing Unit Resources Hardware-Aware+1

May 14, 2026cs.LG

A Hardware-Aware, Per-Layer Methodology for Post-Training Quantization of Large Language Models

Scaled Outer Product (SOP) is a post-training quantization methodology for large language model weights, designed to deliver near-lossless fidelity at 4.5--6 bits per weight on hardware with per-layer LUT decode. The methodology combines per-layer search of fixed and dynamic codebook pairs selected by a per-block selection bit, signed per-block scales, activation-weighted cosine selection, and multiple-choice knapsack promotion of sensitive layers with outlier and sparse-residual correction. Fixed codebooks include NF4, BOF4, Split87, and SH4; per-layer optimized codebooks (DD4) are hosted in LUT SRAM. A new hardware-efficient LUT output format (HIF) is proposed to improve performance, energy, and cost. Across six open model families, the recommended FP6 operating point (E2M3sUE4M4, 6.5 bpw) achieves lower weight reconstruction error than the conventional per-layer-POT FP8 baseline (E4M3, 8.0 bpw) at 1.5 bpw lower storage cost, demonstrating that block-scaled small atoms with carefully chosen scale precision can replace conventionally-deployed FP8. Full evaluation across the 4.5--6 bpw range, including layer promotion and sparse residual correction, is reported in a companion paper.

Earl Killian

Post-Training Quantization Model Weights Hardware-Aware

May 10, 2026cs.LG

A Controlled Diagnostic Study of Hardware-Induced Distortions in Hardware-Aware Training

Hardware-aware training (HAT) is widely used to improve the robustness of neural networks on non-ideal AI accelerators, such as analog in-memory computing (IMC) systems. However, not all hardware-induced distortions are equally compensable by training. This paper presents a diagnostic framework that models hardware non-idealities as structured perturbations of the forward operator and evaluates their compatibility with gradient-based optimization. We analyze six representative perturbation classes--read noise, variability, drift, stuck-at faults, IR-drop, and ADC discretization--and identify three key diagnostics: gradient expectation consistency, bounded gradient variance, and non-degenerate sensitivity. Our results show a clear separation between perturbations that can be compensated by HAT and those that consistently break optimization. This provides practical guidance for hardware-software co-design, clarifying which non-idealities can be addressed at the training level and which require circuit-, architecture-, or calibration-level mitigation. This study should be interpreted as a controlled empirical analysis under vanilla forward-perturbation HAT, rather than as a universal theory of hardware-aware training.

Yunxuan Fang, Xinhe Wang

Beihang University Beijing, China

Hardware-Aware In-Memory Computing

May 9, 2026cs.LG

Non-Monotonic Latency in Apple MPS Decoding: KV Cache Interactions and Execution Regimes

Autoregressive inference is typically assumed to scale predictably with decoding length, with latency increasing smoothly as generated sequence length grows. In this work, we identify unexpected non-monotonic latency behavior in the Apple MPS backend, where latency changes abruptly across nearby decoding configurations during transformer decoding. Using multiple model families (GPT-2, BLOOM, and OPT), we observe latency spikes of up to 21x within specific decoding-budget intervals, followed by recovery at neighboring configurations. Controlled experiments show that these anomalies originate primarily during the decode phase rather than prefill, are not explained by memory pressure alone, and remain absent on CPU and NVIDIA CUDA backends under identical conditions. We further show that key-value (KV) cache interacts strongly with these pathological execution regimes: KV caching remains beneficial overall, but its practical speedup collapses sharply within anomalous configurations, while cache-disabled decoding still exhibits residual non-monotonic behavior. These findings suggest that autoregressive decoding on MPS enters discrete execution regimes that are not captured by coarse-grained benchmarking, highlighting the importance of hardware-aware evaluation for long-context inference.

Willy Fitra Hendria

1Independent Researcher, Seoul, South Korea.

Autoregressive Decoding Decoding Autoregressive Model Key-Value Cache Cache Hardware-Aware

May 3, 2026cs.LG

HERCULES: Hardware-Efficient, Robust, Continual Learning Neural Architecture Search

Neural Architecture Search (NAS) has emerged as a powerful framework for automatically discovering neural architectures that balance accuracy and efficiency. However, as AI transitions from static benchmarks to real-world deployment, the traditional focus on hardware-aware efficiency is no longer sufficient. We observe that modern NAS methods, especially those that target edge AI, are evolving to address a triple objective: Efficiency, Robustness, and Continual Learning. While efficiency ensures feasibility in resource-constrained environments, robustness guarantees reliability under environmental variabilities, and continual learning enables adaptation to sequential tasks without catastrophic forgetting. We propose a taxonomy of NAS approaches through this triple lens, distinguishing between methods targeting resource optimization, environmental resilience, and architectural plasticity. This unified perspective reveals that these axes, though often studied in isolation, are mutually reinforcing. Building on this taxonomy, we map the current landscape of these NAS methods into a new framework called Hardware-Efficient, Robust, and ContinUal LEarning Search (HERCULES). We define the desiderata, the twelve labours of HERCULES, addressing the non-trivial challenge of balancing an adequate search-space exploration with the immense computational costs of a multi-objective NAS, accounting for these crucial objectives of current AI systems. By identifying critical gaps in existing research, this survey outlines a roadmap toward integrated algorithmic, architectural, and hardware-software co-design for truly deployable, lifelong-learning AI systems.

Matteo Gambella, Fabrizio Pittorino, Manuel Roveri

Dipartimento di Elettronica, Informazione e Bioingegneria

Neural Architecture Search Hardware-Aware Continual Learning Plasticity

Apr 29, 2026cs.CL

Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference

We present tensor and sequence parallelism (TSP), a parallel execution strategy that folds tensor parallelism and sequence parallelism onto a single device axis. In conventional multi-dimensional parallelism layouts, tensor parallelism (TP) shards model weights while sequence parallelism (SP) shards tokens, reducing per-device parameter or activation memory, respectively. Traditionally, each scheme is assigned its own mesh dimension. TSP instead assigns each rank both a weight shard and a sequence shard, reducing both parameter and activation memory along the same device axis. We implement this design with two runtime schedules. For attention, ranks iterate over broadcast parameter shards and reconstruct context through a sequence-wise key/value exchange. For gated MLPs, weight shards circulate in a ring while partial outputs accumulate locally. By sharding both weights and activations across the same devices, TSP trades additional communication volume for reduced memory overhead. We provide a theoretical communication and memory analysis, describe our implementation of TSP attention and gated MLP blocks, and benchmark TSP against TP, SP, and TP+SP. These results position TSP as a hardware-aware alternative for long-context and memory-constrained model training, and as a viable axis of parallelism in concert with existing parallelism schemes such as pipeline and expert parallelism for dense and mixture-of-expert models.

Vasu Shyam, Anna Golubeva, Quentin Anthony

Zyphra

Cpu-Gpu Hybrid Designs Shard Transformer Architectures Hardware-Aware

Apr 27, 2026cs.CV

Deployment-Aligned Low-Precision Neural Architecture Search for Spaceborne Edge AI

Designing deep networks that meet strict latency and accuracy constraints on edge accelerators increasingly relies on hardware-aware optimization, including neural architecture search (NAS) guided by device-level metrics. Yet most hardware-aware NAS pipelines still optimize architectures under full-precision assumptions and apply low-precision adaptation only after the search, leading to a mismatch between optimization-time behavior and deployment-time execution on low-precision hardware that can substantially degrade accuracy. We address this limitation by integrating deployment-aligned low-precision training directly into hardware-aware NAS. Candidate architectures are exposed to FP16 numerical constraints during fine-tuning and evaluation, enabling joint optimization of architectural efficiency and numerical robustness without modifying the search space or evolutionary strategy. We evaluate the proposed framework on vessel segmentation for spaceborne maritime monitoring, targeting the Intel Movidius Myriad X Visual Processing Unit (VPU). While post-training precision conversion reduces on-device performance from 0.85 to 0.78 mIoU, deployment-aligned low-precision training achieves 0.826 mIoU on-device for the same architecture (95,791 parameters), recovering approximately two-thirds of deployment-induced accuracy gap without increasing model complexity. These results demonstrate that incorporating deployment-consistent numerical constraints into hardware-aware NAS substantially improves robustness and alignment between optimization and deployment for resource-constrained edge Artificial Intelligence (AI).

Parampuneet Kaur Thind, Vaibhav Katturu, Giacomo Zema +1

1Sapienza University of Rome · 2 Φ-lab, ESA ESRIN · 4Argotec Srl +1

Neural Architecture Search Intelligent Edge Hardware-Aware Nvidia

Mar 12, 2026cs.DC

KernelFoundry: Hardware-aware evolutionary GPU kernel optimization

GPU kernel optimization challenges LLMs beyond standard coding tasks, as it requires an understanding of hardware architecture, parallel computing optimization strategies, and profiling outputs. However, most existing approaches leveraging LLMs for kernel generation apply standard prompting and feedback loops, considering hardware only through profiling feedback. We introduce KernelFoundry, an evolutionary framework that efficiently explores the space of GPU kernels through (1) MAP-Elites quality diversity search with kernel-specific behavioral dimensions to sustain exploration; (2) meta-prompt evolution that co-evolves prompts with kernels to uncover task-specific optimization strategies, and (3) a template-based parameter optimization approach to tune kernels to inputs and hardware. We evaluate this framework on Kernel-Bench, robust-kbench and custom tasks, generating SYCL kernels as a cross-platform GPU programming paradigm, and CUDA kernels for comparison to prior work. Our approach consistently outperforms the baseline methods and achieves an average speedup of 2.3 on KernelBench for SYCL. Moreover, KernelFoundry is implemented as a distributed framework with remote access to diverse hardware, allowing quick benchmarking and featuring a flexible user input layer to support kernel generation for a wide range of real use cases beyond benchmarking.

Nina Wiedemann, Quentin Leboutet, Michael Paulitsch +2

Intel Corporation

Graphics Processing Unit Kernels Hardware-Aware Code Optimization Hardware Efficiency

Feb 28, 2026cs.MM

Amanous: Distribution-Switching for Superhuman Piano Density on Disklavier

A player piano can strike more keys, across more of the register, and faster than any pianist can reach. Three traditions dominate composition in that region, namely Nancarrow's tempo canons, Xenakis' stochastic distributions, and L-system grammars. They have developed in isolation, and none of them accounts for the instrument itself. A Disklavier does not answer instantly, because a loud note reaches the string sooner than a soft one, so music written as if the mechanism were transparent arrives distorted. We present Amanous, a hardware-aware composition system for the Yamaha Disklavier that unifies the three traditions through distribution-switching, in which each grammar symbol selects an entire distributional regime rather than adjusting parameters within a fixed one. A four-layer pipeline carries a symbol from grammar to actuation-ready MIDI. An L-system fixes the macro-form, each symbol is mapped to distributions and a tempo-canon ratio, events are sampled and time-scaled, and a hardware layer pre-compensates velocity-dependent latency and enforces the key-reset time. Convergence points close a feedback loop, letting the music's own temporal structure trigger the next switch. Sections from different symbols remain statistically separable at the output, and ablating the L-system, the tempo canon, or the hardware compensation each degrades a distinct property. Under the modelled latency curve, pre-compensation takes the mean onset error from roughly 18 ms to below the instrument's 1 ms scanning resolution. Structured and random textures are most separable near 25 notes/s, and melodic metrics lose their discriminative power over the 40-100 notes/s band, above which the difference is consistent with distributional content rather than melodic order. All results are computational, nothing was played or measured on an instrument, and a psychoacoustic protocol is proposed for future work.

Joonhyung Bae

Graduate School of Culture Technology KAIST (Korea Advanced Institute of Science and Technology) Daejeon, South Korea

Instrument Hardware-Aware Autonomous Hard Drive Disassembly Ridge