Efficient Neural Network Inference

Follow this topic Ask about this topic

Momentum

39 papers in the last four weeks, up 225% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 314

Jun 13, 2026eess.SP

ECG-LDC: A Hardware-Efficient Low-Dimensional Computing Framework for ECG Arrhythmia Classification

Continuous cardiac monitoring in wearable devices demands classifiers that are simultaneously accurate, energy-efficient, and deployable on resource-constrained hardware. While deep neural network approaches have demonstrated high classification accuracy for electrocardiogram (ECG) arrhythmia detection, their substantial parameter counts and reliance on multiply-accumulate-intensive operations make them impractical for low-cost edge platforms. In this work, we propose ECG-LDC, a hardware-software co-design framework that adapts Low-Dimensional Computing (LDC) for real-time ECG arrhythmia classification. ECG-LDC employs a dual-encoder architecture with dedicated value and feature codebooks to independently encode morphological waveform features and RR-interval temporal features, enabling effective capture of both intra-beat and inter-beat cardiac dynamics. The framework encompasses data preprocessing, model training, and a hardware accelerator architecture prototyped on the Pynq-Z2 platform. Implemented using binary representations and XOR/XNOR-based operations, ECG-LDC achieves

97.18\%

accuracy with a memory footprint of only

3.86\ \text{kB}

. ECG-LDC sacrifices approximately

1.8\%

accuracy versus SOTA TinyML classifiers but achieves

11

~

570\times

reduction in memory usage; among FPGA-based five-class arrhythmia classifiers, it delivers the highest accuracy with up to

2.4\times

fewer LUTs and zero DSP block utilization, affirming its suitability for real-time arrhythmia detection on resource-constrained wearable platforms.

Anh Tran, Khanh Tran, Cuong Do

School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA 19104, USA · VinUni–Illinois Smart Health Center, VinUniversity, Hanoi 100000, Vietnam · College of Engineering and Computer Science, VinUniversity, Hanoi 100000, Vietnam

Efficient Neural Network Inference Electrocardiogram Classification FPGA-Based Neural Network Acceleration

Jun 12, 2026cs.PL

ANEForge: Python for direct computation on the Apple Neural Engine

ANEForge is a Python package that programs the Apple Neural Engine (ANE), the fixed-function neural accelerator on every recent Apple device, directly and without CoreML. In production the engine is reachable only through CoreML, which treats it as a scheduling option: no configuration requires the ANE, and a model can silently run on the CPU or GPU instead. ANEForge compiles a lazy tensor graph, built from 58 fused operators and 19 native bridge operators, into a single ANE program. The program is dispatched through the same ANE daemon and kernel-driver stack as Apple's internal framework. Beyond inference, the package reaches the engine's native fused attention, streams int8, int4, and sparse weights, keeps decoder and optimizer state resident across steps, and runs the forward pass, backward pass, and optimizer update of training on the engine. A small fused program completes a call in about 90us, near the engine's 70us per-program dispatch floor, and a pretrained ResNet-18 forward runs end-to-end in 0.33ms. ResNet-18, a sentence encoder, and a Vision Transformer run end-to-end against framework references, and a Stable Diffusion U-Net validates its forward pass. ANEForge targets Apple Silicon under macOS 14 and later. Each release is verified against a recorded macOS and ANE-compiler version.

Spencer H. Bryngelson

School of Computational Science & Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA · Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA · George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA

Efficient Neural Network Inference AI Accelerator Inference

Jun 12, 2026cs.CV

HumP-KD: A Hybrid Uncertainty-Aware Multi-Stage Progressive Knowledge Distillation Framework for Efficient Fire Classification

Real-time fire classification systems require models that are simultaneously accurate, computationally efficient, and deployable on resource-constrained hardware. This work proposes \textbf{HumP-KD}, a Hybrid Uncertainty-aware Multi-stage Progressive Knowledge Distillation framework for efficient fire classification. Two datasets, FlameVision and Dataset-II, containing 8,600 and 31,309 images, are used. Various CNN and transformer baselines are applied under standard preprocessing, online augmentation, Gaussian noise and motion blur robustness conditions. The proposed HumP-KD model distills knowledge from two frozen heterogeneous transformer teachers, Swin-Tiny and ViT-Base, along with their Meta-MLP ensemble, into a lightweight MobileViT-S student via three tightly integrated components. Hierarchical Progressive Knowledge Distillation employs a Hierarchical Feature Builder. It generates a fused spatial attention mask to guide distillation toward discriminative regions selectively. Multi-Stage Knowledge Distillation progressively activates three distillation stages across training. On Dataset-II, HumP-KD achieves a mean F1 score of

0.9876 \pm 0.0063

across 10 independent trials, significantly outperforming the MobileViT-S baseline trained without distillation (

0.9537 \pm 0.0351

), with statistical significance confirmed by both independent t-test (

p = 0.0195

) and Wilcoxon signed-rank test (

W = 1

,

p = 0.0039

). The proposed method also demonstrates strong generalization across datasets and robustness under degraded visual conditions. The student model retains only 4.94M parameters and 19.01Mb model size, representing a

5.7\times

parameter reduction over Swin-Tiny and a

17.5\times

reduction over ViT-Base, while achieving 37.72 CPU FPS, making it suitable for real-time deployment.

Mohammed Arif Mainuddin, Najifa Tabassum, Omar Ibne Shahid +1

Electrical and Computer Engineering North South University Dhaka, Bangladesh

Efficient Neural Network Inference Cross-Architecture Knowledge Distillation Uncertainty-Aware Knowledge Distillation Multi-Teacher Knowledge Distillation Knowledge Distillation

Jun 11, 2026cs.AR

ReSCom: A Reconfigurable Spiking Neural Network Accelerator Using Stochastic Computing

Spiking Neural Networks (SNNs) provide an attractive framework for energy-efficient inference due to their event-driven computation and biologically inspired dynamics. However, efficient hardware realization of SNNs remains challenging because neuronal computations incur significant power and area costs, and uncontrolled approximate arithmetic can destabilize recurrent state updates when precision is not properly managed. To address these challenges, this paper presents ReSCom, a reconfigurable SNN accelerator that leverages stochastic computing to reduce hardware complexity while maintaining stable inference. The proposed architecture employs stochastic arithmetic for multiplication operations in neuron dynamics, while preserving exact fixed-point addition/subtraction operations. This stochastic strategy enables runtime trade-offs between accuracy, latency, and energy consumption. A unified reconfigurable neuron design supports Integrate-and-Fire (IF), Leaky Integrate-and-Fire (LIF), and Synaptic neuron models within a single hardware framework. Experimental results for MNIST inference on a Xilinx Artix-7 FPGA show that ReSCom achieves

92.80\%

classification accuracy while consuming just

0.05~\mathrm{mJ}

of operational energy per image at

100~\mathrm{MHz}

, outperforming the energy efficiency of recent state-of-the-art implementations. Furthermore, managing the stochastic bit-stream length allows explicit, dynamic control over accuracy-latency-energy trade-offs to meet target application constraints.

Ali Alipour Fereidani, Mohammad Rasoul Roshanshah, Saeed Safari

High-Performance Embedded Architecture Laboratory (HiPEAL) School of Electrical and Computer Engineering College of Engineering, University of Tehran Tehran, Iran · High-Performance Embedded Architecture Laboratory (HiPEAL) School of Electrical and Computer Engineering[cs.AR] College of Engineering, University of Tehran Tehran, Iran

Efficient Neural Network Inference Energy-Efficient ML Spiking Neural Networks FPGA-Based Neural Network Acceleration

Jun 11, 2026cs.LG

WHAR Arena: Benchmarking the State of the Art in Efficient Wearable Human Activity Recognition

Deep learning has become the dominant paradigm in Wearable Human Activity Recognition (WHAR), yet progress is obscured by a comparability crisis. Results are often reported using inconsistent datasets, custom data processing, and varying evaluation protocols, making state-of-the-art claims fragile. We address this with a large-scale, open-source benchmark that integrates 30 diverse datasets under standardized processing, unified model interfaces, and a shared cross-subject evaluation protocol. Evaluating 17 representative architectures across 4760 training runs, we jointly measure predictive performance alongside on-device latency, peak memory, and model size on an Android reference device. Our results reveal that the WHAR state of the art is distributed rather than dominated by a single architecture. While CNN-HAR achieves the highest mean macro-F1, top-performing models cluster tightly, indicating contemporary architectures have converged near a predictive performance ceiling. When accounting for deployment efficiency, compact neural models, such as TinierHAR, and classical Random Forests define the practically relevant Pareto frontier, whereas larger recurrent and hybrid models incur high hardware costs without corresponding performance gains. Consequently, while predictive performance has plateaued, substantial potential for future progress remains in optimizing deployment efficiency and improving adaptation to domain shifts. We release our full framework to support transparent reuse and extension.

Maximilian Burzer, Tobias King, Till Riedel +2

Karlsruhe Institute of Technology, Karlsruhe, Germany · IPAI Foundation gGmbH, Heilbronn, Germany

Efficient Neural Network Inference Human Activity Recognition On-Device Inference

Jun 10, 2026cs.NE

SpikeDecoder: Realizing the GPT Architecture with Spiking Neural Networks

The Transformer architecture is widely regarded as the most powerful tool for natural language processing, but due to a high number of complex operations, it inherently faces the issue of high energy consumption. To address this issue, we consider Spiking Neural Networks (SNNs), which are an energy-efficient alternative to conventional Artificial Neural Networks (ANNs) due to their naturally event-driven approach to processing information. However, this inherently makes them difficult to train. Often, many SNN-based models circumvent this issue by converting pre-trained ANNs. More recently, attempts have been made to design directly trainable SNN-based adaptations of the Transformer model structure. Although the results showed great promise, the application field was computer vision. Moreover, the proposed model incorporates only encoder blocks. In this paper, we propose SpikeDecoder, a fully SNN-based implementation of the Transformer decoder block, for applications in natural language processing. In a series of experiments, we analyze the impact of exchanging different blocks of the ANN model with spike-based alternatives to identify trade-offs and significant sources of performance loss. We further investigate the role of residual connections and the selection of SNN-compatible normalization techniques. Besides the work on the model architecture, we formulate and compare different embedding methods to project text data into spikes. Finally, we demonstrate that our proposed SNN-based decoder block reduces the theoretical energy consumption by 87% to 93% compared to the ANN baseline.

Claas Beger, Florian Walter, Alois Knoll

Chair of Robotics, Artificial Intelligence and Real-time Systems Technical University of Munich, Germany

Efficient Neural Network Inference Spiking Transformers Decoder-Only Language Models Spiking Neural Networks

Jun 10, 2026cs.CV

SpikeTAD: Spiking Neural Networks for End-to-End Temporal Action Detection

Video understanding is a crucial part of computer vision, with numerous application scenarios. With the increasing popularity of mobile devices, an increasing number of efforts are trying to deploy video understanding models on them. However, existing video understanding models are difficult to deploy due to their large size and prohibitive power consumption. Spiking Neural Networks (SNNs) have shown bioplausibility and low power advantages over Artificial Neural Networks (ANNs), especially on neuromorphic chips which are regarded as essential components of future mobile devices. However, excessively long conversion time-steps and severe performance degradation problems limit their application. To solve the problems above, we explore the application of SNNs on temporal action detection (TAD), which is an important task in video understanding, and propose the first SNN-based end-to-end TAD architecture coined as SpikeTAD. While maintaining extremely low power consumption, SpikeTAD achieves an average mAP of 67.2% in THUMOS14 and 37.42% in ActivityNet-1.3, demonstrating the feasibility of a low-power TAD model. Our code is available at https://github.com/MCG-NJU/SpikeTAD.

Min Yang, Mi Zhou, Limin Wang

State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China

Efficient Neural Network Inference Human Activity Recognition Spiking Neural Networks Temporal Action Detection Temporal Video Understanding

Jun 10, 2026cs.SD

SpAArSIST: Sparsified AASIST for Efficient and Reliable Anti-Spoofing

We present SpAArSIST, a deployment-oriented refinement of the widely used AASIST graph pooling backend for self-supervised learning (SSL) based anti-spoofing. Motivated by redundant operations in public implementations, we replace learned pooling and stack-node attention with explicit, lightweight choices: separate train and inference graph pooling ratios

(k_{\mathrm{tr}},k_{\mathrm{inf}})

, magnitude-based node scoring, and mean aggregation of graph nodes. The best overall configuration (rank 1) cuts backend compute by 20.7% (195.045M

\rightarrow

154.706M MACs) and model size by 4.1% (611.8k

\rightarrow

586.4k params), while improving out-of-domain robustness on In-the-Wild to 2.82% EER and 0.078 minDCF (from 4.64% and 0.133) and remaining competitive on ASVspoof5. We further provide a composite selection score that summarizes accuracy, calibration, and compute to support balanced deployment-oriented model choice.

Anton Firc, Vojtěch Staněk, Zbyněk Lička +2

Efficient Neural Network Inference Graph Neural Networks Audio Deepfake Detection Graph Pooling

Jun 9, 2026cs.CV

PENet+: A Lightweight Residual Transformer Framework for Efficient Image Steganalysis

Image steganalysis, the detection of hidden information embedded in digital images, is a core component of modern cybersecurity and digital forensics. Recent residual Transformer architectures, such as the Pixel-Difference-Convolution and Enhanced-Transformer-Network (PENet) [1], achieve strong detection accuracy, but their computational and memory demands hinder deployment in resource-constrained settings. We present PENet+, a lightweight steganalysis framework that preserves PENet's discriminative structure while substantially improving efficiency. Rather than redesigning or compressing the attention blocks, we retain PENet's self-attention topology for reproducibility and add a classifier-streamlining stage that progressively narrows the SPP-to-FC1 input channels (SPP: spatial pyramid pooling; FC1: first fully connected layer), yielding large reductions in parameters and FLOPs with negligible accuracy loss. We further refine the high-pass-filter (HPF) stem with an activation-aware mechanism that aggregates HPF responses early and selects a balanced SRM-Gabor top-K subset, and we replace PENet's backbone with a MobileNetV2-style inverted residual network. A balanced configuration with K=31 filters (16 Gabor + 15 SRM) matches or surpasses heavier settings at lower compute. Finally, we motivate PReLU from a steganalysis standpoint, arguing that preserving negative responses helps capture weak stego cues that ReLU suppresses. On a disjoint ALASKA2 JPEG QF90 protocol at 512x512 resolution (5,000 cover images for training, validation, and internal testing; a separate 19,000-cover evaluation set), PENet+ achieves up to 45.5% fewer parameters and about 97% fewer FLOPs than the re-evaluated PENet baseline, offering a computationally efficient direction for resource-constrained steganalysis. Device-level latency and power measurements remain future work.

Jincheol AN, Dongsu Kim, Haneol Jang +1

Department of Artificial Intelligence, Chung-Ang University, Seoul, Korea · Hanbat National University, Department of Computer Engineering, 125, Dongseo-daero, Yuseong-gu, Daejeon, 34158, Korea · 3SNUAILAB, Seoul, Korea

Efficient Neural Network Inference Hybrid CNN-Transformer Architectures Digital Image Forensics

Jun 8, 2026cs.NE

Spiking Neural Network inference on FPGAs with hls4ml

Spiking Neural Networks (SNNs) provide a naturally temporal machine-learning framework. Their neurons maintain an internal state and propagate information through discrete spikes, enabling low-latency temporal inference. Although SNNs are often associated with asynchronous neuromorphic processors, many scientific real-time inference systems rely on conventional synchronous field-programmable gate arrays (FPGAs) and high-level synthesis (HLS) workflows. In this paper we present an extension of hls4ml that enables clock-driven deployment of SNNs trained in pytorch onto FPGA firmware. We demonstrate the workflow using a dense quantised SNN trained on the Heidelberg Spiking Digits dataset where it achieves inference latencies of approximately

34μ

s. We validate the generated design through software reference comparisons, HLS C simulation, HLS synthesis, export, and Vivado synthesis reports. This work opens up the hls4ml toolkit to neuromorphic computing, allowing streamlined optimisation, synthesis, and deployment of SNN models for real-time inference.

Barry M. Dillon

ISRC, Ulster University, Derry, BT48 7JL, Northern Ireland

Efficient Neural Network Inference Spiking Neural Networks FPGA-Based Neural Network Acceleration

Jun 8, 2026cs.DC

FMplex: Model Virtualization for Serving Extensible Foundation Models

Foundation models (FMs) are increasingly used as backbones for downstream tasks across language, vision, time-series, and multimodal applications. Yet existing model-serving systems deploy each customized task as an independent model instance, thereby replicating heavyweight backbones, wasting accelerator memory, and losing opportunities to amortize batching and loading costs. This paper presents FMplex, a serving system that treats FM backbones as a virtualization substrate for deployment sharing. FMplex presents each task with a virtual foundation model (vFM), a logically private FM instance backed by a shared physical FM. This abstraction lets independently customized tasks share a backbone while preserving task-specific extensions, independent lifecycles, and task-level isolation. In addition, we propose a batch-aware fair-queueing scheduler that combines weighted task-level sharing with inter- and intra-task batching across colocated tasks. We implement a FMplex-based serving stack spanning task construction, sharing-aware deployment, and runtime execution. Across 7 FM backbones (16 variants) and 92 downstream tasks, FMplex reduces latency by up to 80% over spatial partitioning and 33.3% over best-effort co-location, while hosting up to 6x more tasks at cluster scale.

Hetvi Shastri, Pragya Sharma, Walid A. Hanafy +3

University of Massachusetts Amherst · University of California Los Angeles

Efficient Neural Network Inference Inference Acceleration

Jun 8, 2026cs.LG

Efficient Traffic Prediction at Scale: A Systematic Study of STGCN Architectural Depth

Spatio-temporal graph neural networks (STGNNs) have become the dominant approach for traffic prediction, yet their computational requirements pose challenges for practical deployment in intelligent transportation systems (ITS). While recent work has proposed efficient alternatives to STGNNs, a fundamental question remains unexplored: are these architectures themselves over-parameterised? We examine this question using the Spatio-Temporal Graph Convolutional Network (STGCN), one of the most widely adopted models in this domain. Through systematic experiments across four diverse traffic datasets, we compare 1-block, 2-block (standard), and 3-block STGCN variants. Our findings reveal that the single-block architecture achieves optimal performance for short-term prediction (10 mins) on three of four datasets, while incurring only marginal degradation (

\leq

1.8% relative error) at longer horizons. Crucially, the 2-block variant incurs 61% higher CPU inference latency and 37% lower throughput relative to 1-block -- substantial overhead for resource-constrained ITS deployment. The 3-block architecture offers no favourable tradeoff, more than doubling computational cost for

<

0.5% relative improvement. These results suggest that the default 2-block STGCN may be over-parameterised for many applications, with implications for both practitioners deploying traffic prediction systems and researchers benchmarking efficiency-focused methods.

Soban Nasir Lone, Mohamed Abouelela, Taeyoung Yu +2

Technical University of Munich Munich, Germany · The University of Queensland Brisbane, Australia

Efficient Neural Network Inference Intelligent Transportation Systems Traffic Forecasting Spatiotemporal GNNs

Jun 8, 2026cs.LG

Heterophily-Aware Adaptive Knowledge Distillation for Hypergraph Neural Networks

Hypergraph knowledge distillation aims to retain the predictive performance of a hypergraph neural network (HNN) teacher while reducing inference costs through a lightweight student model. In this work, we observe that HNNs exhibit substantially lower prediction performance on heterophilic nodes connected through semantically diverse hyperedges, indicating that the reliability of teacher knowledge varies across nodes. Motivated by this observation, we propose HADES, a heterophily-aware adaptive distillation method for hypergraph neural networks. HADES quantifies node heterophily and leverages it as an estimate of teacher reliability to modulate the transfer of teacher knowledge during distillation. Experimental results on real-world hypergraphs demonstrate that HADES consistently improves student performance across different HNN teachers and distillation objectives. In many cases, the resulting student models surpass the predictive performance of their teachers while achieving up to 12.3 times faster inference.

Joohee Cho, David Yoon Suk Kang, Yunyong Ko

Chung-Ang University · Seoul, South Korea · Chungbuk National University +1

Efficient Neural Network Inference Hypergraph Neural Networks Knowledge Distillation

Jun 7, 2026cs.LG

Sigma-Branch: Hierarchical Single-Path Network Reconstruction for Dynamic Inference with Reduced Active Parameters

Deploying deep neural networks on memory-constrained edge accelerators is bottlenecked by per-inference off-chip weight transfer rather than computation: the dense network cannot be retained on-chip, and every parameter must be loaded for every input. Existing model compression reduces this transfer only at the cost of permanent capacity loss. We propose Sigma-Branch (SigmaB), a framework that restructures a pretrained dense network into a hierarchical binary tree composed of a shared backbone, hierarchical routers, and specialized leaves. Pretrained weights are distributed across the tree via activation-based spherical k-means clustering, which jointly initializes router weights and per-branch channel allocations; soft-routing fine-tuning then aligns each leaf with its routed input subset. At inference, the resulting network executes only a single root-to-leaf path, reducing the active-parameter footprint while storing the complete dense parameter set in memory. Across CIFAR-100 / ResNet-50, ImageNet-1K / ResNet-50, and ModelNet40 / PointNet++, SigmaB-Net reduces per-inference active parameters by 58-60% while remaining within 1.72 percentage points (pp) of the dense baseline Top-1. At comparable ImageNet-1K Top-1, the active-parameter reduction exceeds static structured pruning (FPGM, HRank) by 14-23 pp. The cross-modal evaluation, spanning 2D vision and 3D point-cloud backbones, substantiates a framework-level claim that decouples per-inference memory traffic from the total parameter count.

Kohga Tanaka, Hiroaki Nishi

Graduate School of Science and Technology, Keio University, Kohoku-ku, Yokohama 223-8522, Japan · Department of System Design, Faculty of Science and Technology, Keio University, Kohoku-ku, Yokohama 223-8522, Japan

Efficient Neural Network Inference Dynamic Neural Networks Edge Inference Conditional Computation

Jun 4, 2026cs.SD

DBHN-Net: Dual-Branch Hybrid Neural Network For Low-Complexity Monaural Speech Enhancement

Although artificial neural network (ANN) based speech enhancement (SE) methods demonstrate excellent performance, the high computational complexity and high energy consumption hinder their deployment in practical front-end processing tasks.} Currently, the spiking neural networks (SNNs) have shown potential in reducing power consumption. However, the discrete binary activation and complex spatio-temporal dynamics of SNNs often result in information loss. The current challenge therefore focuses on how to maintain performance and reduce computational complexity. To address this issue, this work propose a Dual-Branch Hybrid Neural (DBHN) Network. 1) In terms of network architecture: A dual-branch network integrating ANN and SNN was designed, where the SNN branch reduces power consumption while the ANN branch addresses information loss; The BandSplit and Time-Frequency (TF) -Mamba modules were developed to simultaneously compress energy consumption and enhance model performance; Spiking Feature Extraction Group (SFEG) and Information Transformation Block (ITB) components were implemented with residual connections to mitigate information loss while further refining feature representations. 2) To facilitate inter-branch information fusion: An Interaction module was designed to promote information exchange at various stages of the dual-branch network; A TF-Cross Attention-Fusion module was designed to perform time-frequency domain fusion of dual-branch information while data-adaptively guiding the SNN branch to retain more critical information. Results show that the proposed model maintains superior performance across three public datasets while achieving an average 7.5 fold reduction in computational complexity compared to baseline models.

Cunhang Fan, Enrui Liu, Jing Zhou +6

State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, (School of Computer Science and Technology), Anhui University, Hefei, 230601, Anhui, P. R. China · China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd · Institute of Acoustics, University of Chinese Academy of Sciences, Beijing 100190, China +1

Efficient Neural Network Inference Speech Enhancement Spiking Neural Networks

Jun 4, 2026cs.LG

GenAutoML: An Agentic Framework for Dynamic Architecture Generation and Optimization in Time-Series Analysis

Designing neural architectures for time-series forecasting and anomaly detection remains a resource-intensive task that often requires substantial domain expertise. Traditional Automated Machine Learning (AutoML) systems typically rely on static, predefined search spaces, limiting their ability to adapt to diverse data characteristics. We present GenAutoML, an agentic framework that leverages Large Language Models (LLMs) as neural architects to bridge natural-language requirements and executable PyTorch implementations. The framework incorporates a Sandboxed Reflection Loop for autonomous code refinement and a Signature-Aware Runtime that enforces architectural consistency and execution safety. To improve robustness under non-stationary conditions, we further introduce a Dynamic Reversible Instance Normalization (Dyn-RevIN) wrapper. Experiments on the ETTh1, ETTm1, and Weather benchmarks demonstrate that GenAutoML can dynamically generate task-specific neural architectures tailored to dataset characteristics. Among the generated models, WaveInterferenceNet achieves inference latency below 0.01 ms per sample while maintaining competitive predictive performance. By emphasizing computational efficiency, architectural adaptability, and stable optimization behavior, GenAutoML enables the creation of ultra-lightweight neural networks suitable for resource-constrained and latency-sensitive Edge AI deployments.

Oleeviya Babu Poikarayil, Cédric Schockaert, Abdulrahman Nahhas +3

Paul Wurth S.A., Luxembourg · Otto-von-Guericke University, Magdeburg, Germany · Technical University of Munich, Germany

Efficient Neural Network Inference Large Language Model-Guided NAS Time Series Forecasting Edge ML Neural Architecture Search Automated Machine Learning+1

Jun 3, 2026cs.CV

LightVesselNet: An Ultra-Lightweight Sub-100K Parameter Network for Retinal Blood Vessel Segmentation

Retinal blood vessel segmentation plays a vital role in the early detection of diabetic retinopathy and glaucoma. While recent deep learning models have achieved great segmentation accuracy, they typically require heavy computational resources, making real-world deployment on edge devices difficult. In this paper, we propose LightVesselNet, an efficient neural network designed for retinal vessel segmentation in a resource-constrained environment. Despite containing only 75K parameters, LightVesselNet performs competitively with much larger models. The network employs a compact encoder decoder architecture enhanced with channel and spatial attention mechanisms, a multi-scale feature aggregation module at the bottleneck, and a subpixel upsampling strategy in the decoder. A dedicated edge residual connection preserves fine vessel detail throughout decoding. Extensive experiments on five publicly available datasets: DRIVE, STARE, CHASEDB1, FIVES, and HRF, yield sensitivity scores of 0.8189, 0.8499, 0.8640, 0.8634, 0.8096, and Dice coefficients of 0.8070, 0.8072, 0.8181, 0.8649, and 0.7686, respectively. LightVesselNet shows improved efficiency (Performance vs Parameter or GFlops) compared to State-of-the-Art models. Cross-dataset evaluation confirms the model's generalisation capability. Overall, LightVesselNet is a strong candidate for deployment in low-resource clinical settings and mobile screening tools.

Shadman Sobhan, Farhana Jalil

Department of Electrical & Electronic Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh.

Efficient Neural Network Inference Retinal Vessel Segmentation Edge Inference Medical Image Segmentation

Jun 3, 2026cs.CV

Data Efficient Complex Feature Fusion Network For Hyperspectral Image Classification

This work presents a data-efficient variant of the Attention-Based Dual-Branch Complex Feature Fusion Network (CFFN) for hyperspectral image classification. The proposed model, termed DE-CFFN, retains the original two-stream structure: the Real-Valued Neural Network (RVNN) processes standard hyperspectral patches, while the Complex-Valued Neural Network (CVNN) handles their Fourier-transformed counterparts. The main contribution of this work lies in the feature extraction process and architectural enhancement. Factor Analysis is used for dimensionality reduction, offering improved latent feature representation over Principal Component Analysis. Additionally, both the RVNN and CVNN streams are structurally modified by successively halving the number of filters in the 3D convolutional layers to reduce complexity. The outputs of both branches are concatenated and passed through a Squeeze and Excitation (SE) block to enhance joint feature representation. Evaluated on the Pavia University and Salinas datasets, DE-CFFN achieves classification performance comparable to CFFN, while significantly reducing model size, memory consumption, and inference latency, making it suitable for real-time hyperspectral imaging applications.

Maitreya Shelare, Atharva Satam, Poonam Sonar +1

Department of Electronics and Telecommunication, Rajiv Gandhi Institute of Technology, University of Mumbai, India

Efficient Neural Network Inference Complex-Valued Neural Networks Hyperspectral Image Classification Neural Network Compression

Jun 3, 2026cs.AR

Characterizing the Impact of NVFP4 Quantization for Low-Power Edge AI Deployment

Energy-efficient neural-network inference at the edge requires reducing arithmetic cost, memory traffic, computation energy, and storage overhead while maintaining acceptable accuracy. This paper presents an ablation-focused study of NVFP4 quantization for edge-efficient neural networks, with emphasis on the relationship between activation precision, weight precision, block-size scaling, retraining, and model accuracy. NVFP4 activations are represented using 4-bit FP4 data, an FP8 block scale, and an FP32 tensor scale, enabling ultra-low precision inference while preserving activation dynamic range. A block-size ablation over six edge-efficient models shows that block size B = 16 provides a practical accuracy/storage trade-off, requiring only 4.5078 bits per input for N = 4096. A weight precision ablation further shows that FP8 and FP16 weights provide only modest gains over FP4 weights under the same NVFP4 activation path, suggesting that activation quantization and scaling dominate much of the accuracy behavior. To isolate the benefit of the NVFP4 data type, this work compares conventional unscaled FP4 activation inference and NVFP4 activation inference with and without retraining. The results show that conventional FP4 inference collapses accuracy for most compact models, while NVFP4 without retraining already recovers substantial accuracy by restoring activation dynamic range through FP8 block scaling and FP32 tensor scaling. When combined with retraining, NVFP4 achieves the best accuracy across the evaluated models, demonstrating the effectiveness of scaling-aware FP4 (NVFP4) inference. These findings provide general design guidance for hardware-software co-design of low power edge inference across a broad range of accelerator platforms, including GPUs, Tensor Cores, FPGAs, domain-specific AI accelerators, near-memory computing systems, and emerging edge-computing architectures.

Ovishake Sen, Venkata Nithin Kamineni, Daniel Lobo +3

Department of Electrical and Computer Engineering, University of Florida, Gainesville, USA

Efficient Neural Network Inference Edge Inference 4-Bit Quantization Activation Quantization Neural Network Quantization

Jun 2, 2026cs.SD

Feasibility of Time-Domain DNN-Based Speech Enhancement on Embedded FPGA for Hearing Aid

Hearing aids impose strict latency and power constraints that current DNN-based speech enhancement systems struggle to meet on embedded hardware. We characterize this gap by deploying both speech separation and denoising using the lightweight SuDoRM-RF++ architecture on the AMD-Xilinx Kria KV260, evaluated at FP32 and 16-bit fixed-point precision for each task. Across these configurations, first-sample latency tracks with on-chip parameter caching rather than arithmetic throughput, identifying data movement as the primary bottleneck. Precision reduction halves the model memory footprint without compromising objective speech quality. The fixed-point denoising accelerator achieves a first-sample latency of 9.7~~ms, meeting the 10~~ms clinical threshold, while speech separation reaches 16.0~ms. These measurements establish concrete resource requirements for embedded DNN-based speech enhancement and quantify the remaining gap to hearing aid deployment.

Feyisayo Olalere, Umut Altin, Kiki van der Heijden +1

Radboud University, Donders Institute for Brain, Cognition, and Behaviour, The Netherlands · Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia University, USA.

Efficient Neural Network Inference Speech Processing Speech Enhancement FPGA-Based Neural Network Acceleration Speech Separation On-Device Inference

Jun 1, 2026cs.AR

CRAM-ER: Error-Resilient Spintronic Computational Random Access Memory for Scalable In-Memory Computation

Deep neural networks (DNNs) have achieved state-of-the-art performance across diverse domains. However, typical Von Neumann compute paradigms face severe memory bottlenecks. Emerging near-memory and compute-in-memory approaches alleviate this but incur significant peripheral overhead. Computational Random Access Memory (CRAM) based on MRAM enables in-situ logic without peripheral overhead, offering a dense, energy-efficient solution. However, probabilistic MRAM switching induces gate-level errors that limit the scalability and reliability of CRAM for accelerating DNN. Moreover, the large number of sequential MRAM writes severely constrains CRAM throughput. To address these challenges, we propose an error-resilient CRAM (CRAM-ER) architecture for scalable in-memory matrix-vector multiplications (MVMs). Our error-aware hardware-software co-design framework leverages a hybrid spintronic-CRAM + CMOS adder-tree architecture to mitigate the impact of device-level errors, demonstrating MVM functionality with high area and energy efficiency. We further develop an error-aware model fine-tuning and fine-grained error correction for enhanced error resilience. Evaluations of the CMOS+spintronic hybrid architecture on DNN benchmarks show near-lossless accuracy while reducing CRAM latency by up to 2 orders of magnitude, outperforming CPU/GPU+high-bandwidth DRAM in both energy efficiency and energy-delay product.

Sohan Salahuddin Mugdho, Md. Shahedul Hasan, Brahmdutta Dixit +3

Electrical and Computer Engineering Iowa State University of Science and Technology Ames, IA, USA · Electrical and Computer Engineering University of Minnesota Twin Cities Minneapolis, MN, USA

Efficient Neural Network Inference Compute-in-Memory

Jun 1, 2026cs.CV

MixerSENet: A Lightweight Framework for Efficient Hyperspectral Image Classification

In this paper, a novel framework, MixerSENet, is introduced for hyperspectral image (HSI) classification, designed to address the challenges of computational efficiency and limited labeled data. The proposed model processes hyperspectral image patches while maintaining consistent size and resolution throughout the network, effectively decoupling the mixing of spatial and channel dimensions. Notably, MixerSENet is lightweight and computationally efficient, requiring fewer parameters compared to traditional models, making it suitable for resource-constrained environments. A squeeze and excitation block is incorporated into the model to refine feature extraction, enhancing the network's ability to capture more informative features. Experimental results on two benchmark datasets demonstrate that MixerSENet achieves superior performance, reaching an overall accuracy (OA) of 82.47% on Houston13 dataset and 96.70% on the Qingyun dataset, outperforming state-of-the-art methods including 3D-CNN, HybridKAN, HSIFormer, SimPoolFormer, and MorphMamba. Furthermore, a detailed analysis of computational efficiency shows that MixerSENet achieves a favorable balance between accuracy and efficiency, with only 53,146 parameters and an low inference time, confirming its practicality for real-world applications. At publication, source code will be publicly available at https://github.com/mqalkhatib/MixerSENet.

Mohammed Q. Alkhatib, Swalpa Kumar Roy, Ali Jamali

College of Engineering and IT, University of Dubai, Dubai, 14143, UAE · Department of Computer Science and Engineering, Alipurduar Government Engineering and Management College, West Bengal 736206, India · Department of Geography, Simon Fraser University, British Columbia 8888, Canada

Efficient Neural Network Inference Hyperspectral Imaging Convolutional Neural Networks Hyperspectral Image Classification

May 31, 2026cs.NE

Spiking and Event-driven Neuromorphic Mamba Models for Efficient Speech Recognition

Deep learning has greatly advanced automatic speech recognition (ASR), enabling widespread deployment on edge devices such as smartphones and smart home systems. However, the computational and energy demands of deep neural networks pose significant challenges for such resource-constrained deployments, introducing latency and limiting real-time interaction. Neuromorphic computing offers a promising solution by introducing activation sparsity through spiking neural networks (SNNs) and event-driven neural networks, converting dense operations into sparse computations. However, a study that evaluates the hardware benefits of different neuromorphic strategies remains lacking for ASR. This paper explores spiking and event-driven neuromorphic neural networks to improve activation sparsity in the state-of-the-art SpeechMamba model for ASR. We introduce an event-driven SpeechMamba with FATReLU activation, achieving over 60% activation sparsity with less than 1% accuracy degradation on LibriSpeech. We also propose a spiking SpeechMamba that attains over 70% sparsity while using 30% fewer parameters than comparable SNNs. Finally, we develop a cycle-accurate event-driven simulator enabling flexible algorithm-hardware co-exploration, which helps us identify computational bottlenecks and yields over 10% additional efficiency improvements.

Tauseef Ahmed, Tao Sun, Jeronimo Castrillon +2

Department of Advanced Computing Sciences, Maastricht University, Netherlands · Hardware-Efficient AI Team, IMEC, Netherlands · Chair for Compiler Construction, TU Dresden, Germany +1

Efficient Neural Network Inference Mamba Automatic Speech Recognition Spiking Neural Networks Neuromorphic Computing

May 29, 2026cs.CV

Where to Refine, When to Stop: Rethinking Redundancy via Latent Discrepancy for Efficient Visual Autoregressive Generation

Visual Autoregressive (VAR) models deliver high-quality image generation but suffer from significant inference latency at high resolutions. Recent acceleration approaches most rely on heuristic measures with layer features to prune tokens. Such heuristics are sensitive to complex contextual semantics, leading to inaccurate identification of redundant computation and poor adaptability across prompts. We rethink redundancy in VAR from the perspective of its impact on pixel-space generation and introduce Latent Discrepancy. This unified metric quantifies a token's contribution by measuring the change in model states during generation. Our analysis shows that redundancy is more accurately identified when guided by image latent or pixel-space signals. We further observed that in classifier-free guidance (CFG), the convergence trend of the discrepancy between conditional and unconditional branches exhibits high dynamics with different prompts. Based on these findings, we propose LD-Pruning (Latent Discrepancy Pruning), a training-free framework that removes redundancy via latent discrepancy by integrating decoding-free region selection and adaptive unconditional-branch skipping. Extensive experiments show that LD-Pruning substantially reduces inference latency while maintaining high generation quality, achieving up to 2.35x speedup on Infinity-8B.

Changwang Mei, Peisong Wang, Zekun Li +7

Nanjing University of Science and Technology · C2DL, Institute of Automation, Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence +2

Efficient Neural Network Inference Visual Autoregressive Models Autoregressive Image Generation Visual Token Pruning

May 29, 2026cs.NE

Memristor-Based Spiking Neural Network Accelerator for Bio-inspired Interception Task

Spiking neural networks (SNNs) provide event-driven and low-power computation inspired by biological neural systems, but current implementations rely on von Neumann graphics processing units (GPUs) and central processing units (CPUs) platforms, where memory and computation bottlenecks limit energy efficiency. To address this challenge, this paper proposes an analog memristor-based spiking neural network (SNN) accelerator that integrates in-memory synaptic computation with analog integrate-and-fire (IF) neurons, eliminating multi-transistor CMOS synapse circuits and enabling asynchronous event-driven operation at the 45nm technology node. Additionally, a digital SNN accelerator is designed and optimized at the 5 nm technology node for comparison. The proposed architecture is evaluated using a predator-prey tracking task that emulates pursuit behavior. In this task, the analog SNN accelerator's inference closely matches the ideal software inference with a mean squared error (MSE) of 0.004. HSPICE simulation results show that the proposed analog SNN accelerator achieves 12.7 times lower energy consumption and 1.26 times lower delay compared to the digital baseline, demonstrating the potential of memristor-based neuromorphic circuits for energy-efficient real-time edge intelligence.

Qianhou Qu, Sheng Lu, Liuting Shang +4

Department of Electrical Engineering The University of Texas at Arlington Arlington, USA · Department of Electrical Engineering and Computer Science South Dakota State University Brookings, USA

Efficient Neural Network Inference Compute-in-Memory Spiking Neural Networks Neuromorphic Computing

May 29, 2026cs.LG

Beyond Accuracy: Evaluating Efficiency, Robustness and Explainability in Deep Learning for Malaria Diagnosis

Malaria remains a leading cause of mortality in sub-Saharan Africa, where scarce diagnostic infrastructure makes timely, accurate diagnosis particularly challenging. While deep learning offers a compelling path toward automated malaria screening, clinical adoption is hindered by computational cost and opacity in decision-making. This work benchmarks four deep learning models spanning a wide range of designed design architectures and model capacities on the NLM-Malaria dataset, jointly evaluating predictive performance, robustness, and post-hoc explainability. We find that lightweight, efficient-by-design models match their heavier counterparts in predictive performance, and the Friedman test confirms no statistically significant performance differences. CAM-based XAI methods consistently localize diagnostically relevant regions, while fine-grained attribution methods produce less targeted explanations, particularly with heavier backbones. Robustness evaluation under three types of image corruption further reveals that model confidence degrades faster than accuracy, providing a practical signal for human review. However, no XAI method is robust to corruption, with explanation reliability degrading at noise levels plausible in clinical practice, even when predictions remain accurate. These findings support the deployment of lightweight architectures for malaria diagnosis in resource-constrained settings, while highlighting the vulnerability of post-hoc explanations as an important consideration for responsible clinical deployment.

Olivier Kanamugire, Kerol Djoumessi

African Institute for Mathematical Sciences, Kigali, Rwanda · Hertie Institute for AI in Brain Health, University of Tübingen, Germany

Efficient Neural Network Inference Image Corruption Robustness Medical Diagnosis Medical Image Classification Explainability Evaluation Medical Image Benchmarks

May 29, 2026cs.LG

Reducing the GPU Memory Bottleneck with Lossless Compression for ML -- Extended

Machine learning (ML) training and inference often process data sets far exceeding GPU memory capacity, forcing them to rely on PCIe for on-demand tensor transfers, causing critical transfer bottlenecks. Lossy compression has been proposed to relieve bottlenecks but introduces workload-dependent accuracy loss, making it complex or even prohibitive to use in existing ML deployments. We explore lossless compression as an alternative that avoids this deployment complexity. We identify where lossless compression can be integrated into ML pipelines while minimizing interference with GPU execution. Based on our findings, we introduce Invariant Bit Packing (IBP), a novel lossless compression algorithm designed to minimize data transfer time for ML. IBP identifies and eliminates invariant bits across groups of tensors, improving throughput through GPU-optimized decompression that leverages warp parallelism, low-overhead bit operations, and asynchronous PCIe transfers. We provide easy-to-use APIs, showcasing them by adding IBP support to GNN training, as well as DLRM and LLM inference frameworks. IBP achieves, on average, 74% faster GNN training, 180% faster DLRM embedding lookup, and 24% faster LLM inference.

Aditya K Kamath, Arvind Krishnamurthy, Marco Canini +1

University of Washington Seattle, WA, USA · KAUST Thuwal, Saudi Arabia

Efficient Neural Network Inference GPU Kernel Optimization LLM Inference Acceleration

May 28, 2026cs.CV

EVL-ECG: Efficient ECG Interpretation With Multi-Aspect Heterogeneous Knowledge Distillation

High-fidelity ECG interpretation is increasingly reliant on massive foundation models, yet their deployment in clinical edge-care remains hindered by extreme computational demands. While knowledge distillation (KD) is a promising solution, traditional methods fail to capture the complex spatio-temporal dependencies of ECG signals when transferring knowledge across heterogeneous architectures. In this paper, we propose EVL-ECG, a framework specifically designed for cross-architecture distillation of cardiac diagnostic logic. EVL-ECG introduces three ECG-aware innovations: (1) Multi-Head Cross-Attention Alignment, which harmonizes architectural discrepancies to preserve fine-grained morphological features; (2) Optimal Transport-based Visual Feature Matching, utilizing optimal transport to maintain global structural relationships across ECG leads despite mismatched token representations; and (3) Geometric Intra-Architecture Relation Matching, which distills the latent diagnostic reasoning of the teacher model. Evaluations across ECG benchmarks demonstrate that EVL-ECG yields improvements of up to 2.4% AUC and 1.1% clinical accuracy over existing baselines. Notably, EVL-ECG establishes an efficient 2B-parameter ECG foundation model, suitable for resource-constrained clinical environments.

Dang Nguyen Hong, Nhi Ngoc-Yen Nguyen, Huy-Hieu Pham

Hanoi University of Science and Technology, Hanoi, Vietnam · VinUni-Illinois Smart Health Center, VinUniversity, Hanoi, Vietnam · College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam +1

Efficient Neural Network Inference Electrocardiogram Foundation Models Electrocardiogram Classification Knowledge Distillation Time-Series Foundation Models

May 28, 2026cs.LG

NeuroEdge: Real-Time Hand Gesture Recognition with High-Density EMG Using Deep Learning at the Edge

High-density electromyography (HD-EMG) has emerged as a powerful modality for decoding fine-grained neuromuscular activity, enabling real-time neural-machine interfaces (NMIs) for applications such as prosthetic control, rehabilitation, and augmented interaction. While deep learning approaches such as convolutional neural networks (CNNs)have demonstrated high classification accuracy for EMG-based gesture recognition, their deployment on embedded hardware remains a major challenge due to computational and memory constraints. This paper presents NeuroEdge, a real-time HD EMG-based NMI system that performs gesture recognition entirely on resource-constrained microcontrollers. The system features two custom-designed modules: the HD-EMG StreamBridge, a wireless communication interface that streams raw HD-EMG data from a Quattrocento amplifier to an ESP32 microcontroller; and the EdgeDL Inference Engine, a lightweight deep learning framework executing on a Sony Spresense microcontroller. A compact 1-dimensional CNN optimized for embedded inference processes, sliding windows of EMG data in real time. Data streaming and inference are pipelined and synchronized through an architecture that utilizes Direct Memory Access (DMA) for data transfer and Serial Peripheral Interface (SPI) burst communication between the ESP32 and Spresense, ensuring low-latency performance. Experimental results show that NeuroEdge achieves a real-time classification accuracy of 90% across seven hand gestures, with a total average latency of 83 ms using 192 channels of HD-EMG recorded from the forearm. Our system demonstrates the feasibility of deploying complex HD-EMG-based gesture recognition on microcontroller-based edge devices, bridging the gap between high-resolution biosignal acquisition and deep learning-based embedded inference for next-generation NMIs.

Peter Chudinov, Zhenyu Lin, Jay Motamarry +3

Department of Biology1 · School of Engineering in Computer Engineering2 at San Francisco State University, San Francisco, CA 94132. · College of San Mateo3, CA 94402. +1

EMG-Based Gesture Recognition Efficient Neural Network Inference Human Activity Recognition Edge Inference Gesture Recognition

May 28, 2026cs.LG

KLAS: Using Similarity to Stitch Neural Networks for Improved Accuracy-Efficiency Tradeoffs

Given the wide range of deployment targets, flexible model selection is essential for optimizing performance within a given compute budget. Recent work demonstrates that stitching pretrained models within a model family enables cost-effective interpolation of the accuracy-efficiency tradeoff space. Stitching transforms intermediate activations from one pretrained model into another, producing a new interpolated stitched network. Such networks provide a pool of deployment options along the accuracy-efficiency spectrum. However, existing stitching approaches often yield suboptimal tradeoffs and lack generalizability, as they primarily rely on heuristics to select stitch configurations. We argue that constructing improved accuracy-efficiency tradeoffs requires explicitly capturing and leveraging the similarity between pretrained models being stitched. To this end, we introduce KLAS, a novel stitch selection framework that automates and generalizes stitch selection across model families by leveraging KL divergence between intermediate representations. KLAS identifies the most promising binary stitches from the

O(k^2n^2)

possibilities for

k

pretrained models of depth

n

. Through comprehensive experiments, we demonstrate that KLAS improves the accuracy-efficiency curve of stitched models at the same finetuning cost as baselines. KLAS achieves up to

1.21\%

higher ImageNet-1K top-1 accuracy at the same computational cost, or maintains accuracy with a

1.33\times

reduction in FLOPs.

Debopam Sanyal, Anantharaman Iyer, Alind Khare +5

Georgia Institute of Technology · Microsoft M365 Research · Cisco Research +1

Efficient Neural Network Inference Representational Similarity Analysis