Efficient Neural Network Inference

Overview

The field studies how to run trained neural networks with less latency, memory, and energy, from phones and microcontrollers to GPUs, FPGAs, and large serving clusters. Common approaches include quantization, pruning, distillation, early exits and conditional computation, alongside compiler and kernel optimization, memory-aware scheduling, and specialized or neuromorphic hardware. Work spans vision, language, graphs, and streaming tasks, with evaluation increasingly measuring end-to-end latency, throughput, and energy on target devices rather than relying only on parameter counts or FLOPs. A central tension is that theoretical savings such as sparsity or fewer operations do not always yield faster real deployments, and efficiency gains must be balanced against accuracy, reliability, and hardware constraints.

Latest papers

CardsList

507 papers30 papers in the last four weeks, down 47% on the four weeks before. 0.2% of all new papers.

  1. LAMP: Look-Ahead Mixed-Precision Inference of Large Language Models

    Jan 29, 2026Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz +5LLM InferenceEfficient Neural Network Inference

  2. Exploring the Potential of Spiking Neural Networks in UWB NLOS Identification

    Dec 30, 2025Youdong Zhang, Xu He, Xiaolin MengSpiking Neural NetworksWireless Localization

  3. Animal Re-Identification on Microcontrollers

    Dec 9, 2025Yubo Chen, Di Zhao, Yun Sing Koh +1Efficient Neural Network InferenceAnimal Re-Identification

  4. SparsePixels: Efficient Convolution for Sparse Data on FPGAs

    Dec 5, 2025Ho Fung Tsoi, Dylan Rankin, Vladimir Loncar +1Efficient Neural Network InferenceConvolutional Neural Networks

  5. Hardware-Algorithm Co-Optimization of Early-Exit Neural Networks for Multi-Core Edge Accelerators

    Dec 4, 2025Alaa Zniber, Arne Symons, Ouassim Karrakchou +2Efficient Neural Network InferenceDynamic Neural Networks

  6. Accelerated Execution of Bayesian Neural Networks using a Single Probabilistic Forward Pass and Code Generation

    Nov 28, 2025Bernhard Klein, Falk Selker, Hendrik Borras +3Bayesian Neural NetworksEfficient Neural Network Inference

  7. AutoTailor: Automatic and Efficient Adaptive Model Deployment for Diverse Edge Devices

    Nov 27, 2025Mengyang Liu, Chenyu Lu, Haodong Tian +7Edge MLEfficient Neural Network Inference

  8. NeuroFlex: Lossless Element-Level ANN-SNN Co-Execution for Efficient Sparse Inference

    Nov 7, 2025Varun Manjunath, Pranav Ramesh, Gopalakrishnan SrinivasanEfficient Neural Network InferenceSpiking Neural Networks

  9. Time-multiplexed layer reuse for physical neural networks

    Oct 28, 2025Kohei Tsuchiyama, Andre Roehm, Takatomo Mihana +1Efficient Neural Network InferencePhysical Neural Networks

  10. BiHDTrans: binary hyperdimensional transformer for efficient multivariate time series classification

    Sep 29, 2025Jingtao Zhang, Yi Liu, Qi Shen +1Efficient Neural Network InferenceTransformer

  11. Physics-informed GNN for medium-high voltage AC power flow with edge-aware attention and line search correction operator

    Sep 26, 2025Changhun Kim, Timon Conrad, Redwanul Karim +6Efficient Neural Network InferenceGraph Attention Networks

  12. GraphMend: Code Transformations for Fixing Graph Breaks in PyTorch 2

    Sep 17, 2025Savini Kashmira, Jayanaka Dantanarayana, Thamirawaran Sathiyalogeswaran +3Efficient Neural Network InferenceCompiler Optimization

  13. Practical and Private Hybrid ML Inference with Fully Homomorphic Encryption

    Sep 1, 2025Sayan Biswas, Philippe Chartier, Akash Dhasade +7Privacy-Preserving MLHomomorphic Encryption

  14. Visual Perception Engine: Fast and Flexible Multi-Head Inference for Robotic Vision Tasks

    Aug 15, 2025Jakub Łucki, Jonathan Becktor, Georgios Georgakis +2Efficient Neural Network InferenceRobotic Perception

  15. Versatile yet Efficient Network Traffic Analysis: Offloading Network Foundation Model to SmartNIC

    Aug 4, 2025Chungang Lin, Xuying Meng, Tianyu Zuo +9Encrypted Network Traffic AnalysisEncrypted Traffic Classification

  16. Adaptive Time-step Training for Enhancing Spike-Based Neural Radiance Fields

    Jul 30, 2025Ranxi Lin, Canming Yao, Jiayi Li +3Efficient Neural Network InferenceNeural Rendering

  17. Efficient Memristive Spiking Neural Networks Architecture with Supervised In-Situ STDP Method

    Jul 28, 2025Santlal Prajapati, Susmita Sur-Kolay, Soumyadeep DuttaEfficient Neural Network InferenceSpiking Neural Networks

  18. evMLP: An Efficient Event-Driven MLP Architecture for Vision

    Jul 2, 2025Zhentan ZhengEfficient Neural Network InferenceConditional Computation

  19. PERTINENCE: Input-based Opportunistic Neural Network Dynamic Execution

    Jul 2, 2025Omkar Shende, Gayathri Ananthanarayanan, Marcello TraiolaEfficient Neural Network InferenceAdaptive Inference

  20. Implementing Keyword Spotting on the MCXN947 Microcontroller with Integrated NPU

    Jun 10, 2025Petar Jakuš, Hrvoje DžapoNPU AccelerationKeyword Spotting

  21. Learning Interpretable Differentiable Logic Networks for Tabular Regression

    May 29, 2025Chang Yue, Niraj K. JhaEfficient Neural Network InferenceDifferentiable Logic Gate Networks

  22. Optimizing Binary and Ternary Neural Network Inference on RRAM Crossbars using CIM-Explorer

    May 20, 2025Rebecca Pelke, José Cubero-Cascante, Nils Bosbach +5Efficient Neural Network InferenceCompute-in-Memory

  23. Sigma-Delta Neural Network Conversion on Loihi 2

    May 9, 2025Matthew Brehove, Sadia Anjum Tumpa, Espoir Kyubwa +2Efficient Neural Network InferenceSpiking Neural Networks

  24. Leveraging Neural Graph Compilers in Machine Learning Research for Edge-Cloud Systems

    Apr 28, 2025Alireza Furutanpey, Carmen Walser, Philipp Raith +2Efficient Neural Network InferenceHeterogeneous Computing

  25. Echo-E3^3Net: Efficient Endocardial Spatio-Temporal Network for Ejection Fraction Estimation

    Mar 21, 2025Moein Heidari, Afshin Bozorgpour, AmirHossein Zarif-Fakharnia +5Efficient Neural Network InferenceLeft Ventricular Ejection Fraction Estimation

  26. Reforge: Low-Latency Distributed GNN Serving with Selective Embedding Recomputation

    Jan 15, 2025Geon-Woo Kim, Donghyun Kim, Jeongyoon Moon +5Efficient Neural Network InferenceGraph Neural Networks

  27. FlowWise: Stateful Fast-Slow Model Serving for Streaming Traffic Intelligence

    Feb 6, 2024Shinan Liu, Ted Shaowang, Gerry Wan +4Cascaded InferenceEfficient Neural Network Inference