Efficient Neural Network Inference

Overview

The field studies how to run trained neural networks with less latency, memory, and energy, from phones and microcontrollers to GPUs, FPGAs, and large serving clusters. Common approaches include quantization, pruning, distillation, early exits and conditional computation, alongside compiler and kernel optimization, memory-aware scheduling, and specialized or neuromorphic hardware. Work spans vision, language, graphs, and streaming tasks, with evaluation increasingly measuring end-to-end latency, throughput, and energy on target devices rather than relying only on parameter counts or FLOPs. A central tension is that theoretical savings such as sparsity or fewer operations do not always yield faster real deployments, and efficiency gains must be balanced against accuracy, reliability, and hardware constraints.

Latest papers

CardsList

507 papers30 papers in the last four weeks, down 47% on the four weeks before. 0.2% of all new papers.

  1. GPU-Accelerated Secure Inference for Drug--Target Interaction Prediction

    Nov 19, 2026Sikha Pentyala, Steven Golob, Daniil Filienko +2Secure Multi-Party ComputationEfficient Neural Network Inference

  2. RSIBench-Infra: Can Agents Optimize Their Own Training and Inference Infrastructure at Production Scale?

    Nov 18, 2026Hanlin Wu, Yifei Wang, Zhicheng Cai +5AI Agent BenchmarksEfficient Neural Network Inference

  3. RespCCT and FreqCCT: Compact, Explainable and Cost-Aware Convolutional Transformers for Multi-Class Respiratory Disease Classification

    Nov 18, 2026Auntor Chakma, Mst. Farhana Akter, Abu Henaf Rashid Ahmad Mufti +1Respiratory Sound ClassificationHybrid CNN-Transformer Architectures

  4. ResoLUT: Distilling Residual Networks into Lookup-Table Networks

    Nov 17, 2026Katarzyna Fojcik, Aidas Mikalauskas, Matthias Kampa +2Efficient Neural Network InferenceNeural Network Hardware Acceleration

  5. EM-SDTrack: Efficient Multi-Scale Spike-driven Tracking for Event Cameras

    Nov 17, 2026Yimeng Shan, Haodi Wu, Wei Zheng +6Event-Based VisionSpiking Neural Networks

  6. Integrated GPU Core for Mamba-3

    Nov 16, 2026Shiva Raja, William Desueza, Ajay JoshiNeural Network Hardware AccelerationInference Acceleration

  7. Advancing Quantization for R&R Models with Microscaling Formats

    Nov 12, 2026Yavuz Yetim, Bor-Yiing Su, Zhe Wang +16QuantizationEfficient Neural Network Inference

  8. Theoretical and Empirical Analysis of Sparse Convolution

    Nov 12, 2026Keshvi Tuteja, Yu-Hsiang Tsai, Gregor Olenik +4Efficient Neural Network InferenceConvolutional Neural Networks

  9. SPARC: Accurate Photonic Matrix Multiplication via Representative Vectors

    Nov 12, 2026Yasuhiro Fujiwara, Tatsuhiro Nakamori, Hideyuki KawashimaPhotonic ComputingEfficient Neural Network Inference

  10. Probabilistic State Space Enumeration in Neural Network Computation

    Nov 11, 2026Victor O. K. Li, Yang Han, Jacqueline C. K. LamEfficient Neural Network InferenceAdaptive Computation

  11. UniPipe: Unified Inference Serving and Fine-Tuning for Deadline-Constrained DNN Surrogate Workflows

    Nov 10, 2026Jaiaid Mobin, Avinash Maurya, M. Mustafa Rafique +1Efficient Neural Network InferenceAdaptive Learning

  12. picoEmergencyNet : Attention-Enhanced Ultra-Small Deep Learning Model for Aerial Image Classification in Emergency Monitoring

    Nov 10, 2026Christos KyrkouLightweight CNNsImage Classification

  13. Benchmarking Vendor-Guided Optimizations on Edge NPUs: From NPU Only to End-to-End Performance. Extended technical report

    Nov 10, 2026Youngjin Jo, Jihwan Kim, Minseok Choi +3NPU AccelerationEfficient Neural Network Inference

  14. ConvLUT: Rethinking Convolutional Architectures for Ultra-Low-Latency LUT-Based Inference

    Nov 8, 2026Marta Andronic, Olivia Weng, Danial Zuberi +2Efficient Neural Network InferenceFPGA-Based Neural Network Acceleration

  15. A Unified Framework for License Plate Identification and Character-Recognition

    Nov 8, 2026Saurabh Kumar, Dewant Katare, Prakriti JainAutomatic License Plate RecognitionOptical Character Recognition

  16. ADC-RANet: Improving Resolution Adaptive Neural Networks with Attention, Distillation and Calibration

    Nov 8, 2026Jeferson Severino de Araujo, Lilian Hollard, Eric Granger +1Dynamic Neural NetworksEfficient Neural Network Inference

  17. Enabling Retraining-Free Accurate Inference in Analog Compute in Memory Accelerators

    Nov 5, 2026George Karfakis, Puneet GuptaCompute-in-MemoryNeural Network Hardware Acceleration

  18. MCUFlowNet: Hardware-Aware Neural Architecture Search for Optical Flow on Microcontrollers

    Nov 5, 2026Enmin Lin, Jona BeysensOptical FlowEfficient Neural Network Inference

  19. HyVox: Rethinking Low-Power Voice Intelligence towards Neuromorphic-von Neumann Edge Computing

    Nov 1, 2026Snehasis Addy, Eduardo Calle Ortiz, Prashant Shenoy +1Neuromorphic ComputingOpen-Set Recognition

  20. TADA: Technology Aware DNN Accelerator Performance and Energy Estimator

    Oct 30, 2026Tanvi Sharma, Surya Selvam, Anand Raghunathan +1Neural Network Hardware AccelerationEfficient Neural Network Inference