Overview

Inference acceleration studies how to make trained models produce outputs faster and at lower cost, from language and video generation to real-time, scientific, and recommendation workloads. Common approaches include pruning or reusing intermediate features, speculative decoding and caching with error correction, efficient kernels and numerical formats, and serving techniques such as batching, scheduling, and off-path computation. Methods are evaluated across benchmarks, production services, and edge devices using metrics such as latency, throughput, memory, and retained quality or task success. The central tension is balancing computation and memory savings against approximation error, while making optimizations work across different models and hardware.

Latest papers

CardsList

22 papers4 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

  1. Integrated GPU Core for Mamba-3

    Nov 16, 2026Shiva Raja, William Desueza, Ajay JoshiNeural Network Hardware AccelerationInference Acceleration

  2. ShiftCache: Fast and Exact Indel Effect Prediction by Reusing Shifted Reference Encodings

    Nov 9, 2026Sina Barazandeh, Arth Banka, Jose Lugo-MartinezInference AccelerationIncremental Computation

  3. Fixed-point acceleration on Neural Wasserstein manifolds

    Nov 6, 2026Vitalii Aksenov, Sebastian Gutierrez HernandezInference AccelerationRiemannian Optimization

  4. TailDART: Compute-Efficient Foundation Models for Financial Time-Series Forecasting

    Nov 6, 2026Chiang Qin Zhi, Ranjan Satapathy, Erik Cambria +1Time Series ForecastingFinancial Time Series

  5. Training-free Acceleration for Blind Compressed Image Restoration via Activation-Calibrated Singular Value Decomposition

    Oct 21, 2026Suhang Yao, Xin Li, Xijun Wang +3Inference AccelerationImage Restoration

  6. ReplaySSM: Cache SSM Inputs, Not State

    Oct 9, 2026Ze-Wei Liou, Harris Nover, Amey Naik +4Inference AccelerationLLM Inference Acceleration

  7. TRAC: Trajectory-aware Reuse and Adaptive Correction for Efficient Autoregressive Video Generation

    Oct 2, 2026Jiaxing Song, Weiqi Yan, You Huang +4Autoregressive Video GenerationInference Acceleration

  8. One Basis to Animate Them All: Gaussian Blendshape Distillation for Real-Time Avatars

    Oct 1, 2026Ramazan Fazylov, Stamatis Lefkimmiatis, Ivan LaptevInference AccelerationHuman Avatar Animation

  9. Cache-Aware Conv3D Lowering Across Embedded World-Model Decoders

    Sep 25, 2026Jiaming Zhang, Wu Yang, Shuai Tao +1Inference AccelerationOn-Device Inference

  10. OMatG-flash: An All-Atom Flow Map with Reinforce Adjoint Matching for Scalable Materials Discovery

    Sep 22, 2026Thomas Egg, Harry Winston Sullivan, Ellad B. Tadmor +1Flow MatchingCrystal Structure Prediction

  11. AMICA-Python: Adaptive Mixture Independent Component Analysis with Anderson Acceleration

    Jul 20, 2026Scott Huberty, Christian O'ReillyUnsupervised LearningIndependent Component Analysis

  12. CSD: Content-aware Speculative Decoding for Efficient Image Generation

    Jun 26, 2026Mingcheng Wang, Junbo Qiao, Yunchen Li +8Speculative DecodingInference Acceleration

  13. MIVE: A Minimalist Integer Vector Engine for Softmax LayerNorm and RMSNorm Acceleration

    Jun 16, 2026Kosmas Alexandridis, Giorgos DimitrakopoulosAI Accelerator InferenceInference Acceleration

  14. FMplex: Model Virtualization for Serving Extensible Foundation Models

    Jun 8, 2026Hetvi Shastri, Pragya Sharma, Walid A. Hanafy +3Efficient Neural Network InferenceInference Acceleration

  15. C3^3ache: Accelerating World Action Models with Cross Inference Chunk Cache

    Jun 8, 2026Weisen Zhao, Lam Nguyen, Zhicong Lu +1World Model LearningAction-Conditioned World Models

  16. OptiLookUp: An Optical ROM-Based Lookup Table Engine for Photonic Accelerators

    May 5, 2026Ankur Singh, Akhilesh JaiswalNeural Network Activation FunctionsInference Acceleration

  17. SURGE: SuperBatch Unified Resource-efficient GPU Encoding for Heterogeneous Partitioned Data

    May 1, 2026Shashank Kapadia, Deep Narayan Mishra, Sujal Reddy Alugubelli +3Efficient Neural Network InferenceGPU Acceleration

  18. SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling

    Apr 13, 2026Zikun Liu, Liang Luo, Qianru Li +31Inference AccelerationRecommender Systems

  19. StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models

    Mar 2, 2026Keli Liu, Zhendong Wang, Wengang Zhou +1Inference AccelerationVisual Autoregressive Models