Efficient Inference

Ask about this topic

Momentum

9 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 61

All topics
CardsList

  1. ReSolve: Reusing Candidate Reasoning through Selective Generative Moderation

    Oct 1, 2026Bangji Yang, Jiajun Fan, Hongba Ma +6CandidateEfficient Inference

    University of Illinois at Urbana-Champaign · Tsinghua University · Rutgers University +2

  2. AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs

    Sep 28, 2026Cheuk Hang Lau, Zeyu Cao, Kevin Wong Cheuk Yin +6Agentic BenchmarksAgentic Inference

    Imperial College London · University of Cambridge · University of Oxford

  3. Accelerating Video Diffusion via Training-Free Trajectory Routing

    Sep 24, 2026Mustafa Munir, Huy Vu, Shreyas Misra +7Video Diffusion ModelsDenoising Trajectory

  4. Adapting Tree-Structured Speculative Decoding to DeepSeek-V4 for Efficient Inference

    Sep 21, 2026Changxu Liu, Zhaogeng LiSpeculative DecodingLLM Inference Optimization

    Baige AI Team, Baidu Inc. · Fudan University

  5. The Free Inference Dimension: Complexity Measure for Zero-Collision Navigation under Hypothesis Mixtures

    Sep 15, 2026Luiz Carlos Castro Guedes, Edward Hermann HaeuslerCollision-Free TrajectoriesProbabilistic Inference

    Departamento de Informática – PUC-Rio, Brazil · Seção de Engenharia de Computação – Instituto Militar de Engenharia, Brazil

  6. LCAP: Population-Informed Latent Chip Adaptation from Few Output Probes for Photonic Neural Networks

    Sep 15, 2026Tianyu Gao, Guantian ZhengParameter-Efficient AdaptationEfficient Inference

    City University of Hong Kong, Hong Kong SAR, China · Sichuan University, Chengdu, China · Nanyang Technological University, Singapore

  7. Attention Quantization for Tabular Foundation Models

    Sep 14, 2026Jonas M. Kübler, Benjamin Jäger, Klemens Flöge +2Tabular Foundation ModelsKey-Value Cache Quantization

    Prior Labs

  8. Self-Orchestrating Language Models: Leveraging Semantic Dependence for Efficient Inference

    Sep 13, 2026Tian JinLLM Inference OptimizationAutoregressive Language Models

    Department of Electrical Engineering and Computer Science, MASSACHUSETTS INSTITUTE OF TECHNOLOGY

  9. Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Composition

    Sep 13, 2026Yecheng Wu, Song Han, Han CaiLarge Reasoning ModelsEfficient Inference

    Massachusetts Institute of Technology · NVIDIA

  10. Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching

    Aug 10, 2026Kristian Schwethelm, Daniel Rueckert, Georgios KaissisEfficient Inference

    Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM), Germany · Department of Computing, Imperial College London, UK · Munich Center for Machine Learning (MCML), Germany +1

  11. BnBERT-iPET: Sparse Few-Shot Language Modeling for Bengali via Lottery Ticket Pruning

    Aug 5, 2026Sajib Hossain, Md Kamrus Samad, Anan Ghosh +2Bert-Based ModelsBangla

    Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh

  12. Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Models

    Aug 4, 2026Paribesh Regmi, Qingshuang Chen, Chi Zhang +3Visual Token PruningVideo-Language Models

    Amazon.com Services LLC

  13. Gecko: Fast Private Inference via Secure Public Encoder Offloading

    Aug 3, 2026Cheng'an Wei, Kai Chen, Yue Zhao +2Neural Network InferenceEfficient Inference

    Institute of Information Engineering, Chinese Academy of Sciences, China, and the School of Cyber Security, University of Chinese Academy of Sciences, China

  14. From Data to Device: ELMOD An Efficient German-First 2.7B Language Model for Mobile Inference

    Jul 27, 2026Darina Gold, Alexander Schwirjow, Viktor Haag +4Small Large Language ModelsEfficient Inference

    IIS Fraunhofer

  15. Are All Tokens Necessary for Visual Place Recognition? An Empirical Study of Token Reduction for Efficient Inference

    Jul 17, 2026Tong Jin, Yunpeng Liu, Shuyu Hu +4Visual Place RecognitionVision Transformer

    Shenyang Institute of Automation, Chinese Academy of Sciences. · University of Chinese Academy of Sciences. · Shenzhen University of Advanced Technology.

  16. Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques

    Jul 14, 2026Daehoon Gwak, Minhyung Lee, Junwoo Park +1Diffusion Language ModelsEfficient Inference

    KAIST AI · Yonsei University

  17. Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata

    Jul 8, 2026Meihua Dang, Stefano ErmonConstrained DecodingDiffusion Language Models

    Stanford University

  18. Quantize the Target, Quantize the Drafter: Efficient Inference with Qwen3.5-4B

    Jul 5, 2026Jaeyeon Kim, Jewon Lee, Bo-Kyeong KimLLM Inference OptimizationQwen3

    Nota Inc.

  19. ROSA: A Robotics Foundation Model Serving System for Robot Factories

    Jul 1, 2026Wenqi Jiang, Jason Clemons, Rowland O'Flaherty +5Cpu-Gpu Hybrid DesignsEfficient Inference

    NVIDIA Research · Stanford University

  20. Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM

    Jun 28, 2026Shuvendu Roy, Mengyao Zhai, Hossein Hajimirsadeghi +1Key-Value Cache EvictionLLM Inference Optimization

    RBC Borealis

  21. KernelSight-LM: A Kernel-Level LLM Inference Simulator

    Jun 26, 2026Xiteng Yao, Taeho Kim, Hengzhi Pei +7LLM Inference OptimizationEfficient Inference

    Amazon Web Services Santa Clara, CA, USA · Boston University Boston, MA, USA

  22. Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation

    Jun 24, 2026DatologyAI, :, Matthew L. Leavitt +8Efficient InferenceData-Curation

    DatologyAI Team1

  23. MemoryWAM: Efficient World Action Modeling with Persistent Memory

    Jun 18, 2026Sizhe Yang, Juncheng Mu, Tianming Wei +8Efficient World-Action ModelWorld Models

    The Chinese University of Hong Kong · Tsinghua University · Zhejiang University

  24. FEnc2^2: Unifying Data Packing for Efficient Private Inference via Convolution and Architecture-Aware Fragment Encoding

    Jun 15, 2026Ran Ran, Zhaoting Gong, Nuo Xu +3Homomorphic EncryptionNeural Network Inference

    North Carolina State University · University of Minnesota · University of California, Santa Cruz +1

  25. AVIS: Adaptive Test-Time Scaling for Vision-Language Models

    Jun 10, 2026Ahmadreza Jeddi, Minh Ngoc Le, Amirhossein Kazerouni +8Fast InferenceEfficient Inference

    AI Center-Toronto, Samsung Electronics · University of Toronto · Vector Institute +1

  26. Multi-task LLMs for Bug Classification: Efficient Inference with Auxiliary Decoding Heads

    Jun 8, 2026Nikolai RozanovBug DetectionCode Generation

    Department of Computing, Imperial College London, London, UK

  27. OctoT2I: A Self-Evolving Agentic Text-to-Image Router

    Jun 1, 2026Xu Jiang, Bin Chen, Gehui Li +3Text-To-ImageText-To-Video Generation Model

    School of Electronic and Computer Engineering, Peking University · Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University

  28. EvoCut: Multi-Layer Evolution-Aware Visual Token Compression for Efficient Large Vision-Language Models

    Jun 1, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Token CompressionVision Encoders

    Harbin Institute of Technology · Xiaohongshu · Fudan University

  29. MURMUR: An Efficient Inference System for Long-Form ASR

    May 31, 2026Wei-Tzu Lee, Keisuke Kamahori, Baris KasikciAutomatic Speech RecognitionEfficient Long-Context Inference

    University of Washington

  30. Efficient Test-time Inference for Generative Planning Models with OCL Search

    May 30, 2026Robert Gieselmann, Mihai Samson, Federico Pecora +1Classical PlanningEfficient Inference

    1Amazon · 2Work done while at Amazon.

  31. Bridging Maximum Likelihood and Optimal Transport for Efficient Inference and Model Selection in Stochastic Block Models

    May 27, 2026Simon Queric, Cédric Vincent-Cuaz, Charles Bouveyron +1Maximum LikelihoodModel Selection

    Universit´e Cˆote d’Azur, Inria, CNRS LJAD, Maasai Nice, France · EPFL Lausanne, Switzerland · Universit´e Cˆote d’Azur, Inria, CNRS CEPAM, Maasai Nice, France

  32. Semiparametrically Efficient Inference for Kernel Measures of Noise Heterogeneity

    May 26, 2026Jakub Wornbard, Zikai Shen, Dimitri Meunier +1HeteroskedasticityMultiplicative Gamma Noise

    University College London

  33. Boosting Inference with Guided Reasoning: Stochastic Exploration for Recursive Models

    May 24, 2026Andrew Corbett, Archit Sood, Anna Tzatzopoulou +2Recursive ModelsReasoning Trajectory

    digiLab, UK · University of Bristol, UK

  34. Your Embedding Model is SMARTer Than You Think

    May 24, 2026Jianrui Zhang, Hyun Jung Lee, Sukanta Ganguly +3Multimodal RetrievalContrastive Learning

    UW-Madison · Korea University · NetApp, Inc.

  35. JT-SAFE-V2: Safety-by-Design Foundation Model with World-Context Data

    May 23, 2026Junlan Feng, Fanyu Meng, Chong Long +12Large Language Model SafetyWireless Foundation Models

    JIUTIAN Research

  36. Can AI Guess What You Know? Performance Comparison of Large Language Models for Human Domain Knowledge Estimation From Communication Logs

    May 21, 2026Ko Watanabe, Shoya IshimaruEfficient Inference

    DFKI, Kaiserslautern, Germany · Osaka Metropolitan University, Osaka, Japan

  37. TIDE: Efficient and Lossless MoE Diffusion LLM Inference with I/O-aware Expert Offload

    May 19, 2026Zhiben Chen, Youpeng Zhao, Yang Sui +2Efficient InferenceDiffusion Language Models

    University of Central Florida · Rice University

  38. PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding

    May 15, 2026Shengyin Sun, Yiming Li, Renxi Liu +7Diffusion Language ModelsSpeculative Decoding

    Department of Computer Science City University of Hong Kong · Huawei Technologies · City University of Hong Kong

  39. PreFT: Prefill-only finetuning for efficient inference

    May 14, 2026Andrew Lanpouthakoun, Aryaman Arora, Zhengxuan Wu +4Parameter-Efficient Fine-Tuning MethodsPrefill

    Stanford University · Tilde Research

  40. Efficient LLM Reasoning via Variational Posterior Guidance with Efficiency Awareness

    May 10, 2026Zizhao Chen, Yuying Li, Siting Lin +1LLM Reasoning StrategiesEfficient Inference

    Guangdong University of Foreign Studies, Guangzhou, China.

  41. DARE: Difficulty-Adaptive Reinforcement Learning with Co-Evolved Difficulty Estimation

    May 9, 2026Yang Zhou, Can Jin, Zihan Dong +7Large Language Model Reinforcement LearningAlignment-Aware Selection

    Rutgers University · Amazon · Washington University +1

  42. HCInfer: An Efficient Inference System via Error Compensation for Resource-Constrained Devices

    May 7, 2026Shen Xu, Xiangwen Zhuge, Zhe Xu +3Efficient InferenceSampling Bias Compensation

    Tsinghua University · Huazhong University of Science and Technology

  43. Budgeted LoRA: Distillation as Structured Compute Allocation for Efficient Inference

    May 5, 2026Mohammed Sabry, Anya BelzEfficient InferenceInference Cost

    ADAPT Centre, Dublin City University, Ireland

  44. Exact and Approximate Algorithms for Polytree Learning

    May 5, 2026Juha Harviainen, Frank Sommer, Manuel SorgeBayesian NetworksApproximation Algorithms

    Department of Computer Science, University of Helsinki, Helsinki, Finland · Institute of Computer Science, Friedrich-Schiller Universität Jena, Jena, Germany · Institute of Logic and Computation, TU Wien, Vienna, Austria

  45. When Less is Enough: Efficient Inference via Collaborative Reasoning

    May 1, 2026Yilei Chen, Sharut Gupta, Yannis Paschalidis +2Efficient InferenceProbabilistic Inference

    Boston University · MIT CSAIL · Biohub

  46. DARE: Diffusion Language Model Activation Reuse for Efficient Inference

    May 1, 2026Natalia Frumkin, Bokun Wang, Hung-Yueh Chiang +3Diffusion Language ModelsEfficient Inference

    Chandra Family Department of Electrical and Computer Engineering The University of Texas at Austin · Cornell University

  47. Density Field State Space Models: 1-Bit Distillation, Efficient Inference, and Knowledge Organization in Mamba-2

    Apr 28, 2026Chirag ShindeMamba-Based ModelsEfficient Inference

    Independent Researcher

  48. Understanding Scattered Forest Search: A Version-Space Perspective on Multi-Turn Program Correction

    Apr 27, 2026Yuto Tanaka, Issei SatoEfficient InferenceIterative Refinement

    Department of Computer Science The University of Tokyo Tokyo, Japan

  49. Reasoning Primitives in Hybrid and Non-Hybrid LLMs: Do Architectural Differences Yield Advantages in State-Tracking and Recall?

    Apr 23, 2026Shivam Rawat, Lucie Flek, Florian Mai +1LLM Reasoning StrategiesSearch-Augmented Reasoning

    Lamarr Institute for Machine Learning and Artificial Intelligence · Rheinische Friedrich-Wilhelms-Universität Bonn

  50. Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees

    Apr 22, 2026Xueyan Li, Johannes Zenn, Ekaterina Fadeeva +3Efficient InferenceReasoning Traces

    Max Planck Institute for Intelligent Systems · ETH Zurich · AI Center Tübingen +3

  51. DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models

    Apr 20, 2026You-Liang Huang, Xinhao Huang, Chengxi Liao +1LLM Inference OptimizationEfficient Inference

    Boston University · Boston, Massachusetts, USA · HKUST (Guangzhou) +2

  52. SecureRouter: Encrypted Routing for Efficient Secure Inference

    Apr 16, 2026Yukuan Zhang, Mengxin Zheng, Qian LouNeural Network InferenceEfficient Inference

    University of Central Florida · Orlando, Florida, USA

  53. DEEP-GAP: Deep-learning Evaluation of Execution Parallelism in GPU Architectural Performance

    Apr 16, 2026Kathiravan PalaniappanNvidiaEfficient Inference

    Independent Researcher (University of Colorado Colorado Springs Alumni)

  54. Beyond Memorization: Distinguishing Between Pattern-Based and Epistemic Reasoning in LLMs Using Epistemic Puzzles

    Mar 22, 2026Adi Gabay, Gabriel Stanovsky, Liat PeterfreundEpistemic AsymmetryEpistemology

    School of Computer Science and Engineering The Hebrew University of Jerusalem

  55. A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMs

    Feb 23, 2026Zijie Liu, Jie Peng, Jinhao Duan +7Load BalancingLLM Inference Optimization

    University of North Carolina at Chapel Hill · University of Minnesota Twin Cities · North Carolina State University +2

  56. Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States

    Feb 3, 2026Ximing Dong, Shaowei Wang, Dayi Lin +2Speculative DecodingEfficient Inference

    Centre for Software Excellence, Huawei, Canada · Department of Computer Science, University of Manitoba, Canada · School of Computing, Queen’s University, Canada

  57. What If We Allocate Test-Time Compute Adaptively?

    Feb 1, 2026Ahsan Bilal, Ahmed Mohsin, Muhammad Umer +4Test-Time ScalingEfficient Inference

    University of Oklahoma, OK, USA · Stanford University, CA, USA · Universitat Pompeu Fabra, Barcelona, Spain +2

  58. DAGGER: Distractor-Aware Graph Generation for Executable Reasoning in Math Problems

    Jan 11, 2026Zabir Al Nazi, Shubhashis Roy Dipta, Sudipta KarMathematical ReasoningMath Problems

    University of California, Riverside · University of Maryland, Baltimore County · Oracle Health AI

  59. Entropy-Aware Token Rejection for Improving Speculative Decoding

    Dec 29, 2025Tiancheng Su, Meicong Zhang, Guoxiu HeSpeculative DecodingToken-Level Entropy

    School of Economics and Management East China Normal University

  60. Fine-Tune, Then Rectify

    Nov 23, 2025Zikun Ye, Jinglong Zhao, Lei WangModel Fine-TuningReweighting

    Foster School of Business, University of Washington, Seattle, WA · Questrom School of Business, Boston University, Boston, MA