LLM Inference Acceleration

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

108 papers in the last four weeks, up 120% on the four weeks before. 1.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 677

All topics

Dual-Dimensional Consistency: Balancing Budget and Quality in Adaptive Inference-Time Scaling
May 14, 2026Rongman Xu, Yifei Li, Tianzhe Zhao +3Inference-Time Scaling LLM Inference Acceleration
Xi’an Jiaotong University
GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding
May 14, 2026Fanxu MengGrouped-Query Attention GPU Acceleration
Institute for Artificial Intelligence, Peking University
Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing
May 14, 2026Jie Jiang, Xing Sun, Ruotian Chen +2Speculative Decoding LLM Inference Acceleration
XFP: Quality-Targeted Adaptive Codebook Quantization with Sparse Outlier Separation for LLM Inference
May 14, 2026Thomas WittLLM Quantization Vector Quantization
Gemini Stiftung Leipzig
BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE
May 14, 2026Juntong Wu, Jialiang Cheng, Qishen Yin +5Sparse Mixture-of-Experts Mixture-of-Experts Inference
Taobao & Tmall Group of Alibaba · Shenzhen Graduate School, Peking University
PreFT: Prefill-only finetuning for efficient inference
May 14, 2026Andrew Lanpouthakoun, Aryaman Arora, Zhengxuan Wu +4Fine-Tuning LLM Serving
Stanford University · Tilde Research
Know When To Fold 'Em: Token-Efficient LLM Synthetic Data Generation via Multi-Stage In-Flight Rejection
May 13, 2026Anjir Ahmed Chowdhury, Syed Zawad, Feng YanSynthetic Data Generation LLM Inference Acceleration
Department of Computer Science University of Houston · IBM Research
Self-Pruned Key-Value Attention: Learning When to Write by Predicting Future Utility
May 13, 2026Gergely Szilvasy, Manuel Faysse, Maria Lomeli +5Self-Attention KV Caching
Meta FAIR · MICS, CentraleSupélec
Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding
May 13, 2026Shuoyang Sun, Chang Dai, Hao Fang +6Adversarial Attacks on LLMs Speculative Decoding
Harbin Institute of Technology, Shenzhen · South China University of Technology · Tsinghua Shenzhen International Graduate School, Tsinghua University +1
MinT: Managed Infrastructure for Training and Serving Millions of LLMs
May 13, 2026Mind Lab, :, Song Cao +60LLM Serving Adapter Tuning
Multi-Scale Dequant: Eliminating Dequantization Bottleneck via Activation Decomposition for Efficient LLM Inference
May 13, 2026Lingchao Zheng, Yuwei Fan, Jun Li +5LLM Quantization LLM Inference Acceleration
Huawei
N-vium: Mixture-of-Exits Transformer for Accelerated Exact Generation
May 13, 2026Aleksander Lorenc, Frédéric Berdoz, Joël Mathys +1Efficient Transformer Inference LLM Inference Acceleration
ETH Zurich
STOP: Structured On-Policy Pruning of Long-Form Reasoning in Low-Data Regimes
May 13, 2026Chenjun Xu, Zhennan Zhou, Zhan Su +3LLM Pruning LLM Fine-Tuning
University of Washington · University of Montreal
Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion
May 12, 2026Chien Van Nguyen, Chaitra Hegde, Van Cuong Pham +3Language Model Decoding Diffusion Language Model Inference
University of Oregon · Google DeepMind · Adobe Research
Search Your Block Floating Point Scales!
May 12, 2026Tanmaey Gupta, Hayden Prairie, Xiaoxia Wu +104-Bit Quantization LLM Inference Acceleration
Cornell University, Ithaca, NY, USA · Together AI, USA · University of California San Diego, La Jolla, CA, USA +1
D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting
May 12, 2026Tianyu Wu, Yu Yao, Zhenting Qi +7Speculative Decoding LLM Inference Acceleration
1Harvard · 2MIT
Efficient LLM-based Advertising via Model Compression and Parallel Verification
May 12, 2026Wenxin Dong, Chang Gao, Guanghui Yu +9LLM Quantization LLM Compression
Baidu Inc. Beijing, China
Ada-MK: Adaptive MegaKernel Optimization via Automated DAG-based Search for LLM Inference
May 12, 2026Wenxin Dong, Mingqing Hu, Guanghui Yu +7GPU Kernel Optimization GPU Acceleration
Baidu Inc.
BitLM: Unlocking Multi-Token Language Generation with Bitwise Continuous Diffusion
May 12, 2026Shaobin Zhuang, Yuang Ai, Jiaming Han +7Multi-Token Prediction Autoregressive Language Modeling
Shanghai Jiao Tong University · MMLab, The Chinese University of Hong Kong · Institute of Automation, Chinese Academy of Sciences +1
Fast MoE Inference via Predictive Prefetching and Expert Replication
May 12, 2026Ankit Jyothish, Ali Jannesari, Aishwarya Sarkar +1Expert Load Balancing GPU Acceleration
Iowa State University Ames, USA
Block-Based Double Decoders
May 11, 2026Asher Labovich, Benjamin Bradley, Vanessa Alexander +1Efficient Transformer Inference LLM Inference Acceleration
Department of Computer Science Brown University Providence, RI 02912
Rethinking LLMOps for Fraud and AML: Building a Compliance-Grade LLM Serving Stack
May 11, 2026Prathamesh Vasudeo Naik, Naresh Dintakurthi, Yue WangLLM Serving Financial Fraud Detection
CATS: Cascaded Adaptive Tree Speculation for Memory-Limited LLM Inference Acceleration
May 11, 2026Yuning Han, Yangchenchen Jin, Dylan Zhao +1On-Device Language Model Inference Memory-Efficient Inference
University of Florida Gainesville, FL 32611
ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
May 11, 2026Chayne Thrash, Ali Abbasi, Soheil KolouriRotation-Based Quantization LLM Inference Acceleration
Department of Computer Science Vanderbilt University
SlimSpec: Low-Rank Draft LM-Head for Accelerated Speculative Decoding
May 11, 2026Anton Plaksin, Sergei Krutikov, Sergei Skvortsov +1LLM Compression Speculative Decoding
Nebius, Amsterdam, Netherlands.
Breaking the Reward Barrier: Accelerating Tree-of-Thought Reasoning via Speculative Exploration
May 11, 2026Shuzhang Zhong, Haochen Huang, Shengxuan Qiu +3Inference-Time Search Tree of Thoughts
Institute for Artificial Intelligence, Peking University · School of Integrated Circuits, Peking University · 3ByteDance Seed
GELATO: Generative Entropy- and Lyapunov-based Adaptive Token Offloading for Device-Edge Speculative LLM Inference
May 11, 2026Zengzipeng Tang, Yuxuan Sun, Wei Chen +2On-Device Language Model Inference Edge Computing
School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China
Attention Drift: What Autoregressive Speculative Decoding Models Learn
May 11, 2026Doğaç Eldenk, Payal Mohapatra, Yigitcan Comlek +3Long-Context Language Model Inference Speculative Decoding
Northwestern University · GE Aerospace · fal +1
Merlin: Deterministic Byte-Exact Deduplication for Lossless Context Optimization in Large Language Model Inference
May 11, 2026Sietse SchelpeLLM Inference Acceleration
Corbenic AI, Inc.
Entropy-informed Decoding: Adaptive Information-Driven Branching
May 10, 2026Benjamin Patrick Evans, Sumitra Ganesh, Leo ArdonLanguage Model Decoding LLM Inference Acceleration
JP Morgan AI Research, London, UK · JP Morgan AI Research, New York, USA

May 14, 2026, cs.AIJ/K move · Enter open · S save

Dual-Dimensional Consistency: Balancing Budget and Quality in Adaptive Inference-Time Scaling

Rongman Xu, Yifei Li, Tianzhe Zhao, Yanrui Wu+2

Xi’an Jiaotong University

Inference-Time Scaling LLM Inference Acceleration

Open paper AskPDF arXiv

Large Language Models (LLMs) have demonstrated remarkable abilities in reasoning. However, maximizing their potential through inference-time scaling faces challenges in trade-off between sampling budget and reasoning quality. Current strategies remain inefficient as they typically treat sampling width and depth as orthogonal objectives, where width consensus methods risk reinforcing hallucinations, while depth pruning mechanisms prematurely truncate complex yet valid reasoning chains. Therefore, we propose Dual-Dimensional Consistency (DDC), a unified framework that bridges path quality with adaptive termination. By coupling Confidence-Weighted Bayesian protocol with a Trend-Aware Stratified Pruning, our method ensures that computational resources are concentrated on high quality reasoning paths, filtering hallucinations while accelerating consensus. Evaluations across five benchmarks demonstrate that this approach reduces token consumption by over 10 times while maintaining or exceeding the accuracy of strong baselines across various LLMs.

Momentum

Related topics

Latest papers 677