LLM Inference

LLM: Large Language Model

Ask about this topic

Momentum

30 papers in the last four weeks, up 275% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 183

All topics
CardsList

  1. DySCo: Dynamic Sharding for Collaborative Edge-Cloud LLM Inference with Depth-Synchronized Batching

    Oct 6, 2026Jingpo Xu, Paul Joe Maliakel, Ivona Brandic +1LLM InferenceEdge Computing

    University of Amsterdam Amsterdam, Netherlands · TU Wien Vienna, Austria

  2. Shared Stopping Decisions Change Answers in HQQ Cache Quantization

    Oct 5, 2026Seunghui Jwa, Minsu Oh, Chanjun Park +1LLM InferenceKV-Cache Quantization

    Department of Artificial Intelligence, Jeju National University · School of Software, Soongsil University

  3. Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offs

    Oct 4, 2026Javad Mirzaei, Jeebak MitraLLM InferenceTensor Parallelism

    Office of CTO Dell Technologies Inc. Canada

  4. LLM-as-Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them

    Oct 1, 2026Yinheng Li, Justin WagleLLM InferenceLLM Fine-Tuning

    Microsoft

  5. From Discovery to Decision: Finite-Budget Recoverability in LLM Voting

    Oct 1, 2026Shaoang Li, Jian LiLLM Inference EfficiencyLLM Inference

    Stony Brook University

  6. Lingtai: What Concept Geometry Reveals--and Does Not Reveal--About LLM Inference

    Sep 30, 2026Jiangang ChenLLM InferenceLLM Interpretability

    Chengdu Beiluoshimen Technology Co., Ltd.

  7. Can Computation from Earlier Problems Help LLMs Solve New Ones?

    Sep 30, 2026Jipei He, Wenhui Tan, Xiaoyi Yu +4Memory-Augmented Language ModelsLLM Inference

    Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · University of Modena and Reggio Emilia, Italy

  8. SparseEngine: Sparse-First Inference Engine

    Sep 30, 2026Jitai Hao, Quansheng Gu, Qiang Huang +1LLM InferenceLong-Context Language Model Inference

  9. SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUs

    Sep 30, 2026Fahao Chen, Linkang Du, Jinhao Zhou +2LLM InferencePrivacy Leakage in Language Models

    Shandong University China · Xi’an Jiaotong University China · Waseda University Japan

  10. StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams

    Sep 29, 2026Jhen-Ke Lin, Chung Chun WangLLM EvaluationLLM Inference

    National Yang Ming Chiao Tung University

  11. Decode-Latency Feedback Prefill: A Model-Free Controller and Its Generalization Limits

    Sep 29, 2026Gaurav Agarwal, Ashish Garg, Isha SinghalLLM ServingLLM Inference

  12. Hermes: Learning Contextual Reasoning Unlocks Test-Time Scaling

    Sep 29, 2026Xinyu Li, Mononito Goswami, Hao Liu +5LLM InferenceInference-Time Scaling

    Carnegie Mellon University · AWS AI Labs · Washington University in St. Louis +1

  13. Bits Under ZK-LLM: Evaluating Zero-Knowledge-Friendly Quantization for Verifiable Private LLM Inference

    Sep 29, 2026Taeung Yoon, Yupeng Zhang, Xiaojing LiaoLLM QuantizationLLM Inference

    Siebel School of Computing and Data Science University of Illinois Urbana-Champaign

  14. AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs

    Sep 28, 2026Cheuk Hang Lau, Zeyu Cao, Kevin Wong Cheuk Yin +6LLM Inference EfficiencyLLM Inference

    Imperial College London · University of Cambridge · University of Oxford

  15. DPS: Dual-Mode Precision LLM Serving with Semi-Unified Memory

    Sep 28, 2026Xuan Truong Nguyen, Tien Son Pham, Tuan Duc Chu +2LLM ServingLLM Inference

    Department of Next Generation Semiconductor Convergence and Open Sharing System (COSS), Seoul National University, Seoul 08826, South Korea · Van-Lang Institute of Semiconductor Technology (VIST), Hanoi, Vietnam · Efficient Computation Research Group, Moreh Vietnam +1

  16. Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction

    Sep 28, 2026Burc GokdenLLM InferenceLanguage Model Scaling Laws

    Fromthesky Research Labs LLC, Oregon, USA

  17. Kafila: Serving Large Language Models on a Trusted Set of Heterogeneous Commodity Machines

    Sep 28, 2026Murtaza Rangwala, Richard O. Sinnott, Rajkumar BuyyaLLM InferenceLLM Inference Scheduling

    Quantum Cloud Computing and Distributed Systems (qCLOUDS) Lab, School of Computing and Information Systems, The University of Melbourne, Australia

  18. Greenpixie's AI Token Methodology: Assessing the Energy, Water and CO2-eq Impact of AI Tokens for Open and Closed Weight Models

    Sep 27, 2026Joshua Horswill, Ross Hunter, Matt Clifford +1LLM Inference EfficiencyLLM Inference

    Greenpixie Ltd., London, UK.

  19. Greedy Decoding Is Not Precision-Invariant: Cross-Precision Output Divergence in LLM Inference

    Sep 22, 2026Gaoyuan Du, Anam Nawaz Khan, Rex Zhou +4Language Model DecodingLLM Inference

    University of Tennessee, Knoxville · University of Chicago · Amazon +1

  20. Disaggregated Quantization: Specializing LLM Prefill and Decode

    Sep 22, 2026Andrei Panferov, Maximilian Kleinegger, Sweta Priyadarshi +2LLM QuantizationLLM Inference

    NVIDIA · ISTA

  21. TriFleetRCA: On-Premise LLM Root Cause Analysis for Kubernetes

    Sep 20, 2026Rohit Patel, Susil Kumar Mohanty, Jeenal ChaudharyLLM InferenceCloud Computing

    Department of Computer Science and Engineering Indian Institute of Technology Jodhpur, Jodhpur, India

  22. PrefixBench-H100: Characterizing Prefix Reuse and Time-to-First-Token in H100 LLM Serving

    Sep 17, 2026Omkar Shewale, Deepak Kumar, Divakar Kumar YadavLLM Inference EfficiencyLLM Inference

    Illinois Institute of Technology · University of Wisconsin–Milwaukee

  23. Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling

    Sep 16, 2026Mobina Kashaniyan, Ali JannesariLLM Inference EfficiencyLLM Inference

    Department of Computer Science Iowa State University, USA

  24. Dynamic Semantic Compression for Efficient Latent-Space Inference in Large Language Models

    Sep 14, 2026Peipei Li, Dongsen Zhang, Yuchen Liu +1LLM CompressionLLM Inference

    Beijing University of Posts and Telecommunications, Beijing, China

  25. Dissecting GPU Utilization for LLM Inference on Nvidia Hopper

    Sep 14, 2026Mohammad Siavashi, Gerald Q. Maguire Jr., Dejan Kostic +1LLM Inference EfficiencyGPU Acceleration

    KTH Royal Institute of Technology, Stockholm, Sweden

  26. The Battery Price of edge AI: A study of the Environmental Impact of LLM Inference on Mobile Devices

    Sep 14, 2026Édouard Guégain, Tristan CoignionOn-Device Language Model InferenceEnergy-Efficient ML

    DRI, GREENSPECTOR · Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, Talence, France

  27. Sampling via Decision-Flow: Training-Free Extraction of Improved Latent Reasoning Paths in Large Language Models

    Sep 14, 2026Zhendong Mi, Shaoyi HuangInference-Time SearchLLM Inference

    Stevens Institute of Technology

  28. Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding

    Sep 14, 2026Minoo Ahmadi, Seyedarmin Azizi, Erfan Baghaei Potraghloo +2Language Model DecodingLLM Inference

    University of Southern California, Los Angeles, USA

  29. RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems

    Sep 14, 2026Ziyue Yang, Yuting Jiang, Lei Qu +1LLM InferenceLarge Language Model-Guided Optimization

    Shanghai Xingyunzhili Artificial Intelligence Institute, Shanghai, China · Microsoft Research, Redmond, WA, USA

  30. Maverick: Private and Verifiable LLM Inference Made Practical via Matrix-Vector Multiplication Delegation

    Sep 9, 2026Ben Merbaum, Mohammad Amin Raeisi, Wenhao Wang +3Privacy-Preserving Language ModelsLLM Inference

    Yale University, IC3

  31. A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware

    Sep 8, 2026Maysam Khatib, Moysis Symeonides, Demetris Trihinas +2LLM Inference EfficiencyLLM Evaluation

    University of Cyprus, Nicosia, Cyprus · University of Nicosia, Nicosia, Cyprus

  32. MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference

    Sep 7, 2026Michael Wang, Keith Li, Roozbeh BostandoostLLM Inference EfficiencyLLM Inference

    Lake Washington School District · XSchool · UMass Amherst

  33. GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving

    Sep 3, 2026Qiankun Ma, Yanjiang Zhou, Zinan Xiong +5LLM InferenceKV Caching

    Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · Pengcheng Laboratory · University of Chinese Academy of Sciences

  34. A Universal Context-Reuse Layer for Cross-Model KV Sharing

    Aug 31, 2026Yi Li, Dongming Jiang, Yi Zhao +1LLM InferenceKV Caching

    University of Texas at Dallas

  35. The Price of Intelligence: A Quality-Adjusted Price Index for AI Services

    Aug 30, 2026Louis Yiven ZhuLLM Inference

    University of Oxford

  36. GVS5H: Zero-Shot Self-Orchestration with Ledger-Based Control for Improved LLM Coding Performance

    Aug 27, 2026Victor Gao, Vida Khosrowshahi, Ali Khosrowshahi +4Zero-Shot LearningMulti-Agent LLM Systems

    Sang Won · Persis Capital Inc.

  37. Semantic Lenia: Emergence of Homeostatic Solitons within the Semantic Space of Large Language Models

    Aug 12, 2026Yoshihiko KayamaLanguage Model SteeringArtificial Life

    BAIKA Women’s University 2–19–5, Ibaraki, Osaka, Japan

  38. Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference

    Aug 10, 2026Burc GokdenGraph Attention NetworksLLM Inference

    Fromthesky Research Labs LLC, Oregon, USA

  39. Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching

    Aug 10, 2026Kristian Schwethelm, Daniel Rueckert, Georgios KaissisLLM InferenceLLM Inference Scheduling

    Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM), Germany · Department of Computing, Imperial College London, UK · Munich Center for Machine Learning (MCML), Germany +1

  40. Beyond Tier Labels: Role- and Deployment-Dependent Model Substitution in Multi-Call LLM Workflows

    Aug 10, 2026Renxiang Wang, Jiaming CuiLLM InferenceLLM Routing

    Virginia Tech, Blacksburg, VA

  41. Measuring and Reducing WebGPU Dispatch Overhead for LLM Inference

    Aug 9, 2026Jędrzej MaczanLLM Inference EfficiencyLLM Inference

    1Independent Researcher

  42. Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families

    Aug 8, 2026Alizishaan Khatri, Dun Li ChanLLM Safety BenchmarksLLM Inference

    Wrynx Inc · INTI International College Penang

  43. Multi-Level Modeling of Large Language Model Inference Latency and Energy via Hybrid Analytical--Machine-Learning Predictors

    Aug 7, 2026Saeid Shokoufa, Mohammad Erfan Sadeghi, Mehdi Kamal +1LLM Inference

    University of Southern California Los Angeles, California, USA

  44. Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference

    Aug 5, 2026Hongsheng Wang, Philipp KoehnMultilingual Language ModelsLanguage Model Steering

    Johns Hopkins University · Baltimore / USA

  45. PPDL: LLM-Based Flows as Probabilistic Programs

    Aug 5, 2026Louis Mandel, Guillaume Baudart, Mandana Vaziri +1LLM InferenceLLM Reliability

    IBM, New York, USA · Université Paris Cité, Inria, CNRS, IRIF, France

  46. What We Observe as LLM Behavior Can Be a Side-effect of Inference Backend

    Aug 5, 2026Shahed Masoudian, Passant Shafaei, Monorama Swain +1LLM EvaluationLanguage Model Generation Evaluation

    Johannes Kepler University Linz

  47. LiveMem: Maintaining Memory State Continuity in Long-Running LLM Inference

    Aug 3, 2026Zhichen Liu, Ruihan Sun, Hengjie Yang +4Memory-Augmented Language ModelsLLM Inference

    Southern University of Science and Technology · Xidian University

  48. TELLER: Non-intrusive Cross-Layer Root-Cause Analysis for LLM Inference

    Aug 3, 2026Ruilin Xu, Junyi Li, Pengfei Chen +1LLM InferenceLLM Inference Serving

    Sun Yat-sen University Guangzhou, China

  49. Output-Aware Rotation for INT2 KV-Cache Quantization

    Aug 3, 2026Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4LLM InferenceKV Caching

    University of Southern California · Seoul National University · Inha University

  50. Celty: SpMspV GPU Kernel and SIMT Co-Design for Efficient Dual-Sparse LLM Inference

    Aug 2, 2026Ruokai Yin, Priyadarshini PandaGPU Kernel OptimizationLLM Inference

    Yale University New Haven, USA · University of Southern California Los Angeles, USA

  51. MOSAIC: Masked Outsourcing of Secure AI Computations

    Jul 31, 2026James Hsin-yu Chiang, Sheila Zingg, Kari Kostiainen +1Efficient Transformer InferenceSecure Multi-Party Computation

    ETH Zurich Switzerland

  52. Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale

    Jul 30, 2026Banruo Liu, Haoran Qiu, Íñigo Goiri +3LLM InferenceAI Coding Agents

    UIUC · Microsoft Azure Research · Microsoft Azure

  53. Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation

    Jul 30, 2026Alexander Boesgaard LorupLLM InferenceKV Caching

    Openhagen

  54. From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

    Jul 29, 2026Tina Vartziotis, Rodopi Kosteli, Elli Vartziotis +5LLM Inference EfficiencyEnergy-Efficient ML

    1National Technical University of Athens, Patission Complex 42, 10682 Athens, Greece · 2Harvard University, 1350 Massachusetts Avenue, 02138 Cambridge, MA, USA · 3TWT GmbH Science & Innovation, Industriestraße 6, 70565 Stuttgart, DE +4

  55. Profiling Lightweight Large Language Models

    Jul 23, 2026Tomohiro Harada, Enrique Alba, Gabriel LuqueLLM Inference EfficiencyLLM Evaluation

    Graduate School of Science and Engineering, Saitama University, Saitama, Japan

  56. Total Variation Distance Estimation in Autoregressive Models

    Jul 21, 2026Eric Price, Kevin Tian, Zhiyang Xun +1LLM EvaluationLLM Inference

    University of Texas at Austin

  57. Cache-Aware Prompt Compression:A Two-Tier Cost Model for LLM API Caching

    Jul 17, 2026Yan SongLLM ServingLLM Inference

    PayPal Inc.

  58. The Cost and Network Limits of Space-Based AI Compute

    Jul 15, 2026Kees van BerkelLLM InferenceCost-Aware Inference

    Technical University Eindhoven, Dept. of Math. & Computer Science P.O. Box 513, 5600 MB Eindhoven, The Netherlands

  59. Automatic Model-Hardware Co-Adaptation for Heterogeneous AI Accelerators

    Jul 14, 2026Tian Chen, Mingheng Mi, Pu Wang +2AI Accelerator InferenceLLM Inference

    Pazhou Laboratory (Huangpu) · Unaffiliated

  60. Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models

    Jul 14, 2026Yi Li, Chen Li, Jiexiong LiuOn-Device Language Model InferencePrivacy-Preserving Language Models

    KunlunMeta