LLM Inference

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 233% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 187

All topics

RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems
Sep 14, 2026Ziyue Yang, Yuting Jiang, Lei Qu +1LLM Inference Large Language Model-Guided Optimization
Shanghai Xingyunzhili Artificial Intelligence Institute, Shanghai, China · Microsoft Research, Redmond, WA, USA
Dissecting GPU Utilization for LLM Inference on Nvidia Hopper
Sep 14, 2026Mohammad Siavashi, Gerald Q. Maguire Jr., Dejan Kostic +1LLM Inference Efficiency GPU Acceleration
KTH Royal Institute of Technology, Stockholm, Sweden
Maverick: Private and Verifiable LLM Inference Made Practical via Matrix-Vector Multiplication Delegation
Sep 9, 2026Ben Merbaum, Mohammad Amin Raeisi, Wenhao Wang +3Privacy-Preserving Language Models LLM Inference
Yale University, IC3
A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware
Sep 8, 2026Maysam Khatib, Moysis Symeonides, Demetris Trihinas +2LLM Inference Efficiency LLM Evaluation
University of Cyprus, Nicosia, Cyprus · University of Nicosia, Nicosia, Cyprus
MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference
Sep 7, 2026Michael Wang, Keith Li, Roozbeh BostandoostLLM Inference Efficiency LLM Inference
Lake Washington School District · XSchool · UMass Amherst
GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving
Sep 3, 2026Qiankun Ma, Yanjiang Zhou, Zinan Xiong +5LLM Inference KV Caching
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · Pengcheng Laboratory · University of Chinese Academy of Sciences
Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM Weights
Sep 2, 2026Pier-Jean MalandrinoLLM Inference Vector Quantization
Scub
A Universal Context-Reuse Layer for Cross-Model KV Sharing
Aug 31, 2026Yi Li, Dongming Jiang, Yi Zhao +1LLM Inference KV Caching
University of Texas at Dallas
The Price of Intelligence: A Quality-Adjusted Price Index for AI Services
Aug 30, 2026Louis Yiven ZhuLLM Inference
University of Oxford
GVS5H: Zero-Shot Self-Orchestration with Ledger-Based Control for Improved LLM Coding Performance
Aug 27, 2026Victor Gao, Vida Khosrowshahi, Ali Khosrowshahi +4Zero-Shot Learning Multi-Agent LLM Systems
Sang Won · Persis Capital Inc.
Semantic Lenia: Emergence of Homeostatic Solitons within the Semantic Space of Large Language Models
Aug 12, 2026Yoshihiko KayamaLanguage Model Steering Artificial Life
BAIKA Women’s University 2–19–5, Ibaraki, Osaka, Japan
Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference
Aug 10, 2026Burc GokdenGraph Attention Networks LLM Inference
Fromthesky Research Labs LLC, Oregon, USA
Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching
Aug 10, 2026Kristian Schwethelm, Daniel Rueckert, Georgios KaissisLLM Inference LLM Inference Scheduling
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM), Germany · Department of Computing, Imperial College London, UK · Munich Center for Machine Learning (MCML), Germany +1
Beyond Tier Labels: Role- and Deployment-Dependent Model Substitution in Multi-Call LLM Workflows
Aug 10, 2026Renxiang Wang, Jiaming CuiLLM Inference LLM Routing
Virginia Tech, Blacksburg, VA
Measuring and Reducing WebGPU Dispatch Overhead for LLM Inference
Aug 9, 2026Jędrzej MaczanLLM Inference Efficiency LLM Inference
1Independent Researcher
Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families
Aug 8, 2026Alizishaan Khatri, Dun Li ChanLLM Safety Benchmarks LLM Inference
Wrynx Inc · INTI International College Penang
Multi-Level Modeling of Large Language Model Inference Latency and Energy via Hybrid Analytical--Machine-Learning Predictors
Aug 7, 2026Saeid Shokoufa, Mohammad Erfan Sadeghi, Mehdi Kamal +1LLM Inference
University of Southern California Los Angeles, California, USA
Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference
Aug 5, 2026Hongsheng Wang, Philipp KoehnMultilingual Language Models Language Model Steering
Johns Hopkins University · Baltimore / USA
PPDL: LLM-Based Flows as Probabilistic Programs
Aug 5, 2026Louis Mandel, Guillaume Baudart, Mandana Vaziri +1LLM Inference LLM Reliability
IBM, New York, USA · Université Paris Cité, Inria, CNRS, IRIF, France
What We Observe as LLM Behavior Can Be a Side-effect of Inference Backend
Aug 5, 2026Shahed Masoudian, Passant Shafaei, Monorama Swain +1LLM Evaluation Language Model Generation Evaluation
Johannes Kepler University Linz
LiveMem: Maintaining Memory State Continuity in Long-Running LLM Inference
Aug 3, 2026Zhichen Liu, Ruihan Sun, Hengjie Yang +4Memory-Augmented Language Models LLM Inference
Southern University of Science and Technology · Xidian University
TELLER: Non-intrusive Cross-Layer Root-Cause Analysis for LLM Inference
Aug 3, 2026Ruilin Xu, Junyi Li, Pengfei Chen +1LLM Inference LLM Inference Serving
Sun Yat-sen University Guangzhou, China
Output-Aware Rotation for INT2 KV-Cache Quantization
Aug 3, 2026Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4LLM Inference KV Caching
University of Southern California · Seoul National University · Inha University
Celty: SpMspV GPU Kernel and SIMT Co-Design for Efficient Dual-Sparse LLM Inference
Aug 2, 2026Ruokai Yin, Priyadarshini PandaGPU Kernel Optimization LLM Inference
Yale University New Haven, USA · University of Southern California Los Angeles, USA
MOSAIC: Masked Outsourcing of Secure AI Computations
Jul 31, 2026James Hsin-yu Chiang, Sheila Zingg, Kari Kostiainen +1Efficient Transformer Inference Secure Multi-Party Computation
ETH Zurich Switzerland
Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale
Jul 30, 2026Banruo Liu, Haoran Qiu, Íñigo Goiri +3LLM Inference AI Coding Agents
UIUC · Microsoft Azure Research · Microsoft Azure
Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation
Jul 30, 2026Alexander Boesgaard LorupLLM Inference KV Caching
Openhagen
From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs
Jul 29, 2026Tina Vartziotis, Rodopi Kosteli, Elli Vartziotis +5LLM Inference Efficiency Energy-Efficient ML
1National Technical University of Athens, Patission Complex 42, 10682 Athens, Greece · 2Harvard University, 1350 Massachusetts Avenue, 02138 Cambridge, MA, USA · 3TWT GmbH Science & Innovation, Industriestraße 6, 70565 Stuttgart, DE +4
Profiling Lightweight Large Language Models
Jul 23, 2026Tomohiro Harada, Enrique Alba, Gabriel LuqueLLM Inference Efficiency LLM Evaluation
Graduate School of Science and Engineering, Saitama University, Saitama, Japan
Total Variation Distance Estimation in Autoregressive Models
Jul 21, 2026Eric Price, Kevin Tian, Zhiyang Xun +1LLM Evaluation LLM Inference
University of Texas at Austin

Sep 14, 2026, cs.DCJ/K move · Enter open · S save

RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems

Ziyue Yang, Yuting Jiang, Lei Qu, Peng Cheng

Shanghai Xingyunzhili Artificial Intelligence Institute, Shanghai, China · Microsoft Research, Redmond, WA, USA

LLM Inference Large Language Model-Guided Optimization LLM Inference Acceleration

Open paper AskPDF arXiv

AI is beginning to make substantive contributions to LLM inference optimization. Existing AI optimizations are predominantly profiling-based. Profiling-bound feedback confines the search to the capabilities and performance of an existing software stack, preventing a fundamentally better architecture of LLM inference systems from being identified. To enable the AI-driven LLM inference system architecting loop, we argue that a general workload representation, a verifiable mutation space, and an implementation-independent evaluator are required. We present the RoofLang domain-specific language (DSL) that provides these features. In our evaluation, RoofLang reveals that DeepSeek V4-series models could achieve 3.5-39.5

\times

higher peak decode throughput than other representative models. This gap is disproportionate to their total parameter counts and arises largely from compact KV-cache designs that support larger batches and reduce memory traffic. A persistent optimizer agent further discovered several new architectures that improved both throughput and interactivity of DeepSeek V4 Pro on NVIDIA B300 by 6.23-50.1%.

Momentum

Related topics

Latest papers 187