Disaggregated LLM Serving

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 17

All topics

Evaluating Inference Compute for Generative AI: A Framework for Enterprise Workloads
Oct 5, 2026Abbas Raza Ali, Muhammad Ajmal Siddiqui, Moona ZahidLLM Inference Efficiency Disaggregated LLM Serving
Citigroup Inc., London, United Kingdom · Ernst & Young LLP, London, United Kingdom · NVIDIA Corporation, London, United Kingdom
SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving
Sep 28, 2026Gunho Park, Kyoungho Jeun, Juntaek Oh +3Mixture-of-Experts Pruning Disaggregated LLM Serving
a2sys · KAIST
Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving
Sep 22, 2026Yi Xu, Ehsan K. Ardestani, Wenyin Fu +7Disaggregated LLM Serving LLM Inference Scheduling
Meta Platforms
MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving
Sep 14, 2026Tiancheng Zhang, Yulin Chen, Yunfeng Zhao +3Disaggregated LLM Serving LLM Inference Scheduling
College of Intelligence and Computing, Tianjin University, Tianjin, China · The International Joint Institute of Tianjin University, Tianjin University, Fuzhou, China · Faculty of Digital Economics and Managements, Tianjin University of Finance and Economics, Tianjin, China
Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
Sep 14, 2026Arya Tschand, Yaosheng Fu, Vikram Sharma Mailthody +6Disaggregated LLM Serving LLM Inference Acceleration
Harvard University Cambridge, Massachusetts, USA · NVIDIA Santa Clara, California, USA
Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Serving
Sep 11, 2026Jae Gon Kim, Donghoon Yoo, Hanyul Ryu +3LLM Inference Efficiency Disaggregated LLM Serving
Xenoscube, Inc.
AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregation
Aug 5, 2026Chengyu Qiu, Xiao Fu, Fengcun Li +6Mixture-of-Experts Language Models Disaggregated LLM Serving
Tsinghua University · Meituan · Beijing, China
Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling
Aug 3, 2026Cunchen Hu, Liangliang Xu, Tian Liu +9Energy-Efficient ML Disaggregated LLM Serving
China Telecom Cloud Computing Research Institute · Xidian University · University of Science and Technology of China +1
Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving
Jul 2, 2026Shrikara Arun, Anjaly Parayil, Srikant Bharadwaj +2LLM Serving Disaggregated LLM Serving
Microsoft
Semantic Cache Distillation: Efficient State Transfer via Reuse and Selective Patching
Jun 5, 2026Qianli Ma, Zhiqing Tang, Hanshuai Cui +2LLM Serving KV Caching
Faculty of Arts and Science, Beijing Normal University, Zhuhai 519087, China · Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai 519087, Guangdong, China · School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China +1
Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving
Jun 1, 2026Jianru Ding, Ryien Hosseini, Pouya Mahdi Gholami +2LLM Serving Disaggregated LLM Serving
University of Chicago
How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
May 27, 2026Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee +9LLM Serving LLM Inference
1Georgia Institute of Technology · 5Infravana · 2Intel +2
Frontier: Towards Comprehensive and Accurate LLM Inference Simulation
May 20, 2026Yicheng Feng, Xin Tan, Yangtao Deng +3LLM Serving Disaggregated LLM Serving
The Chinese University of Hong Kong · Anuttacon · StepFun
KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Serving
May 13, 2026Zedong Liu, Xinyang Ma, Dejun Luo +9LLM Serving KV Caching
University of Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences · Shanghai Jiao Tong University
SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving
May 3, 2026Yipin Guo, Siddharth JoshiLLM Serving KV Caching
University of Notre Dame IN, USA
VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing
Sep 5, 2025Jiahuan Yu, Aryan Taneja, Junfeng Lin +1LLM Serving Energy-Efficient ML
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Champaign, USA · Department of Precision Instrument, Tsinghua University, Beijing, China
Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
Jun 24, 2024Ruoyu Qin, Zheming Li, Weiran He +4LLM Serving Disaggregated LLM Serving
Moonshot AI · Tsinghua University

Oct 5, 2026, cs.ARJ/K move · Enter open · S save

Evaluating Inference Compute for Generative AI: A Framework for Enterprise Workloads

Abbas Raza Ali, Muhammad Ajmal Siddiqui, Moona Zahid

Citigroup Inc., London, United Kingdom · Ernst & Young LLP, London, United Kingdom · NVIDIA Corporation, London, United Kingdom

LLM Inference Efficiency Disaggregated LLM Serving LLM Agent Evaluation Cost-Aware Inference

Open paper AskPDF arXiv

LLM deployment is shifting from single-turn completion to agentic trajectories in which a model plans, calls tools, reads results and reasons at test time before acting. This inverts the economics of inference hardware: chat serving amortises weight reads across large batches, whereas agent trajectories are sequentially dependent, run at effective batch one, and make per-token decode latency (TPOT) the dominant term in task completion time. Using a roofline analysis and a closed-form episode-latency model, we show why this regime favours accelerators that keep weights in on-die SRAM (Cerebras WSE-3/3T, Groq/NVIDIA LPU) or compiler-managed tiered memory (SambaNova SN40L/SN50), and why three vendor ecosystems converged in 2026 on disaggregated prefill/decode serving. We show that per-step reliability compounds exponentially in trajectory length-a 2% per-step failure rate erases a 2x decode advantage for a 20-step agent-so determinism and tail latency are first-order performance variables. We then propose a four-layer evaluation framework (silicon, serving system, agent episode, enterprise) with a metric set built on goodput at an agentic SLO and cost per successful episode, a six-axis benchmark protocol over six task families, a paired-bootstrap statistical design, an attestation protocol for vendor-run benchmarks, and TCO, availability and adoption-timing models with explicit break-even conditions. All performance figures are public and labelled by evidence class; we state seven falsifiable hypotheses and the experiments that test them, and argue that the most likely original result is that token-throughput rankings diverge from cost-per-successful-task rankings on long-horizon work.

Momentum

Related topics

Latest papers 17