Prefill-Decode Disaggregation

Follow this topic Ask about this topic

Latest papers

13 papers5 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Prorsa: Unleashing the Potential of Prefill-Decode Disaggregation in Agentic Reinforcement Learning
Oct 12, 2026Chiheng Lou, Xiangwei Deng, Bangjun Xiao +4Prefill-Decode Disaggregation Agentic RL
†School of Computer Science, Peking University · ‡LLM-Core Xiaomi
FluidPD: In-Place Elasticity for SLO-Aware Prefill-Decode Disaggregated LLM Serving
Oct 2, 2026Kartik Ramesh, Kaidi Fu, Zihan Zheng +4LLM Serving Prefill-Decode Disaggregation
University of Illinois Urbana-Champaign · IBM Research
PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learning
Sep 28, 2026Jiaan Zhu, Wei Gao, Youhui Bai +6Agentic RL Asynchronous RL
USTC China
Crossflow: Prefill-Decode Elasticity for Agentic LLM Serving
Sep 22, 2026Yi Xu, Ehsan K. Ardestani, Wenyin Fu +7Disaggregated LLM Serving LLM Inference Scheduling
Meta Platforms
Disaggregated Quantization: Specializing LLM Prefill and Decode
Sep 22, 2026Andrei Panferov, Maximilian Kleinegger, Sweta Priyadarshi +2LLM Quantization LLM Inference
NVIDIA · ISTA
Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving
Jul 2, 2026Shrikara Arun, Anjaly Parayil, Srikant Bharadwaj +2LLM Serving Prefill-Decode Disaggregation
Microsoft
Prefill/Decode-Aware Evaluation of LLM Inference on Emerging AI Accelerators
Jun 14, 2026Shun Usami, Venkatram Vishwanath, E. Wes BethelAI Accelerator Inference LLM Inference
Department of Computer Science San Francisco State University San Francisco, CA, 94132 · Argonne National Laboratory Lemont, IL, 60439 · Lawrence Berkeley National Laboratory Berkeley, CA, 94720
Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models
Jun 9, 2026Jing Xiong, Qi Han, Shansan Gong +5Lost-in-the-Middle Effect Prefill-Decode Disaggregation
The University of Hong Kong · University of Michigan, Ann Arbor · LMSYS Org
Achieving Cloud-Grade SLOs for Local Mixture-of-Experts Inference through CPU-GPU Hybrid Design
Jun 9, 2026Wenxin Wang, Yule Hou, Yu Ji +2Expert Parallelism On-Device Language Model Inference
1Tsinghua University · 2Xingyun Integrated Circuits Co., Ltd. · 3Beijing National Research Center for Information Science and Technology
SpectrumKV: Per-Token Mixed-Precision KV Cache Transfer for Prefill-Decode Disaggregated LLM Serving
Jun 7, 2026Yang PengjuMixed-Precision Quantization Prefill-Decode Disaggregation
E2LLM: Towards Efficient LLM Serving in Heterogeneous Edge/Fog Environments
Jun 2, 2026Truong-Thanh Le, Amir Taherkordi, Hoang-Loc La +3LLM Serving Prefill-Decode Disaggregation
Department of Informatics University of Oslo Oslo, Norway · Department of Computer Science UiT The Arctic University of Norway Tromsø, Norway
SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving
May 3, 2026Yipin Guo, Siddharth JoshiDisaggregated LLM Serving KV-Cache Compression
University of Notre Dame IN, USA
SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference
Mar 5, 2026Luchang Li, Dongfang Li, Bozhao Gong +1Disaggregated LLM Serving Prefill-Decode Disaggregation
Kingsoft Cloud

Oct 12, 2026J/K move · Enter open · S save

Prorsa: Unleashing the Potential of Prefill-Decode Disaggregation in Agentic Reinforcement Learning

Chiheng Lou, Xiangwei Deng, Bangjun Xiao, Xiaodong Ji+3

†School of Computer Science, Peking University · ‡LLM-Core Xiaomi

Prefill-Decode Disaggregation Agentic RL LLM Inference Scheduling LLM Inference Acceleration Disaggregated LLM Serving

Open paper AskPDF arXiv

Agentic reinforcement learning (RL) is commonly used to enhance the capabilities of large language models (LLMs) in solving complex multi-turn tasks. During rollout, agentic RL exhibits a workload pattern distinct from that of traditional RL: multi-turn interactions repeatedly introduce prefill requests, causing prefill and decode workloads to coexist throughout rollout. This workload makes prefill-decode (PD) disaggregation, a design widely used for online LLM serving, promising for speeding up agentic RL rollout. This paper shows that PD disaggregation can improve rollout efficiency by (a) mitigating interference among data-parallel attention modules, (b) enabling stage-specific parallelism, and (c) reducing end-to-end trajectory completion time. However, a fixed prefill-decode allocation may fail to sustain these benefits as workload composition varies across tasks and over time. We therefore present Prorsa, an agentic RL framework for prediction-guided prefill-decode resource allocation. Prorsa observes the lifecycle of every trajectory to forecast short-horizon prefill and decode demand. These forecasts then guide worker allocation decisions and trigger role switching of rollout engines, aligning prefill and decode capacity with anticipated workload demand. Across synchronous and asynchronous agentic RL settings, Prorsa achieves end-to-end rollout speedups of 1.11x-4.05x over prefill-decode colocation.