Characterizing Software Aging in GPU-Based LLM Serving Systems

Domenico Cotroneo; Bojan Cukic

cs.SEJun 10, 2026

Characterizing Software Aging in GPU-Based LLM Serving Systems

Authors: Domenico Cotroneo, Bojan Cukic

Organizations: University of North Carolina at Charlotte · College of Computing and Informatics

Abstract

This paper proposes an empirical methodology to study software aging in GPU-based LLM serving systems. Traditional aging studies focus on CPU-centric software with relatively regular workloads; LLM serving is different, spanning a Python host and a CUDA device, handling requests whose cost varies by orders of magnitude, and relying on rapidly evolving software stacks. We run a 216-hour campaign across six co-located deployments under identical stress conditions, monitor host, device, and client metrics in parallel, and apply a statistical pipeline that accounts for autocorrelation and multiple testing. Our results reveal statistically significant memory aging in all deployments, with leak rates strongly dependent on the serving runtime and deployment configuration. Beyond these findings, we provide a reproducible framework that opens a research direction at the intersection of the software aging and rejuvenation and LLM serving communities.

Explore similar work

LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs
Aug 4, 2026Forough Majidi, Mohammad Mehdi Morovati, Foutse Khomh +1Large Language Model Serving Empirical Study
Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control
Aug 12, 2026Josef Liyanjun ChenMulti-Gpu Systems Consumer Graphics Processing Units
WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs
Jul 2, 2026Mauricio Fadel Argerich, Jonathan Fürst, Marta Patiño-MartínezLLM Inference Efficiency Inference Workloads

Aug 4, 2026 · cs.SEJ/K move · Enter open · S save

LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs

Forough Majidi, Mohammad Mehdi Morovati, Foutse Khomh, Heng Li

SWAT Lab., Polytechnique Montr´eal, Montr´eal, Canada

Large Language Models (LLMs) are integrated into software systems and AI services, making efficient LLM serving a concern for software engineering. Serving LLMs is challenging because inference requires computation, memory, GPU resources, and execution while maintaining latency and throughput. Although prior research has proposed LLM inference, optimization, and serving techniques and frameworks, little is known about how they are adopted in practice. In this study, we investigate the use of LLM serving frameworks and serving methods in open-source software systems. We identify and analyze five LLM-specific frameworks: vLLM, SGLang, TensorRT-LLM, LMDeploy, and FlashInfer. We examine how these frameworks and techniques are adopted individually and in combination, how adoption varies across categories of LLMs, and how repositories differ in intent, focus, use case, and architectural design. Our results show that vLLM is the most visible framework in popularity and adoption, while parallel computation, memory management, and network pruning are the most frequently used serving-method categories. Multi-framework usage is limited, suggesting that developers rely on a single serving framework; however, combined frameworks connect complementary capabilities across the serving stack. Framework adoption varies across model families, modalities, model sizes, domain specializations, and deployment settings. Repository-level analysis shows that LLM serving frameworks support applications and architectures, including Reinforcement Learning (RL)-based reasoning, multimodal generation and understanding, microservices, and cloud infrastructure. Overall, this study provides a large-scale empirical characterization of LLM serving framework adoption in practice and offers insights for researchers, framework maintainers, and practitioners working on LLM systems.

Large Language Model Serving Empirical Study Ai-Driven Repository-Level Code Understanding

Open paper Ask about it PDF

Abstract

Explore similar work

LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs

Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control

WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs