Customer Side Queue

Customer-side queues model the waiting experienced by users or requests in various systems, with research focusing on optimizing resource allocation and minimizing wait times to improve user experience and system efficiency. Current research employs diverse approaches, including reinforcement learning algorithms to manage admission control and stochastic programming for multi-model queue management in applications like large language model serving. These advancements aim to improve service level objectives, resource utilization, and overall system performance across domains ranging from cloud computing to ride-sharing platforms and electric vehicle charging stations.