PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Response

Yueying Li; Jiayang Chen; Yuanfan Chen; Leo Han; Haoran Qiu; Esha Choukse; Rodrigo Fonseca; Udit Gupta

cs.DCAug 22, 2026

PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Response

Authors: Yueying Li, Jiayang Chen, Yuanfan Chen, Leo Han, Haoran Qiu, Esha Choukse, Rodrigo Fonseca, Udit Gupta

Organizations: Cornell University · Microsoft Azure Research · Cornell Tech

Abstract

AI inference clusters are increasingly constrained by instantaneous power, not just energy: grid operators condition new capacity on demand response, imposing time-varying power caps. Existing LLM serving systems optimize a static energy objective or shed fixed priority tiers under load; either way, goodput collapses when the power envelope moves. An LLM pipeline is not a uniform load: compute-bound prefill loses throughput almost linearly with GPU frequency, memory-bound answer decode sustains it down to $0.57\times$ nominal, and reasoning's thinking phase couples KV-cache capacity to scheduling -- so a cap should be steered to where each watt costs the least performance. PowerSlider does so with a new Flex SLO contract that turns bounded user slack into an optimization constraint, prefill--think--answer disaggregation exposing per-stage frequency and KV control, and a Karush--Kuhn--Tucker (KKT) online solver re-solving within 7.7 ms of every cap change, backed by a consolidated fail-safe that power-gates drained instances when DVFS bottoms out on static power. On SGLang with production traces, \sys{} sustains 78.3% online goodput at a 30% cap reduction versus 47.6% for the best of five baselines ( $1.64\times$ ), holds latency-critical tails within $1.3\times$ of nominal (baselines: $2.3$ -- $6\times$ , up to $12\times$ ), and delivers 92% mean goodput through a replayed CAISO grid-emergency day bottoming at $0.41\times$ (54% at the trough; every baseline below 7%).

Figures & tables

Fig. 1: Demand response is richer than a step cap. (a) California Independent System Operator (CAISO) net load during a grid emergencies; (b) Power caps under 5 min settlement interval in high-demand and typical day [ 18 ] . Black steps: base interruptable program (BIP) events, dispatched 30 min after the alert, deeper on the more severe day [ 55 ] . EEA2/3 are grid emergency levels.

	Scope	Power-Aware	Reasoning	Dynamic Cap
PowerWeave [ 37 ]	GPU	✓
Power Sloshing [ 10 ]	Server	✓
PowerGrad / POLCA / Splitwise DynamoLLM [ 52 , 57 , 56 , 65 ]	Cluster	✓
SLOs-Serve [ 9 ]	Cluster		✓
PowerSlider	Cluster	✓	✓	✓

TABLE I: Comparison of prior work to PowerSlider

Fig. 2: Load-duration trends show that peak power demand is driven by tail behavior: a small amount of service flexibility (5%) substantially reduces required peak power provisioning. We sweep arrival burstiness (larger coefficient of variation gives more power headroom) and quantify how different load-shedding rates flatten the peak. This burst-driven slack is the raw material PowerSlider converts into demand-response headroom (O1).

Fig. 3: Flexible SLOs enable graceful degradation across power-cap reductions. At full power both paradigms deliver the same total goodput. As the cap tightens, the rigid policy protects LC requests by rejecting BE traffic and still incurs LC violations; the flexible-SLO policy allows controlled LC degradation, preserving BE service and higher overall goodput. Rigid tiers waste the slack of under a cap; Flex tier exists to spend it.

Fig. 4: Measured throughput, power, energy, and latency vs. GPU frequency for five batch sizes (BS) on A100-80GB. The frequency at which throughput saturates shifts with batch size, and BS < 128 (memory-bandwidth-bound) saturates within the DVFS range, above

810

MHz, saturation happens lower for lower BS; from BS>128 (compute-bound) the knee lies beyond the maximum 1,410 MHz, so any frequency decrease costs a proportional throughput drop. PowerSlider exploits this differential: it reduces batch-constrained answer-stage frequency first, shedding power at minimal performance cost. This stage and batch-related asymmetry is the hardware lever PowerSlider ’s solver pulls whenever the cap moves (O2).

Dimension	Examples
Stage	Prefill / Think / Answer
SLO class	LC / Flex / BE
GPU allocation	$k_{i}$ GPUs per stage-class
Frequency	210–1410 MHz per stage-class
KV chunk size	e.g., 512 / 1024 / 2048 tokens
Model / TP setting	Llama-70B, CodeLlama-34B, Qwen-32B; TP-4

TABLE II: The per-cap configuration space PowerSlider must navigate.

Fig. 5: Reasoning workloads have shorter prompts (left) but far longer, heavier-tailed output distributions (right), defeating the output-length prediction that prior resource-allocation and routing schemes depend on. These tails are why PowerSlider admits reasoning work by observation rather than by length prediction (O3), which matters in the near-saturation regime.

Component	Cadence	Crit. path?	Overhead
PSRoute (dispatch/admission)	per request	yes	virtual-queue lookup
PSSched: pool reallocation	5 min + 10 s refine	no	drain-bounded
$\hookrightarrow$ vote–commit DVFS	50–100 ms	no	one NVML call/GPU
PSOpt (KKT solve)	DR event / hourly	no	under 100 ms

TABLE III: PowerSlider control components

Fig. 6: Architecture overview of PowerSlider . Dark-headed boxes are PowerSlider ’s components; the gray bar is the existing stack (SGLang runtime, GPU driver, NVML DVFS). Three SLO classes flow left-to-right through the Prefill–Think–Answer pipeline; each pool forms its own frequency domain ( $f_{\mathrm{P}},f_{\mathrm{T}},f_{\mathrm{A}}$ ), and the KV cache is partitioned per class (colors match the user chips). PSOpt turns the grid cap $p_{\max}(t)$ into per-stage allocations, frequencies, and KV budgets; PSSched actuates them; PSRoute admits and dispatches per request.

Fig. 7: Normalized online and BE goodput vs. power-cap reduction for (top) reasoning and (bottom) non-reasoning workloads. PowerSlider sustains

1.64\times

the best baseline’s goodput at a 30% reduction (top) and 100% goodput for all classes through a 60% reduction (bottom). Main end-to-end result: PowerSlider widens the demand-response operating envelope, making the 5–20% reductions grid programs request (§ II-A ) essentially free.

Fig. 8: P90 TTLT and TTFAT vs. power-cap reduction for the LC and Flex classes. PowerSlider preserves near-nominal latency up to a 60% power-cap reduction, while the best-performing baseline incurs up to

12\times

higher LC latency and

18\times

higher Flex latency at a 50% reduction. The Flex disparity is larger because baselines do not isolate execution stages and prioritize LC at Flex’s expense. Near-nominal tails are what keep demand-response participation compatible with latency SLOs.

Fig. 9: Ablation of PowerSlider components ( -KKT , -PTA , -Flex ) for LC (top) and Flex (bottom) classes; TTFAT (left) and TTLT (right). The full design minimizes P90 latency across all regimes. Annotations indicate % latency increase over PowerSlider at a 30% power-cap reduction. Absorbing a cap needs all three mechanisms; no single knob suffices.

Fig. 10: A real grid-emergency day replayed as the 24 h DR day. The cap schedule (gray) replays the Fig. 1 emergency day (Sep 6; trough 0.41

\times

) [ 23 ] . (a) PowerSlider holds

{\geq}98\%

online goodput all day; every baseline collapses to 0–6.6% at the trough and none recovers in-trace (post-22.5 h windows censored). (b) Per-stage GPU allocation under drain-before-reassign (43 reconfigurations).

Fig. 11: Power tracking (a) and Flex contract validation (b, c) on the compressed 24 h DR day. (a) Delivered power on the replayed day in Fig. 1 : PowerSlider stays under the binding cap throughout; (b) Flex E2E latency CDFs (0–60% reductions;

L_{c}{=}128.9

s) stay inside the

(\alpha_{c}{=}3,\rho_{c}{=}0.3)

envelope at every depth; (c) POLCA breaches it beyond 30% power reduction

Fig. 12: Top: PSOpt solve time scales near-linearly with group count, staying inside one 100 ms DVFS interval. Bottom: per-phase P50 latency, PD vs. PTA: under a 40% cap PD develops decode queueing while PTA keeps it near zero (

12.5\times

lower end-to-end latency). Millisecond re-solves track the moving cap; stage isolation keeps it from becoming queueing collapse.

Model	TTF(A)T	TTLT/TBT	Dataset
CodeLlama-34B	72.0s	115.5s	Magpie
QWen-32B	36.0s	121.0s	S1K
DS-R1-Llama-70B	5.0s	0.50s	Azure-Code
QWen-14B	0.30s	0.1s	Azure-Chat

TABLE IV: Workloads in evaluation and latency requirements.

Name	Arch	DVFS Policy	Scheduler	Multi-SLO
B1 Uniform	PD disagg	Uniform	Overlap-KV JSQ	Single class
B2 POLCA	Collocated	Priority-aware	Mixed-JSQ	HP/LP binary
B3 SplitWise+	PD disagg	Stage-aware	Overlap-KV JSQ	+Priority
B4 DynamoLLM+	Collocated	+Uniform	Prediction rtr.	SLO-aware
B5 SLOs-Serve+	Collocated	+Uniform	ProMax	LC/BE priority
PowerSlider	PTA disagg	KKT solver	Multi-SLO JSQ	LC/Flex/BE

TABLE V: Baseline configurations.

Fig. 13: Latency degradation from prediction-based misrouting under increasing per-instance offered load (128-instance cluster). At low load (

\leq

2 rps per instance) misrouting has negligible impact; near saturation (

\sim

5 rps per instance), TTFT mean degrades

>

100% and P99 exceeds 1,000%, while end-to-end and TTFAT remain buffered until saturation.

Fig. 14: Robustness of the PTA benefit across workload distributions. Panels (a)–(c) increase the T/A ratio from 0.5:1 to 16:1 (average prefill 512); panel (d) uses average prefill 2048. The advantage grows with T/A ratio (49% TBAT reduction at 0.5:1 up to 71% at 16:1 at high load) because longer thinking chains cause more severe batch interference under PD and colocated serving, which PTA eliminates by isolating answer decode onto dedicated workers.

Fig. 15: Per-request response times for LC, Flex, and BE classes as the power cap steps from 90% to 60% of nominal at $t=750$ s on a 512-GPU cluster. PowerSlider completes the most requests across all classes with stable response times; baselines either shed BE traffic or accumulate unbounded queueing.

Fig. 16: Sensitivity of PowerSlider ’s Flex SLO parameters on LC goodput, normalized to POLCA. Top: varying the relaxation ratio

\alpha

with

\rho=0.2

. Bottom: varying the violation-budget fraction

\rho

with

\alpha=3.0

. At a 60% power-cap reduction, PowerSlider delivers up to

1.85\times

the goodput of POLCA and remains robust over a wide range of

\alpha

and

\rho

. The contract that funds PowerSlider ’s power headroom does not need careful tuning to deliver it.

Fig. 17: Workload-mix sensitivity under a dynamic user mix (measured). (a) The LC/Flex/BE arrival mix drifts from 5% to 80% Flex within a single run (QPS 14, CoV 7.5 bursty arrivals, 64 GPUs); the gray line is the arrival rate. (b) Online goodput per mix segment at a 30% power-cap reduction ( PowerSlider also shown at 50%): PowerSlider holds 95–100% goodput across the entire drift (95.3% at 80% Flex, shedding BE admissions when the cap makes full service infeasible), while POLCA collapses to 10% at 80% Flex. (c) Mix extremes – LC-dominant (90/5/5), default (30/30/40), Flex-dominant (10/80/10) – at 30% (solid) and 50% (faded) reductions.

Fig. 18: Energy–goodput Pareto and economic break-even under power caps (measured). (a) Cluster energy per generated token vs. online goodput, one point per system and cap depth (0–60%; ideal is bottom-right). PowerSlider alone holds the ideal corner at every depth (0.34–0.44 J/token at 97.9–100% goodput; inset), while at a 60% cap the baselines pay

\approx

\times

the energy per useful token at 12–23% goodput. (b) Net benefit per 2 h DR event vs. cap depth; PowerSlider is net-positive at every depth where the baselines turn negative (§ VII-G ).

Program	Resp. req.	PowerSlider path	Feasible
Freq. regulation (AGC)	seconds	DVFS band ( $<$ 1 s)	✓ ( $\leq$ 20%)
Spinning reserve	10 min	reallocation epoch	✓ (full depth)
Emergency DR	30 min	reallocation epoch	✓ (full depth)
Day-ahead / economic	hours	capacity planning	✓
Stacked (reg. $+$ reserve)	both	both bands	✓

TABLE VI: Grid-program feasibility given PowerSlider ’s actuation paths (7.7 ms solve;

<

1 s DVFS band; 5 min reallocation epoch).

LLM Serving

Large language model serving studies how to deliver model responses with low latency, high throughput, and efficient use of compute and memory.

See all 207 papers

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
May 20, 2026Can Hankendi, Rana Shahout, Minlan Yu +1LLM Inference Efficiency LLM Serving
Boston University · Electrical and Computer Engineering Department · Boston, MA, USA +3
Characterizing Job Power Elasticity for Power-Flexible AI Training
Sep 10, 2026Philip Colangelo, Charles Dawson, Shayan Sengupta +2Energy-Efficient ML Efficient Language Model Training
Emerald AI
VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing
Sep 5, 2025Jiahuan Yu, Aryan Taneja, Junfeng Lin +1LLM Serving Energy-Efficient ML
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Champaign, USA · Department of Precision Instrument, Tsinghua University, Beijing, China

May 20, 2026J/K move · Enter open · S save

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models

Can Hankendi, Rana Shahout, Minlan Yu, Ayse K. Coskun

Boston University · Electrical and Computer Engineering Department · Boston, MA, USA+3

LLM Inference Efficiency LLM Serving Energy-Efficient ML

Open paper AskPDF arXiv

Large language model (LLM) inference has become a dominant workload in modern data centers, driving significant GPU utilization and energy consumption. While prior systems optimize throughput and latency by batching, scheduling, and parallelism, they largely treat GPU power as a static constraint rather than a controllable resource. In this paper, we present a power-aware runtime for LLM serving, PALS, that treats GPU power caps as a first-class control knob and jointly optimizes them with software parameters such as batch size. The system combines lightweight offline power-performance models with a feedback-driven controller to select configurations that satisfy throughput targets while maximizing energy efficiency. We implement PALS within an existing LLM serving framework, vLLM, demonstrating that it requires no model retraining or API changes. Across multi-GPU systems and both dense and mixture-of-experts (MoE) models, PALS improves energy efficiency by up to 26.3%, reduces QoS violations by 4x to 7x under power constraints, and tracks dynamic power budgets. These results highlight the potential of integrating power control directly into LLM inference runtimes, enabling energy-proportional and grid-interactive AI systems.

Abstract

LLM Serving

Explore similar work

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models

Characterizing Job Power Elasticity for Power-Flexible AI Training

VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing