Green or Fast? Learning to Balance Cold Starts and Idle Carbon in Serverless Computing

Serverless computing eliminates infrastructure management overhead but introduces significant challenges regarding cold start latency and resource utilization. Traditional static resource allocation often leads to inefficiencies under variable workloads, resulting in performance degradation or excessive costs. This paper presents an adaptive engineering framework that optimizes serverless performance through event-driven architecture and probabilistic modeling. We propose a dual-strategy mechanism that dynamically adjusts idle durations and employs an intelligent request waiting strategy based on slot survival predictions. By leveraging sliding window aggregation and asynchronous processing, our system proactively manages resource lifecycles. Experimental results show that our approach reduces cold starts by up to 51.2% and improves cost-efficiency by nearly 2x compared to baseline methods in multi-cloud environments.

Modern cloud deployments distribute applications across multiple geographic regions, yet standard routing mechanisms prioritize latency while ignoring the fluctuating carbon intensity of local power grids. Latency-driven routing incurs avoidable carbon emissions, particularly when cleaner regions are within acceptable latency bounds. The proposed model formulates the carbon-aware serverless routing problem as a constrained optimization over geo-distributed cloud regions and introduces an SLA-constrained carbon-aware routing policy that achieves optimal carbon reduction within the SLA-feasible region, evaluated using real carbon intensity measurements across 5 primary AWS deployments. Experimental results show that the proposed policy achieves up to 46.8% carbon reduction while maintaining zero SLA violations across all evaluated thresholds. The system reduces carbon by an average of 27.4% under mixed workloads, and the routing overhead is very low (less than 0.02% of total request latency). A scalability study across 12 AWS regions spanning 6 continents demonstrates that average carbon savings increase from 27.4% to 47.5% as routing flexibility expands under mixed workloads. The proposed work contributes to SDG 13 (Climate Action) and SDG 7 (Affordable and Clean Energy) by enabling low-carbon routing decisions. These results indicate that cloud systems can achieve significant carbon savings without compromising user experience.

The growing deployment of large language models (LLMs) makes per-request routing essential for balancing response quality and computational cost across heterogeneous model pools. Current routing methods rarely consider sustainable energy use and CO2 emissions as optimization objectives, despite grid carbon intensity varying by time and region, and models differing significantly in energy consumption. To address this gap, we introduce Green-Aware Routing (GAR), a constrained multi-objective optimization framework that minimizes per-request CO2 emissions subject to explicit accuracy floors and p95-latency service-level objectives (SLOs). GAR employs adaptive constraint optimization through per-dataset floor tuning and incorporates lightweight estimators for correctness, tail latency, and carbon emissions, enabling real-time routing decisions without additional inference passes. We present GAR-PD, a practical online primal-dual routing algorithm for rolling carbon budgets, alongside heuristic variants that achieve high feasibility coverage while limiting accuracy degradation. Comprehensive experiments across standard NLP benchmarks with heterogeneous LLM pools (7B-70B) demonstrate that GAR achieves substantial carbon reductions while maintaining competitive accuracy and p95 latency guarantees, providing a practical, theoretically grounded approach to sustainable LLM inference.