cs.DCOct 31, 2026

Ripple: Deadline-Aware CPU-GPU Sharing for Embodied Inference

Authors: Jiadong Zhang, Xiaosong Ma

Organizations: MBZUAI.

Abstract

Unified-memory edge devices increasingly run LLM/VLM inference alongside time-critical CPU tasks for perception and control. GPU memory traffic can overwhelm these tasks, yet platforms such as DGX Spark expose neither bandwidth counters nor direct bandwidth regulation. We present Ripple, a CPU-GPU co-scheduler that combines model-derived GPU memory volumes with CPU completion feedback to regulate interference. Ripple divides inference into pausable, layer-based units and translates each CPU job's deadline slack into a GPU-byte budget, admitting work within the budgets of all overlapping jobs. Online updates track changing interference, while deadline-aware pacing and judicious SM footprint narrowing preserve inference progress. Implemented with SGLang engine hooks and a central scheduler, we evaluate Ripple on DGX Spark with four robotic CPU task mixes and LLM/VLM inference. It is the only evaluated approach that consistently meets the 2% CPU deadline-miss target and VLM deadlines across the tested deadline settings while retaining best-effort LLM throughput.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Blink: CPU-Free LLM Inference by Delegating the Serving Stack to GPU and SmartNIC

    Apr 8, 2026Mohammad Siavashi, Mariano Scazzariello, Gerald Q. Maguire +2LLM ServingLLM Inference

  2. Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices

    Jul 11, 2026Yangyijian Liu, Hongyi Ye, Mingyang Li +1On-Device Language Model InferenceLLM Inference Scheduling