Vision-Language-Action (VLA) models have emerged as foundational models for next-generation robotics. High VLA inference throughput is critical for meeting the control-rate requirements of robots. VLA models comprise two phases, a vision-language model (VLM) and an action head, that can be decoupled and executed asynchronously and concurrently across independent robot requests. Through a detailed characterization of four state-of-the-art VLAs, we observe that GPUs are severely underutilized in VLA inference as the Cooperative Thread Array (CTA) scheduler of GPU is unable to fully overlap the two independent phases of VLA execution. We identify that this inefficiency is caused by head-of-line blocking in the hardware thread-block dispatcher. We demonstrate that prior scheduling frameworks do not address the challenges posed by VLA concurrency. First, the independent phases across different robot requests each comprise numerous kernels, and at any given time, there are different combinations of kernels that are executed in parallel. This makes static or ahead-of-time scheduling policies largely ineffective. Second, many of the action-head operators are short-running kernels and there are numerous such kernels. This leaves no headroom for online profiling or preemption-based mechanisms. To address these challenges, we present KerColle, a lightweight GPU scheduling framework that leverages online Streaming Multiprocessor (SM) utilization and individual kernel resource requirements to intelligently and dynamically co-schedule kernels to efficiently overlap the two phases of execution by (1) mitigating head-of-line blocking, and (2) co-scheduling kernels with complementary resource requirements. We demonstrate in simulation, across two GPU architectures, for 4 state-of-the-art VLA models, that KerColle delivers throughput gains of up to 28%.
Figures & tables
Figure 1. Representative VLA architecture decomposed into two phases.
Figure 2. Execution timelines for action chunking and asynchronous inference.
Figure 3. High-level overview of Streaming Multiprocessor (SM) and processing cores on modern GPUs.
Figure 4. CDF of inference time vs. SM throughput for a single-image inference on the RTX 4090.
Figure 5. Resource utilization profiles of the VLM and Action Head phases for CogACT (top) and π0.5 (bottom) across RTX 4090, H100, and RTX 5090.
Figure 6. Roofline of VLA kernels on RTX 4090. Large markers are time-weighted centroids.
Figure 7. Execution patterns for asynchronous VLA inference. (a) Sequential execution serializes the two phases. (b) Naive two-stream overlap leaves the action stream lagging behind. (c) Ideal overlap fully hides action-head latency behind the next frame’s VLM.
Figure 8. (a) Default GPU scheduling. Head-of-line blocking behavior. (b) Ideal co-scheduling. Fully overlapped execution within an SM.
Workload
Median
< 5K
< 10K
< 25K
< 50K
(cyc)
(%)
(%)
(%)
(%)
CogACT ( Li et al., 2024 )
7,156
18.9
67.2
88.9
95.2
Octo ( Team et al., 2024 )
6,832
44.3
69.8
85.4
100.0
π0.5 ( Intelligence et al., 2025 )
9,348
17.9
51.7
73.7
90.2
GRASP-VLA ( Deng et al., 2025 )
12,407
23.9
46.3
77.6
88.4
Table 1. Execution profile of micro-kernels across different VLA workloads after applying a CUDA Graph capture.
Scheduler
Mechanism
Unit Aware
Micro-K. Resp.
Black- Box
SMK ( Wang et al., 2016b )
HW Ctx Switch
×
×
✓
Warped-Slicer ( Xu et al., 2016 )
Epoch + Static
×
×
✓
Maestro ( Park et al., 2017 )
Epoch + 2-Stack
×
×
✓
GPUPool ( Tan et al., 2023 )
ML + TB/SM Cap
✓
×
✓
Kitsune ( Davies et al., 2025 )
Dataflow + L2 Q
✓
✓
×
KerColle (Ours)
Dynamic Score
✓
✓
✓
Table 2. Qualitative comparison of prior GPU co-scheduling frameworks
Figure 9. High-level overview of KerColle design.
Field
Description
τTC
Tensor Core (TC) pipeline intensity. Fraction of peak TC throughput that CTAs consume.
τALU
ALU pipeline intensity. Fraction of peak ALU (CUDA Core) throughput consumed per CTA.
τMEM
Memory bandwidth intensity. Fraction of peak DRAM bandwidth consumed per CTA.
Table 3. A kernel’s Intensity Descriptor. Each field (4 bits) encodes a normalized intensity level in the range 0–15.
Register
Description
UTC[j]
Estimated Tensor Core utilization on SM j .
UALU[j]
Estimated ALU pipeline utilization on SM j .
UMEM[j]
Estimated memory bandwidth utilization on SM j .
Table 4. Per-SM 8-bit utilization registers (range [0,255] ) maintained by the scheduler.
Parameter
NVIDIA RTX 4090
NVIDIA RTX 5090
Generation
Ada Lovelace
Blackwell
SM Clock Frequency
2520 MHz
2407 MHz
SMs/GPU
128
170
Warp Schedulers/SM
4
4
Warp Scheduling Policy
GTO
GTO
Max #Thread Blocks/SM
24
32
Table 5. GPU Microarchitectural Parameters in Accel-Sim
Figure 10. Left: VLA inference throughput on RTX 4090. Right: Percentage increase in throughput gain of KerColle and Warped-Slicer relative to Baseline.
Figure 11. Left: VLA inference throughput on RTX 5090. Right: Throughput gain of KerColle and Warped-Slicer relative to Baseline.
Figure 12. Concurrent SM utilization under the baseline scheduler and KerColle across all models and GPUs (left y-axis), and KerColle throughput gain relative to baseline (right y-axis).
Figure 13. Left: Distribution of concurrent kernel pairs grouped by the smaller kernel’s thread block count. Right: Throughput gains of KerColle over baseline for four models correlated to the median thread block count of each model’s smaller kernels.
Figure 14. Warp occupancy of different KerColle configurations and Warped-Slicer on RTX 4090.
Figure 15. Warp occupancy of different KerColle configurations and Warped-Slicer on RTX 5090.
Vision-Language-Action (VLA) models are promising for generalist robot control, but on-robot deployment is bottlenecked by real-time inference under tight cost and energy budgets. Most prior evaluations rely on desktop-grade GPUs, obscuring the trade-offs and opportunities offered by heterogeneous edge accelerators (GPUs/XPUs/NPUs). We present a systematic analysis for low-cost VLA deployment via model-hardware co-characterization. First, we build a cross-accelerator leaderboard and evaluate model-hardware pairs under CET (Cost, Energy, Time), showing that right-sized edge devices can be more cost-/energy-efficient than flagship GPUs while meeting control-rate constraints. Second, using in-depth profiling, we uncover a consistent two-phase inference pattern: a compute-bound VLM backbone followed by a memory-bound Action Expert, which induces phase-dependent underutilization and hardware inefficiency. Finally, guided by these insights, we propose DP-Cache and V-AEFusion to reduce diffusion redundancy and enable asynchronous pipeline parallelism, achieving up to 2.9x speedup on GPUs and 6x on edge NPUs with only marginal success degradation. The example leaderboard website is available at: https://vla-leaderboard-01.vercel.app/.
Kaijun Zhou, Qiwei Chen, Da Peng +3
School of Computer Science, Shanghai Jiao Tong University
Deploying vision--language--action (VLA) models on robots requires adapting model-specific inference pipelines to heterogeneous processors and limited onboard memory. We present vla.cpp, a unified C++ inference runtime for eleven VLA models, with no PyTorch dependency for model execution. The runtime shares model loading, tensor execution, and serving while retaining architecture-specific attention, conditioning, and action heads. Iterative policies reuse observation-dependent computation across solver steps, while regression policies predict actions directly. We evaluate task success on LIBERO-Object and profile supported configurations on NVIDIA, Apple, and Intel hardware. BitVLA completes 200/200 LIBERO-Object episodes on an 8,GB Jetson Orin Nano. A ternary tensor-core kernel accelerates its client inference by 4.0--4.6× over the CUDA-core baseline on RTX 3060 and AGX Orin. A SmolVLA case study links positional-index precision to gripper commands and task success, showing why fixed-input numerical checks should accompany rollout evaluation. Deployments on UR10e and ALOHA demonstrate physical robot integration; delay and execution-horizon studies characterize synchronous chunked control. The results demonstrate a common deployment path across VLA architectures and hardware, with numerical validation and control settings guiding deployment alongside inference efficiency.
Khanh D. Nguyen, Hung T. Ho, Chinh T. Nguyen +5
VinRobotics, Vietnam · Max Planck Research School for Intelligent Systems, Germany · University of Stuttgart, Germany +3
Vision-language-action (VLA) models translate multimodal observations into low-level robot actions. During robot operation, each control period sets an inference deadline, and overruns leave the robot acting on stale observations, reducing task success. Meeting this deadline motivates on-device or nearby edge execution, where a single robot requires batch-1 inference outside the design point of LLM serving systems. Although VLA architectures combine familiar vision-language, autoregressive, and diffusion-style components, their runtime behavior in this batch-1 control setting remains uncharacterized. We characterize four representative VLA models on an edge GPU server and two onboard SoCs, using single-inference profiling and 43,200 closed-loop episodes. Action tensor dimensionality determines whether a stage is memory- or compute-bound, platform balance can shift that bottleneck, and GPU frequency scaling yields a platform-dependent energy-latency sweet spot. In closed-loop operation, overlapping inference with action execution creates an accuracy-speed-energy tradeoff, and no configuration is Pareto-dominant across deployment SLOs. These results guide joint design of VLA model architectures, hardware, and runtime policies.