cs.RONov 4, 2026

Where Is VLA Inference Time Spent? Bottlenecks and Optimization Interactions Across GPUs

Authors: Muhammad Danish Waseem, Ahmed Ali-Eldin Hassan

Organizations: Department of Computer Science and Engieering, Chalmer University and University of Gothenburg, Gothenburg, Sweden.

Abstract

Vision-language-action (VLA) models map camera observations and language instructions to robot actions. However, their large backbones and repeated action generation can make inference take longer than the robot's real-time requirements for its next action. To reduce the load of VLA computations, researchers have proposed multiple optimization approaches, including quantization and pruning. These optimizations can be combined with other general optimizations such as better kernel compilation or using CUDA graphs instead of the eager mode. However, most of these optimization results are usually reported on a single GPU type, typically for a few VLA models, with no analysis of what the real bottlenecks are for a model on a particular hardware or how these optimizations would generalize beyond the reported results. In this paper, we provide a comprehensive measurement study of seven families of inference optimization methods, from CUDA graphs and compilation to token pruning and quantization, on seven state-of-the-art VLAs evaluated on four GPU families ranging from a datacenter H100 to an embedded Jetson Thor. We analyse how CPU launch overhead, memory bandwidth, and kernel efficiency jointly shape optimization gains, and how applying one method changes the benefit of another. From this analysis, we derive VLA-Triage, guidelines for stacking optimizations under latency budgets. The guidelines identify which stages can provide the required savings and which combinations need testing in the deployed execution mode.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. VLA-HWBench: Where the Time Goes When VLA Models Run on Edge Hardware

    Oct 26, 2026Borui Xu, Hongshi Tan, Wenqi Jiang +4Efficient VLA Model InferenceOn-Device Inference

  2. A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference

    Jul 27, 2026Zhuoran Song, Haozhe Jiang, Chunyu Qi +4AI Accelerator InferenceEfficient VLA Model Inference