Where Is VLA Inference Time Spent? Bottlenecks and Optimization Interactions Across GPUs
Organizations: Department of Computer Science and Engieering, Chalmer University and University of Gothenburg, Gothenburg, Sweden.
Abstract
Vision-language-action (VLA) models map camera observations and language instructions to robot actions. However, their large backbones and repeated action generation can make inference take longer than the robot's real-time requirements for its next action. To reduce the load of VLA computations, researchers have proposed multiple optimization approaches, including quantization and pruning. These optimizations can be combined with other general optimizations such as better kernel compilation or using CUDA graphs instead of the eager mode. However, most of these optimization results are usually reported on a single GPU type, typically for a few VLA models, with no analysis of what the real bottlenecks are for a model on a particular hardware or how these optimizations would generalize beyond the reported results. In this paper, we provide a comprehensive measurement study of seven families of inference optimization methods, from CUDA graphs and compilation to token pruning and quantization, on seven state-of-the-art VLAs evaluated on four GPU families ranging from a datacenter H100 to an embedded Jetson Thor. We analyse how CPU launch overhead, memory bandwidth, and kernel efficiency jointly shape optimization gains, and how applying one method changes the benefit of another. From this analysis, we derive VLA-Triage, guidelines for stacking optimizations under latency budgets. The guidelines identify which stages can provide the required savings and which combinations need testing in the deployed execution mode.
Figures & tables
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Step | Decisions (groups; near ) | ms: false accept/reject | %: false accept/reject |
|---|---|---|---|
| CUDA graphs, pooled | 294 (183; 28) | 0 / 0 | 0 / 4 |
| Compilation | 252 (252; 40) | 6 / 2 | 0 / 7 |
| Pruning then quant., compiled | 582 (582; 55) | 23 / 4 | 10 / 5 |
| Pruning then quant., graphs only | 42 (42; 5) | 2 / 0 | 2 / 0 |
| Compilation, fewer steps (flow) | 96 (96; 0) | 0 / 0 | 0 / 0 |
| Method | tested | filtered | misses | agree |
|---|---|---|---|---|
| Exhaustive | 156 | 0 | 0 | 13/13 |
| Stop when budget met | 107 | 0 | 0 | 13/13 |
| Affected-stage screen | 107 | 0 | 0 | 13/13 |
| Weight-read screen | 70 | 37 | 0 | 13/13 |
| Both screens | 70 | 37 | 0 | 13/13 |
Efficient VLA Model Inference
Efficient vision-language-action (VLA) model inference reduces robot-control latency, memory use, and energy while preserving task performance.
See all 221 papers