How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf

Wenqi Jiang; Jason Clemons; Karu Sankaralingam; Christos Kozyrakis

cs.ROFeb 20, 2026

How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf

Authors: Wenqi Jiang, Jason Clemons, Karu Sankaralingam, Christos Kozyrakis

Organizations: NVIDIA Research

Abstract

Vision-Language-Action (VLA) models have recently demonstrated impressive capabilities across various embodied AI tasks. While deploying VLA models on real-world robots imposes strict real-time inference constraints, the inference performance landscape of VLA remains poorly understood due to the large combinatorial space of model architectures and inference systems. In this paper, we ask a fundamental research question: How should we design future VLA models and systems to support real-time inference? To address this question, we first introduce VLA-Perf, an analytical performance model that can analyze inference performance for arbitrary combinations of VLA models and inference systems. Using VLA-Perf, we conduct the first systematic study of the VLA inference performance landscape. From a model-design perspective, we examine how inference performance is affected by model scaling, model architectural choices, long-context video inputs, asynchronous inference, and dual-system model pipelines. From the deployment perspective, we analyze where VLA inference should be executed -- on-device, on edge servers, or in the cloud -- and how hardware capability and network performance jointly determine end-to-end latency. By distilling 15 key takeaways from our comprehensive evaluation, we hope this work can provide practical guidance for the design of future VLA models and inference systems.

Figures & tables

Figure 1: VLA-Perf enables a comprehensive performance analysis of the VLA inference landscape. Our systematic study explores the interplay between model architectures and deployment configurations, yielding 15 actionable insights for designing future VLA models and serving systems.

Figure 2: An example timeline of synchronous VLA inference. An efficient inference system should aim to match camera ingest rates to provide the robot with real-time action guidance.

Figure 3: VLA-Perf abstracts VLA inference as model components interleaved with data transfers.

Metric	1 camera	2 cameras	3 cameras
Roofline ( VLA-Perf )	14.7 ms	22.5 ms	30.4 ms
Real Perf. (Triton)	20.0 ms	27.3 ms	36.8 ms
Fidelity (Real/Roofline)	73.3%	82.3%	82.6%

Table 1: Roofline model validation against real

\pi_{0}

Triton inference latencies on an RTX 4090 Ma et al. (2025) . This evaluation uses 10 flow-matching steps, action chunk size of 63, and an empty language prompt.

Capacity and Placement	On-Device	Edge Server	Cloud Server
Mobile (Thor)	✓
Consumer (RTX 4090)		✓
Datacenter (B100)		✓	✓

Table 2: We consider systems with various (1) GPU capabilities (rows) and (2) inference location (columns).

Hardware	Vision Lat.	VLM Lat.	Action Lat.	E2E Lat.	E2E Freq.
Jetson Thor	6.06 ms	20.30 ms	26.20 ms	52.57 ms	19.0 Hz
RTX 4090	4.02 ms	19.79 ms	7.25 ms	31.06 ms	32.2 Hz
A100	2.13 ms	10.47 ms	3.60 ms	16.20 ms	61.7 Hz
H100	0.71 ms	3.30 ms	2.14 ms	6.15 ms	162.5 Hz
B100	0.40 ms	1.87 ms	0.91 ms	3.18 ms	314.4 Hz

Table 3: Inference performance of

\pi_{0}

on various GPUs without considering network latency.

Hardware	Balance OI	Vision (OI=321.4)	VLM (OI=542.8)	Action (OI=54.0)
Jetson Thor	1481.5	Memory	Memory	Memory
RTX 4090	163.7	Compute	Compute	Memory
A100	153.0	Compute	Compute	Memory
H100	295.2	Compute	Compute	Memory
B100	218.8	Compute	Compute	Memory

Table 4: Compute- vs. memory-bound analysis of

\pi_{0}

across different hardware. Operator intensity (OI) denotes the ratio between compute operations and memory accesses (FLOPs/Bytes). The balance OI denotes the hardware balance point at which compute throughput and memory bandwidth are equally limiting.

Model	Vision Encoder	VLM	Action Expert	Jetson Thor	RTX 4090	B100
$\pi_{0}$ (2.7B)	SigLIP-So (0.4B)	Gemma-2B (2.0B)	Act-M (0.3B)	19.0 Hz	32.2 Hz	314.4 Hz
$\pi_{0}$ -L (9.1B)	SigLIP-Giant (1.1B)	Llama2-7B (6.5B)	Act-L (1.5B)	3.9 Hz	8.0 Hz	73.6 Hz
$\pi_{0}$ -XL (16.7B)	SigLIP-Giant (1.1B)	Llama2-13B (12.7B)	Act-XL (2.9B)	2.1 Hz	N/A	39.7 Hz
$\pi_{0}$ -XXL (81.3B)	SigLIP-Giant (1.1B)	Llama2-70B (68.5B)	Act-XXL (11.7B)	N/A	N/A	9.6 Hz

Table 5: Inference performance of scaled-up VLA models across different hardware platforms.

Figure 5: Increased model sizes lead to proportional inference latency increases.

Timesteps	Total Memory	KV Cache Size	Jetson Thor	RTX 4090	B100
1	5.1 GB	0.01 GB	52.6 ms (19.0 Hz)	31.1 ms (32.2 Hz)	3.2 ms (314.4 Hz)
10	5.3 GB	0.13 GB	58.4 ms (17.1 Hz)	39.0 ms (25.7 Hz)	3.9 ms (254.6 Hz)
100	6.4 GB	1.3 GB	122.9 ms (8.1 Hz)	117.3 ms (8.5 Hz)	11.3 ms (88.4 Hz)
1000	18.3 GB	13.2 GB	768.3 ms (1.3 Hz)	900.6 ms (1.1 Hz)	85.2 ms (11.7 Hz)
10000	137.0 GB	131.8 GB	N/A	N/A	823.7 ms (1.2 Hz)

Table 6: Inference performance and memory consumption of long-context VLA models.

Panel of Figure 6 — Figure 6: The impact of denoising steps and action chunk sizes to inference performance on B100 GPU.

Panel of Figure 7 — Figure 7: Diffusion vs autoregressive VLA inference performance on B100 GPU.

Metric	Ethernet 1G	Ethernet 10G	WiFi 6	WiFi 7	4G	5G	Slow Cloud	Fast Cloud
Upload BW	1 Gbps	10 Gbps	560 Mbps	2 Gbps	19 Mbps	80 Mbps	1 Gbps	10 Gbps
Download BW	1 Gbps	10 Gbps	800 Mbps	3 Gbps	75 Mbps	500 Mbps	1 Gbps	10 Gbps
Base Latency	0.10 ms	0.05 ms	3.50 ms	2.50 ms	25.00 ms	10.00 ms	100.00 ms	10.00 ms

Table 7: Network configuration specifications.

Figure 8: Inference performance on device, on edge servers, and on cloud servers.

Figure 9: Device-server collaborative inference versus server-only and device-only solutions.

Hardware	Network	Latency	Freq. (Sync)	Freq. (Async)	Speedup
B100	Ethernet 10G	3.3 ms	301.4 Hz	314.4 Hz	1.04 $\times$
B100	Ethernet 1G	3.8 ms	266.5 Hz	314.4 Hz	1.18 $\times$
B100	WiFi 7	8.4 ms	119.7 Hz	314.4 Hz	2.63 $\times$
B100	5G	27.8 ms	35.9 Hz	215.3 Hz	5.99 $\times$
B100	4G	73.0 ms	13.7 Hz	50.5 Hz	3.68 $\times$
B100	Wired + Fast Cloud	23.4 ms	42.8 Hz	314.4 Hz	7.34 $\times$

Table 8: Inference frequency of synchronous and asynchronous systems.

Hardware	Network	S1 Lat.	S2 Lat.	Freq. (Sync)	S2 Cap = 5 Hz		S2 Cap = 10 Hz
					Freq. (Async)	Speedup	Freq. (Async)	Speedup
Jetson Thor	On-device	32.3 ms	20.3 ms	19.0 Hz	27.8 Hz	1.46 $\times$	24.7 Hz	1.30 $\times$
B100	Ethernet 10G	1.5 ms	1.9 ms	301.4 Hz	682.4 Hz	2.26 $\times$	676.0 Hz	2.24 $\times$
B100	WiFi 7	6.5 ms	1.9 ms	119.7 Hz	152.6 Hz	1.28 $\times$	151.2 Hz	1.26 $\times$
B100	5G	26.0 ms	1.9 ms	35.9 Hz	38.2 Hz	1.06 $\times$	37.8 Hz	1.05 $\times$

Table 9: Performance gains by using dual-system inference.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Hardware	FP32	BF16/FP16	INT8	Memory	Memory BW
Jetson Thor	100 TFLOP/s	400 TFLOP/s	800 TOP/s	128 GB	270 GB/s
RTX 4090	83 TFLOP/s	165 TFLOP/s	330 TOP/s	24 GB	1008 GB/s
A100	20 TFLOP/s	312 TFLOP/s	624 TOP/s	80 GB	2039 GB/s
H100	67 TFLOP/s	989 TFLOP/s	1979 TOP/s	80 GB	3350 GB/s
B100	60 TFLOP/s	1750 TFLOP/s	3500 TOP/s	192 GB	8000 GB/s

Appendix

Table 10: Hardware specifications for the GPUs used in our evaluation.

Component	Layers	Hidden Dim	Interm. Dim	Q Heads	KV Heads	Params
Vision Encoder	27	1,152	4,304	16	16	411.19M
VLM Backbone	18	2,048	16,384	8	1	1.98B
Action Expert	18	1,024	4,096	8	1	292.63M

Appendix

Table 11: Parameter specifications for

\pi_{0}

model components (without vocabulary table).

Explore similar work

vla.cpp: A Unified Inference Runtime for Vision-Language-Action Models
Jun 6, 2026Khanh D. Nguyen, Hung T. Ho, Chinh T. Nguyen +5AI Accelerator Inference Efficient VLM Inference
VinRobotics, Vietnam · Max Planck Research School for Intelligent Systems, Germany · University of Stuttgart, Germany +3
Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design
Oct 4, 2026Seonghun Jung, Sieun Moon, Jiyoung Jeong +2Efficient VLA Model Inference Vision-Language-Action Models
KAIST, Daejeon, Republic of Korea
Understanding Asynchronous Inference Methods for Vision-Language-Action Models
May 4, 2026Ayoub AgouzoulRobotic Control Efficient VLM Inference
École polytechnique · Systems Group, ETH Zürich

Jun 6, 2026J/K move · Enter open · S save

vla.cpp: A Unified Inference Runtime for Vision-Language-Action Models

Khanh D. Nguyen, Hung T. Ho, Chinh T. Nguyen, Thanh Q. Duong+4

VinRobotics, Vietnam · Max Planck Research School for Intelligent Systems, Germany · University of Stuttgart, Germany+3

AI Accelerator Inference Efficient VLM Inference Efficient VLA Model Inference Efficient VLA Models Vision-Language-Action Models VLMs for Robotics

Open paper AskPDF arXiv

Deploying vision--language--action (VLA) models on robots requires adapting model-specific inference pipelines to heterogeneous processors and limited onboard memory. We present vla.cpp, a unified C++ inference runtime for eleven VLA models, with no PyTorch dependency for model execution. The runtime shares model loading, tensor execution, and serving while retaining architecture-specific attention, conditioning, and action heads. Iterative policies reuse observation-dependent computation across solver steps, while regression policies predict actions directly. We evaluate task success on LIBERO-Object and profile supported configurations on NVIDIA, Apple, and Intel hardware. BitVLA completes 200/200 LIBERO-Object episodes on an 8,GB Jetson Orin Nano. A ternary tensor-core kernel accelerates its client inference by

4.0

4.6\times

over the CUDA-core baseline on RTX 3060 and AGX Orin. A SmolVLA case study links positional-index precision to gripper commands and task success, showing why fixed-input numerical checks should accompany rollout evaluation. Deployments on UR10e and ALOHA demonstrate physical robot integration; delay and execution-horizon studies characterize synchronous chunked control. The results demonstrate a common deployment path across VLA architectures and hardware, with numerical validation and control settings guiding deployment alongside inference efficiency.

Abstract

Explore similar work

vla.cpp: A Unified Inference Runtime for Vision-Language-Action Models

Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design

Understanding Asynchronous Inference Methods for Vision-Language-Action Models