VLA-HWBench: Where the Time Goes When VLA Models Run on Edge Hardware

Borui Xu; Hongshi Tan; Wenqi Jiang; Bingsheng He; Ming Wu; Jason Zeng; Michael Heinrich

cs.ROOct 26, 2026

VLA-HWBench: Where the Time Goes When VLA Models Run on Edge Hardware

Authors: Borui Xu, Hongshi Tan, Wenqi Jiang, Bingsheng He, Ming Wu, Jason Zeng, Michael Heinrich

Organizations: National University of Singapore

Abstract

Vision-language-action (VLA) models increasingly run on edge devices aboard robots, where inference latency constrains closed-loop robot control. However, it remains unclear how to design VLA models, their inference software and edge hardware so that robots can run VLA inference in real time. Most VLA models report only task success, while prior studies cover limited action-generation strategies, divide inference into at most three stages, and lack comparisons of inference optimizations on the same models. Therefore, we introduce VLA-HWBench, a benchmark that measures 18 open VLA models on NVIDIA Jetson AGX Thor and AMD Ryzen AI Max+ PRO 395, with a seven-stage inference breakdown. Our study reveals three insights. First, action-generation strategy matters more than model size. It dominates 14 of 19 Jetson configurations, latency varies by $82\times$ on Jetson and $157\times$ on AMD, and parameter count explains at most 15% of latency variance. Models that generate long action chunks in a few flow-matching steps keep up with control, whereas token-by-token decoding does not. Second, kernel fusion with 4-bit weights helps most, running $π_{0.5}$ and GR00T N1.7 2.6-2.9 $\times$ faster than PyTorch with torch.compile, but it exists for only these two models and only on the Jetson. Optimization also moves the time to the stages it shortens least, as CPU preprocessing grows from 12% to 39% of the GR00T N1.7 call. Third, after optimization, memory bandwidth limits only the stages that process few tokens on the Jetson, such as the denoising steps, whereas software limits all other stages, from unfused kernels to serial preprocessing on the host CPU. Overall, low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusion and low-bit weights to more models and platforms, and edge hardware needs high memory bandwidth and a fast host CPU.

Figures & tables

Figure 1: End-to-end latency versus loaded parameter count on the Jetson AGX Thor (left) and the Ryzen AI Max+ PRO 395 (right), with one point per reference configuration.

Figure 2: The seven stages of one inference call. S0 and S5 run on the CPU and S2–S4 on the GPU.

Study	Models	Forms	Measured	Non-NVIDIA	Stages	CPU	Backend	Low-bit	Real
				edge		prep.			input
Taherin et al.	5	2	✓	✗	–	✗	✗	✗	✗
Zhou et al.	5	2	✓	✓	3	✗	✗	✓	✗
Vishwanathan et al.	1	1	✓	✗	3	✗	✗	✗	✗
VLA-Perf	–	3	✗	✗	3	✗	✗	✗	✗
VLA-HWBench	18	4	✓	✓	7	✓	✓	✓	✓

Table 1: Comparison with prior VLA edge-inference studies.

Action generation	Model	Params (B)	Weights	Action chunk $H$
Flow matching	X-VLA ( Zheng et al., 2026 )	0.88	BF16	30
	GR00T N1.7 ( Bjorck et al., 2025 )	3.14	BF16	16
	SmolVLA † ( Shukor et al., 2025 )	0.45	BF16 + FP32	50
	InstructVLA ( Yang et al., 2026a )	2.60	BF16 + FP32	8
	$\pi_{0.5}$ † ( Black et al., 2025a )	4.14	BF16 + FP32	50
	NORA-1.5 ( Hung et al., 2025 )	3.99	BF16	5

Table 2: The 18 models, with the parameters and weight types of the loaded model and the chunk size

H

of each release’s LIBERO evaluation. † Runs on LeRobot, the others on the authors’ code.

	Jetson AGX Thor (T5000)	Ryzen AI Max+ PRO 395
GPU	Blackwell, 2560 CUDA cores, 1.57 GHz	RDNA 3.5, 40 CUs, 2.9 GHz
CPU	14 Neoverse-V3AE cores, 2.6 GHz	16 Zen 5 cores, 5.1 GHz
Memory	128 GB LPDDR5X, 273 GB/s	128 GB LPDDR5X-8000, 256 GB/s
FP16 / FP32 (TFLOP/s)	258 / 8.1	59.4 a / 29.7 a
FP8 / FP4 (TFLOP/s)	517 / 1035	–
Power	40–130 W	45–120 W

Table 3: Specifications of the two platforms. a Derived from the CU count and clock.

Figure 3: Stage breakdown of the 19 reference configurations on the Jetson (a) and the AMD platform (b), with a fused stage counted in the stage that contains it.

Figure 4: Latency of the 19 reference configurations. (a) Median latency on the Jetson (bars, 5th–95th percentile) and the AMD platform (circles). (b) Latency over the chunk size

H

Figure 5: Optimizations applied in stages to

\pi_{0.5}

and GR00T N1.7. (a, b) Latency of the fastest configuration with the optimizations up to each stage, labeled with the speed-up of the stage. (c, d) Their stage times on the Jetson.

Figure 6: What limits the fastest configurations. (a) Stage shares, with the stages inside a captured graph separated by its kernels. (b) CPU preprocessing of GR00T N1.7 and its sub-steps with PyTorch’s 14 CPU threads and with one thread. (c, d) Rooflines, with the peak memory bandwidth as the sloped line, the peak of each number format as a flat line and each GPU stage at its arithmetic intensity

F/W

and achieved throughput.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Action generation	Model	Software	Views	$H$	$H_{e}$
Flow matching	X-VLA	authors’ code	2	30	30
	GR00T N1.7	authors’ code	2	16	8
	SmolVLA	LeRobot 0.6.1	3	50	50
	InstructVLA	authors’ code	2	8	8
	$\pi_{0.5}$	LeRobot 0.6.1	2	50	50
	NORA-1.5	authors’ code	1	5	5

Appendix

Table 4: Software, camera views, chunk length

H

and execution horizon

H_{e}

of each configuration.

Model	Code repository (stars)	LIBERO checkpoint	Downloads
X-VLA	2toinf/X-VLA (733)	2toINF/X-VLA-Libero	12,303
GR00T N1.7	NVIDIA/Isaac-GR00T (8,130)	nvidia/GR00T-N1.7-LIBERO	–
SmolVLA	huggingface/lerobot (27,783)	lerobot/smolvla_libero	68,467
InstructVLA	InternRobotics/InstructVLA (117)	ShuaiYang03/instructvla_pretraining_v2_libero_10_wrist-image_aug	220
$\pi_{0.5}$	Physical-Intelligence/openpi (13,997)	lerobot/pi05_libero_finetuned_v044	174,582
NORA-1.5	declare-lab/nora-1.5 (109)	declare-lab/nora-1.5-libero	109

Appendix

Table 5: GitHub stars of the code repository and all-time HuggingFace downloads of the LIBERO checkpoint of each model on 2026-09-26. A dash marks a checkpoint whose downloads HuggingFace does not count.

Stage	Work	Unit	Measured by
S0 Preprocessing	resize, normalize, tokenize	CPU	timer
S1 Host-to-device	copy the inputs	copy engine	timer
S2 Vision encoding	images to tokens	GPU	timer or kernel trace
S3 Prefill	language model, key–value cache	GPU	timer or kernel trace
S4 Action generation	denoising steps or decoded tokens	GPU	timer, step sweep or kernel trace
S5 Postprocessing	unnormalize, copy back	CPU	timer

Appendix

Table 6: The seven stages of one inference call.

Model	$p_{50}$	$p_{5}$ – $p_{95}$	Per action	$p_{50}$	$p_{5}$ – $p_{95}$	Per action	AMD / Jetson
	Jetson AGX Thor			Ryzen AI Max+ PRO 395
X-VLA	76.7	75.8–78.8	2.6	206	206–209	6.9	2.69
VLA-Adapter	92.7	72.8–102	11.6	114	113–115	14.2	1.23
GR00T N1.7	119	111–126	7.4	297	296–300	18.6	2.50
SimpleVLA-RL	129	120–135	16.1	446	444–449	55.7	3.46
SmolVLA	191	187–195	3.8	140	140–142	2.8	0.73

Appendix

Table 7: Median latency (ms), 5th–95th percentile, inference time per action (ms) and AMD-to-Jetson latency ratio of the 19 reference configurations, sorted by latency on the Jetson.

Model	S0	S1	S2	S3	S4	S5	S6	Total
X-VLA	6.0	0.9	$\rightarrow$	19.1	51.7	0.1	0.4	78
VLA-Adapter	22.8	0.8	27.0	20.8	21.7	2.4	0.8	96
GR00T N1.7	14.2	1.0	19.6	19.4	67.7	0.2	1.2	123
SimpleVLA-RL	20.1	0.9	23.5	85.7	$\rightarrow$	2.2	-3.0	129
SmolVLA	4.3	$\rightarrow$	20.9	15.0	146.6	0.1	0.2	187
InstructVLA	9.6	$\rightarrow$	24.8	36.2	149.1	17.5	0.7	238

Appendix

Table 8: Stage times (ms) on the Jetson AGX Thor.

Model	S0	S1	S2	S3	S4	S5	S6	Total
X-VLA	2.8	0.4	$\rightarrow$	51.5	154.3	0.1	0.1	209
VLA-Adapter	5.9	0.3	60.2	29.7	15.8	1.8	0.1	114
GR00T N1.7	2.4	0.9	98.6	49.6	145.3	0.1	0.7	298
SimpleVLA-RL	3.3	0.3	32.7	408.4	$\rightarrow$	1.0	-0.0	446
SmolVLA	0.7	$\rightarrow$	40.8	10.9	87.5	0.1	0.0	140
InstructVLA	3.8	$\rightarrow$	49.5	71.8	224.6	7.5	0.5	358

Appendix

Table 9: Stage times (ms) on the Ryzen AI Max+ PRO 395.

		Jetson AGX Thor			Ryzen AI Max+ PRO 395
Configuration	Swept	$b$ (ms)	$a$ (ms)	$R^{2}$	$b$ (ms)	$a$ (ms)	$R^{2}$
Reference configurations
X-VLA	steps	4.9	29	0.999	15.2	56	1.000
GR00T N1.7	steps	15.7	53	0.996	34.2	161	1.000
SmolVLA	steps	14.5	41	1.000	8.6	54	1.000
InstructVLA	steps	13.3	103	1.000	19.5	163	1.000

Appendix

Table 10: Fits

T=a+bN

of latency over the number of denoising steps or camera views

N

Model	Jetson AGX Thor	Ryzen AI Max+ PRO 395
X-VLA	3	1
VLA-Adapter	25	1
GR00T N1.7	15	0
SimpleVLA-RL	10	0
SmolVLA	5	0
InstructVLA	4	0

Appendix

Table 11: Spread (%) of the per-frame median latency over the 24 frames of one episode.

	Jetson AGX Thor			Ryzen AI Max+ PRO 395
Optimization	Before	After	Speed-up	Before	After	Speed-up
SmolVLA, compile (reduce-overhead)	191	66.4	2.88	140	109	1.28
SmolVLA, compile (max-autotune)	191	71.9	2.66	140	77.7	1.80
$\pi_{0.5}$ , compile (reduce-overhead)	405	157	2.58	781	682	1.14
$\pi_{0.5}$ , compile (max-autotune, shipped)	405	159	2.55	781	442	1.77
$\pi_{0.5}$ , compile (max-autotune), FlashRT’s workload	318	128	2.48	631	347	1.82

Appendix

Table 12: Median latency (ms) before and after each optimization of Q2 and the resulting speed-up.

Model	Platform	Mode	$p_{50}$ (ms)	Speed-up	Traced frames
SmolVLA	Jetson	reduce-overhead	66.4	2.88	14/17
SmolVLA	AMD	reduce-overhead	109	1.28	–
SmolVLA	Jetson	max-autotune	71.9	2.66	–
SmolVLA	AMD	max-autotune	77.7	1.80	–
$\pi_{0.5}$	Jetson	max-autotune (shipped)	159	2.55	–
$\pi_{0.5}$	AMD	max-autotune (shipped)	442	1.77	–

Appendix

Table 13: Compile modes against the eager reference configuration. GR00T N1.7 compiles only its action head, as in the Isaac-GR00T recipe, and traced frames are the frames that the compiler captures out of those it encounters. The aligned rows run FlashRT’s workload of 10 actions and two image slots on both platforms.

Optimization	Platform	min	median	optimized	repeated calls
		Cosine similarity		Max $\|\Delta\|$
$\pi_{0.5}$ , FlashRT FP16 $\rightarrow$ FP8	Jetson	0.99839	0.99938	0.124	0.059
$\pi_{0.5}$ , FlashRT FP16 $\rightarrow$ NVFP4	Jetson	0.93429	0.99812	0.677	0.096
GR00T N1.7, FlashRT FP16 $\rightarrow$ FP8	Jetson	0.99986	0.99993	0.033	0.003
GR00T N1.7, FlashRT FP16 $\rightarrow$ NVFP4	Jetson	0.99974	0.99986	0.047	0.003
X-VLA, FP32 $\rightarrow$ BF16	Jetson	0.99999	0.99999	0.004	0.004

Appendix

Table 14: Agreement of the actions before and after each optimization over 11 frames.

Figure 7: Time of every GPU stage of the fastest configurations over the larger of its two bounds, split by kernel class, with action generation measured per denoising step.

Configuration	Stage	Format	Time	$F$	$W$	$F/P$	$W/B$	Ratio
			(ms)	(GFLOP)	(MB)	(ms)	(ms)
Jetson, $\pi_{0.5}$	Vision (S2)	FP8	5.66	439.7	413	0.88	1.51	3.7
	Prefill (S3)	NVFP4, FP8	15.20	2205.4	1132	2.45	4.15	3.7
	Action (S4)	FP8	1.92	7.1	322	0.02	1.18	1.6
Jetson, GR00T N1.7	Vision (S2)	FP8	4.61	349.5	431	0.72	1.58	2.9
	Prefill (S3)	FP8	5.12	254.4	805	0.50	2.95	1.7

Appendix

Table 15: Time of every GPU stage of the fastest configurations against the compute bound

F/P

and the memory bound

W/B

, with action generation per denoising step.

Efficient VLA Model Inference

Efficient vision-language-action (VLA) model inference reduces robot-control latency, memory use, and energy while preserving task performance.

See all 190 papers

How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf
Feb 20, 2026Wenqi Jiang, Jason Clemons, Karu Sankaralingam +1Efficient VLA Model Inference On-Device Inference
NVIDIA Research
Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures
Mar 1, 2026Manoj Vishwanathan, Suvinay Subramanian, Anand RaghunathanVision-Language-Action Models Efficient VLA Model Inference
Google, Mountain View, CA USA · Purdue University, West Lafayette, IN USA
Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment
Apr 27, 2026Kaijun Zhou, Qiwei Chen, Da Peng +3AI Accelerator Inference Edge Inference
School of Computer Science, Shanghai Jiao Tong University

Feb 20, 2026J/K move · Enter open · S save

How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf

Wenqi Jiang, Jason Clemons, Karu Sankaralingam, Christos Kozyrakis

NVIDIA Research

Efficient VLA Model Inference On-Device Inference

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have recently demonstrated impressive capabilities across various embodied AI tasks. While deploying VLA models on real-world robots imposes strict real-time inference constraints, the inference performance landscape of VLA remains poorly understood due to the large combinatorial space of model architectures and inference systems. In this paper, we ask a fundamental research question: How should we design future VLA models and systems to support real-time inference? To address this question, we first introduce VLA-Perf, an analytical performance model that can analyze inference performance for arbitrary combinations of VLA models and inference systems. Using VLA-Perf, we conduct the first systematic study of the VLA inference performance landscape. From a model-design perspective, we examine how inference performance is affected by model scaling, model architectural choices, long-context video inputs, asynchronous inference, and dual-system model pipelines. From the deployment perspective, we analyze where VLA inference should be executed -- on-device, on edge servers, or in the cloud -- and how hardware capability and network performance jointly determine end-to-end latency. By distilling 15 key takeaways from our comprehensive evaluation, we hope this work can provide practical guidance for the design of future VLA models and inference systems.

Abstract

Efficient VLA Model Inference

Explore similar work

How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf

Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures

Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment