Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures

Manoj Vishwanathan; Suvinay Subramanian; Anand Raghunathan

cs.PFMar 1, 2026

Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures

Authors: Manoj Vishwanathan, Suvinay Subramanian, Anand Raghunathan

Organizations: Google, Mountain View, CA USA · Purdue University, West Lafayette, IN USA

Abstract

Vision-Language-Action (VLA) models are an emerging class of workloads critical for robotics and embodied AI at the edge. As these models scale, they demonstrate significant capability gains, yet they must be deployed locally to meet the strict latency requirements of real-time applications. This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jetson Orin and Thor platforms. Using MolmoAct-7B, a state-of-the-art VLA model, we identify a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the memory-bound action-generation phase. Through analytical modeling and simulations, we project the hardware requirements for scaling to 100B parameter models. We also explore the impact of high-bandwidth memory technologies and processing-in-memory (PIM) as promising future pathways in edge systems for embodied AI.

Figures & tables

Figure 1. VLA System Architecture. The workload consists of a Vision Encoder, a Generation Engine (autoregressive decoding), and an Action Transformer. Block diagram of a VLA system with a vision encoder, a projector, and an autoregressive LLM backbone leading to action outputs.

Figure 2. Performance on current edge platforms. Latency of MolmoAct-7B on Jetson Orin and Jetson Thor. Performance comparison between NVIDIA Jetson Orin and NVIDIA Jetson Thor for the profiled VLA workload.

Commercial hardware	Memory	BW (GB/s)	BF16 TFLOPS
Orin	LPDDR5	203	100
Thor	LPDDR5X	273	500
Hypothetical variants	Memory	BW (GB/s)	BF16 TFLOPS
Orin+LPDDR5X	LPDDR5X	273	100
Orin+GDDR7	GDDR7	1000	100
Orin+PIM	LPDDR6X PIM	2180	1074

Table 1. Commercial edge platforms and hypothetical hardware systems used in our experiments. For systems with PIM, the compute throughput (BF16 TFLOPS) includes both the SoC and PIM.

Figure 3. Control Frequency for various edge system configurations Higher memory bandwidth and PIM increase control frequency, but achieving the 10 Hz target for long horizon action generation at larger model sizes requires new innovations and algorithm-system co-design. Bar chart of control frequency versus model size across Orin and Thor with different memory technologies, showing that higher bandwidth increases frequency but 10 Hz remains out of reach for larger models.

Explore similar work

Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design
Oct 4, 2026Seonghun Jung, Sieun Moon, Jiyoung Jeong +2Efficient VLA Model Inference Vision-Language-Action Models
KAIST, Daejeon, Republic of Korea
Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment
Apr 27, 2026Kaijun Zhou, Qiwei Chen, Da Peng +3AI Accelerator Inference Edge Inference
School of Computer Science, Shanghai Jiao Tong University
How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf
Feb 20, 2026Wenqi Jiang, Jason Clemons, Karu Sankaralingam +1Efficient VLA Model Inference On-Device Inference
NVIDIA Research

Oct 4, 2026J/K move · Enter open · S save

Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design

Seonghun Jung, Sieun Moon, Jiyoung Jeong, Jimin Lee+1

KAIST, Daejeon, Republic of Korea

Efficient VLA Model Inference Vision-Language-Action Models VLMs for Robotics

Open paper AskPDF arXiv

Vision-language-action (VLA) models translate multimodal observations into low-level robot actions. During robot operation, each control period sets an inference deadline, and overruns leave the robot acting on stale observations, reducing task success. Meeting this deadline motivates on-device or nearby edge execution, where a single robot requires batch-1 inference outside the design point of LLM serving systems. Although VLA architectures combine familiar vision-language, autoregressive, and diffusion-style components, their runtime behavior in this batch-1 control setting remains uncharacterized. We characterize four representative VLA models on an edge GPU server and two onboard SoCs, using single-inference profiling and 43,200 closed-loop episodes. Action tensor dimensionality determines whether a stage is memory- or compute-bound, platform balance can shift that bottleneck, and GPU frequency scaling yields a platform-dependent energy-latency sweet spot. In closed-loop operation, overlapping inference with action execution creates an accuracy-speed-energy tradeoff, and no configuration is Pareto-dominant across deployment SLOs. These results guide joint design of VLA model architectures, hardware, and runtime policies.

Abstract

Explore similar work

Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design

Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment

How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf