LiteVLA-Edge: Quantized On-Device Multimodal Control for Embedded Robotics

Justin Williams; Kishor Datta Gupta; Roy George; Mrinmoy Sarkar

cs.ROMar 3, 2026

LiteVLA-Edge: Quantized On-Device Multimodal Control for Embedded Robotics

Authors: Justin Williams, Kishor Datta Gupta, Roy George, Mrinmoy Sarkar

Organizations: Clark Atlanta University, Atlanta, GA, USA · Siemens Corporation, Princeton, NJ, USA

Abstract

Vision-Language-Action (VLA) models provide a unified framework for perception, language conditioning, and action generation, but many existing systems remain difficult to deploy in embedded robotic settings because of their computational requirements and inference latency. In this paper, we present LiteVLA-Edge, a deployment-oriented VLA pipeline for fully on-device inference on Jetson Orin-class hardware. Our approach combines supervised image-to-action fine-tuning in FP32 with post-training 4-bit GGUF quantization and GPU-accelerated inference through the llama.cpp runtime. Under our deployment configuration, LiteVLA-Edge achieves a mean end-to-end latency of 150.5,ms (approximately 6.6,Hz) while operating entirely offline within a ROS~2-integrated perception--reasoning--action pipeline. Rather than introducing a new policy objective, our contribution is a practical systems path for executing compact multimodal control models locally on embedded hardware while preserving modular interfaces between perception, reasoning, and actuation. These results establish timing feasibility for reactive language-conditioned control and provide a reproducible baseline for future task-level evaluation of on-device VLAs in robotics.

Figures & tables

Fig. 1: LiteVLA-Edge system architecture. The multimodal transformer runs fully on-device on the Jetson AGX Orin and publishes structured velocity commands to ROS 2 for closed-loop control.

Model	Type	Params	Evaluated HW	Closed-Loop
Moondream2	VLM	$\sim$ 2B	CPU / Edge GPU	No
TinyLLaVA	VLM	$\sim$ 1B–3B	GPU	No
PaliGemma	VLM	$\sim$ 3B	GPU / TPU	No
Qwen2-VL	VLM	2B–7B	Server GPU	No
OpenVLA	VLA	7B	RTX 4090	Partial ( $\sim$ 5 Hz)
EdgeVLA	VLA	$\sim$ 1B	A100-40GB	Yes ( $\sim$ 10 Hz)

TABLE I: Comparison of Compact Multimodal Models and VLA Systems

Measurement	Result
Total Runs	300
Mean Latency	150.5 ms
Std Deviation	0.13 ms
Minimum	150.4 ms
Maximum	151.0 ms
Reasoning Frequency	6.64 Hz

TABLE II: End-to-End Multimodal Inference Performance on Jetson Orin NX

Efficient VLA Model Inference

Efficient vision-language-action (VLA) model inference reduces robot-control latency, memory use, and energy while preserving task performance.

See all 180 papers

EagleVLA: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference
Jul 14, 2026Zebin Yang, Qi Wang, Yunhe Wang +6Efficient VLA Model Inference Vision-Language-Action Models
Peking University · TLAIC · PrimeBot Research Institute +1
RhinoVLA Technical Report
Jun 5, 2026Huixi Technology, :, Chen Zhang +13Efficient VLM Inference Efficient VLA Model Inference
Huixi Technology
Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment
Apr 27, 2026Kaijun Zhou, Qiwei Chen, Da Peng +3AI Accelerator Inference Edge Inference
School of Computer Science, Shanghai Jiao Tong University

Jul 14, 2026J/K move · Enter open · S save

EagleVLA: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference

Zebin Yang, Qi Wang, Yunhe Wang, Xiurui Guo+5

Peking University · TLAIC · PrimeBot Research Institute+1

Efficient VLA Model Inference Vision-Language-Action Models Vision-Based Robot Control On-Device Inference

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have achieved impressive performance on diverse embodied tasks. However, deploying VLA models on low-power onboard devices, such as the Jetson Orin, remains challenging due to their high computational complexity, which leads to substantial inference latency and low control frequency. Asynchronous inference can partially mask this latency by parallelizing action execution and subsequent inference, but it introduces two critical issues: perception-execution misalignment and long reaction time. In this paper, we propose EagleVLA, a method for efficient VLA deployment on onboard devices via Foresight-Aligned Asynchronous Correction. To address misalignment, we train a lightweight future correction module that predicts future environment representations, allowing the action expert to predict actions from the future time step. To reduce reaction time, we introduce confidence-based scheduling optimization that adaptively balances VLM and action expert invocations. We also build a llama.cpp-based inference engine tailored for onboard VLA deployment, with system-level optimizations including CUDA graph reuse, GPU-resident intermediate buffering, and flow unrolling. Extensive experiments demonstrate that EagleVLA achieves 9.85x and 6.19x improvements in control frequency compared with naive PyTorch and vla.cpp on Jetson Orin, while outperforming VLASH by 14.8% in success rate on LIBERO benchmark. The code of our asynchronous algorithm is available on https://github.com/PKU-SEC-Lab/EagleVLA, and our efficient llama.cpp-based inference engine is available on https://github.com/PKU-SEC-Lab/EagleVLA-Edge.

Abstract

Efficient VLA Model Inference

Explore similar work

EagleVLA: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference

RhinoVLA Technical Report

Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment