cs.ARNov 8, 2026

Efficient VLA Inference via Retrieval-Based Speculation and Hardware Acceleration

Authors: Zhengyang Cheng, He Zhu, Zeyu Guo, Zihan Liu, Anbang Wu, Yiming Gan, Jingwen Leng, Yu Feng, 

Organizations: Shanghai Jiao Tong University · Shanghai Qi Zhi Institute · Institute of Computing Technology, Chinese Academy of Science

Abstract

Vision-language-action (VLA) models enable robots to understand and execute complex instructions, but their autoregressive (AR) action generation introduces significant latency on resource-constrained robotic platforms. Speculative decoding can accelerate AR-based inference by parallelizing action generation and verification; however, existing approaches rely on costly draft models, which are inefficient on edge platforms. In this paper, we propose Zora, a hardware-software co-design framework for accelerating AR-based VLA inference. Specifically, Zora introduces a lightweight retrieval-augmented generation (RAG)-based draft model that exploits robotic dynamics without additional neural draft models. We further reduce batch verification overhead through a kinematics-guided action-acceptance strategy and a KV-cache-sharing attention mechanism. To address the retrieval overhead introduced by RAG-based drafting, we co-design dedicated hardware support, Zora core, to address the memory irregularity in RAG-based drafting. We evaluate Zora on four VLA models with the LIBERO benchmark. Zora retains comparable accuracy while achieving up to 3.4×\times speedup with software-only optimization and up to 7.6×\times speedup with hardware support over GPUs.

Figures & tables

Explore similar work

CardsList
  1. Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification

    Aug 16, 2026Chunyu Qi, Zhuoran Song, Jian Weng +6Efficient VLA Model InferenceRobotic Manipulation

  2. Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference

    Jul 14, 2026Yuzhou Wu, Yuxin Zheng, Muchun Niu +6Efficient VLA Model InferenceEfficient VLA Models