cs.RONov 11, 2026

ActionCodec2: A Streaming Action Tokenizer

Authors: Zibin Dong, Yicheng Liu, Shiduo Zhang, Tianyuan Yuan, Baijun Ye, Jinqiu Wang, Hang Zhao

Organizations: Tsinghua University · Galaxea AI · Fudan University

Abstract

Most action tokenizers for autoregressive vision language action (VLA) models compress motion into chunks that must be decoded in full before their actions become executable. We introduce ActionCodec2, a non-neural streaming tokenizer that releases actions incrementally while keeping trajectory reconstruction error within a prescribed tolerance. Its encoder quantizes each step relative to the decoder's reconstructed state, so the selected motion primitive corrects residual error rather than allowing errors from independently quantized increments to accumulate. Finding the smallest such primitive set is a geometric covering problem; we derive the exact minimum for translation and constant-factor-tight bounds for rotation. Set-BPE then losslessly merges co-occurring primitive symbols across coordinates and neighboring steps, without a fixed coordinate ordering. A universal version fitted on 20k hours of data from 24 embodiments averages 8.8 tokens per second for 15 Hz end-effector motion on this corpus. With the same VLA backbone, ActionCodec2 outperforms the compared tokenizers on three simulation benchmarks and in zero-shot real-robot evaluation. Its streaming representation allows policies to incorporate state feedback within an action chunk, substantially improving performance over open-loop execution under randomized dynamics, external disturbances, and moving objects. The same tokens also can serve as a direct control interface for VLM agents, which compose successful action sequences into reusable macros. These agent-built macros reduce control decisions and thinking-token usage across attempts and transfer to a held-out task. More results are available at https://zibindong.github.io/actioncodec2-page/

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ActionCodec: What Makes for Good Action Tokenizers

    Feb 17, 2026Zibin Dong, Yicheng Liu, Shiduo Zhang +8Efficient VLA ModelsVision-Language-Action Models

  2. ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

    Sep 16, 2026Shijie Lian, Bin Yu, Zhaolong Shen +5Vector QuantizationRobotic Manipulation

  3. NAC: Neural Action Codec for Vision-Language-Action Models

    Jun 19, 2026Ahad Jawaid, Yu XiangNeural Audio CodecsResidual VQ