ReAT: Residual Action Tokenization for Autoregressive Robot Policies

Dijia Zhan; Zhuo Li; Xuemiao Xu; Sun Zheng; Jinyi Li; Wenjun Wang; Jie Tang

cs.ROOct 9, 2026

ReAT: Residual Action Tokenization for Autoregressive Robot Policies

Authors: Dijia Zhan, Zhuo Li, Xuemiao Xu, Sun Zheng, Jinyi Li, Wenjun Wang, Jie Tang

Organizations: South China University of Technology (scut.edu.cn) · The Chinese University of Hong Kong (cuhk.edu.hk) · Hong Kong Polytechnic University (polyu.edu.hk)

Abstract

Autoregressive robot policies rely on action tokenizers to convert continuous action chunks into compact sequences of discrete prediction targets. Existing tokenizers can reconstruct action chunks from discrete codes, but often leave implicit what each individual token contributes to the reconstructed action. We introduce Residual Action Tokenization (ReAT), which gives each token an explicit role in refining the current action reconstruction. It uses the decoded reconstruction as feedback to construct the next token from the remaining action error. This closes the loop between encoding and decoding, making each new token responsive to the errors left by the actual decoded prefix. Across 19 manipulation tasks spanning three simulation benchmarks and real-world settings, 8-token ReAT policies outperform the evaluated baselines in task success; on LIBERO, even 4-token ReAT surpasses the evaluated 8-token action-tokenization baselines at half the token budget. These results suggest a path toward action tokenization that supports self-corrective policy learning, helping autoregressive policies learn to progressively refine their own action plans.

Figures & tables

Figure 1: Comparison of action-token decoding strategies. Full-sequence decoding requires all

M

tokens, while ordered prefix decoding reconstructs a complete action chunk from any nonempty prefix. ReAT combines the existing draft with each new token to produce a learned correction. Orange arrows indicate the corrections applied to the previous reconstruction in ReAT; target trajectories are shown only for reference.

Figure 2: Arm trajectory reconstruction with residual tokens. A 32-step LIBERO example at $K=2,4,6,8$ using an earlier ReAT tokenizer. Five sampled poses show translucent motion history and an opaque final pose. MSE is computed over all steps and normalized action dimensions of this example.

Figure 3: Architecture of ReAT. Top left: residual tokenization with reconstruction feedback across tokens. Top right: residual encoding and draft-conditioned decoding in one residual refinement iteration. Bottom: an observation-conditioned policy predicts tokens, whose prefixes are decoded by the frozen tokenizer.

Table 5

Table 8

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Observation / prediction / execution horizon	2 / 32 / 16
Maximum content tokens / vocabulary	8 / 1000
FSQ scalar levels	$[8,5,5,5]$
Register encoder	width 256, head dimension 64
Edit decoder	width 256, head dimension 64
Autoregressive policy	4 layers, width 256, 4 heads

Appendix

Table 7: Main ReAT settings. EMA is enabled in both stages.

Action dimension / summary	ReAT	OAT	QueST	FAST	Bin
Translation $x$	0.94889	0.93819	0.94711	0.99463	0.99994
Translation $y$	0.96745	0.95924	0.96496	0.99662	0.99996
Translation $z$	0.96882	0.95807	0.96940	0.99708	0.99997
Rotation $x$	0.78624	0.73519	0.35841	0.97437	0.99967
Rotation $y$	0.88375	0.83200	0.53693	0.98556	0.99982
Rotation $z$	0.89567	0.85831	0.68065	0.98659	0.99971

Appendix

Table 8: LIBERO tokenizer reconstruction by action dimension.

R^{2}

values are task-averaged; MSE is measured in original action coordinates.

\bar{K}

counts content tokens.

Method	Translation coordinates	Success
OAT	Per-step	$61.48\mathbin{{\color[rgb]{0.45,0.45,0.45}\pm}}{\color[rgb]{0.45,0.45,0.45}0.92}$
OAT	Cumulative	$62.56\mathbin{{\color[rgb]{0.45,0.45,0.45}\pm}}{\color[rgb]{0.45,0.45,0.45}0.67}$
ReAT	Cumulative	$\mathbf{71.16}\mathbin{{\color[rgb]{0.45,0.45,0.45}\pm}}{\color[rgb]{0.45,0.45,0.45}0.68}$

Appendix

Table 9: LIBERO comparison with matched action coordinates. Success (%) at

K=8

, reported as mean

\pm

SEM across five evaluation rounds. Rotation and gripper channels remain per-step in every row.

Explore similar work

Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?
Oct 6, 2026Haoran Chen, Jingtian Ji, Samuel Wheeler +2Robot Policy Learning Robotic Manipulation
Toyota Technological Institute at Chicago · Argonne National Laboratory
Behavior-Aligned Action Tokenization for Robot Policy Learning
Sep 23, 2026Junbo Dong, Ze Chen, Zhendong Xie +5Robot Policy Learning Trajectory Representation Learning
Southern University of Science and Technology · Dexmal · National University of Singapore +1
TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models
Oct 1, 2026Keisuke Shirai, Tomohiro Motoda, Hanbit Oh +5Data-Efficient Robot Learning Vision-Language-Action Models
AIST

Oct 6, 2026J/K move · Enter open · S save

Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?

Haoran Chen, Jingtian Ji, Samuel Wheeler, Kaylene Caswell Stocking+1

Toyota Technological Institute at Chicago · Argonne National Laboratory

Robot Policy Learning Robotic Manipulation

Open paper AskPDF arXiv

Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space. Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective. However, as we show through extensive analysis, sufficiently accurate action reconstruction is only one part of what makes a downstream robot policy successful. It is also critical that the policy is able to predict the right tokens for new observations, and that unseen policy token predictions still decode into reasonable actions. These properties are downstream of tokenizer training and are not directly incentivized by a reconstruction objective alone. In this work, we introduce Predictable and Robust Action Tokenization (ProAct), a tokenizer training method that strategically augments reconstruction with the goal of improving downstream predictability and robustness. ProAct is policy-agnostic and uses only action datasets for training. Across the Robomimic, LIBERO, and RoboTwin benchmarks and a diverse set of tokenizer architectures, ProAct improves rollout success by an average of 11.3 percentage points. These improvements also translate to vision-language-action policies and real-world robotic manipulation, yielding average gains of 21.8 and 36.7 percentage points, respectively. These results suggest that effective action tokenization should be designed as a policy interface that balances fidelity, predictability, and robustness, rather than as a reconstruction problem alone.

Abstract

Explore similar work

Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?

Behavior-Aligned Action Tokenization for Robot Policy Learning

TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models