Autoregressive robot policies learn continuous control by predicting discrete action tokens from observations. Different tasks often share local motions, yet behavioral correspondence across demonstrations receives limited explicit supervision in existing tokenizers. Motions with different timing can therefore lack a shared representation despite following similar patterns. We propose Behavior-Aligned Action Tokenization (BAAT), which uses soft dynamic time warping (Soft-DTW) to select corresponding action chunks and aligns their quantized coordinates jointly with reconstruction. This objective encourages similar motions across tasks to occupy nearby quantized representations while retaining executable action detail. A history-conditioned diffusion decoder reconstructs continuous action chunks from these tokens, and a downstream autoregressive policy learns to predict them. We evaluate BAAT on selected tasks from three simulation benchmarks and two real robot tasks. BAAT achieves a mean simulation success rate of approximately 45.2%, exceeding OAT by approximately 7.2 percentage points. In the controlled LIBERO-All alignment ablation, policy success rises from 70.2% to 79.0% while trajectory replay success decreases. These results support behavioral correspondence as supervision for organizing shared motion structure in action tokenizers and improving downstream robot policy learning.
Figures & tables
Fig. 1: BAAT tokenizer training and downstream policy learning. Soft-DTW supplies behavioral correspondence between action chunks; alignment constrains their quantized coordinates jointly with reconstruction. Each latent slot contains four FSQ factors. The diffusion decoder conditions on executed action history, indicated by At−1 in the schematic.
Fig. 2: Simulation task overview. Top: example scenes from the LIBERO Spatial, Object, Goal, and long-horizon suites. Middle: RoboTwin 2.0 tasks, comprising Handover Mic, Lift Pot, Pick Dual Bottles, and Stack Blocks. Bottom: RoboCasa tasks, comprising Close Drawer, Press Coffee Button, Turn Off Microwave, and Turn Off Faucet.
Fig. 3: Cross-task behavioral neighborhoods. Left: LIBERO-10 queries; right: nearest neighbors from LIBERO-Goal, Object, and Spatial. Rows show lateral transfer, forward reach, and vertical lift; circles and triangles mark trajectory starts and ends. Plots show the retrieved chunks’ trajectories, without decoder generation or selection from trajectory plots.
Method
LIBERO-All
RoboCasa
RoboTwin 2.0
Avg.
FAST
63.5
4.3
5.3
24.4
OAT
76.5
23.3
14.2
38.0
Bin
45.2
18.7
11.8
25.2
DP
62.9
13.8
25.7
34.1
BAAT (ours)
79.0
29.2
27.3
45.2
TABLE I: Success rates (%) under joint training.
Method
LIBERO-10
Spatial
Goal
Object
Suite avg.
Joint
FAST
32.5
67.8
75.8
71.5
61.9
63.5
OAT
43.7
78.3
85.3
93.6
75.2
76.5
Bin
15.2
39.7
47.6
68.1
42.7
45.2
DP
66.0
73.5
79.3
75.8
73.7
62.9
BAAT (ours)
26.0
81.9
85.6
90.0
70.9
79.0
TABLE II: LIBERO success rates (%): per-suite versus joint training.
Fig. 4: Real robot evaluation setups. The two tasks use different bimanual systems; object sorting includes dexterous-hand control.
Tokenizer
Recon. MSE
Replay SR (%)
FAST
5.408×10−5
90.25
Bin
4.170×10−6
93.0
OAT
2.274×10−3
58.5
BAAT (ours)
5.773×10−4
71.5
TABLE III: Action reconstruction on LIBERO-All.
Policy interface
Toothbrush
Object sorting
FAST
35 (7/20)
20 (4/20)
OAT
65 (13/20)
20 (4/20)
Bin
20 (4/20)
15 (3/20)
DP
55 (11/20)
45 (9/20)
BAAT (ours)
70 (14/20)
45 (9/20)
TABLE IV: Real robot success: percentage (successful trials/20).
Fig. 5: LIBERO-All policy SR during one training run per method. The endpoints represent individual runs and need not match the three-seed means in Table I . “LIBERO All-4” denotes the four-suite setting; “Semantic FSQ” denotes BAAT and “DP (EMA)” denotes DP.
Metric
Without history
With history
Boundary jump p50
2.505
1.367
Boundary jump p95
6.359
3.203
Boundary jump p99
8.767
4.496
Arm reconstruction MAE
0.4941
0.2774
TABLE V: Offline reconstruction with and without decoder history on real-robot data (raw joint-target units; lower is better).
λalign
Recon. MSE
Replay SR
Policy SR
0
4.672×10−4
80.5
70.2
0.05
5.101×10−4
76.3
73.0
0.1 (ours)
5.773×10−4
71.5
79.0
0.3
9.186×10−4
65.5
57.9
0.5
1.157×10−2
0.0
0.0
TABLE VI: Alignment-weight ablation on LIBERO-All (SR in %).
Pair construction
Recon. MSE
Replay SR
Policy SR
No alignment
4.672×10−4
80.5
70.2
Pointwise L2
1.051×10−3
64.6
73.0
Soft-DTW (ours)
5.773×10−4
71.5
79.0
TABLE VII: Trajectory matching on LIBERO-All (SR in %).
Decoder
Recon. MSE
Replay SR
Policy SR
Transformer
2.413×10−4
80.6
69.4
Diffusion (ours)
5.773×10−4
71.5
79.0
Flow matching
2.444×10−4
82.4
74.7
TABLE VIII: Decoder comparison on LIBERO-All (SR in %).
Autoregressive robot policies rely on action tokenizers to convert continuous action chunks into compact sequences of discrete prediction targets. Existing tokenizers can reconstruct action chunks from discrete codes, but often leave implicit what each individual token contributes to the reconstructed action. We introduce Residual Action Tokenization (ReAT), which gives each token an explicit role in refining the current action reconstruction. It uses the decoded reconstruction as feedback to construct the next token from the remaining action error. This closes the loop between encoding and decoding, making each new token responsive to the errors left by the actual decoded prefix. Across 19 manipulation tasks spanning three simulation benchmarks and real-world settings, 8-token ReAT policies outperform the evaluated baselines in task success; on LIBERO, even 4-token ReAT surpasses the evaluated 8-token action-tokenization baselines at half the token budget. These results suggest a path toward action tokenization that supports self-corrective policy learning, helping autoregressive policies learn to progressively refine their own action plans.
Dijia Zhan, Zhuo Li, Xuemiao Xu +4
South China University of Technology (scut.edu.cn) · The Chinese University of Hong Kong (cuhk.edu.hk) · Hong Kong Polytechnic University (polyu.edu.hk)
Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space. Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective. However, as we show through extensive analysis, sufficiently accurate action reconstruction is only one part of what makes a downstream robot policy successful. It is also critical that the policy is able to predict the right tokens for new observations, and that unseen policy token predictions still decode into reasonable actions. These properties are downstream of tokenizer training and are not directly incentivized by a reconstruction objective alone. In this work, we introduce Predictable and Robust Action Tokenization (ProAct), a tokenizer training method that strategically augments reconstruction with the goal of improving downstream predictability and robustness. ProAct is policy-agnostic and uses only action datasets for training. Across the Robomimic, LIBERO, and RoboTwin benchmarks and a diverse set of tokenizer architectures, ProAct improves rollout success by an average of 11.3 percentage points. These improvements also translate to vision-language-action policies and real-world robotic manipulation, yielding average gains of 21.8 and 36.7 percentage points, respectively. These results suggest that effective action tokenization should be designed as a policy interface that balances fidelity, predictability, and robustness, rather than as a reconstruction problem alone.
Haoran Chen, Jingtian Ji, Samuel Wheeler +2
Toyota Technological Institute at Chicago · Argonne National Laboratory
Autoregressive Vision-Language-Action models often represent continuous robot actions as discrete token sequences, enabling action prediction with standard next-token objectives. FAST has substantially improved this representation by compactly encoding action containing diverse temporal frequencies into relatively few tokens. However, while such compression reduces the number of action tokens required for autoregressive prediction, it does not necessarily improve the efficiency of policy learning from limited demonstrations. In particular, FAST typically assigns a single deterministic tokenization to each quantized action sequence, although multiple token sequences can represent and decode to the same robot motion. We investigate whether exploiting this representational redundancy can improve policy learning. In this paper, we propose TOkenization of Action sequences with STochastic sampling (TOAST), a stochastic action tokenization method that samples alternative tokenizations of the same quantized action sequence during policy training. This diversifies the discrete supervision while preserving the underlying robot action and requires no additional demonstrations. Experiments on LIBERO show that TOAST consistently improves over its deterministic counterpart, with the improvement increasing as training data decreases, achieving a 6.8 point gain in success rate when only 1/16 of training data is available. Across four real-robot manipulation tasks, TOAST further improves mean success rate by 15.8 points over the deterministic counterpart. These results demonstrate the effectiveness of stochastic action tokenization for autoregressive robot policy learning, particularly when training data are limited.