Organizations: Keio University · National Institute of Informatics Research and Development Center for Large Language Models · National Institute of Informatics
Coarse-to-fine autoregressive modeling has recently shown strong promise for visuomotor policy learning, combining the inference efficiency of autoregressive methods with the global trajectory coherence of diffusion-based policies. However, existing approaches rely on discrete action tokenizers that map continuous action sequences to codebook indices, a design inherited from image generation where learned compression is necessary for high-dimensional pixel data. We observe that robot actions are inherently low-dimensional continuous vectors, for which such tokenization introduces unnecessary quantization error and a multi-stage training pipeline. In this work, we propose Hierarchical Flow Policy (HiFlow), a tokenization-free coarse-to-fine autoregressive policy that operates directly on raw continuous actions. HiFlow constructs multi-scale continuous action targets from each action chunk via simple temporal pooling. Specifically, it averages contiguous action windows to produce coarse summaries that are refined at finer temporal resolutions. The entire model is trained end-to-end in a single stage, eliminating the need for a separate tokenizer. Experiments on MimicGen, RoboTwin 2.0, and real-world environments demonstrate that HiFlow consistently outperforms existing methods including diffusion-based and tokenization-based autoregressive policies.
Figures & tables
Fig. 1 : Tokenized vs tokenization-free scale-wise policy learning. Prior next-scale autoregressive policies discretize multi-scale action chunks using a VQ-VAE tokenizer and learn to predict code indices with cross-entropy, which introduces a separate tokenizer pretraining stage and incurs quantization error. HiFlow instead refines raw continuous actions from coarse to fine temporal scales. A scale-wise autoregressive transformer provides coarse-to-fine conditioning, and a shared conditional flow matching module generates continuous actions at each scale, eliminating the tokenizer while preserving fine-grained control.
Fig. 2 : Architecture overview of HiFlow . Given visual observations, proprioceptive states, and a task identifier, the scale-wise autoregressive Transformer (ScaleAR) produces conditioning features at progressively finer temporal scales via a scale-wise causal mask. A shared ActionFlowNet then generates continuous actions at each scale through conditional flow matching, progressively refining the trajectory from a single-token global summary (scale 1) to the full T -step action chunk. The entire pipeline operates in continuous action space without any discrete tokenization.
D
1024
T
8
Le
12
Lf
6
Batch size
128
Optimizer
AdamW
Learning rate
1×10−4
TABLE I : Experimental settings.
Fig. 3 : Task overview across the three evaluation benchmarks. Top: 8 single-arm manipulation tasks from MimicGen. Middle: 3 dual-arm tasks from RoboTwin 2.0. Bottom: 5 real-world tasks with a mobile manipulator, spanning object grasping, relocation, and target placement.
Policy
Coffee
Hammer
Mug
Nut
Square
Stack
Stack3
Thread
Avg.
TCD [ 15 ]
77
92
53
44
63
95
62
56
68
SDP [ 14 ]
82
100
62
54
82
96
80
70
78
CARP [ 6 ]
86
98
74
78
90
100
82
70
85
HiFlow (Ours)
100
100
76
80
82
98
76
90
88
TABLE II : Quantitative results on MimicGen in success rate (%). Best results in bold .
Policy
Click alarm
Move can
Place basket
ACT [ 2 ]
32
22
1
DP [ 4 ]
61
32
18
HiFlow (Ours)
69
42
39
TABLE III : Quantitative results on RoboTwin benchmark. Success rates (%). Best results in bold . “Click alarm”, “Move can”, and “Place basket” denote “click alarmclock”, “move can pot”, and “place can basket”, respectively.
Grasp
Relocation
Target Placement
Policy
Apple
Mustard
Coke
Ball → Dish
Orange → Plate
ACT [ 2 ]
50.0
33.3
8.3
16.7
8.3
DP [ 4 ]
66.7
58.3
66.7
25.0
16.7
CARP [ 6 ]
66.7
58.3
58.3
33.3
25.0
HiFlow (Ours)
75.0
83.3
75.0
58.3
41.7
TABLE IV : Quantitative results on real-world experiments. Success rates (%) over 12 trials per task. Best results in bold .
Fig. 4 : Qualitative results of HiFlow on representative tasks from each simulation benchmark. (a) Threading from MimicGen. (b) Place can basket from RoboTwin 2.0.
Fig. 5 : Qualitative results of HiFlow on the Orange → Plate task from the real-world experiments. Note that the third-person view are not included in the observation of the model.
Fig. 6 : Visualization of coarse-to-fine action generation. Each panel shows the predicted cumulative end-effector displacement trajectory in Cartesian task space (position components only) at a different scale level on the Mimicgen benchmark. ∘ , ⋆ , and ⋄ indicate token boundaries, trajectory start point and the end point, respectively.
Scale configuration
Avg. SR
{1,8}
84
{1,4,8}
86
{1,2,4,8,16}
85
{1,2,4,8}
88
TABLE V : Sensitivity analysis on temporal downsampling scales. Average success rate (%) on MimicGen. Best result in bold .
Visuomotor policy learning trains robots to turn camera observations, often with language and proprioception, into actions for manipulation and navigation.
Autoregressive robot policies rely on action tokenizers to convert continuous action chunks into compact sequences of discrete prediction targets. Existing tokenizers can reconstruct action chunks from discrete codes, but often leave implicit what each individual token contributes to the reconstructed action. We introduce Residual Action Tokenization (ReAT), which gives each token an explicit role in refining the current action reconstruction. It uses the decoded reconstruction as feedback to construct the next token from the remaining action error. This closes the loop between encoding and decoding, making each new token responsive to the errors left by the actual decoded prefix. Across 19 manipulation tasks spanning three simulation benchmarks and real-world settings, 8-token ReAT policies outperform the evaluated baselines in task success; on LIBERO, even 4-token ReAT surpasses the evaluated 8-token action-tokenization baselines at half the token budget. These results suggest a path toward action tokenization that supports self-corrective policy learning, helping autoregressive policies learn to progressively refine their own action plans.
Dijia Zhan, Zhuo Li, Xuemiao Xu +4
South China University of Technology (scut.edu.cn) · The Chinese University of Hong Kong (cuhk.edu.hk) · Hong Kong Polytechnic University (polyu.edu.hk)
Autoregressive Vision-Language-Action models often represent continuous robot actions as discrete token sequences, enabling action prediction with standard next-token objectives. FAST has substantially improved this representation by compactly encoding action containing diverse temporal frequencies into relatively few tokens. However, while such compression reduces the number of action tokens required for autoregressive prediction, it does not necessarily improve the efficiency of policy learning from limited demonstrations. In particular, FAST typically assigns a single deterministic tokenization to each quantized action sequence, although multiple token sequences can represent and decode to the same robot motion. We investigate whether exploiting this representational redundancy can improve policy learning. In this paper, we propose TOkenization of Action sequences with STochastic sampling (TOAST), a stochastic action tokenization method that samples alternative tokenizations of the same quantized action sequence during policy training. This diversifies the discrete supervision while preserving the underlying robot action and requires no additional demonstrations. Experiments on LIBERO show that TOAST consistently improves over its deterministic counterpart, with the improvement increasing as training data decreases, achieving a 6.8 point gain in success rate when only 1/16 of training data is available. Across four real-robot manipulation tasks, TOAST further improves mean success rate by 15.8 points over the deterministic counterpart. These results demonstrate the effectiveness of stochastic action tokenization for autoregressive robot policy learning, particularly when training data are limited.
Autoregressive Vision-Language-Action (VLA) models offer a scalable path to robot learning, yet existing action tokenizers treat tokenization as a compression problem, producing representations that are semantically misaligned with the autoregressive backbone. We propose CATok, a causal action tokenizer that reframes tokenization as a causally structured generative process. CATok introduces a conditional annealing mechanism that extracts action tokens by progressively annealing a flow-matching process: each token is conditioned on all preceding tokens and encodes the residual reconstruction signal at a specific noise level, establishing a coarse-to-fine causal token space whose generative semantics are structurally aligned with autoregressive modeling. A token-conditioned flow-matching decoder built on Multimodal Diffusion Transformer (MMDiT) reconstructs continuous action chunks from these discrete tokens with the precision of hybrid diffusion-head architectures. This discrete bottleneck enforces knowledge insulation by design, cleanly separating high-level semantic reasoning from low-level motor execution without requiring explicit attention masking. Extensive evaluations across three simulation benchmarks and real-world robotic manipulation tasks demonstrate that CATok consistently surpasses existing tokenization methods in both reconstruction fidelity-compression tradeoff and inference efficiency, while improving VLA task success rate and training efficiency, establishing a high-performance, scalable foundation for purely autoregressive VLA systems.
Chenyu Zhang, Yuhang Cao, Daru Du +8
IIIS, Tsinghua University · Shanghai Qizhi Institute