$τ_0$-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation

Xiaowei Cai; Yunuo Cai; Bingao Chen; Jingxiao Chen; Zhi Chen; Siyuan Feng; Tengyu Hou; Jingshun Huang

cs.ROAug 17, 2026

$τ_0$ -VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation

Authors: Xiaowei Cai, Yunuo Cai, Bingao Chen, Jingxiao Chen, Zhi Chen, Siyuan Feng, Tengyu Hou, Jingshun Huang,

Organizations: Agibot Finch · Shanghai Innovation Institute · The Chinese University of Hong Kong

Abstract

Long-horizon robot manipulation requires a robot to both execute individual skills reliably and sequence them coherently over extended tasks. Most hierarchical vision-language-action (VLA) models make each such decision with a single forward pass, leaving no mechanism to allocate additional computation to difficult or consequential choices. We introduce $τ_0$ -VLA, a hierarchical robot foundation model that formulates high-level subtask generation as a compute-scalable inference problem through world-model-guided test-time computation. At each inference step, the high-level policy uses execution memory to generate a subtask and, when needed, searches over alternatives before committing to its output. A low-level policy then executes the generated subtask across multiple robot embodiments. The policy is trained on 40,115 hours of heterogeneous real-world data with multimodal co-training. Across in-domain and distribution-shifted settings, allocating additional test-time computation substantially improves next-subtask prediction accuracy, and these gains translate into higher closed-loop success on long-horizon robot manipulation tasks.

Figures & tables

Fig. 2: The hierarchical $\tau_{0}$ -VLA architecture. (a) At a high-level inference step $t$ , the proposal model $P$ conditions on the latest multi-view observation $o_{t}$ , task instruction $\ell$ , carried execution memory $\mathcal{M}_{t-1}$ , and previously generated subtask $z_{t-1}^{\star}$ . It produces observation-aligned memory $\mathcal{M}_{t}$ and a direct proposal $z_{t}^{\mathrm{dir}}$ . (b) The low-level policy conditions on the generated subtask $z_{t}^{\star}$ , multi-view observation $o_{t}$ , proprioceptive state $\mathbf{s}_{t}$ , and textual control metadata $\eta$ . A vision-language backbone and Mixture-of-Transformers (MoT) action expert generate the action chunk $\mathbf{a}_{t:t+H-1}$ through conditional flow matching from a noisy action chunk. (c) On the TTC route, the proposal model is invoked $N$ times for each retained branch to generate $N$ candidates. Given the branch’s head-camera image and a candidate, the world model predicts the terminal head-camera image, and the value model assigns a candidate-quality score conditioned on the task instruction, candidate, and predicted image. Beam search globally retains the top- $B$ branches by cumulative score and recursively expands them to depth $D$ . The figure illustrates the root expansion with $N=3$ and $B=2$ , where local and cumulative scores coincide. Deeper expansion is omitted for clarity. The reflective model then conditions on $\bar{h}_{t}$ and the final branch summaries $\mathcal{C}_{t}$ to generate the final subtask $z_{t}^{\star}$ . This output may coincide with a retained proposal but is not restricted to the retained set and is passed to the low-level policy for execution.

Fig. 3: Representative physical-robot evaluation tasks. (a) Clean Room requires collecting two dirty garments, placing them in a laundry basket, hanging a handbag, handing a blanket to a person, and disposing of table trash. (b) Prepare Ingredients requires retrieving a tomato and an egg, then cracking and stirring the egg while returning the tools. (c) Tomato and Egg Stir Fry requires chopping and transferring the tomatoes, cooking and seasoning the ingredients, plating the dish, and returning the cookware and utensils. (d) Make Milk Tea requires adding toppings, pouring milk and tea, sealing the cup, and inserting a straw. (e) Collect Laundry requires transferring a T-shirt from the bedside table to a laundry basket. (f) Tidy Makeup Table comprises three separately scored instruction-following groups that require different object selections and action sequences from matched visual states.

Method	Clean Room		Prepare Ingredients		Tomato and Egg Stir Fry		Make Milk Tea		Avg.
	SR $\uparrow$	Progress $\uparrow$	SR $\uparrow$	Progress $\uparrow$	SR $\uparrow$	Progress $\uparrow$	SR $\uparrow$	Progress $\uparrow$	SR $\uparrow$	Progress $\uparrow$
$\mathrm{GR00T}$ N1.7 [ 26 ]	0/10	59.80%	1/10	68.57%	0/10	24.32%	0/10	28.46%	2.50%	45.29%
LingBot-VLA [ 40 ]	0/10	66.60%	0/10	35.00%	0/10	12.27%	0/10	63.85%	0.00%	44.43%
$\pi_{0.5}$ [ 2 ]	4/10	86.20%	2/10	73.93%	0/10	49.77%	3/10	82.31%	22.50%	73.05%
$\tau_{0}$ -VLA	4/10	92.80%	2/10	66.43%	0/10	65.00%	5/10	96.15%	27.50%	80.10%
$\tau_{0}$ -VLA (Hierarchical System, Plan Once)	5/10	94.80%	4/10	82.86%	4/10	81.82%	5/10	91.92%	45.00%	87.85%

TABLE I: Long-horizon task performance. Each method–task setting uses 10 independently collected physical-robot trials. SR reports successful trials as

x/10

, and Progress is the normalized milestone-completion score. Avg. is the unweighted mean of the four task-level rates. The first four rows use direct execution. The final row uses the Hierarchical System with Plan Once and no beam search.

Method	Collect Laundry		Tidy Makeup Table
	T-shirt		Cotton Pad		Eyelash Curler		Makeup Puff
	SR $\uparrow$	Progress $\uparrow$	SR $\uparrow$	Progress $\uparrow$	SR $\uparrow$	Progress $\uparrow$	SR $\uparrow$	Progress $\uparrow$
$\mathrm{GR00T}$ N1.7 [ 26 ]	4/10	76.00%	10/10	87.50%	8/10	77.50%	7/10	52.50%
LingBot-VLA [ 40 ]	2/10	35.00%	9/10	67.50%	3/10	22.50%	3/10	33.75%
$\pi_{0.5}$ [ 2 ]	9/10	88.00%	9/10	85.00%	8/10	85.00%	7/10	73.75%
$\tau_{0}$ -VLA	10/10	97.00%	10/10	95.00%	9/10	92.50%	10/10	95.00%

TABLE II: Direct-execution performance across embodiments. Tidy Makeup Table comprises three independently scored instruction-following groups. All methods execute the full task instruction without a high-level policy. Each task or task group is evaluated over 10 trials. SR reports successful trials as

x/10

, and Progress is the normalized milestone-completion score.

Fig. 4: Next-subtask prediction accuracy under different high-level inference methods. We compare Plan Once, Best-of-

N

, and TTC (Ours) across four evaluation settings: Make Milk Tea, Book Organization (In-Domain), Book Organization (OOD), and Clean Room.

Method	Make Milk Tea		Book Organization		Clean Room
Method	SR $\uparrow$	Progress $\uparrow$	SR $\uparrow$	Progress $\uparrow$	SR $\uparrow$	Progress $\uparrow$
Plan Once	5/10	91.92%	6/10	66.67%	5/10	94.80%
TTC	7/10	95.38%	9/10	93.33%	7/10	97.60%

TABLE III: Closed-loop physical-robot performance with test-time computation. Each entry uses 10 independently collected trials. Book Organization uses shuffled initial arrangements and is reported without the in-domain and OOD split used in the open-loop evaluation. SR denotes task success rate, and Progress is the normalized milestone-completion score.

Fig. 5: Relationship between computational cost and subtask-prediction accuracy. The accuracy increases with additional computation for the Make Milk Tea and Book Organization tasks, shown in panels (a) and (b), respectively. Each point represents an experimental result obtained at a different computational cost. The orange dashed curves show saturation fits to these results, and the gray dashed lines indicate the Plan Once baselines.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Coordinates	Dimensions	State representation
Left EEF position	1–3	Cartesian position in meters
Left EEF orientation	4–9	$\operatorname{Rot6D}(\mathbf{R}^{L})$
Right EEF position	10–12	Cartesian position in meters
Right EEF orientation	13–18	$\operatorname{Rot6D}(\mathbf{R}^{R})$
Left gripper	19	native opening coordinate
Right gripper	20	native opening coordinate

Appendix

TABLE IV: Canonical 40-D state and action layout. Dimensions are one-indexed. For a rotation matrix

\mathbf{R}=[\mathbf{r}_{1},\mathbf{r}_{2},\mathbf{r}_{3}]

, we use

\operatorname{Rot6D}(\mathbf{R})=[\mathbf{r}_{1}^{\top},\mathbf{r}_{2}^{\top}]^{\top}

Task	Maximum duration
Clean Room	20 min
Prepare Ingredients	20 min
Tomato and Egg Stir Fry	20 min
Make Milk Tea	10 min
Book Organization	5 min
Collect Laundry	5 min

Appendix

TABLE V: Maximum duration of each physical-robot trial.

Family	Sampling position	Input $\rightarrow$ target memory	Target subtask	Deployment failure countered	Mix
within-subtask	anywhere in seg. $n$	$\mathcal{M}_{n}\rightarrow\mathcal{M}_{n}$	seg. $n$	— (aligned, normal progression)	58%
transition	tail of seg. $n$	$\mathcal{M}_{n}\rightarrow\mathcal{M}_{n+1}$	seg. $n{+}1$	starting a new subtask after completion	15%
catch-up	head of seg. $n$	$\mathcal{M}_{n-1}\rightarrow\mathcal{M}_{n}$	seg. $n$	memory lag (behind the visual state)	10%
rollback	late in seg. $n$	$\mathcal{M}_{n+1\ldots n+3}\rightarrow\mathcal{M}_{n}$	retry seg. $n$	memory run-ahead (over-optimistic)	12%
error-think	annotated failure frame	$\mathcal{M}_{n}\rightarrow$ type-dependent	recovery step	unnoticed execution failure	5%

Appendix

TABLE VI: High-level instance families, synthesized by perturbing only the input memory while reading the corrected target from the demonstration.

\mathcal{M}_{n}

is the memory upon entering segment

n

. A single unified <think> / <memory> / <subtask> format instantiates all of them at zero extra annotation.

Robot Policy Learning

Robot policy learning develops controllers that map sensory observations, task goals, and interaction history to actions for embodied robots.

See all 352 papers

AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Models
Mar 9, 2026Xiaoquan Sun, Zetian Xu, Chen Cao +9Vision-Language-Action Models Robotic Manipulation
INFIFORCE Intelligent Technology Co., Ltd. Hangzhou, China · Huazhong University of Science and Technology, Wuhan, China · The University of Hong Kong, Hong Kong SAR, China +1
Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models
May 13, 2026Zixing Lei, Changxing Liu, Yichen Xiong +5Long-Horizon Robotic Manipulation Tool-Using Vision-Language Agents
Shanghai Jiao Tong University · Zhongguancun Academy · Beihang University
ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models
May 28, 2026Ye Li, Huanan Liu, Kangye Ji +7Efficient VLM Inference Efficient VLA Model Inference
Tsinghua University · University of Illinois at Urbana-Champaign

Mar 9, 2026J/K move · Enter open · S save

AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Models

Xiaoquan Sun, Zetian Xu, Chen Cao, Zonghe Liu+8

INFIFORCE Intelligent Technology Co., Ltd. Hangzhou, China · Huazhong University of Science and Technology, Wuhan, China · The University of Hong Kong, Hong Kong SAR, China+1

Vision-Language-Action Models Robotic Manipulation Long-Horizon Robotic Manipulation Robustness of VLA Models

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models demonstrate remarkable potential for generalizable robotic manipulation. The execution of complex multi-step behaviors in VLA models can be improved by robust instruction grounding, a critical component for effective control. However, current paradigms predominantly rely on coarse, high-level task instructions during supervised fine-tuning. This instruction grounding gap leaves models without explicit intermediate guidance, leading to severe compounding errors in long-horizon tasks. Therefore, bridging this instruction gap and providing scalable post-training for VLA models is urgent. To tackle this problem, we propose \method, the first subtask-aware VLA framework integrated with a scalable offline post-training pipeline. Our framework leverages a large language model to decompose high-level demonstrations into fine-grained atomic subtasks. This approach utilizes a pretrained predictive world model to score candidate action chunks against subtask goals in the latent space, mitigating error accumulation while significantly improving long-horizon robustness. Furthermore, this approach enables highly efficient Group Relative Policy Optimization without the prohibitive expenses associated with online rollouts on physical robots. Extensive simulations validate that our AtomVLA maintains strong robustness under perturbations. When evaluated against fundamental baseline models, it achieves an average success rate of 97.0% on the LIBERO benchmark and 48.0% on the LIBERO-PRO benchmark. Finally, experiments conducted in the real world using the Galaxea R1 Lite platform confirm its broad applicability across diverse tasks, especially long-horizon tasks. All datasets, checkpoints, and code will be released to the public domain following the acceptance of this work for future research.

Abstract

Robot Policy Learning

Explore similar work

AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Models

Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models

ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models