We introduce Z-sight, a unified robot foundation model that couples vision-language-action (VLA) and generative latent world modeling. Video-based world action models (WAMs) show the value of joint future-and-action generation for control, yet inherit the heavy cost of video generators. Recent latent-space policies avoid pixel generation, yet typically align intermediate features or rely on frozen target encoders, decoupling future prediction from the VLM's native feature space. Z-sight instead predicts the future directly in the visual token space in which its VLM perceives the present. Coupling a foresight expert and a continuous action expert with the VLM in a mixture-of-transformers, Z-sight jointly denoises future spatial tokens and actions at inference without pixel reconstruction or external encoders. Trained on human video and robot trajectories, Z-sight achieves 81.9% success on SIMPLE (+9.6 points over the strongest baseline) and 73.3% across six Unitree G1 tasks (+20.0 points) at 110 ms per chunk, while remaining competitive on tabletop benchmarks.
Figures & tables
Figure 1: Z -sight predicts the future in the policy’s native representation space. Top: Z -sight performs diverse loco-manipulation tasks in a real-world setting. Bottom: comparison of policy designs. (a) Vision-language-action models (VLAs) map the current observation to actions through a VLM and an action expert. (b) World action models (WAMs) build on a pretrained video generator to predict future video alongside actions. (c) Z -sight (ours) co-denoises future spatial tokens and actions in the co-evolving visual space of its own VLM, allowing the action expert to read the forming prediction during generation.
Figure 2: Overview of Z -sight. (a)–(c) Training stages: VLM pre-training on human video, mid-training all streams on mixed robot-human data, and post-training on the target embodiment. (d) Architecture: The VLM processes observation history, state, and instructions, supervised by discrete FAST tokens ( LFAST ). A foresight expert denoises future spatial tokens against an EMA teacher ( Lview ). Simultaneously, an action expert generates continuous actions via flow matching ( Lflow ) while attending to the predicted future features.
Figure 3: Attention mask of Z -sight. The VLM prefix attends causally and cannot see either expert. The foresight expert attends to the prefix and to itself. The action expert attends to the prefix, the foresight stream, and itself.
Figure 4: Training data. Pre-training uses 1,977 h of egocentric human video (inner arc) and mid-training uses all 4,731 h (outer circle), combining robot data with 20% human-data replay.
Figure 5: Results on SIMPLE. (a) Success rates by difficulty level and overall average. (b) Success rates across the 12 teleoperation tasks (averaged over L0–L2). Values are means over three evaluation passes of one checkpoint per method, with error bars showing ± one sample standard deviation across passes (ten episodes per task and level per pass). Z -sight leads at every difficulty level (81.9% average, +9.6 points over GR00T N1.7) and ranks first or tied on 11 of the 12 tasks.
Figure 6: Real-world tasks. The six evaluation tasks on the Unitree G1 with task instructions overlaid. Dog and Puffs are short pick-and-place tasks, whereas Chair , Book , Trash , and Fruit are multi-stage loco-manipulation tasks requiring coordinated whole-body motion and dexterous grasping.
Figure 7: Real-world results on the Unitree G1. Success rate (SR) and progress score (%) over 10 trials per task. Solid bars and labels show SR, overlaid on translucent bars showing progress. Average weights all six tasks equally. We compare Fast-WAM ( Yuan et al., 2026 ) , π0.5 ( Black et al., 2025a ) , GR00T N1.7 ( NVIDIA, 2026 ) , and Ψ0 ( Wei et al., 2026a ) . Z -sight matches or exceeds every baseline on every task, raising overall success to 73.3% (+20.0 points over Fast-WAM) with the largest gains on multi-stage tasks.
Table 8
SIMPLE
RoboTwin 2.0
Variant
Hard
C2C
C2R
FLARE-style ( Zheng et al., 2025 )
67.5
74.6
20.9
DINOv2-B target ( Oquab et al., 2024 )
70.8
76.7
26.6
Z -sight (ours)
73.1
82.4
40.1
Table 4: Future-prediction designs. Success rates (%) on SIMPLE Hard and RoboTwin 2.0. Z -sight consistently outperforms both implicit feature alignment and frozen external targets.
Table 10Table 11
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Item
Setting
VLM backbone
Qwen3.5-2B ( Qwen Team, 2026 )
Foresight / action expert width
1024 / 1024
Parameters (VLM / foresight / action)
2.22B / 0.48B / 0.48B (3.18B total)
Temporal encoding
Space-time attention every fourth ViT block
Future prediction target
Spatial tokens of the final future frame
Target normalization
Parameter-free per-token layer normalization
Appendix
Table A.1 : Shared architecture and prediction settings.
Stage
Data
Losses
Trainable modules
Visual noise σ
Pre-training
EgoDex, EgoVerse
F
VLM and state projection
Not used
Mid-training
AgiBot World, Humanoid Everyday, 20% human replay
F + A + 0.1 V
VLM via F, foresight via A and V, action expert via A
σ=max(τ,u)
Post-training
Target-domain demonstrations
A + 0.1 V
VLM via A, foresight via A and V, action expert via A
σ=max(τ,u)
Appendix
Table A.2 : Training stages and gradient routing. F, A, and V denote FAST cross-entropy, action flow matching, and future-feature prediction, respectively. Future targets are always stop-gradient EMA features. During mid-training only F updates the VLM, whereas during post-training A updates the VLM while V remains insulated from it. Experts and modality projections are trainable whenever present.
Setting
Steps
Batch
LR
LM scale
Action horizon
History
Stride
Unitree G1
20k
128
10−4
0.5×
30
3
10
LIBERO
80k
128
5×10−5
1.0×
16
2
8
RoboTwin 2.0
100k
256
10−4
0.5×
50
2
25
SIMPLE
50k
256
10−4
0.5×
30
3
10
Appendix
Table A.3 : Target-domain adaptation schedules. Batch sizes are global. LR denotes peak expert learning rate, with a 0.1× multiplier on the vision tower and the listed multiplier on the language model (LM). History and stride refer to the causal video encoder in frames and dataset timesteps.
Metric group
Metric
EgoDex
EgoVerse
Fitted tokenizer
Tokens per chunk
120 → 105
160 → 126
Reduction (mean)
13.1%
20.9%
Reduction (99th pct.)
18.9%
≈ 32.0%
Released tokenizer
Tokens per chunk
124 → 107
169 → 130
Reduction (mean)
13.1%
22.9%
Reduction (99th pct.)
18.4%
≈ 31.6%
Appendix
Table B.2 : Effect of temporal smoothing ( σs=0→2 ) on human action chunks. Token counts are means per chunk. Values marked ≈ are EgoVerse quantiles averaged across its three subsets.
Task
Demos
Duration
Camera
Stages (progress credit)
Dog
60
15.9 s
RealSense
Place the toy dog on the box (0.5), contain the dog within the box (0.5)
Puffs
49
27.7 s
RealSense
Pick up the puffs box (0.5), turn around and place it in the drawer (0.5)
Chair
70
35.4 s
ZED 2i
Push the chair under the table (0.5), reach target pose and return to standing (0.5)
Book
61
54.7 s
ZED 2i
Pick up and hold the book (0.5), walk and place it on the shelf (0.5)
Trash
58
56.8 s
ZED 2i
Pick up bottle (0.25), pick up paper (0.25), throw bottle in bin (0.25), throw paper in bin (0.25)
Fruit
57
83.7 s
ZED 2i
Pick up fruit plate (0.25), open fridge door (0.25), place plate inside (0.25), close door (0.25)
Table D.2 : GR00T N1.7 trainable-backbone control on SIMPLE. Success rates (%) averaged over three evaluation passes.
Method
Teleop (12)
MP (5)
All 17
GR00T N1.7 ( NVIDIA, 2026 )
72.3
30.0
59.9
Fast-WAM ( Yuan et al., 2026 )
66.0
25.3
54.1
Ψ0 ( Wei et al., 2026a )
65.8
20.0
52.4
Z -sight (ours)
81.9
52.0
73.1
Appendix
Table E.1 : Full SIMPLE benchmark summary. Success rates (%) averaged over the 12 Teleop tasks, 5 MP tasks, and all 17 tasks.
Future-feature input to action expert
Normalized action MSE ↓
Relative change
Model’s own prediction α ( σ=τ )
0.0495
Reference
Ground-truth future y , noised to σ=τ
0.0436
−11.9%
Clean present frame β ( σ=0 )
0.0635
+28.3%
Appendix
Table F.1 : Offline action-chunk error under controlled future-feature inputs. Normalized MSE against held-out G1 demonstration chunks with shared initial action noise.
Real-world G1 Dog
LIBERO
Variant
SR
Progress
Avg. SR
Long
Model prediction ( σ=τ )
90
95.0
97.70
96.6
Gaussian noise ( σ=1 )
60
60.0
96.50
95.4
Gaussian noise ( σ=τ )
–
–
75.15
50.8
Present frame (Copy, σ=τ )
0
0.0
7.05
0.0
Appendix
Table F.2 : Additional test-time interventions on physical G1 and LIBERO. Scores (%) with fixed model weights on physical G1 Dog (10 trials) and LIBERO (2,000 episodes across four suites, including 500 on Long ).
The vision-language-action (VLA) and world-action model (WAM) communities debate whether control policies should inherit pretrained knowledge from vision-language models or from visual generators. The debate, however, bundles three separable choices: backbone initialization, the objective applied to future observations, and whether the machinery for that future loss is kept at inference. A gap between a VLA and a WAM therefore cannot simply be attributed to any one of them. We show that decoupling these choices yields a common framework for both families, which we term Just Action Model (JAM): a pretrained visual backbone with an action expert, trained with an action loss and a foresight loss. The foresight loss trains current-observation features to predict frozen encodings of the recorded future through a prediction head discarded after training, so no visual generation is involved in either training or inference, and the backbone can be chosen freely. We first compare objectives within fixed policy architectures, then evaluate the foresight recipe across backbone families. The resulting JAM policies outperform their matched FastWAM and ImageWAM counterparts on LIBERO and LIBERO-Plus, and achieve similar or slightly higher success rates on RoboTwin. Compared with FastWAM, JAM improves the success rates from 97.0%/55.3%/91.2% to 98.2%/64.5%/91.4%, respectively. With cached foresight targets, JAM reduces estimated online training FLOPs per sample per update by 73% relative to FastWAM. Our results show that future supervision improves control without visual generation or additional inference machinery, and strong foresight performance is attainable with either generator-initialized or vision-language backbones. Project Page: https://y0uroy.github.io/JAM
Yonghao Yu, Lang Huang, Runyi Li +1
The University of Tokyo · National Institute of Informatics
Visual-Language-Action models (VLAs) have advanced generalist robot control by mapping multimodal observations and language instructions directly to actions, but sparse action supervision often encourages shortcut mappings rather than representations of dynamics, contact, and task progress. Recent world-action models introduce future prediction through video rollouts, yet pixel-space prediction is a costly and indirect substrate for control, as it may model visual details irrelevant to action generation and introduces substantial training or inference overhead. We present Being-H0.7, a latent world-action model that brings future-aware reasoning into VLA-style policies without generating future frames. Being-H0.7 inserts learnable latent queries between perception and action as a compact reasoning interface, and trains them with a future-informed dual-branch design: a deployable prior branch infers latent states from the current context, while a training-only posterior branch replaces the queries with embeddings from future observations. Jointly aligning the two branches at the latent reasoning space leads the prior branch to reason future-aware, action-useful structure from current observations alone. At inference, Being-H0.7 discards the posterior branch and performs no visual rollout. Experiments across six simulation benchmarks and diverse real-world tasks show that Being-H0.7 achieves state-of-the-art or comparable performance, combining the predictive benefits of world models with the efficiency and deployability of direct VLA policies.
Vision-language-action (VLA) policies excel at generalist robotic manipulation but generate actions without explicitly modeling their environmental consequences. World action models (WAMs) incorporate future prediction, yet most rely on costly video generation, while latent alternatives either use predicted futures only as auxiliary targets or infer latent actions from suboptimal visual transitions through costly large-scale video pretraining. We introduce WAVE-0 (World-model-Augmented VLA Execution), a framework that bridges VLAs and WAMs through a lightweight latent world model. By connecting the semantic understanding of pretrained VLAs with explicit future prediction, WAVE-0 allows anticipated action consequences to directly guide action generation. Built on a pretrained VLA, its 257M world model translates VLM-generated latent action plans into predicted future representations that condition the action expert, unifying latent planning, future prediction, and action generation. The world model is trained directly on ground-truth robot actions with a counterfactual ranking objective, requiring neither pixel reconstruction nor large-scale video pretraining. WAVE-0 achieves success rates of 91.47% on RoboTwin 2.0 and 98.7% on LIBERO, along with 88.0%, 61.4%, and 75.0% on LIBERO-Plus, LIBERO-Pro, and LIBERO-Para, respectively. On real-world tasks, it improves the success rate of π0.5 from 38.3% to 73.3%. These results demonstrate strong manipulation performance and generalization across diverse distribution shifts.
Haomin Wang, Anqiong Ji, Ke Wang +4
AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University · Baidu AI Cloud · Awaken Robotics +1