We introduce Z-sight, a unified robot foundation model that couples vision-language-action (VLA) and generative latent world modeling. Video-based world action models (WAMs) show the value of joint future-and-action generation for control, yet inherit the heavy cost of video generators. Recent latent-space policies avoid pixel generation, yet typically align intermediate features or rely on frozen target encoders, decoupling future prediction from the VLM's native feature space. Z-sight instead predicts the future directly in the visual token space in which its VLM perceives the present. Coupling a foresight expert and a continuous action expert with the VLM in a mixture-of-transformers, Z-sight jointly denoises future spatial tokens and actions at inference without pixel reconstruction or external encoders. Trained on human video and robot trajectories, Z-sight achieves 81.9% success on SIMPLE (+9.6 points over the strongest baseline) and 73.3% across six Unitree G1 tasks (+20.0 points) at 110 ms per chunk, while remaining competitive on tabletop benchmarks.
Figures & tables
Figure 1: Z -sight predicts the future in the policy’s native representation space. Top: Z -sight performs diverse loco-manipulation tasks in a real-world setting. Bottom: comparison of policy designs. (a) Vision-language-action models (VLAs) map the current observation to actions through a VLM and an action expert. (b) World action models (WAMs) build on a pretrained video generator to predict future video alongside actions. (c) Z -sight (ours) co-denoises future spatial tokens and actions in the co-evolving visual space of its own VLM, allowing the action expert to read the forming prediction during generation.
Figure 2: Overview of Z -sight. (a)–(c) Training stages: VLM pre-training on human video, mid-training all streams on mixed robot-human data, and post-training on the target embodiment. (d) Architecture: The VLM processes observation history, state, and instructions, supervised by discrete FAST tokens ( LFAST ). A foresight expert denoises future spatial tokens against an EMA teacher ( Lview ). Simultaneously, an action expert generates continuous actions via flow matching ( Lflow ) while attending to the predicted future features.
Figure 3: Attention mask of Z -sight. The VLM prefix attends causally and cannot see either expert. The foresight expert attends to the prefix and to itself. The action expert attends to the prefix, the foresight stream, and itself.
Figure 4: Training data. Pre-training uses 1,977 h of egocentric human video (inner arc) and mid-training uses all 4,731 h (outer circle), combining robot data with 20% human-data replay.
Figure 5: Results on SIMPLE. (a) Success rates by difficulty level and overall average. (b) Success rates across the 12 teleoperation tasks (averaged over L0–L2). Values are means over three evaluation passes of one checkpoint per method, with error bars showing ± one sample standard deviation across passes (ten episodes per task and level per pass). Z -sight leads at every difficulty level (81.9% average, +9.6 points over GR00T N1.7) and ranks first or tied on 11 of the 12 tasks.
Figure 6: Real-world tasks. The six evaluation tasks on the Unitree G1 with task instructions overlaid. Dog and Puffs are short pick-and-place tasks, whereas Chair , Book , Trash , and Fruit are multi-stage loco-manipulation tasks requiring coordinated whole-body motion and dexterous grasping.
Figure 7: Real-world results on the Unitree G1. Success rate (SR) and progress score (%) over 10 trials per task. Solid bars and labels show SR, overlaid on translucent bars showing progress. Average weights all six tasks equally. We compare Fast-WAM ( Yuan et al., 2026 ) , π0.5 ( Black et al., 2025a ) , GR00T N1.7 ( NVIDIA, 2026 ) , and Ψ0 ( Wei et al., 2026a ) . Z -sight matches or exceeds every baseline on every task, raising overall success to 73.3% (+20.0 points over Fast-WAM) with the largest gains on multi-stage tasks.
Table 8
SIMPLE
RoboTwin 2.0
Variant
Hard
C2C
C2R
FLARE-style ( Zheng et al., 2025 )
67.5
74.6
20.9
DINOv2-B target ( Oquab et al., 2024 )
70.8
76.7
26.6
Z -sight (ours)
73.1
82.4
40.1
Table 4: Future-prediction designs. Success rates (%) on SIMPLE Hard and RoboTwin 2.0. Z -sight consistently outperforms both implicit feature alignment and frozen external targets.
Table 10Table 11
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Item
Setting
VLM backbone
Qwen3.5-2B ( Qwen Team, 2026 )
Foresight / action expert width
1024 / 1024
Parameters (VLM / foresight / action)
2.22B / 0.48B / 0.48B (3.18B total)
Temporal encoding
Space-time attention every fourth ViT block
Future prediction target
Spatial tokens of the final future frame
Target normalization
Parameter-free per-token layer normalization
Appendix
Table A.1 : Shared architecture and prediction settings.
Stage
Data
Losses
Trainable modules
Visual noise σ
Pre-training
EgoDex, EgoVerse
F
VLM and state projection
Not used
Mid-training
AgiBot World, Humanoid Everyday, 20% human replay
F + A + 0.1 V
VLM via F, foresight via A and V, action expert via A
σ=max(τ,u)
Post-training
Target-domain demonstrations
A + 0.1 V
VLM via A, foresight via A and V, action expert via A
σ=max(τ,u)
Appendix
Table A.2 : Training stages and gradient routing. F, A, and V denote FAST cross-entropy, action flow matching, and future-feature prediction, respectively. Future targets are always stop-gradient EMA features. During mid-training only F updates the VLM, whereas during post-training A updates the VLM while V remains insulated from it. Experts and modality projections are trainable whenever present.
Setting
Steps
Batch
LR
LM scale
Action horizon
History
Stride
Unitree G1
20k
128
10−4
0.5×
30
3
10
LIBERO
80k
128
5×10−5
1.0×
16
2
8
RoboTwin 2.0
100k
256
10−4
0.5×
50
2
25
SIMPLE
50k
256
10−4
0.5×
30
3
10
Appendix
Table A.3 : Target-domain adaptation schedules. Batch sizes are global. LR denotes peak expert learning rate, with a 0.1× multiplier on the vision tower and the listed multiplier on the language model (LM). History and stride refer to the causal video encoder in frames and dataset timesteps.
Metric group
Metric
EgoDex
EgoVerse
Fitted tokenizer
Tokens per chunk
120 → 105
160 → 126
Reduction (mean)
13.1%
20.9%
Reduction (99th pct.)
18.9%
≈ 32.0%
Released tokenizer
Tokens per chunk
124 → 107
169 → 130
Reduction (mean)
13.1%
22.9%
Reduction (99th pct.)
18.4%
≈ 31.6%
Appendix
Table B.2 : Effect of temporal smoothing ( σs=0→2 ) on human action chunks. Token counts are means per chunk. Values marked ≈ are EgoVerse quantiles averaged across its three subsets.
Task
Demos
Duration
Camera
Stages (progress credit)
Dog
60
15.9 s
RealSense
Place the toy dog on the box (0.5), contain the dog within the box (0.5)
Puffs
49
27.7 s
RealSense
Pick up the puffs box (0.5), turn around and place it in the drawer (0.5)
Chair
70
35.4 s
ZED 2i
Push the chair under the table (0.5), reach target pose and return to standing (0.5)
Book
61
54.7 s
ZED 2i
Pick up and hold the book (0.5), walk and place it on the shelf (0.5)
Trash
58
56.8 s
ZED 2i
Pick up bottle (0.25), pick up paper (0.25), throw bottle in bin (0.25), throw paper in bin (0.25)
Fruit
57
83.7 s
ZED 2i
Pick up fruit plate (0.25), open fridge door (0.25), place plate inside (0.25), close door (0.25)
Table D.2 : GR00T N1.7 trainable-backbone control on SIMPLE. Success rates (%) averaged over three evaluation passes.
Method
Teleop (12)
MP (5)
All 17
GR00T N1.7 ( NVIDIA, 2026 )
72.3
30.0
59.9
Fast-WAM ( Yuan et al., 2026 )
66.0
25.3
54.1
Ψ0 ( Wei et al., 2026a )
65.8
20.0
52.4
Z -sight (ours)
81.9
52.0
73.1
Appendix
Table E.1 : Full SIMPLE benchmark summary. Success rates (%) averaged over the 12 Teleop tasks, 5 MP tasks, and all 17 tasks.
Future-feature input to action expert
Normalized action MSE ↓
Relative change
Model’s own prediction α ( σ=τ )
0.0495
Reference
Ground-truth future y , noised to σ=τ
0.0436
−11.9%
Clean present frame β ( σ=0 )
0.0635
+28.3%
Appendix
Table F.1 : Offline action-chunk error under controlled future-feature inputs. Normalized MSE against held-out G1 demonstration chunks with shared initial action noise.
Real-world G1 Dog
LIBERO
Variant
SR
Progress
Avg. SR
Long
Model prediction ( σ=τ )
90
95.0
97.70
96.6
Gaussian noise ( σ=1 )
60
60.0
96.50
95.4
Gaussian noise ( σ=τ )
–
–
75.15
50.8
Present frame (Copy, σ=τ )
0
0.0
7.05
0.0
Appendix
Table F.2 : Additional test-time interventions on physical G1 and LIBERO. Scores (%) with fixed model weights on physical G1 Dog (10 trials) and LIBERO (2,000 episodes across four suites, including 500 on Long ).