Vision-language-action (VLA) models have advanced robot manipulation through large-scale pretraining, but real-world deployment remains challenging due to partial observability and delayed feedback. Reinforcement learning addresses this via value functions, which assess task progress and guide policy improvement. However, existing value models built on vision-language models (VLMs) struggle to capture temporal dynamics and physical interactions, undermining reliable value estimation in long-horizon tasks. In this paper, we propose ViVa, a video-generative value model that repurposes a pretrained video generator to jointly predict future proprioception and a scalar value. By grounding value estimation in anticipated embodiment dynamics, ViVa leverages spatiotemporal priors to intrinsically couple value with foresight beyond static snapshots. ViVa achieves state-of-the-art results in metric-based evaluation across three tasks, producing reliable value signals that accurately track task progress and detect execution errors. Integrated into RECAP, it achieves an average success rate of 80%, highlighting the promise of video-generative models for value estimation.
Figures & tables
Figure 2: Overall architecture of ViVa. Left: Proprioception and the scalar value are embedded into latent frames via repeat-padding and broadcast. Right: The latent sequence comprises clean conditioning frames (blank token, proprioception, multi-view images) and noisy target frames (future proprioception and value). The diffusion Transformer denoises the targets conditioned on the clean prefix, jointly predicting future embodiment state and value.
Figure 3: Illustration of the three real-world tasks. For each task, we show the initial state (left), key intermediate stages (middle), and the final successful state (right).
Figure 4: Error type distribution for shirt folding, box packaging, and paper roll organization. Each pie chart shows the relative frequency of failure modes, based on 50 held-out episodes per task.
Shirt
Box
Paper
Method
Pearson
Spearman
Pearson
Spearman
Pearson
Spearman
Sparse-sampled
GVL
0.276
0.263
-0.081
-0.114
-0.099
-0.096
TopReward
0.118
0.090
0.503
0.499
0.318
0.241
Dense-supervised
π0.6∗
0.740
0.720
0.946
0.945
0.926
0.936
Table 1: Comparison with robotics SOTA value models. Sparse - sampled methods show low or negative correlation. Dense - supervised approaches perform much better, and ViVa establishes new SOTA on two of three tasks.
Figure 5: Detection of subtle manipulation errors by ViVa. The model produces clear value drops at failure moments, confirming sensitivity to fine-grained execution errors.
Table 6
Figure 6: Value trajectory visualization across all tasks. Trajectories for π0.6∗ and ViVa are smoothed by 20%. ViVa tracks task progress with clear responses at key events and maintains sensitivity on the unseen task, demonstrating stronger generalization.
Method
Pearson
Spearman
MS
MDR
π0.6∗
0.740
0.721
0.016
0.397
ViVa
0.819
0.799
0.037
0.643
Table 4: Object generalization on the out-of-domain pants folding task. ViVa consistently outperforms π0.6∗ across all evaluation metrics.
Table 9
Figure 7: Rollout-wise performance evolution across three manipulation tasks. Both methods improve over successive rounds, with RECAP (ViVa) achieving the best overall performance.
Table 11
Method
Pretrained
MS
ES
MDR
ErrDR
ViVa w/o pretrain
✗
0.009
0.023
0.523
0.524
ViVa
✓
0.034
0.026
0.733
0.612
Table 9: Ablation on pretrained weights for box packaging. Removing pretrained weights degrades performance across all metrics, confirming the importance of spatiotemporal priors.
Table 13
Figure 8: Loss weight analysis for balancing value and future proprioception predictions. Setting λprop=0.5 achieves the best trade-off and is adopted across all experiments.
Figure 9: Hyperparameter analysis on the prediction horizon K for future proprioception prediction. Setting K=50 achieves the best trade-off across tasks and is adopted for all experiments.
Method
Backbone
Training cost (GPU ⋅ days)
Inference cost (s / frame)
π0.6∗
VLM
6
0.32
ViVa w/o future-state
VG
3
0.11
ViVa
VG
4
0.18
Table 12: Computational cost comparison. Training time (GPU ⋅ days) and inference time (s) are reported. ViVa achieves faster training and inference than the VLM-based baseline.
Generalist value models play a pivotal role in scaling robotic policy learning from large-scale, mixed-quality data. Mathematically, accurate value estimation demands deep temporal understanding, requiring models to both ground the current belief using historical context and plan over future outcomes. However, most existing robotic value models are built on Vision-Language Model (VLM) backbones that are pretrained primarily on static or temporally sparse visual observations, lacking the requisite temporal modeling capabilities for value estimation. Unlike VLMs, world models naturally excel at temporal modeling and future planning, making them ideal foundations for learning generalizable value functions. Driven by this insight, we marry world models with value estimation to construct a new generalist robotic value model, World Value Model (WVM), that offers accurate task progressions to assess data quality. On standard benchmarks, WVM delivers state-of-the-art (SOTA) Value-Order Correlation (VOC) results. Complementing standard evaluation suites that contains only expert data, we further introduce Suboptimal-Value-Bench, a multi-embodiment benchmark consisting of 800 suboptimal trajectories with high-fidelity, human-labeled frame annotations. Our evaluations show that WVM maintains its SOTA performance on Suboptimal-Value-Bench, establishing its robustness in handling both expert and suboptimal data. When deployed for policy learning, WVM improves manipulation performance across various policy extraction approaches in both simulated and real-world deployment, providing robust guidance for learning from mixed-quality data.
Zhihao Wang, Jianxiong Li, Yu Cui +4
ByteDance Seed · Tsinghua University · Peking University
Vision-Language-Action (VLA) models remain brittle in long-horizon, contact-rich manipulation because success-only imitation provides little supervision for execution drift, while failed rollouts are often discarded. We introduce RePO-VLA, a recovery-driven policy optimization framework that assigns distinct roles to success, recovery, and failure trajectories. RePO-VLA first applies Recovery-Aware Initialization (RAI), slicing each corrective suffix into a new episode so corrective actions depend on the current adverse state rather than the preceding failure. It then learns a Progress-Aware Semantic Value Function (PAS-VF), aligning spatiotemporal trajectory features with instructions and successful references. The resulting labels salvage useful failure prefixes via reliability decay, while low-value labels mark drift and terminal breakdowns, teaching differences among nominal, failed, and corrective actions. The data engine turns adverse states into planner-generated or human-collected corrective rollouts, teaching recovery to the success manifold. Value-Conditioned Refinement (VCR) trains the policy to prefer high-progress actions. At deployment, a fixed high-value token biases actions toward the learned success manifold without online failure detectors or heuristic retries. We introduce the Failure-Recovery Benchmark (FRBench), with standardized error injection and recovery-focused evaluation. Across simulated and real-world bimanual tasks, RePO-VLA improves robustness, raising adversarial success from 20% to 75% on average and up to 80% in scaled real-world trials. Code and video results are available at https://1018weijia.github.io/RePO-VLA-Web/.
Weijia Liufu, Xiaoyu Guo, Ruiyi Chen +16
Sun Yat-sen University · South China University of Technology · Peng Cheng Laboratory +3
Pretrained video generation models provide strong priors for robot control, but existing unified world action models still struggle to decode reliable actions without substantial robot-specific training. We attribute this limitation to a structural mismatch: while video models capture how scenes evolve, action generation requires explicit reasoning about where to interact and the underlying manipulation intent. We introduce AIM, an intent-aware unified world action model that bridges this gap via an explicit spatial interface. Instead of decoding actions directly from future visual representations, AIM predicts an aligned spatial value map that encodes task-relevant interaction structure, enabling a control-oriented abstraction of future dynamics. Built on a pretrained video generation model, AIM jointly models future observations and value maps within a shared mixture-of-transformers architecture. It employs intent-causal attention to route future information to the action branch exclusively through the value representation. We further propose a self-distillation reinforcement learning stage that freezes the video and value branches and optimizes only the action head using dense rewards derived from projected value-map responses together with sparse task-level signals. To support training and evaluation, we construct a simulation dataset of 30K manipulation trajectories with synchronized multi-view observations, actions, and value-map annotations. Experiments on RoboTwin 2.0 benchmark show that AIM achieves a 94.0% average success rate, significantly outperforming prior unified world action baselines. Notably, the improvement is more pronounced in long-horizon and contact-sensitive manipulation tasks, demonstrating the effectiveness of explicit spatial-intent modeling as a bridge between visual world modeling and robot control.
Liaoyuan Fan, Zetian Xu, Chen Cao +3
INFIFORCE Intelligent Technology Co., Ltd., Hangzhou, China · The University of Hong Kong, Hong Kong SAR, China · Shanghai Jiao Tong University, Shanghai, China