Vision-Language Models (VLMs) hold promise for zero-shot video task-progress estimation in robot learning, yet existing frameworks frequently suffer from confirmation bias, hallucinating progress in failed executions due to implicit success assumptions or relative temporal ranking designs. Naive prompting of frontier models collapses into a binary success detector that fails to capture nuanced, non-linear progress and failures. While post-training approaches can mitigate this through explicit training, they demand expensive real-world robotics data and require computationally intensive post-training for every individual base model. To resolve these limitations, we introduce D&A (Describe & Assess): our approach decouples progress estimation into an objective, task-grounded textual description of scene dynamics followed by an assessment phase using explicit rubrics. By compelling the VLM to synthesize an explicit, text-based definition of absolute success before scoring, the framework anchors evaluations to an external, global criterion rather than the video's internal timeline. Empirical results demonstrate that our method calibrates zero-shot estimates and mitigates confirmation bias, providing fine-grained, partial progress metrics that accurately penalize failed executions while correctly interpolating successful runs. Being purely prompt-based, D&A generalizes across tasks, embodiments, and benchmarks---matching or surpassing post-trained reward models---and its reward estimates enable efficient downstream reinforcement learning.
Figures & tables
Fig. 1 : Confirmation Bias in Naïve VLM Reward Estimation: Zero-shot querying causes models to hallucinate progress even on failed tasks, resulting in high prediction variance and no clear difference in reward estimates for successful and failed trajectories.
Fig. 2 : Example reasoning trace generated by GPT-5.3 (queried April 11, 2026) for the opposite instruction “unfold the shirt.”
Fig. 3 : Overview of D&A. Given a video o1:T and instruction l , D&A decomposes reward prediction R(o1:T,l)=A(l,D(o1:T)) into a Describer D and an Assessor A . D narrates the trajectory blind to l to avoid confirmation bias, producing a temporally consistent description d1:T ; A then scores each timestep of d1:T against an explicit success criterion for l , yielding dense per-frame rewards r1:T∈[0,1] as well as rich language feedback.
Fig. 4 : Real-world RL Results. D&A improves real-world online ( left ) and offline reinforcement learning ( middle, right ) while other zero-shot and post-trained models struggle. Left: in the online setting, TOPReward fails to distinguish between success and failure, predicting high reward for both picking up the ice cream cone and the banana. Middle, right: Behavior cloning on suboptimal data hurts performance, while offline RL with D&A leads to a higher success rate and faster completion time. TOPReward and Robometer lack generalization and assign similar reward to optimal and suboptimal action chunks.
Fig. 5 : Results on D&A-Bench , RBM-EVAL-OOD [ 16 ] , and RoboRewardBenchmark [ 15 ] . D&A (Gemini 3 Flash) emerges as the strongest zero-shot method, outperforming ( SOLE-R1 ) and matching the performance ( Robometer / RoboReward ) of post-trained models. Scaling up the base model, D&A ∗ (GPT-6 Astra), tops every benchmark.
Fig. 6 : Relative gains of D&A over Naive VLM across various base models on RBM-EVAL-OOD and D&A-Bench .
Fig. 7 : Qualitative D&A-Bench predictions for “hit the enter key”: Completion (an on-screen change) is not visible in the final frame ( green : success, red : failure). D&A separates success ( rT→1 ) from the visually similar failure ( rT→0 ), while Robometer scores both alike, RoboReward / SOLE-R1 score the failure as a success, and GVL / TOPReward score both high. In the failure case, D&A captures visual nuance of the gripper close to the correct key without pressing it.
Fig. 8 : D&A scales with model capability across four open- and closed-source model families. GVL scales much worse than D&A , while TOPReward and RoboReward show an inverse correlation between performance and model capability.
Fig. 9 : Video–language reward confusion matrices over all video–instruction pairings from D&A-Bench (top) RBM-EVAL-OOD (bottom). The reported value is the column-normalized diagonal mean (fraction of reward on correct matches).
Fig. 10 : CaP with rich feedback. Example of iterative improvement with D&A feedback in LIBERO.
Success Criteria ( Sec. IV-B )
Decoupling ( Sec. IV-A )
Grounding ( Sec. IV-A )
Kendall τα↑
✗
✗
✗
0.486
✓
✗
✗
0.561
✗
✓
✗
0.343
✗
✓
✓
0.546
✓
✓
✗
0.457
✓
✓
✓
0.626
TABLE I : Ablations on RBM-EVAL-OOD .
Table 12
Fig. 11 : D&A gains on frontier models
Fig. 12 : D&A qualitative gains on GPT-6 Astra
Fig. 13 : D&A qualitative gains on Gemini 4 Argon
Figure 16
Model
Cost ($)
Wall Clock Time (s)
Tokens
Gemini 3.1 Pro
0.035
23.7
7,358
Gemini 3.5 Flash
0.039
25.9
8,842
Gemini 2.5 Pro
0.060
58.0
10,843
Gemini 2.5 Flash
0.015
36.9
11,177
TABLE VI : Average compute, latency, and financial cost per trajectory evaluated across 567 trajectories on RBM-EVAL-OOD .
Fig. 15 : Qualitative per-trajectory predictions for “assemble the car”.
Fig. 16 : Qualitative per-trajectory predictions for “spell MAP with the blocks”.
Fig. 17 : Qualitative per-trajectory predictions for “build a 5 block tower”.
Fig. 18 : Qualitative per-trajectory predictions for “sort the candy”.
Fig. 19 : Qualitative per-trajectory predictions for “move the bowl of skittles without crashing”.
Fig. 20 : Qualitative per-trajectory predictions for “pull out ‘Thinking, Fast and Slow’ ”.
Fig. 21 : Qualitative per-trajectory predictions for “hit the enter key”.
Fig. 22 : Qualitative per-trajectory predictions for “make sure the two block towers are the same height”.
Fig. 23 : Qualitative per-trajectory predictions for “measure exactly 16g of coffee”.
Fig. 24 : Qualitative per-trajectory predictions for “turn on the light bulb”.
Fig. 25 : Qualitative per-trajectory predictions for “put all three MAOAM blocks in the bowl”.
Fig. 26 : Qualitative per-trajectory predictions for “unplug the white cable only”.
Fig. 27 : Qualitative per-trajectory predictions for “pour half a cup of coke”.
Fig. 28 : Qualitative per-trajectory predictions for “put all four MAOAM blocks in the bowl”.
General-purpose robot learning requires dense, instruction-conditioned feedback that can distinguish meaningful task progress from stalled, failed, or partially completed behavior. Yet obtaining such feedback at scale remains difficult, since existing approaches often rely on manual progress annotations, task-specific demonstrations, or reward models trained on curated robot datasets. We introduce TOPReward, a training-free progress reward method that probes pretrained Video-Language Models (VLMs) through their internal token probabilities rather than asking them to generate numerical progress values. Given a video prefix and a language instruction, TOPReward measures the model's likelihood that the instructed task has been completed, converting latent video-language understanding into a dense reward signal without task-specific reward-model training or manually annotated progress labels. We evaluate TOPReward on ManiRewardBench, our real-world manipulation benchmark spanning 130 unique tasks and four robot platforms, as well as on Open X-Embodiment datasets. Across these settings, TOPReward substantially outperforms prior training-free VLM reward methods on open-source models and is competitive with a trained reward-model baseline on progress-estimation metrics, while requiring no reward-model training. Additional analyses show that the reward is sensitive to the specified instruction and is not explained by time index alone. Finally, TOPReward supports downstream applications including success detection and offline reward-weighted behavior cloning.
Shirui Chen, Cole Harrison, Ying-Chun Lee +6
University of Washington · Allen Institute for AI · Amazon +1
Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness, typically relying on hand-crafted heuristics for task termination. This limitation is particularly severe in long-horizon tasks involving cascaded sub-goals. In this work, we investigate the estimation and integration of task progress, proposing a novel model named {\textbf \vla}. Our technical contributions are twofold: (1) robust progress estimation: We pre-train a progress estimator on large-scale, unsupervised video-text robotic datasets. This estimator achieves a low prediction residual (0.07 on a scale of [0,1]) in simulation and demonstrates zero-shot generalization to unseen real-world samples, and (2) differentiable progress guidance: We introduce an inverse dynamics world model that maps predicted action tokens into future latent visual states. These latents are then processed by the progress estimator; by applying a maximal progress regularization, we establish a differentiable pipeline that provides progress-piloted guidance to refine action tokens. Extensive experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates and generalization over strong baselines.
Hongyu Yan, Qiwei Li, Jiaolong Yang +1
Peking University · Microsoft Research Asia (MSRA)
Reinforcement learning relies on accurate reward functions, which are often hand-crafted or even unavailable in real-world applications, such as robotics. Recent work has explored the zero-shot reasoning capabilities of pre-trained Vision-Language Models (VLMs) as reward models. However, without careful prompt engineering, these approaches tend to produce suboptimal rewards, where false positive predictions can severely degrade downstream policy learning. In robotics, limited datasets comprising expert demonstrations are often collected to bootstrap policy learning. This scenario provides an opportunity to optimize a reward model prior policy training. We propose Demo2Reward a test-time adaptation technique to optimize the language instruction of a reward model based on a few demonstrations (3-10 trajectories) to reduce false positives while preserving true positives. Crucially, this requires no additional model training or computation resources during policy learning. We show that Demo2Reward consistently outperforms existing zero- and few-shot VLM reward models across a range of simulated robotic tasks and policy backbones. Finally, we demonstrate that Demo2Reward effectively transfers to a real-world robotic learning scenario, enabling policy learning without manually engineering a reward function.
Christian Gumbsch, Leonardo Barcellona, Lennard Schünemann +7
University of Amsterdam · 2Catholic University of Leuven · 3Toyota Research Institute +1