D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models

Marius Memmel; Shawn Yin; Urvi Rutia; Angad Wadhwa; Eldon Li; Gavin Angeloff; Ankit Goyal; Jesse Zhang

cs.RONov 2, 2026

D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models

Authors: Marius Memmel, Shawn Yin, Urvi Rutia, Angad Wadhwa, Eldon Li, Gavin Angeloff, Ankit Goyal, Jesse Zhang,

Organizations: University of Washington · NVIDIA

Abstract

Vision-Language Models (VLMs) hold promise for zero-shot video task-progress estimation in robot learning, yet existing frameworks frequently suffer from confirmation bias, hallucinating progress in failed executions due to implicit success assumptions or relative temporal ranking designs. Naive prompting of frontier models collapses into a binary success detector that fails to capture nuanced, non-linear progress and failures. While post-training approaches can mitigate this through explicit training, they demand expensive real-world robotics data and require computationally intensive post-training for every individual base model. To resolve these limitations, we introduce D&A (Describe & Assess): our approach decouples progress estimation into an objective, task-grounded textual description of scene dynamics followed by an assessment phase using explicit rubrics. By compelling the VLM to synthesize an explicit, text-based definition of absolute success before scoring, the framework anchors evaluations to an external, global criterion rather than the video's internal timeline. Empirical results demonstrate that our method calibrates zero-shot estimates and mitigates confirmation bias, providing fine-grained, partial progress metrics that accurately penalize failed executions while correctly interpolating successful runs. Being purely prompt-based, D&A generalizes across tasks, embodiments, and benchmarks---matching or surpassing post-trained reward models---and its reward estimates enable efficient downstream reinforcement learning.

Figures & tables

Fig. 1 : Confirmation Bias in Naïve VLM Reward Estimation: Zero-shot querying causes models to hallucinate progress even on failed tasks, resulting in high prediction variance and no clear difference in reward estimates for successful and failed trajectories.

Fig. 2 : Example reasoning trace generated by GPT-5.3 (queried April 11, 2026) for the opposite instruction “unfold the shirt.”

Fig. 3 : Overview of D&A. Given a video $o_{1:T}$ and instruction $l$ , D&A decomposes reward prediction $R(o_{1:T},l)=A\big(l,\,D(o_{1:T})\big)$ into a Describer $D$ and an Assessor $A$ . $D$ narrates the trajectory blind to $l$ to avoid confirmation bias, producing a temporally consistent description $d_{1:T}$ ; $A$ then scores each timestep of $d_{1:T}$ against an explicit success criterion for $l$ , yielding dense per-frame rewards $r_{1:T}\in[0,1]$ as well as rich language feedback.

Fig. 4 : Real-world RL Results. D&A improves real-world online ( left ) and offline reinforcement learning ( middle, right ) while other zero-shot and post-trained models struggle. Left: in the online setting, TOPReward fails to distinguish between success and failure, predicting high reward for both picking up the ice cream cone and the banana. Middle, right: Behavior cloning on suboptimal data hurts performance, while offline RL with D&A leads to a higher success rate and faster completion time. TOPReward and Robometer lack generalization and assign similar reward to optimal and suboptimal action chunks.

Panel of Figure 5 — Fig. 5 : Results on D&A-Bench , RBM-EVAL-OOD [ 16 ] , and RoboRewardBenchmark [ 15 ] . D&A (Gemini 3 Flash) emerges as the strongest zero-shot method, outperforming ( SOLE-R1 ) and matching the performance ( Robometer / RoboReward ) of post-trained models. Scaling up the base model, D&A ∗ (GPT-6 Astra), tops every benchmark.

Panel of Figure 6 — Fig. 6 : Relative gains of D&A over Naive VLM across various base models on RBM-EVAL-OOD and D&A-Bench .

Panel of Figure 7 — Fig. 7 : Qualitative D&A-Bench predictions for “hit the enter key”: Completion (an on-screen change) is not visible in the final frame ( green : success, red : failure). D&A separates success ( $r_{T}\!\to\!1$ ) from the visually similar failure ( $r_{T}\!\to\!0$ ), while Robometer scores both alike, RoboReward / SOLE-R1 score the failure as a success, and GVL / TOPReward score both high. In the failure case, D&A captures visual nuance of the gripper close to the correct key without pressing it.

Fig. 8 : D&A scales with model capability across four open- and closed-source model families. GVL scales much worse than D&A , while TOPReward and RoboReward show an inverse correlation between performance and model capability.

Panel of Figure 9 — Fig. 9 : Video–language reward confusion matrices over all video–instruction pairings from D&A-Bench (top) RBM-EVAL-OOD (bottom). The reported value is the column-normalized diagonal mean (fraction of reward on correct matches).

Fig. 10 : CaP with rich feedback. Example of iterative improvement with D&A feedback in LIBERO.

Success Criteria ( Sec. IV-B )	Decoupling ( Sec. IV-A )	Grounding ( Sec. IV-A )	Kendall $\tau_{\alpha}$ $\uparrow$
✗	✗	✗	0.486
✓	✗	✗	0.561
✗	✓	✗	0.343
✗	✓	✓	0.546
✓	✓	✗	0.457
✓	✓	✓	0.626

TABLE I : Ablations on RBM-EVAL-OOD .

Table 12

Panel of Figure 13 — Fig. 11 : D&A gains on frontier models

Panel of Figure 14 — Fig. 12 : D&A qualitative gains on GPT-6 Astra

Panel of Figure 15 — Fig. 13 : D&A qualitative gains on Gemini 4 Argon

Model	Cost ($)	Wall Clock Time (s)	Tokens
Gemini 3.1 Pro	0.035	23.7	7,358
Gemini 3.5 Flash	0.039	25.9	8,842
Gemini 2.5 Pro	0.060	58.0	10,843
Gemini 2.5 Flash	0.015	36.9	11,177

TABLE VI : Average compute, latency, and financial cost per trajectory evaluated across 567 trajectories on RBM-EVAL-OOD .

Panel of Figure 18 — Fig. 15 : Qualitative per-trajectory predictions for “assemble the car”.

Panel of Figure 19 — Fig. 16 : Qualitative per-trajectory predictions for “spell MAP with the blocks”.

Panel of Figure 20 — Fig. 17 : Qualitative per-trajectory predictions for “build a 5 block tower”.

Panel of Figure 21 — Fig. 18 : Qualitative per-trajectory predictions for “sort the candy”.

Panel of Figure 22 — Fig. 19 : Qualitative per-trajectory predictions for “move the bowl of skittles without crashing”.

Panel of Figure 23 — Fig. 20 : Qualitative per-trajectory predictions for “pull out ‘Thinking, Fast and Slow’ ”.

Panel of Figure 24 — Fig. 21 : Qualitative per-trajectory predictions for “hit the enter key”.

Panel of Figure 25 — Fig. 22 : Qualitative per-trajectory predictions for “make sure the two block towers are the same height”.

Panel of Figure 26 — Fig. 23 : Qualitative per-trajectory predictions for “measure exactly 16g of coffee”.

Panel of Figure 27 — Fig. 24 : Qualitative per-trajectory predictions for “turn on the light bulb”.

Panel of Figure 28 — Fig. 25 : Qualitative per-trajectory predictions for “put all three MAOAM blocks in the bowl”.

Panel of Figure 29 — Fig. 26 : Qualitative per-trajectory predictions for “unplug the white cable only”.

Panel of Figure 30 — Fig. 27 : Qualitative per-trajectory predictions for “pour half a cup of coke”.

Panel of Figure 31 — Fig. 28 : Qualitative per-trajectory predictions for “put all four MAOAM blocks in the bowl”.

Robot Task Progress Estimation

See all 23 papers

TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics
Feb 22, 2026Shirui Chen, Cole Harrison, Ying-Chun Lee +6Robot Task Progress Estimation Robot Learning
University of Washington · Allen Institute for AI · Amazon +1
ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation
Mar 29, 2026Hongyu Yan, Qiwei Li, Jiaolong Yang +1Robotic Manipulation Vision-Language-Action Models
Peking University · Microsoft Research Asia (MSRA)
From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models
May 22, 2026Christian Gumbsch, Leonardo Barcellona, Lennard Schünemann +7RL for Robotics Multimodal Reward Modeling
University of Amsterdam · 2Catholic University of Leuven · 3Toyota Research Institute +1

Feb 22, 2026J/K move · Enter open · S save

TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics

Shirui Chen, Cole Harrison, Ying-Chun Lee, Angela Jin Yang+5

University of Washington · Allen Institute for AI · Amazon+1

Robot Task Progress Estimation Robot Learning Multimodal Reward Modeling VLMs for Robotics Robot Learning from Video

Open paper AskPDF arXiv

General-purpose robot learning requires dense, instruction-conditioned feedback that can distinguish meaningful task progress from stalled, failed, or partially completed behavior. Yet obtaining such feedback at scale remains difficult, since existing approaches often rely on manual progress annotations, task-specific demonstrations, or reward models trained on curated robot datasets. We introduce TOPReward, a training-free progress reward method that probes pretrained Video-Language Models (VLMs) through their internal token probabilities rather than asking them to generate numerical progress values. Given a video prefix and a language instruction, TOPReward measures the model's likelihood that the instructed task has been completed, converting latent video-language understanding into a dense reward signal without task-specific reward-model training or manually annotated progress labels. We evaluate TOPReward on ManiRewardBench, our real-world manipulation benchmark spanning 130 unique tasks and four robot platforms, as well as on Open X-Embodiment datasets. Across these settings, TOPReward substantially outperforms prior training-free VLM reward methods on open-source models and is competitive with a trained reward-model baseline on progress-estimation metrics, while requiring no reward-model training. Additional analyses show that the reward is sensitive to the specified instruction and is not explained by time index alone. Finally, TOPReward supports downstream applications including success detection and offline reward-weighted behavior cloning.

Abstract

Robot Task Progress Estimation

Explore similar work

TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics

ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation

From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models