cs.ROOct 19, 2026

RoboValue: A Fine-Grained Sim-and-Real Benchmark for Unified Evaluation of Robotic Value Models

Authors: Shengbang Liu, Zhengye Du, Zhilong Wan, Honghao Su, Chenxiang Xia, Chang Ge, Jinyang Xiao, Nan Wang, 

Organizations: Tsinghua University · AgiBot · Xspark AI · Sun Yat-sen University · 

Abstract

Robotic value models assess task execution from visual observations and instructions, providing feedback for policy learning and execution monitoring. However, existing evaluations cover only selected aspects of execution understanding, leaving the reliability of value feedback insufficiently tested. We introduce RoboValue, a fine-grained sim-and-real benchmark for unified evaluation of robotic value models. Its framework evaluates four complementary capability dimensions through shared interfaces: task-state understanding, temporal progress monitoring, failure and recovery reasoning, and value consistency. Diagnostic trajectories and instruction variations test whether value judgments accurately reflect task requirements and changes throughout execution. RoboValue-Dataset covers 15 simulation and 20 real-world manipulation tasks, providing expert demonstrations for training and annotated trajectories for fine-grained evaluation. We evaluate representative robotic value models in zero-shot and one-shot settings under standard conditions and separate shifts in robot embodiment and environment. Our results reveal errors in fine-grained instruction grounding, history-dependent progress tracking, and recovery assessment, even in models with strong outcome and forward-progress scores. The same semantic subtask can also receive substantially different value gains across valid execution orders. These findings identify concrete targets for value supervision, while the dataset and public leaderboard support continued assessment of feedback reliability. The website is available at https://robovalue-benchmark.github.io/.

Explore similar work

CardsList
  1. World Value Models for Robotic Manipulation

    Jun 23, 2026Zhihao Wang, Jianxiong Li, Yu Cui +4Robot Policy LearningWorld Models for Robotics

  2. RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance

    Aug 10, 2026Dongchi Huang, Hongyin Zhang, Bohan Hou +11Reward ModelingRobot Foundation Models

  3. RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies

    Apr 10, 2026Jenai Xuning Yang, Rishit Dagli, Alex Zook +5Robotic Policy EvaluationRobot Policy Generalization