Overview

Robotic policy evaluation asks whether a learned controller can complete tasks reliably, recover from disturbances, follow instructions, and transfer across objects, environments, embodiments, and long-horizon settings. Researchers use simulation and real-robot benchmarks, paired perturbation and recovery tests, process-level video or trajectory metrics, and statistically rigorous comparisons rather than relying only on binary success rates. Other approaches estimate performance with offline fitted-value methods or action-conditioned world models and calibrated simulators, aiming to reduce costly hardware rollouts. The central tension is scalability versus validity: simulators and learned judges can cut evaluation costs but may miss physical or compositional failures, so their predictions need real-world validation and richer diagnostics.

Latest papers

CardsList

149 papers35 papers in the last four weeks, down 8% on the four weeks before. 0.3% of all new papers.

  1. Practice Is Not Free: Measuring the Embodied Cost of Reset-Free Robot Practice

    Nov 15, 2026Vedant Telrandhe, Sushil SharmaRobot LearningRobotic Policy Evaluation

  2. JustRUA: A Controlled Study of GPT-6 Robot-Use Agents

    Nov 11, 2026Zefan Wang, Yufan Dang, Xuanwei Shi +9Robotic ControlRobotic Manipulation

  3. Embodied First Exam

    Nov 9, 2026Kangrui Wang, Qineng Wang, Changhe Chen +27Embodied Agent BenchmarksRobotic Policy Evaluation

  4. Look Ahead: Search-Driven Reasoning in Embodied AI

    Nov 7, 2026Ahmed Zeer, Osama Shbib, Muzaffer Kaan Yuce +2Model-Based PlanningRobotic Policy Evaluation

  5. Goodhart in the Planner: Sampling-Based Control Exploits Rewards Learned from Human Preferences

    Nov 6, 2026Soovadeep Bakshi, Apurba BoseReward HackingPreference-Based RL

  6. RoboScope Kit: Unified Interfaces for Fast and Scalable Embodied-AI Evaluation

    Nov 2, 2026Deng-Ke Wu, Yifan Jin, Yida Yang +8Robotic Policy EvaluationEmbodied Agent Benchmarks

  7. Beyond L∞L_\infty-Norm Sets: Synthesizing Interpretable Non-Uniform Uncertainty Sets for Evaluation-Time State-Adversarial Robustness

    Nov 2, 2026Xulin Chen, Ruipeng Liu, Garrett E. KatzRobust RLRobotic Policy Evaluation

  8. RoboColosseum: Evaluating Physical Intelligence at Scale In-The-Wild

    Nov 2, 2026Tan Sang Nguyen, Yi Ru Wang, Shailesh Shailesh +24Robotic Policy EvaluationEmbodied Agent Benchmarks

  9. Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift

    Nov 1, 2026Qurban Ali, Badaruddin Chachar, Maqsood Ali +1Safe RLRobotic Policy Evaluation

  10. Beyond Validation Loss: Coverage and Reliability Dynamics in Robot Policy Pretraining

    Nov 1, 2026Zhanyi Sun, Max Simchowitz, Shuran SongRobot LearningRobotic Policy Evaluation

  11. When the Simulator Fails, Not the Policy: Auditing Contact Artifacts in Simulated Robot Benchmarks

    Oct 30, 2026Alex Zhou, Junhao Wu, Xueyang Zhou +5Robotics SimulationRobot Manipulation Benchmarks

  12. From Teleoperation to Long-Horizon Dexterity: Evaluating Visuomotor Skills and Offline Proxies for Dexterous Manipulation

    Oct 29, 2026Ammar N. Abbas, Aayush Jain, Prashanth Viswanath +3Visuomotor Policy LearningRobotic Manipulation

  13. CuraEval: Evaluating Demonstration Curation on Mixed-Skill Robot Data

    Oct 28, 2026Jiuyi Xu, Meida Chen, Yang Ye +1demonstration selectionRobot Imitation Learning

  14. Kinematic Constraint-Referenced Evaluation for Articulated Object Manipulation

    Oct 27, 2026Baptiste Bonin, Caro Strickland, Audrey DurandArticulated Object ManipulationRobotic Policy Evaluation

  15. Evaluation-Guided Demonstration Refinement for Vision-Language-Action Policies: The Winning Entry in the MetaFine Open Competition

    Oct 27, 2026Lifan Zheng, Guotao Chen, Zhenyang Li +3VLA Model TrainingLearning from Demonstration

  16. Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy

    Oct 26, 2026Yifei Yao, Yude Zou, Guanbo Jiao +12Language Model-Based ControlRobotic Policy Evaluation

  17. SPECOLA: A Tool for Specification Driven Robot Evaluation

    Oct 26, 2026Immanuel Ampomah Mensah, Hadas Kress-GazitRobotic Policy EvaluationRobotics Simulation