Robotic Policy Evaluation
Overview
Robotic policy evaluation asks whether a learned controller can complete tasks reliably, recover from disturbances, follow instructions, and transfer across objects, environments, embodiments, and long-horizon settings. Researchers use simulation and real-robot benchmarks, paired perturbation and recovery tests, process-level video or trajectory metrics, and statistically rigorous comparisons rather than relying only on binary success rates. Other approaches estimate performance with offline fitted-value methods or action-conditioned world models and calibrated simulators, aiming to reduce costly hardware rollouts. The central tension is scalability versus validity: simulators and learned judges can cut evaluation costs but may miss physical or compositional failures, so their predictions need real-world validation and richer diagnostics.
Latest papers
126 papers34 papers in the last four weeks, up 26% on the four weeks before. 0.2% of all new papers.