cs.RONov 5, 2026

Closed-Loop Physics-Based Manipulation Evaluation at a Commodity Price: What 4.4 L4 GPU hours at $3.50 Buys You

Authors: Rohun Kshirsagar, Diana Rypkema

Abstract

Running evaluations of contact-rich robotic manipulation at statistical scale has largely been possible only at well-resourced labs, highlighting the need for affordable and efficient options for broader audiences. We present a reproducible evaluation harness wrapper based on a previous study at 200-rollout scale on serverless commodity GPUs, at a cost of approximately $3.50 USD and 38 minutes of wall-clock per replication (4.4 L4-GPU-hours across eight parallel shards). Using the harness, we report a partially controlled negative result on action chunking with sparse data: three independently trained chunked checkpoints produce zero genuine successes across 780 rollouts, while three single-step checkpoints range from 2.6% to 11.8% success rate. A per-step-replanning control partially isolates chunk prediction, rather than open-loop chunk execution as the mechanism, with the failure localized to the training formulation and multi-step prediction. This work further democratizes physics-based policy development and evaluation through our reusable harness.

Figures & tables

Explore similar work

CardsList
  1. PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunking

    May 30, 2026Junnan Nie, Jiayi Li, Chenghao Liu +4Robotic ManipulationAction Chunking

  2. RoboScope Kit: Unified Interfaces for Fast and Scalable Embodied-AI Evaluation

    Nov 2, 2026Deng-Ke Wu, Yifan Jin, Yida Yang +8Robotic Policy EvaluationEmbodied Agent Benchmarks

  3. Contact-Rich Motion Planning via GPU-Parallel Mode Evaluation

    Sep 18, 2026Jiayun Li, Georgia ChalvatzakiContact-Aware Motion PlanningMotion Planning