The Relative Value of Demonstration Data: Execution Time Limits and Operator Mixtures in Behavior Cloning
Organizations: Stanford University · The Chinese University of Hong Kong · University of Maryland, College Park
Abstract
Robot demonstration datasets combine demonstrations from teleoperators of uneven experience, so collection teams need to know how many demonstrations from one group match those from another. We measure this with a data multiple: the ratio of the numbers of demonstrations two groups need for their policies to reach the same success rate. On robomimic's simulated Square and Can tasks, we train behavior-cloning policies on demonstrations from the two teleoperators robomimic labels Better and the two it labels Worse, varying dataset size and the share of Worse demonstrations. On Square, within robomimic's 25-second time limit, an all-Worse training set needs 1.55 times as many demonstrations as an all-Better set. Most of this difference is late completion: policies trained on Worse demonstrations finish later, as the Worse teleoperators do, and with a 100-second limit the multiple falls to 1.11. On Can, most of the difference remains at 100 seconds. If each Worse demonstration were worth 1/1.55 of a Better one, as in an all-Worse set, a half-Better, half-Worse set on Square would need 1.22 times as many demonstrations as an all-Better set. In fact, it needs 13% more than that, and this excess persists at every time limit from 20 to 200 seconds. The excess remains when training draws demonstrations uniformly instead of in proportion to their length. On Square, within the 25-second limit, switching to these Worse teleoperators is cheaper at equal success only if they earn less than a third of the Better teleoperators' hourly pay, because Worse demonstrations also take longer to record.
Figures & tables
| Estimate | 25 s | 100 s | 200 s |
|---|---|---|---|
| Shared slope, additive prediction | 13 [6, 21] | 13 [7, 20] | 14 [8, 20] |
| Shared slope, training-sample share prediction | 6 [ 1, 13] | 11 [5, 18] | – |
| Can : shared slope, additive prediction | 9 [1, 17] | 9 [1, 17] | 10 [2, 18] |
| Monotone-curve inversion, | 15 [2, 28] | – | – |
| Monotone-curve inversion, | 13 [5, 25] | – | – |
| Monotone-curve inversion, | 16 [8, 27] | 18 [8, 30] | 16 [8, 29] |
Appendix figures & tables34 assets
Supplementary material from the paper’s appendix.
Appendix
| Term | Definition |
|---|---|
| Dataset size | Number of demonstrations in a training set. |
| Worse share | Fraction of a training set’s demonstrations recorded by the Worse teleoperators: for the all- Better set, for the half- Better , half- Worse set, and for the all- Worse set. |
| Time limit | Longest execution time within which a success counts: 15, 20, 25, 40, 100, and 200 s are 300, 400, 500, 800, 2000, and 4000 control steps at 20 Hz. |
| Late completion | A success after 25 s and within a longer time limit. |
| Data multiple | Factor by which a set with Worse share must be larger than the all- Better set for its policies to reach the same success rate, read from the shared-slope fit of each seed as ; is the geometric mean over seeds. |
| All- Worse multiple | The data multiple of the all- Worse set. |
| Design | Values |
|---|---|
| Square | , ; 24 seeds for and the first 12 for the other two shares; 768 policies. |
| Can | , ; 12 seeds; 216 policies. |
| Evaluation | Checkpoints, episodes per policy, time limits |
| Final checkpoint | Epoch 2000; 100 initial states, a subset of the five-checkpoint states; up to 200 s. |
| Five checkpoints | 5 checkpoints at epochs 1600–2000; 500 episodes; 15–40 s. |
| Experiment | Seeds | Policies | Five-ckpt. episodes | Final-ckpt. episodes |
| Six teleoperators, | 3 | 18 | 500 | 100 |
| Teleoperator pairs, | 6 | 48 | 500 | 100 |
| Sampler experiment, both samplers, eight sizes | 12 | 576 | – | 100 |
| Single teleoperators and groups, | 12 | 72 | 500 | 100 |
| Teleoperator pairs and groups, | 12 | 72 | 500 | 100 |
| Single teleoperators and groups, three sizes | 6 | 108 | 500 | – |
| Dataset | Demonstrations | Seeds | Ours | Published |
| Square MH Better | 90 | 3 | 0.660 | |
| Square MH Worse | 90 | 3 | 0.393 | |
| Square PH 20% | 36 | 3 | 0.387 | |
| Square PH 50% | 90 | 1 | 0.673 | |
| Square PH 100% | 180 | 1 | 0.840 | |
| Can MH Better | 90 | 3 | 0.987 [0.958, 1.015] |
| Five-checkpoint evaluation, 25 s | Estimate |
|---|---|
| Prediction RMSE, shared / separate | 0.34 / 0.58 |
| Paired RMSE difference; seeds favoring shared | 0.24 [ 0.37, 0.11]; 10/12 |
| AIC decrease, shared separate, 20/25/40 s | 7.0 / 4.5 / 5.7 |
| Likelihood-ratio statistic; bootstrap | 12.5; 0.12 |
| Evaluation | Limit (s) | Growth, % [joint interval] | Fixed interval |
|---|---|---|---|
| Final checkpoint | 25 | 31 | |
| Final checkpoint | 200 | 22 | |
| Five checkpoints | 25 | 24 |
| Limit (s) | Model | ||
|---|---|---|---|
| 25 | OLS | 1.55 [1.39, 1.73] | 0.13 [0.06, 0.19] |
| Bin. | 1.63 [1.52, 1.74] | 0.16 [0.10, 0.21] | |
| BB | 1.62 [1.51, 1.74] | 0.16 [0.10, 0.21] | |
| 40 | OLS | 1.30 [1.18, 1.43] | 0.13 [0.07, 0.19] |
| Bin. | 1.35 [1.27, 1.44] | 0.16 [0.10, 0.21] | |
| BB | 1.35 [1.26, 1.44] | 0.16 [0.10, 0.21] |
| Task, evaluation | Monotone-curve | No solution | Separate-line | No solution | |
|---|---|---|---|---|---|
| Square , five checkpoints, 25 s | 0.05 | 1.516 [1.332, 1.708] | 0% | 1.523 [1.369, 1.706] | 0% |
| Square , five checkpoints, 25 s | 0.18 | 1.565 [1.457, 1.691] | 0% | 1.600 [1.515, 1.693] | 0% |
| Square , five checkpoints, 25 s | 0.32 | 1.713 [1.621, 1.804] | 5% | 1.640 [1.566, 1.712] | 5% |
| Square , final checkpoint, 100 s | 0.07 | 1.098 [0.971, 1.294] | 0% | 1.040 [0.894, 1.226] | 0% |
| Square , final checkpoint, 100 s | 0.28 | 1.129 [1.049, 1.213] | 0% | 1.138 [1.070, 1.209] | 0% |
| Square , final checkpoint, 100 s | 0.50 | 1.152 [1.054, 1.264] | 4% | 1.195 [1.096, 1.269] | 13% |
| Excess, % | |||
| Five-checkpoint evaluation, 25 s | |||
| 0.10 | 1.509 [1.317, 1.692] | 1.425 [1.288, 1.556] | 18 [10, 26] |
| 0.20 | 1.598 [1.488, 1.723] | 1.387 [1.277, 1.527] | 13 [5, 23] |
| 0.30 | 1.704 [1.618, 1.802] | 1.492 [1.406, 1.593] | 18 [12, 26] |
| Final-checkpoint evaluation, 25 s | |||
| 0.10 | 1.628 [1.376, 1.869] | 1.423 [1.230, 1.591] | 15 [2, 28] |
| Fit / seeds | , 25 100 s | , 25 s | , 25 100 s |
|---|---|---|---|
| All sets / 24 | 0.33 [ 0.40, 0.27] | 0.13 [0.04, 0.21] | 0.00 [ 0.03, 0.03] |
| / 24 | 0.33 [ 0.39, 0.27] | 0.12 [0.04, 0.21] | 0.00 [ 0.03, 0.03] |
| All sets / first 12 | 0.30 [ 0.41, 0.19] | 0.10 [0.01, 0.19] | 0.00 [ 0.05, 0.05] |
| / first 12 | 0.29 [ 0.38, 0.20] | 0.10 [0.01, 0.18] | 0.00 [ 0.05, 0.05] |
| Main fit: percentile intervals by resampled unit | |||
| Seeds | [ 0.39, 0.28] | [0.06, 0.20] | [ 0.03, 0.03] |
| Quantity | Task | 100 s | 200 s | Paired change |
|---|---|---|---|---|
| Square | [0.97, 1.27] | [0.95, 1.23] | [ 0.06, 0.004] | |
| Can | [1.42, 1.86] | [1.39, 1.81] | [ 0.04, 0.01] | |
| Square | [0.05, 0.21] | [0.06, 0.21] | [ 0.02, 0.02] |
| Square | Can , 6 sizes | ||||
|---|---|---|---|---|---|
| Limit (s) | |||||
| 25 | 1.37 [1.26, 1.49] | 1.55 [1.39, 1.73] | 0.13 [0.06, 0.19] | 1.88 [1.72, 2.07] | 0.09 [0.01, 0.16] |
| 40 | 1.28 [1.18, 1.39] | 1.30 [1.18, 1.43] | 0.13 [0.07, 0.19] | 1.72 [1.58, 1.90] | 0.09 [0.02, 0.16] |
| 100 | 1.19 [1.10, 1.29] | 1.11 [1.02, 1.22] | 0.13 [0.07, 0.19] | 1.61 [1.48, 1.77] | 0.09 [0.01, 0.16] |
| 200 | 1.18 [1.10, 1.26] | 1.08 [0.99, 1.18] | 0.13 [0.07, 0.19] | 1.58 [1.45, 1.73] | 0.09 [0.02, 0.17] |
| Task | Worse share | Seeds | ||
|---|---|---|---|---|
| Square | 0.25 | 12 | 1.17 [1.07, 1.27] | 0.05 [ 0.02, 0.13] |
| 0.5 | 24 | 1.41 [1.33, 1.50] | 0.14 [0.10, 0.19] | |
| 0.75 | 12 | 1.58 [1.41, 1.77] | 0.12 [0.07, 0.17] | |
| 1 | 24 | 1.60 [1.47, 1.73] | – | |
| Can | 1 | 12 | 1.95 [1.76, 2.16] | – |
| Late, interrupted/total | Late, interrupted % | Early, interrupted/total | Early, interrupted % | ||
|---|---|---|---|---|---|
| 24 | 0 | 28/33 | 85 [73, 95] | 57/294 | 19 [16, 23] |
| 24 | 0.5 | 43/61 | 70 [59, 82] | 36/185 | 19 [15, 24] |
| 24 | 1 | 63/103 | 61 [54, 69] | 27/142 | 19 [13, 26] |
| 100 | 0 | 38/55 | 69 [57, 81] | 219/1308 | 17 [14, 19] |
| 100 | 0.5 | 133/211 | 63 [55, 71] | 157/915 | 17 [15, 20] |
| 100 | 1 | 210/412 | 51 [45, 57] | 129/841 | 15 [13, 18] |
| Quantity | Better | Worse |
|---|---|---|
| Square median success time (s), five-checkpoint evaluation | 9.3 | 17.0 |
| Square late completions at , % of successes within 100 s | 4 | 33 |
| Can late completions, % of successes within 100 s | 2 | 14 |
| Square Worse / Better time ratios | Estimate | |
| Policies, grasp to success | 1.95 [1.84, 2.06] | |
| Demonstrations, grasp to success | 1.74 [1.57, 1.91] | |
| Event rule | One-grasp fraction, % |
|---|---|
| Raw contact losses; no release exclusion | 8 [4, 12] |
| Raw contacts; releases within 5 steps excluded | 40 [34, 47] |
| Contact breaks under 5 steps merged; releases within 5 steps excluded | 51 [45, 57] |
| Contact breaks under 5 steps merged; releases within 20 steps excluded | 53 [47, 59] |
| Half- Better , half- Worse minus all- Worse , | 25 s | 100 s |
|---|---|---|
| Success difference, percentage points | 3 [ 1, 7] | 5 [ 9, 1] |
| Task/teleoperator | Pause % | Interrupted/ total | Interrupted % | Regrasp count (%) | Grasp loss | Loss then regrasp, % |
|---|---|---|---|---|---|---|
| Square /B1 | 6/50 | 12 [4, 22] | 1 ( ) | 5 | ||
| Square /B2 | 3/50 | 6 [0, 14] | 1 ( ) | 3 | ||
| Square /W1 | 24/50 | 48 [34, 62] | 10 ( ) | 20 | ||
| Square /W2 | 14/50 | 28 [16, 40] | 9 ( ) | 10 | ||
| Can /B1 | – | – | – | – | – | |
| Can /B2 | – | – | – | – | – |
| Task | Pause, % | Regrasp, % | Loss then regrasp, % | ||
|---|---|---|---|---|---|
| Square | 24 | 0 | |||
| Square | 24 | 0.5 | |||
| Square | 24 | 1 | |||
| Square | 100 | 0 | |||
| Square | 100 | 0.5 | |||
| Square | 100 | 1 |
| Can | Success at | ||
|---|---|---|---|
| 0 | 0.93 | 2.02 | – |
| 1 | 0.69 | 1.64 | 0.38 [ 0.59, 0.16] |
| Evaluation | Log excess over the training- sample share prediction, | Ratio to the log excess over the additive prediction | |
|---|---|---|---|
| Five checkpoints, 25 s | 0.25 | 0.01 [ 0.08, 0.07] | – |
| Five checkpoints, 25 s | 0.5 | 0.07 [0.02, 0.12] | 48% [22%, 62%] |
| Five checkpoints, 25 s | 0.75 | 0.06 [0.01, 0.11] | 51% [11%, 72%] |
| Final checkpoint, 25 s | 0.5 | – | 43% [ 16%, 64%] |
| Final checkpoint, 40 s | 0.5 | – | 67% [36%, 80%] |
| Final checkpoint, 100 s | 0.5 | – | 86% [70%, 97%] |
| Sampler | Limit (s) | Excess (%) | ||||
|---|---|---|---|---|---|---|
| Default | 25 | 1.64 [1.46, 1.85] | 1.43 [1.25, 1.63] | 1.24 [1.19, 1.30] | 0.14 [0.04, 0.25] | 15 [4, 28] |
| Default | 40 | 1.37 [1.23, 1.53] | 1.32 [1.16, 1.51] | 1.16 [1.10, 1.21] | 0.14 [0.04, 0.23] | 15 [4, 26] |
| Default | 100 | 1.18 [1.06, 1.32] | 1.23 [1.08, 1.41] | 1.08 [1.03, 1.14] | 0.13 [0.04, 0.23] | 14 [4, 25] |
| Default | 200 | 1.15 [1.03, 1.28] | 1.20 [1.05, 1.36] | 1.07 [1.01, 1.12] | 0.12 [0.03, 0.20] | 12 [3, 23] |
| Uniform | 25 | 1.56 [1.41, 1.72] | 1.39 [1.28, 1.50] | 1.22 [1.17, 1.26] | 0.13 [0.06, 0.21] | 14 [6, 23] |
| Uniform | 40 | 1.34 [1.23, 1.47] | 1.32 [1.22, 1.43] | 1.15 [1.10, 1.19] | 0.14 [0.07, 0.22] | 15 [7, 24] |
| Default sampler | Demonstration-uniform sampler | |||
|---|---|---|---|---|
| Limit (s) | ||||
| 25 | 0.09 [ 0.26, 0.08] | 0.13 [ 0.37, 0.12] | 0.13 [ 0.36, 0.09] | 0.14 [ 0.35, 0.06] |
| 40 | 0.08 [ 0.26, 0.10] | 0.09 [ 0.36, 0.18] | 0.15 [ 0.34, 0.05] | 0.10 [ 0.26, 0.07] |
| 100 | 0.06 [ 0.27, 0.15] | 0.12 [ 0.35, 0.12] | 0.15 [ 0.34, 0.04] | 0.06 [ 0.24, 0.11] |
| 200 | 0.08 [ 0.27, 0.10] | 0.11 [ 0.32, 0.11] | 0.14 [ 0.33, 0.06] | 0.07 [ 0.24, 0.10] |
| Evaluation | Limit (s) | Better | Worse |
|---|---|---|---|
| Five checkpoints | 25 | 0.64 [0.39, 0.88] | 0.27 [0.07, 0.47] |
| Five checkpoints | 40 | 0.65 [0.41, 0.90] | 0.25 [0.08, 0.43] |
| Final checkpoint | 25 | 0.60 [0.20, 0.99] | 0.34 [ 0.02, 0.70] |
| Final checkpoint | 40 | 0.62 [0.23, 1.02] | 0.23 [ 0.03, 0.48] |
| Final checkpoint | 100 | 0.60 [0.19, 1.01] | 0.22 [ 0.03, 0.46] |
| Set | Single-teleoperator log excess | |
|---|---|---|
| , baseline: average single Better teleoperator | ||
| B1+B2 | – | 0.34 [0.16, 0.52] |
| W1+W2 | – | 0.19 [0.03, 0.35] |
| Four teleoperators, | – | 0.42 [0.21, 0.64] |
| Four teleoperators, | – | 0.41 [0.29, 0.54] |
| Four teleoperators, | – | 0.37 [0.20, 0.55] |
| Pair | Five checkpoints, 25 s | Final checkpoint, 100 s |
|---|---|---|
| B1+W1 | 0.26 [0.17, 0.36] | 0.29 [0.17, 0.41] |
| B1+W2 | 0.05 [ 0.05, 0.15] | 0.10 [ 0.02, 0.23] |
| B2+W1 | 0.26 [0.19, 0.33] | 0.33 [0.18, 0.48] |
| B2+W2 | 0.16 [ 0.26, 0.06] | 0.15 [ 0.34, 0.04] |
| Mean | 0.10 [0.02, 0.18] | 0.14 [0.01, 0.27] |
| B1 | B2 | O1 | O2 | W1 | W2 | |
|---|---|---|---|---|---|---|
| Success log-odds | 0.96 | 0.70 | 1.11 | 1.56 | 2.30 | 1.37 |
| Median success time (s) | 7.6 | 11.2 | 16.8 | 14.1 | 18.5 | 18.8 |
| Multiple | 1.09 | 0.92 | 1.20 | 1.58 | 2.53 | 1.41 |
| Policies compared | Seeds/ policies | Success-rate change, pp | Slope-contrast change | ||
|---|---|---|---|---|---|
| 25 s | 40 s | 25 s | 40 s | ||
| All seeds, both sets | 9/72 | 2 [ 3, 0.2] | 2 [ 4, 0.4] | 0.16 [ 0.46, 0.13] | 0.11 [ 0.46, 0.23] |
| Same evaluation machine, both sets | 6/48 | 3 [ 4, 1] | 3 [ 4, 1] | 0.21 [ 0.71, 0.29] | 0.12 [ 0.72, 0.47] |
| Same evaluation machine, | 6/24 | 2 [ 3, 1] | 2 [ 3, 1] | – | – |
| Same evaluation machine, | 9/36 | 2 [ 4, 1] | 2 [ 5, 1] | – | – |
| Paired change | ratio | ratio | , 25 s | , 20 s | , 40 s |
|---|---|---|---|---|---|
| 2000 4000 epochs | 1.05 [1.01, 1.08] | 1.02 [0.98, 1.06] | 0.04 [0.004, 0.07] | 0.04 [0.01, 0.08] | 0.03 [ 0.0004, 0.07] |
| 8 seeds | ||
| Metric | Diffusion | BC-RNN-GMM |
| 0.40 [ 0.49, 0.31] | 0.31 [ 0.37, 0.25] | |
| , 25 s | 0.10 [ 0.09,0.30] | 0.12 [ 0.06,0.30] |
| , 100 s | 0.07 [ 0.11,0.24] | 0.08 [ 0.06,0.23] |
| Limit (s) | Diffusion, % | BC-RNN-GMM, % | Diffusion BC-RNN-GMM, pp | ||
|---|---|---|---|---|---|
| 24 | 0 | 25 | |||
| 24 | 0 | 100 | |||
| 24 | 1 | 25 | |||
| 24 | 1 | 100 | |||
| 100 | 0 | 25 | |||
| 100 | 0 | 100 |
| Limit (s) | All- Better | All- Worse | Worse / Better |
|---|---|---|---|
| 25 | 1.618 [1.514, 1.728] | 0.796 [0.735, 0.861] | 0.492 [0.441, 0.551] |
| 100 | 0.571 [0.526, 0.621] | 0.425 [0.389, 0.463] | 0.744 [0.652, 0.852] |
| Break-even pay ratio | |||
|---|---|---|---|
| Limit (s) | Replace | Add | |
| 25 | 0.33 [0.29, 0.38] | 0.24 [0.18, 0.32] | |
| 40 | 0.40 [0.35, 0.45] | 0.29 [0.23, 0.37] | |
| 100 | 0.47 [0.41, 0.53] | 0.35 [0.28, 0.44] | |
| 100 | 0.31 [0.28, 0.34] | 0.23 [0.19, 0.29] | |
Learning from Demonstration
Learning from Demonstration teaches agents and robots new behaviors by extracting actions, goals, and task structure from examples.
See all 154 papers