BiDex-100: A Comprehensive Benchmark for Bimanual Dexterous Manipulation Across 100+ Simulation Tasks
Organizations: Zhejiang University · Ant Group
Abstract
Bimanual manipulation with dexterous hands has attracted growing attention in research on robotic action models. However, few studies have developed benchmarks specifically tailored for dexterous manipulation, leaving the dexterous manipulation capabilities of existing models insufficiently characterized. In this paper, we present BiDex-100, a simulation benchmark for bimanual dexterous manipulation that combines broad task coverage, multi-dimensional evaluation, and high-DoF hand embodiments. Concretely, the benchmark comprises 106 tasks organized along five capability axes: generalization, instruction following, limited demonstrations, long horizon, and memory. Each task is paired with an out-of-domain variant featuring controlled visual and spatial changes, allowing systematic assessment of robustness. In addition, the benchmark supports two embodiments and provides 8,080 success-validated teleoperated demonstrations, totaling 6.25 million frames. We evaluate seven recent Vision-Language-Action models under a unified protocol, establishing baselines for capability-specific performance, robustness, and data efficiency. The website is available at https://BiDex-100.github.io/BiDex-100/.
Figures & tables
| Benchmark | Biman. | DoF | Tasks | Progress | OOD | Demos |
| Meta-World [ 6 ] | × | – | 50 | ✓ | ✓ | × |
| RLBench [ 7 ] | × | – | 100 | × | × | synth. |
| LIBERO [ 8 ] | × | – | 130 | × | × | teleop. |
| CALVIN [ 9 ] | × | – | 34 | partial | ✓ | teleop |
| RoboCasa [ 10 ] | × | – | 100 | × | ✓ | teleop+synth. |
| ManiSkill2 [ 11 ] | ✓ | – | 20 | × | ✓ | synth. |
| Model | VLM backbone | Action expert | Chunk | Train |
| [ 4 ] | PaliGemma-2B | 300M flow-matching | 50 | 60k |
| GR00T-N1.7 [ 5 ] | Cosmos-R2-2B | DiT flow-matching | 16 | 400k |
| InternVLA-A1.5 [ 36 ] | Qwen3.5-2B | 1B flow-matching | 50 | 400k |
| Galaxea G0.5 [ 37 ] | Qwen3.5-2B | flow-matching | 32 | 400k |
| OpenDM-DM05 [ 38 ] | Gemma3-4B | rectified flow | 50 | 120k |
| LingBot-VLA-v2 [ 39 ] | Qwen3-VL-4B | MoE flow (L1) | 50 | 60k |
| Model | Generalization | Instruction-Foll. | Limited-Demo. | Long-Horizon | Memory | Overall |
| LingBot-VLA-v2 [ 39 ] | 61.35 / 75.80 | 35.90 / 66.57 | 66.15 / 76.14 | 38.20 / 66.78 | 26.00 / 43.20 | 52.21 / 70.22 |
| [ 4 ] | 55.45 / 72.70 | 36.60 / 64.48 | 66.23 / 75.63 | 34.00 / 64.51 | 33.00 / 50.16 | 50.40 / 68.97 |
| GR00T-N1.7 [ 5 ] | 46.55 / 65.75 | 24.60 / 59.28 | 61.38 / 73.97 | 32.80 / 62.32 | 20.80 / 36.10 | 42.32 / 63.43 |
| OpenDM-DM05 [ 38 ] | 33.05 / 53.18 | 1.80 / 23.53 | 35.69 / 49.11 | 3.20 / 17.45 | 12.60 / 26.13 | 23.06 / 40.66 |
| Galaxea G0.5 [ 37 ] | 24.90 / 44.51 | 5.70 / 36.33 | 39.62 / 53.20 | 5.20 / 25.10 | 10.60 / 28.22 | 21.68 / 41.73 |
| Xiaomi XR1 [ 40 ] | 6.40 / 24.48 | 0.90 / 21.06 | 7.15 / 21.71 | 0.00 / 15.63 | 0.60 / 7.00 | 4.40 / 20.67 |
| Model | Generalization | Instruction-Foll. | Limited-Demo. | Long-Horizon | Memory | Overall |
| LingBot-VLA-v2 [ 39 ] | 1.60 / 10.76 | 0.80 / 19.90 | 3.46 / 10.06 | 0.00 / 7.15 | 0.80 / 6.18 | 1.68 / 11.54 |
| [ 4 ] | 2.40 / 13.19 | 2.40 / 23.96 | 6.31 / 13.54 | 0.40 / 10.02 | 3.80 / 10.70 | 3.30 / 14.77 |
| GR00T-N1.7 [ 5 ] | 4.65 / 17.59 | 3.10 / 30.25 | 10.23 / 21.34 | 2.60 / 19.96 | 3.80 / 12.59 | 5.45 / 20.65 |
| OpenDM-DM05 [ 38 ] | 0.85 / 10.89 | 0.10 / 17.04 | 3.92 / 10.97 | 0.00 / 5.72 | 1.80 / 7.28 | 1.47 / 11.24 |
| Galaxea G0.5 [ 37 ] | 0.20 / 4.40 | 0.20 / 15.41 | 2.62 / 5.75 | 0.00 / 3.66 | 0.20 / 1.57 | 0.77 / 6.47 |
| Xiaomi XR1 [ 40 ] | 0.05 / 2.26 | 0.00 / 14.95 | 0.00 / 1.47 | 0.00 / 2.89 | 0.00 / 2.18 | 0.02 / 4.51 |
| Predicate | Semantics (tolerance unless noted) |
|---|---|
| Spatial placement | |
| on | object center within a target’s horizontal radius and above it in height |
| stacked_on | object one layer above another within horizontal and vertical windows |
| on_table | object directly supported from below by the scene table |
| inside | object center within a circular horizontal radius and a vertical window |
| inside_contact | upward support contact from a container plus a minimum height |
| Modality | Content | Dim |
| Observations | ||
| Arm end-effector pose (right, left) | xyz + quaternion, world or head-camera frame | 14 |
| Arm joint position / velocity | 7 per arm | 14 each |
| Hand joint position / velocity | 22 per hand | 44 each |
| Actions (one 58-dimensional control representation) | ||
| Arm component | joint targets or end-effector pose in world/head-camera frame | 14 |
| Model | Train | Optimizer | Peak LR | Warmup | Min LR | GBS | Dtype | Chunk | Frozen modules |
| 60k | AdamW (optax) | 5e-5 | 100 | 5e-5 | 256 | bf16 | 50 | none (EMA 0.999) | |
| GR00T-N1.7 | 400k | AdamW | 1e-4 | 1500 | 0 | 256 | bf16 | 16 | VLM + vision encoder |
| InternVLA-A1.5 | 400k | AdamW | 5e-5 | 2000 | 5e-6 | 256 | bf16 | 50 | none |
| Galaxea G0.5 | 400k | AdamW | 1e-5 | 1500 | 1e-6 | 256 | bf16 | 32 | none |
| OpenDM-DM05 | 120k | AdamW | 2.5e-5 | 1000 | 2.5e-6 | 256 | bf16 | 50 | VLM token embeddings |
| LingBot-VLA-v2 | 60k | Muon (+AdamW) | 1e-4 | 0 | 5e-5 | 256 | fp32 | 50 | none |
Bimanual Robotic Manipulation
Bimanual robotic manipulation coordinates two robot arms to handle objects, tools, and tasks that require collaboration.
See all 178 papers