Organizations: University of Chinese Academy of Sciences · Beijing Zhongguancun Academy · Institute of Automation, Chinese Academy of Sciences · University of Science and Technology of China · Nankai University · Harbin Institute of Technology · Eastern Institute of Technology, Ningbo
Most simulation benchmarks for robotic manipulation keep target objects static and treat policy inference as instantaneous. These assumptions simplify evaluation but hide how a moving target evolves during computation and how stale an observation is when an action is executed. We introduce LIBERO-Dynamic, a dynamic extension of all 40 LIBERO tasks. It drives objects through contact friction along three trajectory families and speed tiers, while advancing the physical world according to calibrated, model-specific inference latency. The same mechanism supports training by replaying verified static demonstrations and compensating robot actions online for target displacement, producing 9,000 quality-filtered dynamic trajectories. We evaluate six pretrained policies on 1,800 dynamic conditions per configuration, compare execution strategies and action horizons with fixed checkpoints, and fine-tune three architectures using dynamic and mixed data. Faster motion degrades every policy but not uniformly; execution timing and horizon can change the behavior of the same policy; and asynchronous replanning benefits from frequent feedback only when the action queue avoids underflow. Dynamic demonstrations improve each architecture under at least one recipe, with gains varying by temporal conditioning, data composition, and training budget. A small-scale physical study shows task-dependent execution-strategy differences consistent with simulation. LIBERO-Dynamic makes object motion, inference latency, and action execution explicit and controllable for evaluating and improving robot policies in moving-target manipulation.
Figures & tables
Figure 1: Overview of LIBERO -Dynamic: (a) dynamic real-world manipulation and conventional simulation evaluation; (b) latency-aware evaluation; and (c) comparisons across execution and training settings.
Benchmark
Sim. tasks
Controlled dynamics
Latency- aware
Execution comparison
Dynamic demos
Data mixture
LIBERO-Plus
40
✗
✗
✗
✗
✗
LIBERO-X
600
✗
✗
✗
✗
✗
LIBERO-Safety
75
✓
✗
✗
✓
✗
DOMINO
35
✓
✗
✗
✓
✓
DynamicVLA/DOM
–
✓
✓
✓
✓
✗
ReflexBench
6
✓
✓
✓
✓
✗
Table 1: Comparison of simulation benchmarks. Task counts exclude perturbation variants; “–” indicates that DOM v1 reports nine evaluation dimensions rather than a comparable semantic-task count. Controlled dynamics denotes prescribed within-rollout motion; data mixture compares dynamic-only and mixed static–dynamic training. Checks indicate reported features or analyses; crosses indicate not reported.
Figure 2: Benchmark dimensions, dynamic-data construction, execution protocols, and representative results. Blue denotes inference; alternating light and dark green denote actions from consecutive predictions.
Figure 3: Normalized end-effector position density in the robot base frame for original LIBERO demonstrations (left) and generated LIBERO -Dynamic demonstrations (right).
τ
Static baseline
Dynamic SR (%)
Model
(ms)
D
speed =0
Ret. (%)
0.03
0.06
0.09
All
ET
EP
EI
π0.5 (JAX)
60
2
96.9
59.6
64.8
57.5
51.2
57.8
.715
.727
3.14
FastWAM-Joint
64
2
98.5
41.2
49.3
39.5
33.0
40.6
.593
.606
3.70
VLA-JEPA
56
2
97.2
36.7
41.0
37.5
28.7
35.7
.566
.577
1.59
FastWAM
63
2
97.6
32.9
34.2
32.5
29.5
32.1
.503
.533
2.68
π0 (JAX)
57
2
94.2
33.4
36.2
32.7
25.7
31.5
.507
.530
2.65
Table 2: Six pretrained policies under asynchronous execution at their best measured K∗ on the fixed 1,800 episodes. Ret. is dynamic All SR divided by the speed =0 result.
Figure 4: π0.5 (PyTorch) SR versus action horizon under (a) Sync, (b) Async, (c) RTC, and (d) VLASH. Panel (a) includes a zero-latency diagnostic; panel (b) overlays steady underflow (right axis).
Figure 5: Dynamic-data fine-tuning under asynchronous execution. (a–c) Measured K sweeps for three models and steady underflow (right axis). (d) Best-measured SR for each model and training recipe.
Figure 6: Real-robot task sequences. Each panel shows three keyframes from one successful rollout; complete sequences are in the appendix.
Figure 7: Real-robot results at K∈{4,8} . The manipulation tasks report successful trials out of 50; whack-a-mole reports successful presses over 50×30 s sessions.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Check
Rejection condition
Task and rollout
non-successful BDDL goal, runtime error, assisted step, or forbidden state edit
Motion evidence
platform/fixture motion below the thresholds above
Carrier path
maximum platform path error >0.01 m
Target following
pre-contact object–carrier relative error >0.08 m
Object stability
relative tilt >35∘ or downward displacement >0.04 m
Visibility
target outside the diagnostic bird-view projection for more than 25% of sampled pre-contact steps
Appendix
Table 3: Strict physical checks used for generated demonstrations. Crossing any rejection threshold prevents the episode from entering the released dataset.
Model
K∗
linear ping-pong
Catmull-Rom spline
multi-waypoint zigzag
All
π0.5 (JAX)
4
62.7
51.2
59.7
57.8
FastWAM-Joint
8
45.2
36.3
40.3
40.6
VLA-JEPA
4
39.3
33.3
34.5
35.7
FastWAM
8
34.8
29.8
31.5
32.1
π0 (JAX)
8
37.5
27.5
29.5
31.5
OpenVLA-OFT
4
30.8
25.0
26.0
27.3
Appendix
Table 4: Asynchronous success rate by motion family at each model’s overall best measured eligible horizon K∗ . Each family contains 600 episodes; All contains 1,800.
Figure 9: Cross-model action-horizon analysis on LIBERO -Dynamic: (a) synchronous and (b) asynchronous K sweeps; (c) SR by object speed at each model’s best measured asynchronous K .
Figure 10: Extended π0.5 (PyTorch) strategy sweeps, including two RTC settings and asynchronous underflow. Markers are measured results; smooth SR curves are visual guides.
τ
SR (%)
Strategy
(ms)
D
0.03
0.06
0.09
All
ET
EP
EI
VLASH
104
3
65.7
54.7
46.0
55.4
.671
.689
5.61
Sync Hold (zero-lat.)
0
0
57.7
49.8
43.7
50.4
.658
.664
10.11
Async Queue
147
3
55.7
45.2
44.3
48.4
.634
.648
3.56
Sync Hold
147
3
48.7
42.5
38.0
43.1
.593
.618
9.85
RTC (fixed = 10)
170
4
44.7
38.3
30.5
37.8
.556
.565
5.02
Appendix
Table 5: Execution-strategy results on LIBERO -Dynamic at the best measured eligible horizon for each setting (1,800 episodes per row). Sync, Async, and RTC use π0.5 (PyTorch); VLASH uses its official checkpoint. Zero-latency Sync is an idealized timing diagnostic.
Strategy
K
Steps
Path (m)
Δa
AEEF (m/s 2 )
Async Queue
6
167.1
1.159
0.142
0.379
Sync Hold
10
183.5
1.170
0.133
0.605
RTC (fixed = 10)
16
174.3
1.236
0.114
0.364
Appendix
Table 6: Behavioral metrics on 343 matched evaluation conditions successful under all compared strategies. Δa measures action change, and AEEF is a control-sampled acceleration estimate.
Model
FT data
Steps
Batch
Peak LR
Warm-up
π0.5
Dynamic only
1,500
256
5×10−5
75
Dynamic + static
3,000
256
5×10−5
150
Joint
Dynamic only
3,000
128
1×10−4
150
Dynamic + static
3,000
128
1×10−4
150
History Joint
Dynamic only
3,000
128
1×10−4
150
Dynamic + static
3,000
128
1×10−4
150
Appendix
Table 7: Dynamic fine-tuning configurations. Batch is the effective global batch. The mixed set is the union of the dynamic and repaired static demonstrations described above.
τ
SR (%)
Model
Training data
(ms)
D
K∗
0.03
0.06
0.09
All
ET
EP
EI
π0.5 (JAX)
Static
60
2
4
64.8
57.5
51.2
57.8
.715
.727
3.14
Dynamic only
4
70.5
63.3
45.0
59.6
.726
.719
3.38
Dynamic + static
4
81.5
70.8
55.7
69.3
.788
.779
4.50
FastWAM-Joint
Static
64
2
8
49.3
39.5
33.0
40.6
.593
.606
3.70
Dynamic only
4
64.7
48.5
35.0
49.4
.651
.671
2.51
Appendix
Table 8: Results for three training recipes under asynchronous execution. Each row uses its best measured horizon K∗ and 1,800 episodes.
Model
Training
Sync (%)
Async (%)
Single-frame Joint
Static base
43.0
40.6
Dynamic-only FT
52.0
45.5
History Joint
Static base
43.5
43.8
Dynamic-only FT
62.4
67.1
π0.5
Official (static)
59.3
55.9
Mixed FT
65.4
63.8
Appendix
Table 9: Fixed- K comparison before and after dynamic-data fine-tuning on LIBERO -Dynamic ( K=8 ; 1,800 episodes per configuration). Sync is zero-latency, and Async uses calibrated latency.
Model
Exec.
K
Conv. /50
Smooth /50
Zigzag /50
Whack presses
π0.5
Async
4
30
16
18
391
8
23
12
16
374
RTC
4
28
13
15
408
8
20
10
14
393
FastWAM-Joint
Async
4
19
8
12
369
8
16
7
10
348
Appendix
Table 10: Real-robot results for six model–execution configurations at K∈{4,8} . Conveyor and the two car-pursuit conditions report successful trials out of 50; whack-a-mole reports total successful presses across 50×30 s sessions.
Model
Execution
Input
H
Denoising
Evaluated K
π0.5
Async
Current
16
10
4, 8
RTC
Current + RTC
16
10
4, 8
Joint
Async
Current
32
1
4, 8
RTC
Current + RTC
32
10
4, 8
History Joint
Async
5 frames + 16 actions
16 †
1
4, 8
RTC
History + RTC
16 †
10
4, 8
Appendix
Table 11: Deployment configuration for the real-robot results in Table 10 . All six settings run at 15 Hz and are evaluated at K∈{4,8} . “History” denotes the five-frame input and 16-action clean prefix.
Figure 11: Conveyor handover: seven keyframes from one successful episode.
Figure 12: Car pursuit with a smooth-polynomial trajectory: seven keyframes from one successful episode.
Figure 13: Car pursuit with a zigzag trajectory: seven keyframes from one successful episode.
Figure 14: Whack-a-mole: seven keyframes from one successful episode.
Dynamic manipulation is a critical capability for robots operating in complex and dynamic environments, where robots must interact with objects that are moving or require rapid adjustments. However, learning models for dynamic manipulation tasks face two major challenges: (1) the combinatorial complexity of dynamic scenarios leads to substantial data requirements, and (2) rapid variations in dynamics require real-time and accurate policy execution. In this paper, we propose DynamicManip to address these challenges through an efficient data augmentation pipeline and a low-latency imitation policy. We first propose a static-to-dynamic augmentation pipeline that synthesizes diverse dynamic manipulation demonstrations from a single static demonstration. Second, we introduce a dynamic-aware adaptive policy that adjusts its inference frequency according to task dynamics, enabling responsive and effective dynamic manipulation. Third, we build a dynamic manipulation benchmark, which includes diverse dynamic tasks with an automatic evaluation system for scalable and consistent assessment. Extensive experiments in both simulation and the real world demonstrate that DynamicManip not only provides significant improvements in data efficiency but also achieves better performance in dynamic manipulation tasks, with a mean success rate 18.4 percentage points higher and policy-query latency 32.9% lower.
Haoran Liao, Pengyue Wang, Shuoyu Chen +10
1Sun Yat-sen University · 4Jiangxing Intelligence · 2Stanford University +1
Industrial manipulation in flexible manufacturing, e.g., high-mix, low-volume (HMLV) electronics assembly, requires robot policies to adapt to changing products and production conditions. Such settings impose industrial process constraints, including product variants, moving material flows, finite execution windows, and tight geometric tolerances. However, existing evaluation frameworks do not systematically capture these constraints when assessing generalist robot policies. To address this gap, we introduce LIBERO-Industry, a novel benchmark built on the widely used robot learning benchmark LIBERO, that translates the aforementioned industrial constraints into controllable task conditions and measurable success criteria. Specifically, LIBERO-Industry comprises three classic task scenarios: Static-to-Dynamic (S2D) loading, Dynamic-to-Static (D2S) unloading, and Dynamic-to-Dynamic (D2D) transfer. The suites contain 216 tasks in total that systematically vary object geometry, conveyor speed, interaction-window duration, and geometric acceptance region size. Extensive experiments across five generalist robot policies (including vision-language-action models and world-action models) show that performance progressively degrades as industrial constraints are introduced and combined. Under a shared fine-tuning and evaluation protocol, the LIBERO-Industry co-training condition achieved higher observed target-transfer completion than the original-LIBERO condition.
Ming Shin Cindy Sung, Duan Leijiahao, Haoxiang Wang +9
Institute of AI for Engineering, Tongji University · The Chinese University of Hong Kong · University of Aberdeen +1
General-purpose agents can plan, use tools, and revise their behavior from feedback, but it remains unclear whether these capabilities transfer from digital environments to embodied manipulation. To investigate this question, we introduce LIBERO-Agent, an agent-native benchmark for evaluating these agents in robot manipulation tasks. Rather than asking agents to submit task-level Python control programs or operate through high-level robot skills, LIBERO-Agent provides an interactive robotic environment where agents can select which observations to inspect, process them with their own tools, and issue native action commands. LIBERO-Agent integrates 200 tasks into a common interaction framework and provides a 30-task primary suite that separates perception, short-horizon execution, and long-horizon composition. Results reveal a pronounced reliability gap: while agents perform well on perception and easy short-horizon tasks, their performance degrades substantially on hard short-horizon and long-horizon tasks. Richer observations improve short-horizon manipulation, while demonstration benefits depend on the agent and format. Among these agents, GPT-6 Astra achieves the strongest overall performance. Further analysis shows its major advantage lies in mechanism interaction, especially when sustained physical contact is needed, while its remaining failures stem from cross-stage interference and geometric errors.
Zijie Diao, Yitong Chen, Sicheng Xie +7
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Innovation Institute