Video-Action Models for Data-Efficient Industrial Manipulation
Organizations: Mimic Robotics · BMW Group
Abstract
Industrial manipulation requires precise alignment and controlled handling of rigid, deformable, and flexible components. Yet, the benefits of pretrained visuomotor policies remain underexplored in settings where limited access to specialized hardware, components, and fixtures constrains demonstration collection and evaluation. Understanding task-specific data requirements is important for assessing their scalability across diverse processes on the factory floor. We investigate whether video pretraining can support reliable industrial manipulation with limited action data. We propose FLUX-mimic, a video-action model, motivated by the potential of video pretraining to capture motion and interaction patterns that enable sample-efficient training. FLUX-mimic is built on a backbone pretrained on large-scale web and industrial task data, and an action expert to generate robot trajectories. We train our policy by joint video-action training. Across four automotive manufacturing benchmark tasks and two bimanual robot platforms, FLUX-mimic outperforms all considered baselines on the dexterous robot platform and matches pi_{0.5}'s average success rate on the gripper-based robot platform. On a deformable part kitting task, FLUX-mimic reaches 77.5% success with as little as two hours of task-specific action-labeled demonstrations.
Figures & tables
| Method | Control unit assembly | Control unit fixture insertion | Soft part kitting | Wire harness routing | Avg. |
|---|---|---|---|---|---|
| Franka with dexterous hands | |||||
| FM | 50 | 35 | 70 | 55.6 | 52.6 |
| 0 | 30 | 75 | 5.6 | 27.6 | |
| FLUX-mimic (ours) | 85 | 85 | 100 | 61.1 | 82.8 |
| Trossen with grippers | |||||
| 60 | 60 | - | - | 60.0 | |
| Task | Language instruction |
|---|---|
| Control unit assembly | Place the metal piece on the jig and place the control unit component on top of it. |
| Control unit fixture insertion | Pick the black fixture, place it on the white jig, pick the metal control unit, and insert it onto the black fixture. |
| Soft part kitting | Pick a black strut cover from the purple bin and fill the middle narrow row from left to right, with the yellow part facing downward. |
| Wire harness routing | Clip the black cable into the three white clips on the black panel. |
Data-Efficient Robot Learning
Data-efficient robot learning develops ways for robots to acquire reliable skills with fewer demonstrations, interactions, and human annotations.
See all 209 papers