Recent progress in robotic learning has been fueled by large-scale datasets collected in everyday environments. However, most existing datasets emphasize short-horizon, low-contact tasks such as pick-and-place, and therefore do not capture the precision control, force/torque or tactile regulation, and multimodal feedback required for industrial assembly. To address this gap, we introduce PRISM, a large-scale multimodal dataset for contact-rich industrial operations. The dataset spans more than 25 manipulation tasks (e.g., electronic components plug/unplug, conveyor-based sorting) and covers diverse mechanical constraints. PRISM includes more than 5,000 trajectories totaling 45 hours of teleoperated demonstrations, recorded using synchronized multi-view RGB-D, force/torque, tactile, and robot-state measurements. In contrast to datasets collected in household or laboratory settings, PRISM provides a realistic benchmark for multimodal perception and control under high-precision industrial constraints, and serves as a foundation for contact-rich, generalizable manipulation in real-world manufacturing environments. The dataset is open-sourced at: https://tengbo-yu.github.io/PRISM/
Figures & tables
Fig. 1 : Overview of PRISM dataset. We collect data using multiple robot platforms in diverse environments. Each manipulation episode includes synchronized multimodal observations, including visual, force, and proprioceptive signals, with tactile signals available for a subset of episodes.. For each episode, the full manipulation process is captured with well-calibrated multi-view cameras. The dataset spans a wide range of manipulation skills, and each episode is paired with a corresponding language description. In total, we provide more than 5,000 robot trajectories and 5,000 paired human demonstrations, covering over 45 hours of interaction across more than 25 tasks.
TABLE I : Datasets comparison. We compare several commonly used datasets based on the number of demonstrations, the sensors used during data collection, the robotic platform, the data collection method, and whether the dataset contains contact-rich tasks.
Conf.
Robot
Gripper
Teleoperation
Cfg 1
Franka
3D Printed
Tracker
Cfg 2
Franka
Visuotactile Gripper
Tracker
Cfg 3
Franka
Visuotactile Dexterous Hand
Tracker
Cfg 4
Franka
3D Printed
Exoskeleton
Cfg 5
Realman
3D Printed
Exoskeleton
Cfg 6
Realman
3D Printed
VR
TABLE II : Hardware specification of different configurations.
Modal
Size
Frequency
RGB image
540×960×3
15 Hz
Depth image
540×960
15 Hz
Visuotactile image
256×256
30 Hz
Robot joint angle
6 / 7
15 Hz
Robot joint torque
6 / 7
15 Hz
Gripper Cartesian pose
6 / 7
15 Hz
TABLE III : Data information of different modals.
Fig. 2 : Illustration of our data collection platform. Including three different teleoperation platforms with multi-view RGB-D cameras.
Fig. 3 : Statistics on the execution time of different robotic manipulations in our dataset.
Fig. 4 : Real-World Experiments with two Realman manipulators.
Electronic plug/unplug
Calipers packaging
Conveyor sorting
Method
100
200
100
200
100
200
ACT
5
10
30
45
15
25
DP
0
5
25
50
10
25
π0
10
20
65
80
50
75
TABLE IV: Average success rates (%) of three different policies trained with 100 or 200 demonstrations per task and evaluated over 20 episodes.
Method
Pretrain
Electronic plug/unplug
Calipers packaging
Conveyor sorting
ACT
✗
10
45
25
✓
10
55
30
DP
✗
5
50
25
✓
10
55
20
π0
✗
20
80
75
✓
25
85
85
TABLE V : Average success rates (%) of three different policies w/ and w/o pretraining, trained using 200 demonstrations per task and evaluated over 20 episodes.
Method
Collection
Electronic plug/unplug
Calipers packaging
Conveyor sorting
ACT
VR
5
20
10
Exoskeleton
10
45
25
DP
VR
0
15
20
Exoskeleton
5
50
25
π0
VR
5
35
40
Exoskeleton
20
80
75
TABLE VI : Average success rates (%) of three different policies trained on 200 demonstrations per task collected via VR vs. Exoskeleton teleoperation, evaluated over 20 episodes.