LABIT: Long-Horizon Robotic Assembly Benchmark for Industrial Tasks
Authors: Constantin Schempp, Christian Friedrich, Luca Kofler, Laura Suhm
Organizations: Institute for Robotics and Intelligent Production Systems, University of Applied Sciences Karlsruhe (HKA), Moltkestraße 30, Karlsruhe, 76133, Germany
Robot learning has undergone significant advancements in the domain of compliant control and robotic assembly. The development of novel methodologies has transcended the capacity to address discrete tasks, such as grasping or inserting, and has evolved to encompass the comprehensive construction of entire assemblies. Conventional benchmarks, which are predominantly focused on single tasks, are no longer adequate to address the complexity and interconnected nature of modern manufacturing processes. Consequently, the necessity for benchmarks to encompass a broader spectrum of assembly and subassembly steps is becoming apparent. In light of these developments, we propose LABIT, an industrial robotic assembly benchmark for long-horizon manipulation. This benchmark encompasses DIN-norm assembly tasks, including screwing, placing, inserting, and snapping, under both high and low tolerance conditions. The successive nature of these assembly steps implies that prior errors can influence subsequent manipulation. Consequently, this benchmark is particularly relevant for assessing the robustness of intelligent robotic systems. The proposed benchmarking protocols and novel performance metric, based on a taxonomy of assembly complexity in conjunction with the comprehensive list of parts and their corresponding CAD models, serve as essential tools for research and development in the field. To provide a baseline, we evaluate the benchmark using traditional force-position control and report reliability and score metrics per primitive, subtask, assembly category, and subassembly. Accompanying data is available at https://irp-hka.github.io/robotic-assembly-benchmark/.
Figures & tables
Figure 1: Experimental setup for LABIT. A robot arm equipped with a parallel-jaw gripper, a wrist-mounted RGB-D camera, and a force-torque sensor executes the assembly tasks on a dedicated worktable. The benchmark housing is secured in a fixture; individual parts are provisioned via a part feeder, corresponding to planning complexity level 1.
Figure 2: Hierarchical structure of the LABIT benchmark. (a) shows explosion views (top row) and corresponding primitive-level assembly graphs (bottom row) for three exemplary assemblies: cover plate (left), PCB module (center), and middle housing (right). Nodes represent individual assembly parts, where node colors match with part colors in the explosion views; edges display the spatial relationship and type of connections. The SCREW_M5x16 nodes of the housing middle assembly are not depicted here, because they are part of a later assembly stage. (b) shows the top-level graph connecting all assembly parts from LABIT, where the highlighted areas correspond to the assemblies shown in (a).
Benchmark
Ops.
Seq.
Ind.
Force
Reor.
Sim.
Phys.
LABIT (ours)
7
●
◐
●
●
●
●
FurnitureBench [ 16 ]
4
●
○
○
◐
●
●
FMB [ 17 ]
3
◐
○
◐
◐
●
●
RAMP [ 18 ]
3
●
◐
○
○
●
●
REASSEMBLE [ 19 ]
4
○
●
●
○
○
●
AutoMate [ 20 ]
2
○
○
○
○
●
●
Table 1: Comparison of robotic assembly benchmarks. Ops. : number of distinct operation types. Seq. : long-horizon sequential assembly with hierarchical step dependencies (failure in step i directly affects step i+1 ). Ind. : industrial off-the-shelf parts used. Force : force-sensitive/compliant control required. Reor. : explicit subassembly reorientation required. Sim. : simulation environment provided. Phys. : physical hardware provided. ● yes ◐ partial ○ no.
Figure 3: Different initial benchmark setup depending on the chosen complexity level. (a) level 1, parts are provided by a part feeder, (b) level 2, parts are provided unoriented on a plate and (c) level 3, parts are provided cluttered in a box.
Figure 4: Overview of assembly tasks required to successfully assemble the benchmark. Tasks occurring multiple times like peg insertion are only depicted once. Blue colored objects show the interacting components during task execution.
Figure 5: Taxonomy for the categorical assessment of the complexity of robot-guided assembly tasks. The complexity of each assembly task can be evaluated using the three properties tolerance T , geometry Ω and material combination M , where pi encodes the complexity of each group. In the latter, the colors stand for the stiffness of the material, namely for soft, for medium and for hard.
Figure 6: Examples of point allocation for different geometric complexities ord(V) . The blue line indicates the orientation of the object.
Eeff in GPa
pM
Regime
Typical pairings
<0.1
3
Large deformation
Elastomer involved (o-ring, tube)
0.1 – 1
2
Noticeable deformation
Soft thermoplastics (PE, PTFE)
1 – 10
1
Rigid, force-tolerant
Engineering polymers, polymer–metal
10 – 100
2
Increased force sensitivity
Light metals (Al–Al, Al–steel)
≥100
3
High force sensitivity
Steel–steel, ceramics
Table 2: Point allocation for the material combination based on the effective Young’s modulus Eeff according to ( 8 ).
Figure 7: Young’s modulus over density for families of engineering materials (after [ 37 ] ) with the bin edges of the material combination. Solid lines mark the primary edges placed in the gaps between the elastomer, polymer and metal families; dashed lines mark the secondary edges obtained by decade subdivision. The resulting points pM are given on the right axis.
Symbol
Meaning
pik
complexity points of property k of primitive i
Pi
total points of primitive i
Ri
reliability of primitive i
Rtask
reliability of subtask ( =∏i∈taskRi )
Rsa
reliability of subassembly
Rc
reliability of primitive category
Table 3: Overview of the metrics used in this work. The table defines symbols for complexity points ( pik ), reliability measures ( Ri , Rsa , Rc , R ), scores ( Si , Ssa , Sc , S ), timing metrics ( Tsa , T ), and force interaction metrics ( Exy , Ez , σxy , σz ).
Figure 8: Benchmark setup in real environment (left) and simulation (right), showing the robot, gripper, wrist camera and force-torque sensor. The parts are provided by the part feeder, corresponding to planning complexity level 1.
Figure 9: Convex decomposition of assembly parts for accurate contact calculation in simulation. Areas where contact is expected during assembly are modeled with higher detail by using smaller sphere radii (right).
Figure 10: Visualization of executed primitive sequences as seen by the wrist-mounted camera during baseline execution. Each row shows consecutive frames spanning the approach, contact, and completion phase of a single primitive. Top row: PCB insertion, where the board is grasped at its connector socket, aligned above the spring-loaded pins, and pushed into its seat. Center row: o-ring placement, where the ring is picked from the provisioning board, transported to the housing notch, and seated around the rotor shaft. Bottom row: screwing, where the screw is grasped, aligned with the through hole, and driven in by rotating the end effector about the assembly axis.
Parameter
Value
Timestep
0.5 ms (2 kHz)
Integrator
implicitfast
Solver
Newton
Solver tolerance
10−10
Gravity
[0,0,−9.81] m/s 2
Friction cone
elliptic
Table 4: MuJoCo simulation parameters used in LABIT.
Figure 11: Results of the baseline implementation using the task-level metrics from Section 3.3.2 . Top: Scores achieved for each primitive, assembly category and subassembly. Bottom: Reliability for each primitive, assembly category and subassembly.
Figure 12: Force interaction metrics for each subtask executed by the baseline method in the real environment. The first and second rows show the force energy in z -direction and xy -direction and the third and fourth rows show the force smoothness in z -direction and xy -direction respectively. The letter in parentheses after the component name refers to the visual representation of the subtask in Fig. 4 .
Figure 13: Example failure cases during execution of the baseline. (a) inserting a positioning pin fails due to jamming, (b) misalignment of the middle housing during placement, (c) insertion of the internal plug fails due to misalignment, (d) placement of the top housing fails due to slipping of the housing in the gripper, (e) screwing fails due to misalignment of the screw with the through hole and (f) reorienting and placing the module fails due to slipping of the housing in the gripper.
Figure 14: Comparison of the forces incurred during task execution in the real environment (top row) and the simulated environment (middle row). The bottom row shows the corresponding force interaction metrics. (a) insertion of gear wheel 1; (b) alignment and insertion of the top housing onto the middle housing; (c) insertion of the outer plug into the socket on the middle housing; and (d) screwing together of the top and middle housings.
Chair of Machine Learning and Reasoning (i6), RWTH Aachen University, Aachen, Germany · MASCOR Institute, FH Aachen University of Applied Sciences, Aachen, Germany
Nanyang Technological University, Singapore · Beijing University of Posts and Telecommunications, Beijing, China · Carnegie Mellon University, Pittsburgh, USA +1