Interlocking brick assemblies provide a standardized yet challenging testbed for contact-rich and long-horizon robotic manipulation, but existing rigid-body simulators do not faithfully capture snap-fit mechanics. We present BrickSim, the first real-time physics-based simulator for interlocking brick assemblies. BrickSim introduces a compact force-based mechanics model for snap-fit connections and solves the resulting internal force distribution using a structured convex quadratic program. Combined with a hybrid architecture that delegates rigid-body dynamics to the underlying physics engine while handling snap-fit mechanics separately, BrickSim enables real-time, high-fidelity simulation of assembly, disassembly, and structural collapse. On 150 real-world assemblies, BrickSim achieves 100% accuracy in static stability prediction with an average solve time of 5 ms. In dynamic drop tests, it also faithfully reproduces real-world structural collapse, precisely mirroring both the occurrence of breakage and the specific breakage locations. Built on Isaac Sim, BrickSim further supports seamless integration with a wide variety of robots and existing pipelines. We demonstrate robotic construction of brick assemblies using BrickSim, highlighting its potential as a foundation for research in dexterous, long-horizon robotic manipulation. BrickSim is open-source, and the code is available at https://github.com/intelligent-control-lab/BrickSim.
Figures & tables
Fig. 2: Comparison of the temporal behavior of a physically stable brick assembly in MuJoCo [ 27 ] and BrickSim . The real assembly remains intact. Starting from the same initial state, the assembly simulated in MuJoCo progressively collapses, while BrickSim preserves the structure, consistent with real-world behavior.
Rigid-body
FSA
BrickFEM [ 22 ]
BrickSim (Ours)
Snap-fit Mechanics
✗
✓
✓
✓
Temporal Dynamics
✓
✗
✓
✓
Efficient Runtime
✓
✓
✗
✓
Robot Integration
✓
✗
✗
✓
TABLE I: Comparison of representative methods for brick assembly analysis and simulation. FSA: Force-based Stability Analysis [ 17 , 14 ] . Rigid-body simulators include Gazebo [ 10 ] , MuJoCo [ 27 ] , and Isaac Sim [ 20 ] .
Fig. 3: BrickSim Architecture. BrickSim is built on top of Isaac Sim and augments its simulation stack (blue). The Brick Topology Graph (yellow) maintains a snap-fit connectivity graph between bricks and synchronizes it to the physics engine. The Assembly Monitor (green) detects valid stud-hole engagements and adds new connections, while the Breakage Detector (red) evaluates existing connections under external loads and removes those that fail. BrickSim provides a configurable simulation environment that supports diverse robotic platforms and a rich set of brick assets, including individual bricks and predefined assemblies. Bricks can be manipulated directly from the UI or by a robot controlled via teleoperation or custom policies.
Fig. 4: Brick and Connection Modeling. (a) Brick geometries in the real world and the approximated collision models in BrickSim . (b) Depending on the dimensions of the upper brick, each stud has 3 or 4 contact points with the corresponding hole, represented as red spheres. (c) Bricks are assembled by inserting the studs of the lower brick (green) into the holes of the upper brick (blue). At each contact point f , the contact force on the lower brick is decomposed into three components: the effective radial force Fr,eff(f)=Fr(f)+F0 , where Fr(f) is the variable radial force and F0 is the preload force due to snap-fit, together with the axial component Fa(f) and the tangential component Ft(f) of the static friction force which resist separation and twisting.
Figure 5
Fig. 6: Comparison of Predicted Force Distributions. Stress values range from 0 to 1, where lower values indicate smaller internal stress and higher values indicate greater stress, suggesting a higher risk of structural failure. Red: predicted collapsing bricks.
Figure 7
Fig. 9: Robotic Brick Manipulation using BrickSim . Left: a Franka robot is building an assembly by sequentially stacking bricks onto a baseplate on the table. Right: two Franka robots are cooperating to assemble two bricks in hand. One robot is holding the red brick, while the other robot is inserting the blue brick into it.
Autonomous robotic assembly of interlocking bricks demands seamless integration of long-horizon task reasoning, spatial grounding, and fine-grained manipulation. This paper presents BrickCraft, a compositional framework designed for long-horizon and generalizable interlocking brick assembly. BrickCraft models the assembly process using a relative formulation, where each step is anchored to a reference brick within the partial structure, thereby decomposing complex tasks into a finite set of reusable primitive skills. BrickCraft bridges the gap between high-level assembly plans and physical execution through situated manuals, which provide explicit spatial guidance for learned visuomotor skills by projecting the assembly intent onto real-time robot observations. Finally, BrickCraft employs a modular execution pipeline that chains these spatially grounded skills to accomplish long-horizon assembly tasks. Extensive experimental validations demonstrate that BrickCraft acquires proficient assembly skills from a limited set of demonstrations and effectively composes them for diverse long-horizon assembly tasks. The project website is available at https://intelligent-control-lab.github.io/BrickCraft.
It has been difficult to accurately model the physics of the interaction between a robot and the objects it manipulates, including contact mechanics, coupled contact dynamics, deformations, stiffness and friction. We present RoboSimStudio (RSS), an interactive toolkit that makes choices of solver, contact model, and physical parameters explicit and editable, allowing both humans and coding agents to quickly create and iteratively refine simulations based on observed physical behavior. We implement RSS with the open Newton physics simulation system and describe experiments with human-guided real2sim using (1) visual evaluation, (2) force-feedback and (3) agentic Real2Sim2Real. Code, video and additional information can be found on the project website: https://robosimstudio.github.io
Interlocking brick assembly provides a representative testbed for evaluating real-world robotic manipulation capabilities, where diverse structural designs, complex inter-step dependencies, intricate mechanical interactions and tight insertion tolerances pose substantial challenges. We present BrickCraft-Duo, a modular framework for long-horizon dual-arm collaborative assembly of interlocking bricks through data-efficient skill learning and composition. BrickCraft-Duo learns reusable single- and dual-arm assembly skills from diverse demonstrations, with bilateral symmetry alignment facilitating skill sharing across symmetric arms and assembly--support role assignments. Guided by stability-aware assembly reasoning, BrickCraft-Duo composes heterogeneous skills to achieve autonomous long-horizon execution, and further integrates human-in-the-loop correction for targeted skill refinement. The resulting system achieves long-horizon success rates of at least 60% and step-level completion rates of at least 95% across five real-world assembly tasks involving partially supported configurations, with horizons of up to nine steps. Project website: https://jichuan-yu.github.io/BrickCraft-Duo.