BrickSim: A Physics-Based Simulator for Manipulating Interlocking Brick Assemblies

Haowei Wen; Ruixuan Liu; Weiyi Piao; Siyu Li; Changliu Liu

cs.ROMar 17, 2026

BrickSim: A Physics-Based Simulator for Manipulating Interlocking Brick Assemblies

Authors: Haowei Wen, Ruixuan Liu, Weiyi Piao, Siyu Li, Changliu Liu

Organizations: Robotics Institute, Carnegie Mellon University

Abstract

Interlocking brick assemblies provide a standardized yet challenging testbed for contact-rich and long-horizon robotic manipulation, but existing rigid-body simulators do not faithfully capture snap-fit mechanics. We present BrickSim, the first real-time physics-based simulator for interlocking brick assemblies. BrickSim introduces a compact force-based mechanics model for snap-fit connections and solves the resulting internal force distribution using a structured convex quadratic program. Combined with a hybrid architecture that delegates rigid-body dynamics to the underlying physics engine while handling snap-fit mechanics separately, BrickSim enables real-time, high-fidelity simulation of assembly, disassembly, and structural collapse. On 150 real-world assemblies, BrickSim achieves 100% accuracy in static stability prediction with an average solve time of 5 ms. In dynamic drop tests, it also faithfully reproduces real-world structural collapse, precisely mirroring both the occurrence of breakage and the specific breakage locations. Built on Isaac Sim, BrickSim further supports seamless integration with a wide variety of robots and existing pipelines. We demonstrate robotic construction of brick assemblies using BrickSim, highlighting its potential as a foundation for research in dexterous, long-horizon robotic manipulation. BrickSim is open-source, and the code is available at https://github.com/intelligent-control-lab/BrickSim.

Figures & tables

Fig. 2: Comparison of the temporal behavior of a physically stable brick assembly in MuJoCo [ 27 ] and BrickSim . The real assembly remains intact. Starting from the same initial state, the assembly simulated in MuJoCo progressively collapses, while BrickSim preserves the structure, consistent with real-world behavior.

	Rigid-body	FSA	BrickFEM [ 22 ]	BrickSim (Ours)
Snap-fit Mechanics	✗	✓	✓	✓
Temporal Dynamics	✓	✗	✓	✓
Efficient Runtime	✓	✓	✗	✓
Robot Integration	✓	✗	✗	✓

TABLE I: Comparison of representative methods for brick assembly analysis and simulation. FSA: Force-based Stability Analysis [ 17 , 14 ] . Rigid-body simulators include Gazebo [ 10 ] , MuJoCo [ 27 ] , and Isaac Sim [ 20 ] .

Fig. 3: BrickSim Architecture. BrickSim is built on top of Isaac Sim and augments its simulation stack (blue). The Brick Topology Graph (yellow) maintains a snap-fit connectivity graph between bricks and synchronizes it to the physics engine. The Assembly Monitor (green) detects valid stud-hole engagements and adds new connections, while the Breakage Detector (red) evaluates existing connections under external loads and removes those that fail. BrickSim provides a configurable simulation environment that supports diverse robotic platforms and a rich set of brick assets, including individual bricks and predefined assemblies. Bricks can be manipulated directly from the UI or by a robot controlled via teleoperation or custom policies.

Fig. 4: Brick and Connection Modeling. (a) Brick geometries in the real world and the approximated collision models in BrickSim . (b) Depending on the dimensions of the upper brick, each stud has 3 or 4 contact points with the corresponding hole, represented as red spheres. (c) Bricks are assembled by inserting the studs of the lower brick (green) into the holes of the upper brick (blue). At each contact point $f$ , the contact force on the lower brick is decomposed into three components: the effective radial force $F_{r,\mathrm{eff}}^{(f)}=F_{r}^{(f)}+F_{0}$ , where $F_{r}^{(f)}$ is the variable radial force and $F_{0}$ is the preload force due to snap-fit, together with the axial component $F_{a}^{(f)}$ and the tangential component $F_{t}^{(f)}$ of the static friction force which resist separation and twisting.

Fig. 6: Comparison of Predicted Force Distributions. Stress values range from 0 to 1, where lower values indicate smaller internal stress and higher values indicate greater stress, suggesting a higher risk of structural failure. Red: predicted collapsing bricks.

Fig. 9: Robotic Brick Manipulation using BrickSim . Left: a Franka robot is building an assembly by sequentially stacking bricks onto a baseplate on the table. Right: two Franka robots are cooperating to assemble two bricks in hand. One robot is holding the red brick, while the other robot is inserting the blue brick into it.

Physics-Based Simulation

Physics-based simulation models physical systems with equations and numerical solvers, increasingly augmented by learned models for speed and control.

See all 228 papers

BrickCraft: Visuomotor Skill Composition with Situated Manual Guidance for Long-Horizon Interlocking Brick Assembly
May 8, 2026Jichuan Yu, Bowei Li, Zhenran Tang +4Robotic Construction Long-Horizon Robotic Manipulation
Tsinghua University · Carnegie Mellon University
RoboSimStudio: An Interactive Toolkit for Human and Agent Rapid Real2Sim of Robot Manipulation Tasks
Oct 13, 2026Kaiyuan Chen, Shuangyu Xie, Uksang Yoo +9Real-to-Sim Transfer Robotics Simulation
University of California, Berkeley · NVIDIA
BrickCraft-Duo: Efficient Dual-Arm Skill Learning and Refinement for Compositional Long-Horizon Assembly
Sep 23, 2026Jichuan Yu, Zhenyu Xiao, Ze Wang +3Robotic Assembly Long-Horizon Robotic Manipulation
Tsinghua University · Carnegie Mellon University

May 8, 2026J/K move · Enter open · S save

BrickCraft: Visuomotor Skill Composition with Situated Manual Guidance for Long-Horizon Interlocking Brick Assembly

Jichuan Yu, Bowei Li, Zhenran Tang, Guanxing Lu+3

Tsinghua University · Carnegie Mellon University

Robotic Construction Long-Horizon Robotic Manipulation Robot Skill Learning Generalization in Robotic Manipulation Visuomotor Policy Learning Robotic Manipulation+2

Open paper AskPDF arXiv

Autonomous robotic assembly of interlocking bricks demands seamless integration of long-horizon task reasoning, spatial grounding, and fine-grained manipulation. This paper presents BrickCraft, a compositional framework designed for long-horizon and generalizable interlocking brick assembly. BrickCraft models the assembly process using a relative formulation, where each step is anchored to a reference brick within the partial structure, thereby decomposing complex tasks into a finite set of reusable primitive skills. BrickCraft bridges the gap between high-level assembly plans and physical execution through situated manuals, which provide explicit spatial guidance for learned visuomotor skills by projecting the assembly intent onto real-time robot observations. Finally, BrickCraft employs a modular execution pipeline that chains these spatially grounded skills to accomplish long-horizon assembly tasks. Extensive experimental validations demonstrate that BrickCraft acquires proficient assembly skills from a limited set of demonstrations and effectively composes them for diverse long-horizon assembly tasks. The project website is available at https://intelligent-control-lab.github.io/BrickCraft.

Abstract

Physics-Based Simulation

Explore similar work

BrickCraft: Visuomotor Skill Composition with Situated Manual Guidance for Long-Horizon Interlocking Brick Assembly

RoboSimStudio: An Interactive Toolkit for Human and Agent Rapid Real2Sim of Robot Manipulation Tasks

BrickCraft-Duo: Efficient Dual-Arm Skill Learning and Refinement for Compositional Long-Horizon Assembly