Generating a LEGO model from a user request is more than a plausible 3D shape generation. LEGO models require physical support per brick, and each brick should be placed unblocked. Current methods address this issue by training an expert large language model. We reduce the problem of generating extremely long context on all brick connections to that of writing a program that builds LEGO models like a human, considering each intermediate step could be executed. We introduce DailyBrick , a multi-stage agentic system that produces a LEGO model together with a brick-by-brick assembly instruction for human building and robot assembly. We design a multi-stage harness with roles to help LLMs generate more effectively. A versioned construction SDK compiles multi-level decisions, from semantic subparts to individual bricks, into canonical operations. An external Meta-Reviewer agent replays and inspects the generated package, verifies the model and its construction process, identifies failures, traces them to the responsible stage, and routes them to bounded repairs or to editable candidates for later skill or tool updates. Extensive experiments demonstrate that DailyBrick generates LEGO models based on simple user requests better than recent agentic systems, and show its utility in downstream applications such as robot manipulations on model assembly from instructions and brick sets (https://SpatialAgent.github.io/DailyBrick/).
Figures & tables
Figure 1: LEGO models and assembly instructions generated by DailyBrick. Given a user request, DailyBrick produces a LEGO model together with step-by-step assembly instructions. We recommend watching the video (in the anonymous link) for a better overview.
Figure 2: System overview. DailyBrick turns a user request into a model, as well as an assembly instruction. Independent prefix replay and an external Meta-Reviewer close the loop by locating failures and routing targeted updates to the responsible stage or SDK component.
Figure 3: Overview of DailyBrick. A multi-stage Planner–Designer–Critic workflow uses partial assembly skills and a construction SDK to generate a shared assembly trace, from which the LEGO model and step-by-step instructions are derived. Independent verify replay provides evidence for the Self Meta-Reviewer to locate errors and guide targeted refinements.
Figure 4: Generated Examples. Rows (a)–(c) show randomly outputs from BrickGPT ( Pun et al., 2025 ) , BrickNet ( Kulits and Schmid, 2026 ) , and DailyBrick, respectively.
Figure 5: Text-conditioned generation results. Outputs for 16 evaluation prompts are arranged in two blocks of eight columns. Within each block, BrickGPT ( Pun et al., 2025 ) , BrickNet ( Kulits and Schmid, 2026 ) , GPT 6-Astra ( OpenAI, 2026 ) and DailyBrick appear from top to bottom. Each column corresponds to the numbered prompt below.
Method
Tokens/case ↓
Time (s) ↓
Prefix pass ↑
Repair success ↑
Accept ↑
BrickGPT ( Pun et al., 2025 )
2.312k
65.62
0.706
–
0.684
BrickNet (14B) ( Kulits and Schmid, 2026 )
3.544k
56.68
0.690
–
0.743
GPT 6-Astra ( OpenAI, 2026 )
16.24k
324.6
0.884
0.525
0.714
DailyBrick (w/ Qwen3.6-27B)
7.760k
207.7
0.960
0.576
0.834
DailyBrick (w/ GPT 6-Astra)
4.346k
49.09
0.974
0.677
0.885
Table 3: Efficiency and Cost Analysis. Results are reported on the 18,653-case test split. Tokens and time are measured per brick among different model scales. A dash indicates that repair success is not applicable. The best value in each column is bold .
Figure 6: Step-by-Step Assembly in Simulation. Left: the generated model. Middle: eight snapshots of a simulated manipulator following the assembly instructions. Right: the manipulator holding the final brick before placement.
Figure 8
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Staged planning and local assembly in DailyBrick. The large house example illustrates how DailyBrick plans the main structure and its supports, then completes local components, including an interior sofa, an exterior balcony, and a decorative tree. Unlike approaches that fill a target shape with bricks while overlooking its interior, the staged construction represents both internal layout and external details.
Figure 10: Generation with a restricted brick inventory. Examples generated by DailyBrick (w/ Qwen3.6-27B) when specialized parts are excluded from the available brick inventory.
Figure 11: Step-by-Step Assembly Instructions from DailyBrick. Intermediate assembly states generated by DailyBrick (w/ Qwen3.6-27B), shown in construction order from the initial placements to the completed model.
Figure 12: Step-by-Step Assembly Instruction Visualization of LEGO Models . The television cabinet and white cabinet are each shown using five methods: BrickGPT, BrickNet, GPT 6-Astra, ours (with Qwen3.6-27B) and ours w/ GPT 6-Astra. Each row presents eight stages of one model: the first assembly step, six uniformly sampled intermediate steps, and the completed model.
Frontier coding agents are becoming increasingly adept at 3D modeling. Yet, they still face challenges when building objects meant to be assembled in the physical world, often creating invalid assemblies with floating, colliding, or disconnected parts. We argue that providing agents with tools for assembling builds and verifying their physical feasibility is sufficient for them to produce physically valid assemblies that still adhere to input prompts. Focusing on the domain of LEGO brick builds, we present BrickWright, a framework for generating physically feasible brick builds from text or image prompts using pre-trained agents, where a build is physically feasible if it uses only real parts, has no colliding or floating parts, and forms a connected structure if possible. BrickWright guides agents to create builds by writing code with BrickLayer, a Python toolkit with two main components: an ergonomic brick assembly API that halves inference cost compared to working in raw LDraw, and a validator that provides feedback on physical feasibility. Whereas baseline agents achieve physical validity rates ranging from only 9% (GPT-6 Luna) to 69% (GPT-6 Astra), BrickWright-guided agents achieve 100% validity on a sample of 512 prompts, while improving prompt adherence for lightweight agents and maintaining it for stronger agents. Additionally, they outperform specialized models on the text-to-brick task. Code is available on our project page: https://avalovelace1.github.io/brickwright/
Ava Pun, Kevin Dai, Sally Chen +7
Carnegie Mellon University · Independent Researcher · Stanford University
We train a language model to generate LEGO-brick build sequences. While prior work has been restricted to discrete, voxel-like towers, we consider a much broader set of pieces, encompassing thousands of part types with diverse connection semantics. To enable this, we first collect a large-scale dataset of over 100,000 human-designed LDraw brick objects and scenes. The complexity of our setting makes it challenging to autoregressively assemble structures that satisfy physical constraints. When predicting block pose directly, build sequences quickly become invalid after a small number of steps. Although pieces are placed in 3D space, it is the spatial relationships of the parts which define the whole. With this in mind, we design a graph-based program representation that parametrizes structure through connectivity, improving the physical grounding of generated sequences. To enable future applications, we make our dataset and models available for research purposes. https://kulits.github.io/BrickNet
Peter Kulits, Cordelia Schmid
Inria, `Ecole Normale Sup`erieure, CNRS, PSL Research University · Max Planck Institute for Intelligent Systems, Tübingen
We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints. We score validity, alignment, and design across three settings that vary in scale and part availability. We provide BrickAgent, an environment for coding agents to construct, inspect, and validate their designs. We find that leading agents largely satisfy verifiable physical and semantic requirements, but fall short of human designs. We release our benchmark and environment at http://www.brickben.ch
Peter Kulits, Yiqing Xu, R. Kenny Jones +2
Stanford University · Max Planck Institute for Intelligent Systems · Inria