Automated Assembly Instruction Generation from CAD Models Using Grounded Large Language Models: A Human-in-the-Loop Framework
Authors: Aaron Dsouza, Mohammed Azeez Khan, Ashutosh Mishra, Arshaan Khan, Neha K. Nair, Amar Kumar Behera
Organizations: Department of Electronics and Communication Engineering, National Institute of Technology Warangal, India · Department of Computer Science and Engineering, National Institute of Technology Warangal, India · Department of Computer Science and Engineering, Nawab Shah Alam Khan College of Engineering and Technology, Hyderabad, India · Department of Physics, National Institute of Technology Warangal, India · Department of Design, Indian Institute of Technology Kanpur, Kanpur, India
Assembly documentation is a downstream manufacturing artifact that is still usually authored by interpreting CAD models by hand. Structured product data and large language models are both available, yet studies of CAD interpretation, assembly sequence planning, instruction writing, and human oversight have largely proceeded separately. This paper formulates CAD-grounded assembly instruction generation: the production of natural-language assembly procedures constrained by structured engineering information extracted from CAD models. The proposed framework maps a STEP assembly to a typed ProductGraph intermediate representation, derives a precedence order by deterministic topological sorting, realizes each step as language conditioned only on selected graph context, attaches per-step visual documentation, and applies rule-based and model-assisted checks. PDF export remains disabled until a human reviewer resolves every quality flag. The case study establishes endto-end feasibility on a built-in six-part reference assembly: the pipeline preserves a reported assembly order and carries quantity, material, and torque into an exported manual page. Generalization and geometric validation remain open empirical questions. The contribution is an architecture that separates engineering state, deterministic reasoning, grounded language realization, verification, and human release.
Figures & tables
Study
CAD
Order
Text
Ground.
Visual
Gate
Pan et al. [ 18 ]
STEP
GA
–
–
–
–
Wang et al. [ 21 ]
CAD
Interactive
–
Semantic
–
–
Jiang et al. [ 7 ]
–
Case reuse
LLM
KG/RAG
–
–
Kelm et al. [ 10 ]
–
–
LLM
MTM text
Assist.
–
Jonek et al. [ 8 ]
Design
RAG eval.
LLM
RAG
Sim. eval.
–
Lin et al. [ 13 ]
Demo./3D
Demo.
LLM
Demo.
AR
–
Table 1: Placement of the proposed framework relative to related work. “Gate” means a reported control that blocks release until a person resolves review items. Dashes mark functions that are not the reported contribution of that study.
Figure 1: Overview of the CAD-grounded assembly instruction generation framework. STEP-derived engineering information is written into a typed ProductGraph, ordered by deterministic assembly reasoning, realized as grounded instructions, and released only after automated checks and human verification. Gold marks the ProductGraph. The remaining shades distinguish CAD input, ordering, language realization, and verification.
Field
Type
Role
parts
list of parts
Identity, material, quantity, bounding box
constraints
list of constraints
Precedence and optional torque
bom
list of entries
Aggregated bill of materials
assembly_steps
list of steps
Order, sentence, diagram, flags
metadata
dictionary
Job and pipeline metadata
Table 2: ProductGraph fields used as the contract between pipeline stages.
Figure 2: Grounding path for the fastener step of the reference assembly. Structured fields enter a step-specific ProductGraph slice. The language-realization rules permit those fields and exclude facts the slice does not contain. The sentence is the fastener instruction visible on the exported manual page, which preserves quantity and torque.
Figure 3: Implemented precedence for the six-part reference assembly. The left panel is the mapping from constraint categories to directed edges. The right panel is the unique topological order returned by Kahn’s algorithm; arrows follow that order, and the fastener record stores a torque of 2.5 Nm. The interpreter does not parse STEP AP214 kinematic mates, so the figure does not assign a kinematic type to each arrow.
Figure 4: Human-in-the-loop verification path. Deterministic rules and a model-assisted review attach flags to each generated instruction. A reviewer may edit, resolve, flag, or reorder. PDF export stays closed until the required flags are resolved.
Stage
Component
Instantiation
ΓS
CAD parser
CadQuery/OCCT, or the reference assembly
State
ProductGraph
Pydantic schema; JSON in SQLite
ΓO
Sequence stage
NetworkX; Algorithm 1
ΓV
Diagram stage
Blender/Cycles, else Pillow
ΓL
Writer stage
Llama 3.3 70B via Groq; SHA-256 cache
ΓQ
QA stage
Regex rules, then a JSON model review
Table 3: Mapping from framework stages to the proof-of-concept implementation.
Figure 5: Exported manual bands for the motor bracket, the DC motor, and the fastener of the reference assembly, taken from the stored PDF page and rendered with the schematic fallback. Each band shows the instruction. The motor-related bands also show the electrical warning, and the fastener band shows the 2.5 Nm torque callout.
Stage
Wall-clock time
CAD parsing (reference assembly)
<0.1 s
Sequence inference
<0.1 s
Diagram rendering (Pillow)
∼ 3–5 s
Instruction writing (6 calls)
∼ 8–15 s
QA review (6 calls)
∼ 8–15 s
Total
∼ 20–35 s
Table 4: Approximate pipeline latency for the six-part reference assembly on one laptop, using reference-assembly parsing, the Pillow schematic renderer, and remote model calls. Ranges reflect observed run-to-run variation, not a confidence interval.
Figure 6: Implemented framework and open research directions. Solid boxes are stages exercised by the prototype. Dashed boxes are proposed extensions, and the dashed link indicates where richer CAD interpretation would enter the ProductGraph.