Organizations: Transcengram · Beihang University · The University of Hong Kong · Shenzhen Loop Area Institute · ShanghaiTech University · DeepSeek · University of California, Berkeley
Constructing computer-aided design (CAD) models is labor-intensive but essential for engineering and manufacturing. Recent advances in Large Language Models (LLMs) have inspired the LLM-based CAD generation by representing CAD as command sequences. But these methods struggle in practical scenarios because command sequence representation does not support entity selection (e.g. faces or edges), limiting its ability to support complex editing operations such as chamfer or fillet. Further, the discretization of a continuous variable during sketch and extrude operations may result in topological errors. To address these limitations, we present Pointer-CAD, a novel LLM-based CAD generation framework that leverages a pointer-based command sequence representation to explicitly incorporate the geometric information of B-rep models into sequential modeling. In particular, Pointer-CAD decomposes CAD model generation into steps, conditioning the generation of each subsequent step on both the textual description and the B-rep generated from previous steps. Whenever an operation requires the selection of a specific geometric entity, the LLM predicts a Pointer that selects the most feature-consistent candidate from the available set. Such a selection operation also reduces the quantization error in the command sequence-based representation. To support the training of Pointer-CAD, we develop a data annotation pipeline that produces expert-level natural language descriptions and apply it to build a dataset of approximately 575K CAD models. Extensive experimental results demonstrate that Pointer-CAD effectively supports the generation of complex geometric structures and reduces segmentation error to an extremely low level, achieving a significant improvement over prior command sequence methods, thereby significantly mitigating the topological inaccuracies introduced by quantization error.
Figures & tables
Figure 1: Illustration of the strength of our proposed pointer-based command sequence compared to the previous command sequence-based CAD representation. Command sequences suffer from the inability to refer to specific edges or faces, and discretization-induced quantization errors. In contrast, Pointer-CAD leverages edge pointers to directly refer to B-rep entities, enabling precise operations such as sketch snapping, thereby reducing quantization errors and faithfully following complex text instructions.
Figure 2: Pointer-CAD Pipeline. At each generation step, the full user prompt is tokenized, while the B-rep is updated with all geometry generated so far. A multimodal fusion module combines the textual prompt with the evolving B-rep, which is further encoded via a graph neural network over its faces and edges. The fused representation is then processed by a large language model to predict the vector for the current step, which is subsequently translated into geometry to update the B-rep.
Figure 3: Dataset construction pipeline. Raw JSONs are converted into a minimal format containing only annotation-relevant elements. Sketch planes and models are rendered, and Qwen2.5-VL generates textual descriptions for integration into the JSON. Finally, Qwen2.5 produces step-by-step instructions, with dimension parameters wrapped in special tags for future data augmentation.
Figure 4: Qualitative performance comparison on Recap-DeepCAD dataset. Our method consistently produces accurate and faithful geometry aligned with the ground truth, while competing methods often miss details or collapse entirely. Notably, Pointer-CAD achieves superior results among LLM-based methods despite a significantly smaller size than CADmium.
Figure 5: Qualitative performance comparison on Recap-OmniCAD + dataset. Our method accurately recovers detailed structures that closely match the ground truth for complex CAD models involving chamfer or fillet operations. Conversely, competing methods often miss fine-grained features or fail entirely.
Figure 6: Showcase of complex CAD model generation.
Model
Text2CAD
Text-to-
CADmium
Pointer-CAD
CadQuery-1.5B
1.5B
3B
0.5B
1.5B
IR ↓
9.58
12.27
9.12
8.51
6.74
6.59
Line F1 ↑
83.78
-
79.00
80.30
84.01
85.77
Arc F1 ↑
41.54
-
37.72
44.22
37.51
46.83
Circle F1 ↑
78.38
-
76.42
78.39
77.93
78.66
Extrusion F1 ↑
93.76
-
88.53
88.06
95.72
95.50
Table 10: Quantitative comparison on Text2CAD datasets. Pointer-CAD clearly surpasses Text2CAD and CADmium, and remains highly competitive with Text-to-CadQuery despite the latter’s reliance on the mature CadQuery engine.
Figure 7: Prompt comparison. Recap-DeepCAD dataset includes dimensional values with explicit units, whereas Text2CAD dataset uses normalized, unit-free geometric parameters.
Figure 8: Quantization Error. We directly measure quantization error by computing the median Chamfer Distance between each representation before and after quantization, where Pointer-CAD exhibits substantially smaller error than Text2CAD.
Figure 9: Illustration of our interactive editing functionality. Users can directly click on a face or edge of the CAD model and provide a text prompt to specify the desired operation.
Notation
ID
Type
Description
< em >
1
Label Token
End of model (this step is the final step of the model)
< es >
2
Label Token
End of step (additional steps are required after this one)
< ss >
3
Label Token
Start of sketch
< se >
4
Label Token
Start of extrusion
< sc >
5
Label Token
Start of chamfer
< sf >
6
Label Token
Start of fillet
Table 12: Special Token Definitions. This table provides a comprehensive list of all Special Tokens used in our command sequence representation, along with their semantic descriptions.
Notation
Sequence
Description
[ P ]
< nv > < nv > < pe / pd >
2D point (x,y) , snapped to reference or placed freely
[ L ]
< sx > [ P ]
Line starting from a 2D point (x,y)
[ C ]
< sx > [ P ] < nv >
Circle with center at (x,y) and radius r
[ A ]
< sx > [ P ] < ag > < or >
Arc starting from (x,y) with angle α and orientation
[ Loop ]
< sl > [ L / C / A ] +
Closed loop composed of multiple curves
[ Profile ]
< sp > [ Loop ] +
2D Region defined by one or more loops
Table 13: Sequence definitions. Each sequence is defined by a specific combination of commands. The superscript [ + ] denotes that the element appears one or more times, while the symbol [ / ] indicates that one of the alternatives can be chosen.
Symbol
Direction
Auxiliary Direction
14
X+
Y+
15
X-
Z+
16
Y+
Z+
17
Y-
X+
18
Z+
X+
19
Z-
Y+
Table 14: Direction mapping. In command < dr >, each symbol corresponds to a primary direction and its auxiliary direction.
Figure 10: Sketch coordinate system construction. The sketch plane, axes, origin, and rotation are defined step by step to form the local coordinate system UVW .
Figure 11: A non-manifold topology leads to multiple valid interpretations of a fillet operation.
Figure 12: Representative samples from the Recap-OmniCAD + dataset. The figure displays a range of models with varying complexity, from simpler parts with basic features to intricate components incorporating numerous fillets, chamfers, and complex sketches.
Figure 13: Distribution of modeling operations across datasets. The figure illustrates the total count of each modeling operation type for the DeepCAD, OmniCAD, and Recap-OmniCAD + datasets.
Figure 14: Distribution of modeling steps per model. The figure compares the number of solid modeling operations required per model across the datasets.
Figure 15: Prompt for visual description. This prompt is used with the Qwen2.5-vl-72B model to generate a description of the CAD model’s visual appearance.
Figure 16: Prompt for sketch plane description. This prompt guides the model to describe the relative position of the sketch plane, with placeholders for the normal vector and facing direction being dynamically replaced.
Figure 17: Examples of the minimal JSON structure. This figure illustrates two structured ’minimal JSONs’ format, which integrates visual annotations and key modeling parameters for the language model.
Figure 18: Prompt for generating the final natural language description. This prompt is used with the ’minimal JSON’ to generate the final natural language description of the modeling process.
The field of Computer-Aided Design (CAD) generation has made significant progress in recent years. Existing methods typically fall into two separate categories: parametric CAD modeling and direct boundary representation (B-Rep) synthesis. In modern feature-based CAD systems, parametric modeling and B-Rep are inherently intertwined, as advanced parametric operations (e.g., fillet and chamfer) require explicit selection of B-Rep geometric primitives, and the B-Rep itself is derived from parametric operations. Consequently, this paradigm gap remains a critical factor limiting AI-driven CAD modeling for complex industrial product design. This paper presents FutureCAD, a novel text-to-CAD framework that leverages large language models (LLMs) and a B-Rep grounding transformer (BRepGround) for high-fidelity CAD generation. Our method generates executable CadQuery scripts, and introduces a text-based query mechanism that enables the LLM to specify geometric selections via natural language, which BRepGround then grounds to the target primitives. To train our framework, we construct a new dataset comprising real-world CAD models. For the LLM, we apply supervised fine-tuning (SFT) to establish fundamental CAD generation capabilities, followed by reinforcement learning (RL) to improve generalization. Experiments show that FutureCAD achieves state-of-the-art CAD generation performance. Code and dataset are available at: https://github.com/JohanStackk/FutureCAD
Automatic generation of computer-aided design (CAD) models is a core technology for enabling intelligence in advanced manufacturing. Existing generation methods based on large language models (LLMs) often fall short when handling complex CAD models characterized by long operation sequences, diverse operation types, and strong geometric constraints, primarily because reasoning chains break and effective error-correction mechanisms are lacking. To address this problem, this paper proposes a memory-augmented reinforcement learning framework for CAD generation agents. The framework encapsulates the underlying geometric kernel into a structured toolchain callable by the agent and builds a closed-loop mechanism of design intent understanding, global planning, execution, and multi-dimensional verification. It also designs a dual-track memory module consisting of a case library and a skill library, and proposes a dynamic utility retrieval algorithm. By introducing reinforcement learning into retrieval and policy optimization, the agent can effectively avoid retrieval traps in which examples are semantically similar but geometrically infeasible, enabling online self-correction and continual evolution without additional large-scale annotated data. Experiments show that the proposed method significantly improves both the success rate and geometric consistency on complex CAD model generation tasks.
The construction of CAD models has traditionally relied on labor-intensive manual operations and specialized expertise. Recent advances in large language models (LLMs) have inspired research into text-to-CAD generation. However, existing approaches typically treat generation and editing as disjoint tasks, limiting their practicality. We propose PR-CAD, a progressive refinement framework that unifies generation and editing for controllable and faithful text-to-CAD modeling. To support this, we curate a high-fidelity interaction dataset spanning the full CAD lifecycle, encompassing multiple CAD representations as well as both qualitative and quantitative descriptions. The dataset systematically defines the types of edit operations and generates highly human-like interaction data. Building on a CAD representation tailored for LLMs, we propose a reinforcement learning-enhanced reasoning framework that integrates intent understanding, parameter estimation, and precise edit localization into a single agent. This enables an "all-in-one" solution for both design creation and refinement. Extensive experiments demonstrate strong mutual reinforcement between generation and editing tasks, and across qualitative and quantitative modalities. On public benchmarks, PR-CAD achieves state-of-the-art controllability and faithfulness in both generation and refinement scenarios, while also proving user-friendly and significantly improving CAD modeling efficiency.
Jiyuan An, Jiachen Zhao, Fan Chen +6
National Language Resources Monitoring and Research Center for Print Media, Beijing Language and Culture University, China · School of Information Science, Beijing Language and Culture University, China · School of Computer Science and Technology, Beijing Jiaotong University, China +3