DIP-CAD: A Design-Intent-Preserving Agent Harness for Parametric CAD Modeling
Authors: Xi Cheng, Chenxi Zhai, Hang Cheng, Mingyu Fan, Pingfa Feng, Long Zeng
Organizations: Division of Advanced Manufacturing, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China · Department of Mechanical Engineering, Tsinghua University, Beijing, 100084, China
Parametric CAD modeling requires not only matching the target shape, but also preserving design intent under parameter edits. However, most existing methods focus on improving static shape generation accuracy, while the dependencies that govern later edits are often overlooked. To tackle this issue, we present DIP-CAD, a design-intent-preserving agent harness for generating parametric CAD models from images. DIP-CAD combines vision-language model reasoning with CAD kernel operations to establish dependencies that encode the design intent. In parametric CAD, these dependencies are mainly established through geometry references in modeling commands and constraints in parametric sketches. At the command level, ID-based explicit geometry reference binds modeling commands to selected faces, edges, or vertices. At the sketch level, a local-coordinate bridge ensures accurate sketch placement, providing the basis for correctly establishing sketch constraints. Constraints among sketch primitives and constraints to external projected geometry further preserve both internal sketch relationships and dependencies on existing geometry. To provide a more comprehensive data foundation for parametric CAD modeling tasks, we introduce the Geometry Explicit Reference Dataset (GERD), whose structured data format links stepwise modeling commands to intermediate CAD models through explicit geometry references. On GERD-EVL and a DeepCAD test subset, DIP-CAD improves mIoU for static model generation by 11.1% and 5.6%, respectively. In parameter-editing evaluations, the corresponding mIoU improvements are 7.0% and 4.2%. Case studies further demonstrate how parameter edits drive shape updates while preserving design intent.
Figures & tables
Figure 1: DIP-CAD overview. Given a part image, DIP-CAD understands the design intent, and then interacts with CAD software to generate native parametric CAD models. 4-Left: referencing the back blue face makes the back plate follow the back face when base width increases. 4-Middle: placing a slot on the inclined plane requires locating its profile relative to the local coordinate system of the sketch plane. 4-Right: after base height decreases, constraints to projected external geometry keep the rib connected to the base.
Figure 2: Key Elements of Parametric CAD Model. Parametric sketches provide profiles for commands. The local coordinate system of sketch plane determines the sketch placement. Sketch Primitives determine sketch profiles, e.g., line, arc, circle. External constraints between sketch primitives and projected external geometry determine how sketch primitives update after upstream parameters are edited. The three key challenges are Geometry Reference, Sketch Placement (Local coordinate system), and External Constraints.
Figure 3: Geometry Reference. With the intended references, the sketch and swept solid in (A1) and (B1) update with their source geometry in (A2) and (B2). Without those references, the initially identical constructions in (a1) and (b1) fail to retain the intended relationships in (a2) and (b2).
Figure 4: Local coordinate system of different sketch planes. Cyan marks the selected sketch planes, red and green arrows indicate their local coordinate axes, and the orange triangles denote sketch profiles. The sketches are defined identically in their respective local coordinate systems. Sketches are defined in plane-specific local coordinate systems, whose origins and axis orientations may vary. Accurately identifying these coordinate systems is therefore crucial for precise sketch placement.
Figure 5: Sketch external constraints. (a) Sketch primitives are constrained to projected external geometry. (b, c) They update with that geometry after upstream parameters are edited.
Figure 6: DIP-CAD overall workflow. Given a target image I∗ , DIP-CAD progressively updates an initially empty FreeCAD parametric part document D0 into the complete document DN through a sequence of modeling steps. Each step consists of up to three stages. Stage 1: Command Generation predicts the modeling command, including its command type, geometry references, and parameters. If the predicted command requires a new sketch, the process proceeds to Stage 2: Sketch Generation , which constructs the required parametric sketch; otherwise, Stage 2 is skipped. Stage 3: Modeling Verification pre-executes the outputs of the preceding stages to generate a modeling preview, which is verified by the VLM. The accepted result is committed to the FreeCAD document, updating Dt to Dt+1 ; otherwise, the rejection reason is fed back, and the current modeling step is restarted from Stage 1. The VLM context is preserved across all stages within a modeling step and reset between successive modeling steps.
Figure 7: Stage 1: Command Generation workflow. The current document Dt is rendered as a depth image Itdepth , and registered to the target image I∗ . Under the registered camera Ctreg , we generate the tag image Ittag and geometry table Ttgeom ( Figure 8 ). Retrieval uses Itdepth and I∗ to provide modeling examples Itret . The VLM utilizes these inputs to predict the modeling command data Xt(1) . The pseudo-bounding sphere St is constructed under Ctreg for subsequent sketch-view framing.
Figure 8: Tag image Ittag and geometry table Ttgeom . Ittag shows depth, coordinate ticks, and edge ID tags. Visible and occluded edges use solid and dashed magenta lines; vertices use magenta squares and asterisks. ID tags for very short edges and hidden edges after hidden-line removal (HLR) are omitted. Ttgeom lists all edge types and uniformly sampled view coordinates. Face entries contain surface types and boundary edge IDs. The VLM selects faces and edges by ID, and vertices by IDEdge.Start or IDEdge.End . Edge samples in Ttgeom are ordered from start to end. Tag leader lines in Ittag are anchored at random positions within [0.1,0.25] of the normalized edge parameter domain, typically closer to the start. These cues distinguish the endpoints.
Figure 9: An example retrieval instance. Each instance is {Iidepth,Ii∗;Iiret,oiret} , where i is the instance index. (Q1), (Q2): the retrieval queries, which contain the current depth image and completed-part image. (V1), (V2): the retrieved results, which contain a pseudo-preview and its command type. In (V1), yellow highlights the sketch support, red and blue outlines mark the starting and ending profiles, and green segments connect corresponding profile points. Existing geometry retains depth shading and magenta edge outlines.
Figure 10: Stage 2: Sketch Generation workflow. The sketch plane in Xt(1) determines the camera orientation. After the camera is aligned with the normal to the sketch plane, the pseudo-sphere St determines the camera zoom and pan. We render Ittag,s and export Ttgeom,s in the resulting view. The VLM uses these inputs to predict sketch-primitive data Xt(2) . The predicted primitives are subsequently converted into sketch plane local coordinates, and constrained automatically.
Figure 11: Effect of the pseudo-bounding sphere St on sketch viewport framing. The back plate highlighted by the red frame is constructed through sketching and extrude. Including the pseudo-bounding sphere St in FitAll() leaves space for the full sketch profile. But fitting only the existing base enlarges it within the viewport and clips the upper part of the intended profile.
Figure 12: Stage 3: Modeling Verification workflow. FreeCAD executes Xt(1) and, when needed, the sketch derived from Xt(2) against Dt to render Itpv . The VLM evaluates the candidate using Xt(1) , Itpv , and the target I∗ , producing a decision Xt(3) . Acceptance commits the command and produces Dt+1 ; rejection returns to Stage 1 with reasons. The VLM context is retained across retries within the same modeling step and reset after an accepted step.
Figure 13: Modeling previews for verification. From left to right: adding the base, cutting two holes, adding the back plate, and rounding the base corners. The previews expose the geometric change and operation direction, so that the VLM can compare the candidate with the target.
Figure 14: GERD data structure. Each model contains MainRef, Geometry, and Command. MainRef stores three main reference planes and their coordinate systems: Z is normal to the plane, and X/Y define local coordinates. Geometry indexes four levels by ID. Bodies store boundary face IDs and volume. Faces store surface type, boundary edge IDs, and area. Edges store curve type, endpoint vertex IDs, and length. Vertices store 3D coordinates. Faces and edges also provide approximate B-spline representations for reconstruction across CAD systems. MainRef, Geometry, and Command share the same ID namespace.
Figure 15: Static shape accuracy and parameter edit performance.
Figure 16: Generated model examples. The leftmost column shows the target image, the remaining columns show the generated models. The upper four rows are GERD-EVL examples; the lower four are from the DeepCAD test subset. Entries labeled “Compile Failed” indicate unsuccessful model generation or execution. DIP-CAD more consistently recovers the illustrated supports, holes, and curved profiles.
Figure 17: Parameter edit propagation case. Blue arrows link the feature tree to the corresponding modeling preview. Column O shows the original states; columns A–I show previews obtained during parameter editing. Rows 1-9 denote stages in the feature tree. Column-row identifiers (e.g., O3 and A3) are also used in Figure 18 .
Figure 18: Mechanism-level analysis of parameter-edit propagation. The upper and lower rows show the original and updated features following an increase in the center-hole diameter (O1 → A1), respectively. Left: Explicit geometric references for fillet and revolve commands. Middle: Sketch placement on different support planes through the coordinate bridge. Right: Persistent constraints between sketch primitives and projected external geometry (Sketches in the right panels are offset from their sketch planes for visibility only).
Parametric CAD modeling represents 3D designs as editable, constraint-driven construction histories and develops methods to generate, reconstruct, and revise them.
Large language models have shown considerable potential for natural-language-driven parametric CAD modeling. However, a fundamental contradiction exists between their probabilistic generation and the deterministic requirements of CAD modeling, resulting in limitations in reliability, design-intent preservation, and geometric validity. Existing methods typically rely on large-scale annotated datasets, lack explicit modeling of design intent, and underutilize the deterministic capabilities of CAD kernels. To address these limitations, we propose ReliCAD, a unified framework that transforms uncertain LLM generation into reliable parametric CAD modeling. Through explicit design-intent modeling, ReliCAD converts user requirements into structured design specifications and explicitly models geometric relations, topological dependencies, and feature construction order. It then generates constraint-aware parametric instructions and invokes the CAD kernel through an Agent-ready API to perform geometric construction and constraint solving. ReliCAD further records runtime evidence and employs a verification-feedback mechanism to assess consistency between the generated model and the design specifications, enabling error localization and iterative repair. Experiments on the public HistCAD generation dataset and our multi-granularity CAD editing dataset demonstrate that ReliCAD significantly outperforms baseline methods, achieving 99.8% validity rate and 0.8753 IoU. ReliCAD provides a verifiable, repairable, and generalizable approach to natural-language-interactive CAD modeling.
Peng Zheng, Xintong Dong, Chuanyang Li +5
University of Science and Technology of China, Hefei, Anhui 230026, China
Parametric CAD reconstruction requires recovering both precise geometry and editable modeling operations from visual observations, making it challenging under limited and ambiguous views. Existing methods mainly rely on 2D appearance cues and lack strong multi-view geometric priors. In this work, we present VGGT-CAD, a geometry-aware framework for parametric CAD reconstruction from single- and multi-view observations. We transfer pretrained 3D geometric priors into CAD reconstruction by encoding camera parameters as condition tokens and jointly modeling them with image tokens. To handle varying numbers of viewpoints, we introduce a variable-view cross-view context aggregation module that adaptively fuses multi-view features. We further develop a training-free geometry-aware view selection strategy to select complementary and reliable frames during inference. The resulting representation is decoded into CAD command sequences using a non-autoregressive decoder. We also develop VideoCAD, a large-scale multi-view video benchmark derived from existing CAD data through multi-view re-rendering. Extensive experiments demonstrate the effectiveness of VGGT-CAD for visual CAD reconstruction under different observation configurations.
Chunan Yu, Tianrun Chen, Fu Shen +3
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · KOKONI3D, Moxin (Huzhou) Technology Co., Ltd., China · Nanjing Institute of Agricultural Mechanization, Ministry of Agriculture and Rural Affairs, Nanjing 210014, China
Computer-Aided Design (CAD) models are defined by their construction history: a parametric recipe that encodes design intent. However, existing large-scale 3D datasets predominantly consist of boundary representations (B-Reps) or meshes, stripping away this critical procedural information. To address this scarcity, we introduce Zero-to-CAD, a scalable framework for synthesizing executable CAD construction sequences. We frame synthesis as an agentic search problem: by embedding a large language model (LLM) within a feedback-driven CAD environment, our system iteratively generates, executes, and validates code using tools and documentation lookup to promote geometric validity and operation diversity. This agentic approach enables the synthesis of approximately one million executable, readable, editable CAD sequences, covering a rich vocabulary of operations beyond sketch-and-extrude workflows. We also release a curated subset of 100,000 high-quality models selected for geometric diversity. To demonstrate the dataset's utility, we fine-tune a vision-language model on our synthetic data to reconstruct editable CAD programs from multi-view images, outperforming strong baselines, including GPT-5.2, and effectively bootstrapping sequence generation capabilities without real construction-history training data. Zero-to-CAD bridges the gap between geometric scale and parametric interpretability, offering a vital resource for the next generation of CAD AI.
Mohammadmehdi Ataei, Farzaneh Askari, Kamal Rahimi Malekshan +1