DIP-CAD: A Design-Intent-Preserving Agent Harness for Parametric CAD Modeling
Authors: Xi Cheng, Chenxi Zhai, Hang Cheng, Mingyu Fan, Pingfa Feng, Long Zeng
Organizations: Division of Advanced Manufacturing, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China · Department of Mechanical Engineering, Tsinghua University, Beijing, 100084, China
Parametric CAD modeling requires not only matching the target shape, but also preserving design intent under parameter edits. However, most existing methods focus on improving static shape generation accuracy, while the dependencies that govern later edits are often overlooked. To tackle this issue, we present DIP-CAD, a design-intent-preserving agent harness for generating parametric CAD models from images. DIP-CAD combines vision-language model reasoning with CAD kernel operations to establish dependencies that encode the design intent. In parametric CAD, these dependencies are mainly established through geometry references in modeling commands and constraints in parametric sketches. At the command level, ID-based explicit geometry reference binds modeling commands to selected faces, edges, or vertices. At the sketch level, a local-coordinate bridge ensures accurate sketch placement, providing the basis for correctly establishing sketch constraints. Constraints among sketch primitives and constraints to external projected geometry further preserve both internal sketch relationships and dependencies on existing geometry. To provide a more comprehensive data foundation for parametric CAD modeling tasks, we introduce the Geometry Explicit Reference Dataset (GERD), whose structured data format links stepwise modeling commands to intermediate CAD models through explicit geometry references. On GERD-EVL and a DeepCAD test subset, DIP-CAD improves mIoU for static model generation by 11.1% and 5.6%, respectively. In parameter-editing evaluations, the corresponding mIoU improvements are 7.0% and 4.2%. Case studies further demonstrate how parameter edits drive shape updates while preserving design intent.
Figures & tables
Figure 1: DIP-CAD overview. Given a part image, DIP-CAD understands the design intent, and then interacts with CAD software to generate native parametric CAD models. 4-Left: referencing the back blue face makes the back plate follow the back face when base width increases. 4-Middle: placing a slot on the inclined plane requires locating its profile relative to the local coordinate system of the sketch plane. 4-Right: after base height decreases, constraints to projected external geometry keep the rib connected to the base.
Figure 2: Key Elements of Parametric CAD Model. Parametric sketches provide profiles for commands. The local coordinate system of sketch plane determines the sketch placement. Sketch Primitives determine sketch profiles, e.g., line, arc, circle. External constraints between sketch primitives and projected external geometry determine how sketch primitives update after upstream parameters are edited. The three key challenges are Geometry Reference, Sketch Placement (Local coordinate system), and External Constraints.
Figure 3: Geometry Reference. With the intended references, the sketch and swept solid in (A1) and (B1) update with their source geometry in (A2) and (B2). Without those references, the initially identical constructions in (a1) and (b1) fail to retain the intended relationships in (a2) and (b2).
Figure 4: Local coordinate system of different sketch planes. Cyan marks the selected sketch planes, red and green arrows indicate their local coordinate axes, and the orange triangles denote sketch profiles. The sketches are defined identically in their respective local coordinate systems. Sketches are defined in plane-specific local coordinate systems, whose origins and axis orientations may vary. Accurately identifying these coordinate systems is therefore crucial for precise sketch placement.
Figure 5: Sketch external constraints. (a) Sketch primitives are constrained to projected external geometry. (b, c) They update with that geometry after upstream parameters are edited.
Figure 6: DIP-CAD overall workflow. Given a target image I∗ , DIP-CAD progressively updates an initially empty FreeCAD parametric part document D0 into the complete document DN through a sequence of modeling steps. Each step consists of up to three stages. Stage 1: Command Generation predicts the modeling command, including its command type, geometry references, and parameters. If the predicted command requires a new sketch, the process proceeds to Stage 2: Sketch Generation , which constructs the required parametric sketch; otherwise, Stage 2 is skipped. Stage 3: Modeling Verification pre-executes the outputs of the preceding stages to generate a modeling preview, which is verified by the VLM. The accepted result is committed to the FreeCAD document, updating Dt to Dt+1 ; otherwise, the rejection reason is fed back, and the current modeling step is restarted from Stage 1. The VLM context is preserved across all stages within a modeling step and reset between successive modeling steps.
Figure 7: Stage 1: Command Generation workflow. The current document Dt is rendered as a depth image Itdepth , and registered to the target image I∗ . Under the registered camera Ctreg , we generate the tag image Ittag and geometry table Ttgeom ( Figure 8 ). Retrieval uses Itdepth and I∗ to provide modeling examples Itret . The VLM utilizes these inputs to predict the modeling command data Xt(1) . The pseudo-bounding sphere St is constructed under Ctreg for subsequent sketch-view framing.
Figure 8: Tag image Ittag and geometry table Ttgeom . Ittag shows depth, coordinate ticks, and edge ID tags. Visible and occluded edges use solid and dashed magenta lines; vertices use magenta squares and asterisks. ID tags for very short edges and hidden edges after hidden-line removal (HLR) are omitted. Ttgeom lists all edge types and uniformly sampled view coordinates. Face entries contain surface types and boundary edge IDs. The VLM selects faces and edges by ID, and vertices by IDEdge.Start or IDEdge.End . Edge samples in Ttgeom are ordered from start to end. Tag leader lines in Ittag are anchored at random positions within [0.1,0.25] of the normalized edge parameter domain, typically closer to the start. These cues distinguish the endpoints.
Figure 9: An example retrieval instance. Each instance is {Iidepth,Ii∗;Iiret,oiret} , where i is the instance index. (Q1), (Q2): the retrieval queries, which contain the current depth image and completed-part image. (V1), (V2): the retrieved results, which contain a pseudo-preview and its command type. In (V1), yellow highlights the sketch support, red and blue outlines mark the starting and ending profiles, and green segments connect corresponding profile points. Existing geometry retains depth shading and magenta edge outlines.
Figure 10: Stage 2: Sketch Generation workflow. The sketch plane in Xt(1) determines the camera orientation. After the camera is aligned with the normal to the sketch plane, the pseudo-sphere St determines the camera zoom and pan. We render Ittag,s and export Ttgeom,s in the resulting view. The VLM uses these inputs to predict sketch-primitive data Xt(2) . The predicted primitives are subsequently converted into sketch plane local coordinates, and constrained automatically.
Figure 11: Effect of the pseudo-bounding sphere St on sketch viewport framing. The back plate highlighted by the red frame is constructed through sketching and extrude. Including the pseudo-bounding sphere St in FitAll() leaves space for the full sketch profile. But fitting only the existing base enlarges it within the viewport and clips the upper part of the intended profile.
Figure 12: Stage 3: Modeling Verification workflow. FreeCAD executes Xt(1) and, when needed, the sketch derived from Xt(2) against Dt to render Itpv . The VLM evaluates the candidate using Xt(1) , Itpv , and the target I∗ , producing a decision Xt(3) . Acceptance commits the command and produces Dt+1 ; rejection returns to Stage 1 with reasons. The VLM context is retained across retries within the same modeling step and reset after an accepted step.
Figure 13: Modeling previews for verification. From left to right: adding the base, cutting two holes, adding the back plate, and rounding the base corners. The previews expose the geometric change and operation direction, so that the VLM can compare the candidate with the target.
Figure 14: GERD data structure. Each model contains MainRef, Geometry, and Command. MainRef stores three main reference planes and their coordinate systems: Z is normal to the plane, and X/Y define local coordinates. Geometry indexes four levels by ID. Bodies store boundary face IDs and volume. Faces store surface type, boundary edge IDs, and area. Edges store curve type, endpoint vertex IDs, and length. Vertices store 3D coordinates. Faces and edges also provide approximate B-spline representations for reconstruction across CAD systems. MainRef, Geometry, and Command share the same ID namespace.
Figure 15: Static shape accuracy and parameter edit performance.
Figure 16: Generated model examples. The leftmost column shows the target image, the remaining columns show the generated models. The upper four rows are GERD-EVL examples; the lower four are from the DeepCAD test subset. Entries labeled “Compile Failed” indicate unsuccessful model generation or execution. DIP-CAD more consistently recovers the illustrated supports, holes, and curved profiles.
Figure 17: Parameter edit propagation case. Blue arrows link the feature tree to the corresponding modeling preview. Column O shows the original states; columns A–I show previews obtained during parameter editing. Rows 1-9 denote stages in the feature tree. Column-row identifiers (e.g., O3 and A3) are also used in Figure 18 .
Figure 18: Mechanism-level analysis of parameter-edit propagation. The upper and lower rows show the original and updated features following an increase in the center-hole diameter (O1 → A1), respectively. Left: Explicit geometric references for fillet and revolve commands. Middle: Sketch placement on different support planes through the coordinate bridge. Right: Persistent constraints between sketch primitives and projected external geometry (Sketches in the right panels are offset from their sketch planes for visibility only).
Parametric CAD modeling represents 3D designs as editable, constraint-driven construction histories and develops methods to generate, reconstruct, and revise them.
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · KOKONI3D, Moxin (Huzhou) Technology Co., Ltd., China · Nanjing Institute of Agricultural Mechanization, Ministry of Agriculture and Rural Affairs, Nanjing 210014, China