DreamCAD: Scaling Multi-modal CAD Generation using Differentiable Parametric Surfaces

Mohammad Sadil Khan; Muhammad Usama; Rolandos Alexandros Potamias; Didier Stricker; Muhammad Zeshan Afzal; Jiankang Deng; Ismail Elezi

cs.CVMar 5, 2026

DreamCAD: Scaling Multi-modal CAD Generation using Differentiable Parametric Surfaces

Authors: Mohammad Sadil Khan, Muhammad Usama, Rolandos Alexandros Potamias, Didier Stricker, Muhammad Zeshan Afzal, Jiankang Deng, Ismail Elezi

Organizations: DFKI, Germany · RPTU Kaiserslautern, Germany · Imperial College London, United Kingdom · Huawei London Research Center, United Kingdom

Abstract

Computer-Aided Design (CAD) relies on structured and editable geometric representations, yet existing generative methods are constrained by small annotated datasets with explicit design histories or boundary representation (BRep) labels. Meanwhile, millions of unannotated 3D meshes remain untapped, limiting progress in scalable CAD generation. To address this, we propose DreamCAD, a multi-modal generative framework that directly produces editable BReps from point-level supervision, without CAD-specific annotations. DreamCAD represents each BRep as a set of parametric patches (e.g., Bézier surfaces) and uses a differentiable tessellation method to generate meshes. This enables large-scale training on 3D datasets while reconstructing connected and editable surfaces. Furthermore, we introduce CADCap-1M, the largest CAD captioning dataset to date, with 1M+ descriptions generated using GPT-5 for advancing text-to-CAD research. DreamCAD achieves state-of-the-art performance on ABC and Objaverse benchmarks across text, image, and point modalities, improving geometric fidelity and surpassing 75% user preference. Code and dataset will be publicly available.

Figures & tables

Figure 1: Our proposed DreamCAD (left) is a multimodal generative framework that can reconstruct CAD models from text, images, and point clouds using parametric patches. CADCap-1M (right) provides 1M+ GPT-5–generated captions.

Dataset	Samples	CADCap	Dataset	Samples	CADCap
ABC [ 23 ]	757,433	✓	3D-Future [ 13 ]	16,990	✓
Automate [ 19 ]	380,124	✓	ModelNet [ 63 ]	12,308	✓
ShapeNet [ 4 ]	52,458	✗	ABO [ 5 ]	7,944	✗
CADParser [ 71 ]	40,989	✓	Fusion360 [ 59 ]	4,603	✓
HSSD [ 22 ]	30,078	✗	Toys4K [ 53 ]	3,482	✗
Total				1,306,409

Table 1: Training datasets for DreamCAD comprising 1.3M 3D meshes. ✓ denotes if the dataset is included in proposed CADCap-1M dataset.

Figure 2: Bézier surface representation and differentiable tessellation.

Figure 3: DreamCAD Overview: (A). Sparse Transformer VAE from an input mesh, active voxels $v_{i}$ are generated with local features $f_{i}$ , from DINOv2 [ 45 ] embeddings, normal images, and SDF values and encodes it to generate structured latents $z_{i}$ . These are then decoded into parametric (rational bézier) surfaces and optimized using Chamfer loss. (B). Initial $C^{0}$ -continuous Parametric Surface generation from sparse voxels via flood-fill and quad conversion using grid control points and unit weights. (C). Multi-modal CAD generation from images, or points using a flow-matching framework from coarse voxel grid to parametric surface refinement.

Figure 4: Examples of metadata-augmented captions from CADCap-1M showing object type, part names, and hole counts.

Figure 5: Qualitative comparison on Point2CAD (Top-Right), Image2CAD (Bottom-Left) and Text2CAD (Right) tasks. For each task, the first four examples are from the ABC dataset, while the last two from Objaverse dataset. ✗ indicates invalid models.

Task	Models	ABC						Objaverse
Task	Models	F1 $\uparrow$	NC $\uparrow$	CD $\downarrow$	HD $\downarrow$	JSD $\downarrow$	MMD $\downarrow$	F1 $\uparrow$	NC $\uparrow$	CD $\downarrow$	HD $\downarrow$	JSD $\downarrow$	MMD $\downarrow$
Point2CAD	DeepCAD [ 61 ]	19.31	0.49	51.10	0.37	783.94	29.63	7.05	0.48	320.33	0.41	855.14	34.62
	CAD-Recode [ 49 ]	75.99	0.79	3.73	0.13	271.89	2.94	53.24	0.66	7.92	0.19	479.50	6.27
	Cadrille [ 24 ]	78.86	0.80	2.98	0.12	236.10	2.51	57.49	0.67	6.28	0.17	445.23	5.24
	DreamCAD (Ours)	92.12	0.94	0.93	0.06	96.13	0.84	87.31	0.89	1.25	0.11	189.12	1.86
Img2CAD		GPT $\uparrow$	User $\uparrow$	CD $\downarrow$	HD $\downarrow$	JSD $\downarrow$	MMD $\downarrow$	GPT $\uparrow$	User $\uparrow$	CD $\downarrow$	HD $\downarrow$	JSD $\downarrow$	MMD $\downarrow$

Table 2: Quantitative comparison across the Point2CAD, Img2CAD, and Text2CAD tasks on the ABC and Objaverse datasets. For readability, F1 is scaled by

10^{2}

and CD, JSD, and MMD by

10^{3}

. For text- and image-to-CAD, GPT and User ratings measure visual alignment.

Table 3: Ablation studies on regularization (left), and voxel resolution choice (right).

Figure 6: Examples showing VAE reconstructions for different regularizers.

Figure 7: Left: Captions from GPT-5 with and without metadata-augmented prompting. Right: Images from pretrained vs. fine-tuned SD-3.5 given the same prompts.

Table 4: Ablation studies on initialization quality and coarse-to-fine generation.

Figure 8: ( Left ) Topology recovery from patch-based outputs to hybrid NURBS CAD representation [ 54 ] . ( Right ) For multimodal inputs, DreamCAD reconstructions and the corresponding recovered topology.

Figure 9: Wordcloud of captions from CADCap-1M

Mean Length	Unigram	Bigram	Trigram
19.63	21042	446,061	2,368,731

Table 5: Summary statistics of the CADCap-1M dataset, including average caption length and diversity measured through unigram, bigram, and trigram counts.

Figure 10: Distribution of patch sizes in training datasets.

VAE	Sparse Structure	Coarse		Fine-Grained
		Image	Point	Image	Point
71M	133M	354M	280M	400M	325M

Table 6: Parameter counts for all components of DreamCAD.

Task	Models	DeepCAD
Task	Models	F1 $\uparrow$	NC $\uparrow$	CD $\downarrow$	HD $\downarrow$	JSD $\downarrow$	MMD $\downarrow$
Point	DeepCAD	23.18	0.53	47.63	0.34	729.11	26.80
	CAD-Recode	90.95	0.92	1.38	0.06	126.15	1.13
	Cadrille	92.12	0.93	0.30	0.05	112.31	1.05
	DreamCAD	94.91	0.94	0.12	0.03	98.51	0.90
Img		GPT $\uparrow$	User $\uparrow$	CD $\downarrow$	HD $\downarrow$	JSD $\downarrow$	MMD $\downarrow$

Table 7: Quantitative comparison on Point2CAD, Img2CAD, and Text2CAD tasks over the DeepCAD test set. F1 is scaled by

10^{2}

, while CD, JSD, and MMD by

10^{3}

. For text - and image-to-CAD , GPT and User ratings measure visual alignment.

Figure 11: Some failure cases showcasing missing geometric details and invalid models.

Table 8: Ablation studies on initialization quality and coarse-to-fine generation.

Figure 12: Examples of captions for complex (top) and simple (bottom) CAD parts.

Figure 13: Examples of captions for different types of fasteners .

Figure 14: Examples of captions for letter -based (top) and engraving -style (bottom) CAD models.

Figure 15: More qualitative results for the point-to-CAD reconstruction task using DreamCAD. Each row shows input point clouds (top) and the corresponding reconstructed CAD models (bottom) generated by DreamCAD across a wide range of shapes, including mechanical components, furniture, utensils, and free-form surfaces. As seen in the examples, DreamCAD successfully recovers clean parametric geometry from input point clouds, preserving fine structural details (e.g., tubular connectors, circular cutouts, brackets ), smooth surfaces (e.g., bowls, vases, seats ), and complex topologies (e.g., multi-part assemblies and chair frames ).

Figure 16: More qualitative results for the image-to-CAD generation task using DreamCAD. Each row shows the ground-truth reference image (top) and the CAD reconstruction produced by DreamCAD (bottom) across a wide range of object categories, including furniture, mechanical parts, consumer products, and free-form designs. As seen in the examples, DreamCAD consistently recovers accurate 3D geometry from a single image, capturing fine structural details (e.g., chair backrests, table legs, nozzle openings ), complex mechanical features (e.g., gear teeth, connectors, threads ), and overall proportions with high fidelity.

Figure 17: More qualitative text-to-CAD results on the test set. Each row shows an input text description (bottom) and the corresponding CAD geometry generated by DreamCAD (top). DreamCAD successfully reconstructs shapes from complex text prompts, ranging from mechanical parts ( gear assemblies, flange plates, couplings ), tools ( pyramidal axe head ), and structural elements ( DIN rail, stepped platforms ), to free-form designs ( seat frames, turbine blades ), and consumer objects ( USB cable ).

Text-to-CAD Generation

Text-to-CAD generation turns natural-language design intent into executable, editable parametric 3D models and assemblies.

See all 48 papers

GuideCAD: A Lightweight Multimodal Framework for 3D CAD Model Generation via Prefix Embedding
Jun 5, 2026Minseong Kim, Jinyeong Park, Sungho Park +13D Asset Generation Parametric CAD Modeling
Convergence Research Center for Insect Vectors Incheon, South Korea · Department of Computer Science and Engineering Incheon National University Incheon, 22012, South Korea · Department of Computer Science and Engineering Center for Brain-Machine Interface Incheon National University Incheon, South Korea
Zero-to-CAD: Agentic Synthesis of Interpretable CAD Programs at Million-Scale Without Real Data
Apr 27, 2026Mohammadmehdi Ataei, Farzaneh Askari, Kamal Rahimi Malekshan +1Synthetic Data Generation Parametric CAD Modeling
Autodesk Research
Towards High-Fidelity CAD Generation via LLM-Driven Program Generation and Text-Based B-Rep Primitive Grounding
Mar 12, 2026Jiahao Li, Qingwang Zhang, Qiuyu Chen +3Text-to-CAD Generation Parametric CAD Modeling
Fudan University · Shanghai Jiao Tong University

Jun 5, 2026J/K move · Enter open · S save

GuideCAD: A Lightweight Multimodal Framework for 3D CAD Model Generation via Prefix Embedding

Minseong Kim, Jinyeong Park, Sungho Park, Jibum Kim

Convergence Research Center for Insect Vectors Incheon, South Korea · Department of Computer Science and Engineering Incheon National University Incheon, 22012, South Korea · Department of Computer Science and Engineering Center for Brain-Machine Interface Incheon National University Incheon, South Korea

3D Asset Generation Parametric CAD Modeling Prompt Tuning Multimodal Generation Text-to-CAD Generation Prefix Tuning

Open paper AskPDF arXiv

Multi-modal approaches used for 3D CAD generation require substantial computational resources, necessitating efficient training. To address this, we propose GuideCAD, which leverages semantically rich visual-textual representations having only a small number of trainable parameters to generate 3D CAD models. Specifically, GuideCAD uses a mapping network that converts image embeddings into prefix embeddings, enabling a pretrained large language model to integrate visual and textual information. As a result, a transformer-based decoder predicts the construction sequence using the visual-textual embeddings in order to generate the 3D CAD model. For experimental evaluation, we construct a new dataset, referred to as GuideCAD, which consists of text-image pairs. Each pair includes a text prompt that represents a 3D CAD construction sequence and its corresponding 3D CAD image. Our experimental results show that GuideCAD generates comparably high-quality 3D CAD models while using approximately four times fewer parameters and achieving twice the training efficiency compared to fine-tuning approaches. We have released the source code and dataset for our method at: https://github.com/mskimS2/GuideCAD

Abstract

Text-to-CAD Generation

Explore similar work

GuideCAD: A Lightweight Multimodal Framework for 3D CAD Model Generation via Prefix Embedding

Zero-to-CAD: Agentic Synthesis of Interpretable CAD Programs at Million-Scale Without Real Data

Towards High-Fidelity CAD Generation via LLM-Driven Program Generation and Text-Based B-Rep Primitive Grounding