MIRAGE-CAD: Construction-Mediated Multimodal Generation of Executable CAD Programs

Jizong Zhan

cs.CVAug 24, 2026

MIRAGE-CAD: Construction-Mediated Multimodal Generation of Executable CAD Programs

Authors: Jizong Zhan

Abstract

Recovering an executable parametric CAD program from an observed object is fundamentally ambiguous, because the same final geometry can result from different construction procedures. We study this problem from four types of input: natural-language descriptions, rendered images, point clouds, and STEP/B-Rep geometry. MIRAGE-CAD maps each input to a shared construction representation and mediates program generation through an explicit construction-plan interface. The resulting Python CAD code is executed by an OpenCASCADE kernel to build the solid and export it as STEP. On 2,500 held-out queries per modality, the system achieves 55.4-70.0% build success and 52.3-66.2% STEP export success without retrieval at inference. Controlled comparisons show that strong reconstruction does not depend on expressing the construction representation as text: a decoder conditioned directly on the continuous representation also reconstructs strongly, while an exposure-matched plan-based decoder shows no detected material loss in per-part geometric fidelity. The explicit plan instead provides a readable and separately measurable intermediate representation whose agreement with the reference construction is informative about downstream execution success. Finally, we show that executable validity, geometric fidelity, and parametric responsiveness can diverge substantially and should therefore be evaluated separately.

Figures & tables

Figure 1: Overview of the MIRAGE-CAD architecture. One input modality is active for each query. Its encoder

f_{m}

produces

z_{m}

, which is mapped by the modality-specific prior

\pi_{m}

to the predicted construction latent

\hat{z}_{\mathrm{ir}}

. The prefix adapter

\Psi

converts this latent into a continuous prefix for the plan decoder, and the generated plan is then passed to the code decoder to produce an executable Python CAD program. A query-derived observation block also conditions the decoders. It is empty for image queries, so the plan forms a strict information bottleneck in that case. Text and STEP/B-Rep queries retain additional observation information, while point-cloud queries use a placeholder at the plan stage and normalised statistics at the code stage. The dashed retrieved/reference-plan branch is used only for evaluation and is not part of deployed generation.

Modality	Observation content	Absolute scale
Text	Query description (up to 400 characters)	When explicitly stated
Image	Empty	No
Point cloud	Normalised shape and PCA statistics	No
STEP/B-Rep	Kernel-derived geometric statistics	Yes

Table 1: Query-derived observation information used by the decoders.

Stage	Purpose	Trainable modules	Objective
1	Representation alignment	Projection heads, point/STEP encoders, and $f_{\mathrm{ir}}$	$\mathcal{L}_{\mathrm{align}}$
2	Latent prior fitting	Four modality-specific priors $\pi_{m}$	$\mathcal{L}_{\mathrm{prior}}$
3	Plan-decoder adaptation	Prefix adapter $\Psi$ and LoRA-IR	Plan NLL
3b	Cross-modal continuation	Prefix adapter $\Psi$ and LoRA-IR	Plan NLL
4	Code-decoder training	LoRA-Code	Program NLL
4b	Predicted-plan continuation	LoRA-Code	Program NLL

Table 2: Training schedule of MIRAGE-CAD. Stages 3b and 4b continue the preceding checkpoints rather than starting new models.

Arm	Construction conditioning	Explicit plan	Role
B0	None	No	Direct query-to-code baseline
B1a	Continuous $\hat{z}_{\mathrm{ir}}$	No	One-epoch direct-latent arm
B1b	Continuous $\hat{z}_{\mathrm{ir}}$	No	Extended direct-latent arm
B2	Latent followed by plan	Generated	Deployed MIRAGE-CAD
B2-Pred	Latent followed by plan	Generated	One-epoch plan arm
B3	Retrieved training construction	Retrieved	Retrieval reference

Table 3: Experimental conditions used in the main comparisons.

ID	Configuration	Syntax (%)	Build (%)	STEP Export (%)
B0	Direct query $\rightarrow$ code	97.6	35.4	34.6
B2	Generated plan $\rightarrow$ code	94.5	70.0	66.2

Table 4: STEP results on the 2,500-sample held-out set. B0 removes the construction-mediated pathway, while B2 is deployed MIRAGE-CAD.

Figure 2: Comparisons used to separate construction conditioning, training exposure, and textualisation. B0–B1a tests construction conditioning without plan text. B2–B2-Pred tests the effect of matching code-decoder training to predicted-plan inference. B1a–B2-Pred gives the closest comparison between continuous and textual construction interfaces. These are not factorial ablations: training budgets differ in the first two comparisons, while B1a–B2-Pred retains a small checkpoint difference (

3{,}000

versus

3{,}125

updates) and an interface asymmetry.

Metric	B1a latent	B2-Pred plan	B2 deployed
Syntax (%)	95.8	95.8	95.2
Build (%)	86.2	90.4	71.4
STEP export (%)	85.6	90.0	67.4
Unconditional F@1 / $F_{\mathrm{self}}$ (%)	73.0	76.8	53.6
Bbox within $\pm 10\%$ (%)	94.4	97.1	90.6

Table 5: Continuous construction conditioning and textual plan conditioning on the same 500 STEP samples. B1a and B2-Pred form the closest comparison, evaluated at 3,000 and 3,125 updates, respectively; B2 is the deployed curriculum reference at approximately 4,307 updates.

Figure 3: Diagnostic value of agreement at the construction-plan interface. Panel A reports AUROC for Build and STEP-export success on the

500

exposure-matched samples. Panel B reports Spearman correlations with Chamfer distance and F@1 on the

450

samples with scoreable geometry. Negative correlation is favourable for Chamfer distance and positive correlation for F@1. Whiskers show 95% paired bootstrap intervals. Plan-based diagnostics require the reference construction and are therefore available only for offline evaluation, not as deployment-time confidence scores.

Diagnostic	Build AUROC $\uparrow$	Export AUROC $\uparrow$	$\rho$ with CD	$\rho$ with F@1
Latent cosine	0.484	0.482	$\mathbf{-0.133}$	$-0.051$
Plan cosine	0.708	0.700	$+0.022$	$+0.179$
Plan Op-Set F1	0.727	0.725	$-0.004$	$\mathbf{+0.340}$
Plan Op-Seq LCS	0.727	0.724	$-0.002$	$\mathbf{+0.357}$

Table 6: Diagnostic value of plan-level agreement. AUROC is computed on all 500 exposure-matched STEP rows; geometric correlations use the 450 rows with both CD and F@1 scores.

Figure 4: Prefix and observation interventions on STEP queries. Panel A reports Build for the four combinations of correct or shuffled construction prefix and present or suppressed observation. Each condition includes all

500

queries. Panel B shows the median produced-to-target bounding-box diagonal ratio before and after observation suppression for the correct- and shuffled-prefix conditions. The two comparisons use their own jointly scoreable subsets (

n=211

and

n=200

). Observation is suppressed only at inference; the decoders were trained with the observation block present.

Modality	Syntax (%)	Prog-Op-F1 (%)	Build (%)	STEP Export (%)
Text	98.3	80.1	57.2	53.1
Image	94.3	79.1	57.8	52.3
Point cloud	89.5	73.9	55.4	52.6
STEP/B-Rep	94.5	85.1	70.0	66.2

Table 7: Generated-path results on the four held-out modalities,

2{,}500

queries each.

MIRAGE pathway	Coverage	Paired CD	Paired IoU
Generated plan	226/400 vs 392/400	55.78 / 0.117 ( $n=222$ )	0.0604 / 0.9226 ( $n=200$ )
Prior-NN-IR	392/400 vs 392/400	17.59 / 0.187 ( $n=385$ )	0.1621 / 0.9426 ( $n=369$ )

Table 8: External point-cloud positioning on 400 Fusion 360 parts. Coverage is reported as MIRAGE versus CAD-Recode over all inputs. CD and IoU values are paired medians on the subsets scoreable by both systems for the corresponding metric.

Figure 5: Coverage and fidelity answer different questions. Panel A compares the generated and retrieved STEP pathways on the family-held-out split. Among the

1{,}330

parts scoreable by both arms, generation has the higher conditional median F@1. When all

2{,}923

parts are included and non-scoreable outputs are assigned zero, retrieval has the higher mean. Panel B compares retrieval-assisted MIRAGE with CAD-Recode on Fusion 360 parts. Both produce measurable geometry for

392/400

inputs, but CAD-Recode has a much higher paired median IoU on the

369

parts scoreable by both systems.

Parts	Perturbations	Rebuild-valid (%)	Responsive (%)	Reachability (%)
74	1,608	99.8	82.2	29.3

Table 9: Parametric behaviour of the generated STEP pathway.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Operation	Documents	Document coverage (%)	Total occurrences
OP_HOLE_PATTERN	22,094	88.38	24,303
OP_TRANSLATE	9,623	38.49	15,545
OP_BOOLEAN_JOIN	11,667	46.67	15,065
OP_SKETCH	11,254	45.02	13,287
OP_BOX	8,255	33.02	9,637
OP_PLATE	7,905	31.62	7,905

Appendix

Table A.1 : Complete Construction IR operation vocabulary over the

25{,}000

STEP-modality training samples. Documents counts samples containing the operation at least once; document coverage is the corresponding percentage of the training split. Total occurrences counts all associated F statements.

Slots	Count	Content
0	1	Validity : brep_valid , represented as 0 or 1.
1–6	6	Topology counts : numbers of solids, shells, faces, wires, edges, and vertices.
7–9	3	Bounding box : extents along the three coordinate axes.
10	1	Surface area : total B-Rep surface area.
11	1	Volume : solid volume.
12–22	11	Surface-type counts : plane , cylinder , cone , sphere , torus , bezier_surface , bspline_surface , surface_of_revolution , surface_of_extrusion , offset_surface , and other_surface .

Appendix

Table B.1 : Layout of the

50

-dimensional global STEP/B-Rep descriptor. Slot indices are zero-based and follow the extractor order. All entries except the validity flag are transformed using

\log(1+x)

Modality	before (%)	after (%)	$\Delta$
STEP	74.3 [72.6, 76.0]	70.0 [68.1, 71.7]	$-4.4$
Point cloud	29.9 [28.1, 31.7]	55.4 [53.5, 57.4]	$+25.6$
Text	28.5 [26.7, 30.3]	57.2 [55.3, 59.1]	$+28.7$
Image	38.8 [36.9, 40.8]	57.8 [55.8, 59.7]	$+18.9$
Average	42.9	60.1	$+17.2$

Appendix

Table C.1 : Effect of Stage 3b on Build, with

2{,}500

held-out rows per modality and 95 % Wilson intervals. The

\Delta

column is computed from unrounded values.

Prefix source	plan cos	Op-Set F1 (%)	Op-Seq LCS (%)	Build (%)
zero	0.057	38.1	28.3	—
constant	0.082	35.9	26.8	—
shuffled	0.047	34.6	26.0	68.6 [64.4, 72.5]
deployed	0.886	88.0	80.1	71.4 [67.3, 75.2]
oracle	0.898	88.7	79.8	69.2 [65.0, 73.1]

Appendix

Table D.1 : Prefix intervention on

500

paired STEP queries. Zero supplies

K

zero vectors with

\Psi

bypassed; constant is

\Psi(\mathbf{0})

; shuffled uses another sample’s prefix; deployed uses the predicted latent from

\pi_{m}

; and oracle encodes the reference plan before applying

\Psi

. Build is reported with a 95 % Wilson interval and was not evaluated for the two synthetic-prefix conditions.

Condition	Plan	Obs.	STEP Build (%)	Text Build (%)
C3 deployed	Yes	Yes	71.4 [67.3, 75.2]	59.6 [55.2, 63.8]
C2 plan-only	Yes	No	58.0 [53.6, 62.2]	58.0 [53.6, 62.2]
C1 observation-only	No	Yes	0.0	0.2
C0 neither	No	No	0.0	0.0

Appendix

Table D.2 : Observation intervention on the same 500 held-out rows. Suppression is applied at both decoder inputs. Build is reported with a 95% Wilson interval. C1 and C0 are inference-time suppression conditions and should not be interpreted as retrained architectures.

Modality	Variant	Op-Set F1 (%)	Build (%)	STEP Export (%)	plan cos
STEP	A	79.2	100.0	100.0	0.634
	B	76.6	100.0	99.97	0.534
	C	64.4	46.7	46.0	0.499
Text	A	53.7	98.1	98.1	0.342
	B	54.4	98.3	98.3	0.328
	C	45.5	45.0	41.8	0.125

Appendix

Table E.1 : Family-held-out structural diagnostic on the

2{,}923

held-out-family samples. Variant A is Direct-NN-IR, B is Prior-NN-IR, and C is the generated pathway.

Accounting	$n$	Retrieved	Generated	Wins (Gen.:Ret.)	$p$
Conditional	1,330	0.111 (45.6%)	0.162 (66.2%)	982:347	$1.7\times 10^{-70}$
Unconditional	2,923	0.099 (40.6%)	0.066 (26.9%)	983:1,939	$4.1\times 10^{-71}$

Appendix

Table E.2 : Two accountings of the same STEP comparison on the family-held-out split. Conditional uses the 1,330 parts scoreable in both arms and reports a median. Unconditional uses all 2,923 parts, assigns zero to non-scoreable outputs for this accounting only, and reports a mean. Percentages of

F_{\mathrm{self}}

are shown in parentheses.

Pathway	Conditioning at inference	Autonomous
CAD-Recode	Point cloud $\rightarrow$ code	Yes
MIRAGE generated plan	Point cloud $\rightarrow$ construction latent $\rightarrow$ plan $\rightarrow$ code	Yes
MIRAGE Prior-NN-IR	Plan retrieved from the training index at inference	No

Appendix

Table F.1 : Information available to each pathway in the external point-cloud comparison.

Pathway	CADR	MIRAGE	Both	CADR only	MIR. only	Neither	$p$
Generated plan	392/400	226/400	222	170	4	4	$3.2\times 10^{-45}$
Prior-NN-IR	392/400	392/400	385	7	7	1	1.0

Appendix

Table F.2 : External measurable-geometry coverage over all 400 Fusion 360 inputs. The count columns report parts measurable by both systems, by CAD-Recode only, by MIRAGE only, and by neither. McNemar tests are exact. CADR denotes CAD-Recode.

Pairing	$n$	MIRAGE	CADR	$\Delta$	95% CI	MIR. wins	$p$
Gen.-plan CD	222	55.78	0.117	$+55.64$	$[+49.29,+63.84]$	1/222	$6.6\times 10^{-65}$
Gen.-plan IoU	200	0.0604	0.9226	$-0.785$	$[-0.810,-0.751]$	5/200	$1.2\times 10^{-50}$
Prior-NN-IR CD	385	17.59	0.187	$+17.28$	$[+15.51,+19.84]$	5/385	$1.8\times 10^{-105}$
Prior-NN-IR IoU	369	0.1621	0.9426	$-0.684$	$[-0.704,-0.658]$	10/369	$2.8\times 10^{-91}$

Appendix

Table F.3 : External paired fidelity on jointly scoreable subsets.

\Delta\mathrm{CD}=\mathrm{CD}_{\text{MIRAGE}}-\mathrm{CD}_{\text{CAD-Recode}}

, so positive values favour CAD-Recode.

\Delta\mathrm{IoU}=\mathrm{IoU}_{\text{MIRAGE}}-\mathrm{IoU}_{\text{CAD-Recode}}

, so positive values favour MIRAGE. The reported

\Delta

is the paired median of per-part differences. Exact ties are excluded from the sign tests. CADR denotes CAD-Recode.

Implementation detail	Consequence	Possible correction
The point-cloud input is divided by its maximum radius before encoding	Absolute scale is removed before the encoder; the produced-to-target bounding-box ratio is 0.695 internally and 0.339 on the cross-source set.	Provide the pre-normalisation scale as an additional input and retrain from the alignment stage.
The point-cloud path contains a train–inference prompt inconsistency in the cross-modal continuation	Point-cloud plan quality is confounded by this prompt mismatch; its magnitude and direction are not isolated.	Correct the prompt and retrain the cross-modal continuation.
Prog-Op-F1 matches operation identifiers wherever they occur	61.6% of matched identifiers are parameter names inside string literals; the metric’s absolute value should not be interpreted as operation-call accuracy.	Restrict scoring to canonical operation invocations or operation families; this requires rescoring but not retraining.
Each trained condition is represented by a single run	Small differences between trained configurations include unmeasured seed-to-seed variation.	Repeat the comparisons supporting the headline claims across multiple training seeds.
Stages 3, 4, and 4b use STEP rows and their adapters are reused across modalities	The multimodal results combine modality effects with a decoder-training distribution that is not symmetric across modalities.	Train or continue the code decoder with modality-balanced conditioning, or evaluate modality-specific decoder adaptations.
The STEP face, edge, and relation streams contain no independent per-entity records in the present corpus	Reported STEP results effectively depend on the 50-dimensional global descriptor (Appendix B.2 ); they do not establish the performance of a learned entity-level B-Rep encoder.	Export per-entity descriptors during corpus construction and retrain from the alignment stage.

Appendix

Table G.1 : Implementation-specific limitations and the changes required to address them.

Parametric CAD Modeling

Parametric CAD modeling represents 3D designs as editable, constraint-driven construction histories and develops methods to generate, reconstruct, and revise them.

See all 81 papers

StepCAD: Mesh-to-CAD Code Generation via LLM Policy and Geometry-Guided Search
Oct 1, 2026Ghadi Nehme, Faez AhmedParametric CAD Modeling Program Synthesis
Department of Mechanical Engineering Massachusetts Institute of Technology Cambridge, MA 02139
CADFit: Precise Mesh-to-CAD Program Generation with Hybrid Optimization
May 2, 2026Ghadi Nehme, Eamon Whalen, Faez Ahmed3D Reconstruction Parametric CAD Modeling
Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Siemens Digital Industries Software, Plano, TX 75024, USA
CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation
May 11, 2026Anna C. Doris, Jacob Thomas Sony, Ghadi Nehme +33D Reconstruction Parametric CAD Modeling
Massachusetts Institute of Technology

Oct 1, 2026J/K move · Enter open · S save

StepCAD: Mesh-to-CAD Code Generation via LLM Policy and Geometry-Guided Search

Ghadi Nehme, Faez Ahmed

Department of Mechanical Engineering Massachusetts Institute of Technology Cambridge, MA 02139

Parametric CAD Modeling Program Synthesis 3D Reconstruction 3D Mesh Reconstruction

Open paper AskPDF arXiv

Recovering executable CAD programs from 3D meshes is challenging due to the compositional nature of CAD construction and the interaction between discrete modeling choices and continuous parameters. Many learning-based methods predict complete programs in a single pass and rely predominantly on sketch-extrude representations, limiting operation diversity and opportunities to correct geometric errors during reconstruction. We introduce StepCAD, a generative optimization approach that combines a state-conditioned CAD policy with geometry-guided search. Given an input mesh, the policy predicts construction actions conditioned on both target and intermediate geometry, and an IoU-guided tree search refines the resulting program through local edits. We also introduce ARCADE-1.5M, a large-scale dataset of 1.5M executable CAD programs spanning diverse operations, sequences with a maximum length of 150+ counted operations, and 12.5M intermediate state-action transitions. Experiments across multiple CAD reconstruction benchmarks show that StepCAD achieves state-of-the-art geometric reconstruction accuracy with consistently high validity, yielding up to 87.2% relative IoU improvement over the strongest evaluated baseline, with particularly large gains on complex shapes. Project page: https://ghadinehme.com/stepcad.github.io/

Abstract

Parametric CAD Modeling

Explore similar work

StepCAD: Mesh-to-CAD Code Generation via LLM Policy and Geometry-Guided Search

CADFit: Precise Mesh-to-CAD Program Generation with Hybrid Optimization

CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation