Automatic CAD generation is commonly evaluated with volumetric IoU, Chamfer Distance, and surface F-score, which barely penalize local errors that occupy only a small fraction of a part's volume or surface area, such as a misplaced hole or a slot on the wrong face, even though such errors are critical to manufacturing. We formalize these metrics in a unified integral form and show that the weight of a local error decays as the (k+q)-th power of its size, where k is the dimension of the integration domain and q is determined by the penalty function. Guided by this analysis, we propose DrawScore, which projects both models into three orthographic views with visible and hidden lines, as in engineering drawings, and applies a saturating penalty along the projected curves, so that the weight of a local error decays only linearly. Controlled experiments confirm the predicted decay orders, and on outputs of existing CAD generation agents, DrawScore exposes errors in outputs that IoU rates as nearly perfect. DrawScore further improves generation as a reinforcement learning reward and, for CAD generation from engineering drawings, serves as a test-time verifier without 3D ground truth or an additional reward model. Our code is available at https://github.com/UESTC-Simon/DrawScore.
Figures & tables
Figure 1: Comparison of two candidate CAD models generated by an existing CAD generation agent against the ground truth.
Metric
Integration domain
Penalty ρ(d)
Score
Decay order
Supports TTS
Volumetric IoU [ 20 ]
Solid ( k=3 )
1[d>0]
RP/(R+P−RP)
3
✗
Chamfer Distance [ 20 ]
Surface ( k=2 )
d/L
D→+D←
3
✗
Surface F-score [ 24 ]
Surface ( k=2 )
1[d≥τ]
2RP/(R+P)
2
✗
Edge Chamfer Distance [ 4 ]
Model edges ( k=1 )
d/L
D→+D←
2 ∗
✗
DrawScore (ours)
Projected curves ( k=1 )
1−e−d2/(2σ2)
minv2RvPv/(Rv+Pv)
1
✓
Table 1: Common geometric similarity metrics as instances of Eq. 1 . Score : aggregation of the two directions; lower is better for CD and ECD. Decay order : the weight of a local error of size s scales as (s/L)order ; ∗ Decreases to 1 if no corresponding curve lies near the error. Supports TTS : whether the metric can score candidates against an input drawing.
Figure 2: Sensitivity of geometric similarity metrics to different local errors. In each column, the top row shows an error to which the metric in bold is sensitive, and the bottom row an error to which it is insensitive.
Figure 3: Weight of a local error versus its size on the blind-hole sweep of RQ1. Left: the removed blind hole of radius r∈{1,2,4,8,12,16} mm and depth 2r . Right: w is the metric value for CD and ECD and 1−score for the other metrics.
Figure 4: Representative controlled errors of RQ1: (a) tests the choice of curves (Finding 1) and (b) the saturating penalty (Finding 2). The box on the left marks the error and is magnified on the right. p is the error fraction of Eq. 2 on the solid ( k=3 ), the surface ( k=2 ), and the projected curves ( k=1 ).
Figure 5: Bidirectional comparison of IoU and DrawScore on agent outputs (RQ2). Blue shows DrawScore for outputs with IoU ≥0.95 , and orange shows IoU for outputs with DrawScore ≥0.95 . Insets compare the GT and candidate at the marked percentiles within each subset. The horizontal axis is clipped at 0.70.
Figure 6: Sensitivity of DrawScore to the positional tolerance σ ; the gray band marks the default σ=0.005L . (a) Fitted decay order k+q on the blind-hole sweep of RQ1 ( Fig. 3 ); the dashed line is k+q=1 . (b) Spearman correlation between DrawScore at each σ and at the default over the 1,188 agent outputs of RQ2.
Producing an executable CAD script is not sufficient to establish that an agent has satisfied an engineering design request. Evaluation must also consider the resulting geometry, specified dimensions, and native CAD modeling history. We present Parametric CAD Bench v3, a 100-task benchmark comprising 30 text-to-CAD creation tasks, 30 text-to-CAD create-and-edit tasks, and 40 tasks that reconstruct mechanical parts from engineering drawings. Agents operate in a pinned FreeCAD environment and submit native CAD artifacts to a separate programmatic verifier. The primary metric is mean continuous task reward, combining geometry similarity and specification consistency rather than binary task success. A September 24, 2026 snapshot covers 13 model-agent systems; Claude Code with Claude Opus 5.5 achieves the highest overall point estimate, 61.03%, and an image-to-CAD mean reward of 84.66%. These measurements describe complete systems, not isolated model capabilities. We release versioned tasks, evaluation configurations, and inspectable result artifacts.
Text-to-CAD has recently emerged as an important task with the potential to substantially accelerate design workflows. Despite its significance, there has been surprisingly little work on Text-to-CAD evaluation, and assessing CAD model generation performance remains a considerable challenge. In this work, we introduce a new evaluation perspective for Text-to-CAD based on automated testing. We propose CADTestBench, the first test-based benchmark for Text-to-CAD, based on CADTests, executable software tests that verify whether a generated CAD model satisfies the geometric and topological requirements of the input prompt. Using CADTestBench, we conduct comprehensive benchmarking of recent Text-to-CAD methods and further demonstrate that CADTests can also guide CAD model generation, yielding simple baselines that surpass performance of current methods. CADTestBench code and data are available at GitHub and Hugging Face dataset.
Dimitrios Mallis, Marco Wang, Ahmet Serdar Karadeniz +3
SnT, University of Luxembourg · University of Trento · Fondazione Bruno Kessler
Engineering drawings specify mechanical parts through orthographic projections. Recognizing which part a drawing depicts underlies part search, design reuse, and digitization, yet no public benchmark measures whether vision models can do it. We introduce Blueprint2CAD, a benchmark for drawing-to-render correspondence: given an orthographic line drawing and a shaded render, usually from another viewpoint, decide whether they show the same part. Its data layer, BlueCAD-30K, applies an open drafting pipeline to 29,925 Zero-to-CAD models, adding view-aligned renders, clean and dimensioned drawings from six viewpoints, and surface-type metadata. Frozen encoders retrieve the matching part from a 10,000-model gallery with up to 65% Recall@1 when viewpoints are known but 43% when they are not, and the four encoders we test lie within six points of each other: viewpoint, not backbone, is the bottleneck. Six zero-shot vision-language models reach only 0.52-0.67 verification accuracy, with model-specific Yes/No biases that chain-of-thought shifts but does not remove; a quick LoRA fine-tuning on in-domain pairs improves verification accuracy by a few points but sharpens these biases and leaves the viewpoint failure intact.
Alexander Valverde, Danny Xie-Li, Zhichao Wang +1
Novineer · Instituto Tecnol´ogico de Costa Rica · University of Würzburg