CANTO: CAD-Native Transformer Operators for AI-Aided Engineering

Daniel Leibovici; Nikola Borislavov Kovachki; Dawon Ahn; Ruben Ohana; Ira J. S. Shokar; Abouzar Ghasemi; Semih Akkurt; Rishikesh Ranade

cs.AISep 29, 2026

CANTO: CAD-Native Transformer Operators for AI-Aided Engineering

Authors: Daniel Leibovici, Nikola Borislavov Kovachki, Dawon Ahn, Ruben Ohana, Ira J. S. Shokar, Abouzar Ghasemi, Semih Akkurt, Rishikesh Ranade, +3 more

Organizations: NVIDIA

Abstract

Modern engineering systems, from automobiles to aircraft, are designed by using precise, continuous parametric computer-aided design (CAD) models. Evaluating design changes through numerical simulation requires meshing the continuous geometry, a computationally expensive and often brittle process that can require manual intervention and replaces the continuous representation with a discrete approximation. Most neural surrogates accelerate the simulation, but inherit this representation gap by relying on meshes, point clouds, voxels, or other sampled approximations of geometry. We introduce CANTO, a transformer neural operator that maps directly from continuous CAD geometry to physical fields, without meshing the input geometry. We develop a theoretical framework for learning operators from geometric manifolds to function spaces of physical fields, representing geometry through sequences of parametric patches. CANTO instantiates this framework by directly tokenizing non-uniform rational B-spline (NURBS) patches from their control points, knot vectors, and weights, and predicts continuous surface and volume fields at arbitrary query locations. We evaluate CANTO on four automotive and aircraft aerodynamics industry benchmarks: AhmedML, WindsorML, DrivAerML, and HiLiftAeroML. CANTO achieves state-of-the-art accuracy on most evaluated surface and volume prediction tasks, including a 19.8% reduction in surface-pressure relative $L_2$ error compared with AB-UPT on HiLiftAeroML. Differentiability with respect to CAD parameters further enables gradient-based inverse design of designs. On AhmedML, CANTO identifies designs with 4.4 to 20.4% lower drag than the best dataset designs satisfying the same volume and lift constraints, with the improvements verified using the same CFD setup used to generate the original dataset.

Figures & tables

Figure 1: Overview of our approach. CANTO obviates the need for meshing step used by traditional AI surrogate pipelines by instead directly tokenizing continuous CAD NURBS geometries. First, the original CAD geometry is expressed as a sequence of NURBS surfaces, before being encoded by their control points, knot vectors, and weights. These parametrized NURBS surfaces are then tokenized with a Perceiver-based encoder and used to condition our neural operator to predict the target physical fields.

	AhmedML				DrivAerML			WindsorML			HiLiftAeroML
Method	$\boldsymbol{p}_{s}$	$\boldsymbol{u}$	$\boldsymbol{\omega}$	$\boldsymbol{\tau}$	$\boldsymbol{p}_{s}$	$\boldsymbol{u}$	$\boldsymbol{\tau}$	$\boldsymbol{p}_{s}$	$\boldsymbol{u}$	$\boldsymbol{\tau}$	$\boldsymbol{p}_{s}$	$\boldsymbol{u}$	$\boldsymbol{\tau}$
PointNet	8.02	5.44	66.04	10.09	23.63	28.13	41.85	–	–	–	–	–	–
Graph U-Net	6.46	4.15	53.66	7.29	16.13	17.98	27.84	–	–	–	–	–	–
GINO	7.90	6.23	71.81	8.18	13.03	40.58	21.71	–	–	–	–	–	–
LNO	12.95	7.59	72.49	11.50	20.51	23.27	36.44	–	–	–	–	–	–
UPT	4.25	2.73	15.03	5.80	7.44	8.74	12.93	–	–	–	–	–	–

Table 1: Comparison of CANTO with prior methods. We report the relative L2 errors (%) for the surface pressure

\boldsymbol{p}_{s}

, volume velocity

\boldsymbol{u}

, volume vorticity

\boldsymbol{\omega}

, and wall shear stress

\boldsymbol{\tau}

fields on the AhmedML, DrivAerML, WindsorML, and HiLiftAeroML datasets.

Method	$\boldsymbol{p}_{s}$	$\boldsymbol{u}$	$\boldsymbol{\omega}$	$\boldsymbol{\tau}$
Fixed-size	2.50	2.27	5.80	3.54
Variable-size	2.66	2.04	4.83	3.53

Table 2: Evaluation of CANTO with fixed-size and variable-size NURBS features on the AhmedML dataset.

Figure 2: Robustness to sampling procedure on the HiLiftAeroML dataset. Here, uniform-on-mesh denotes anchors sampled uniformly from the surface mesh points, as in AB-UPT, while uniform-on-NURBS surfaces samples anchors uniformly in

(u,v)

parameter space on each NURBS surface, with the number of anchors allocated to each surface proportional to its area.

Figure 3: Optimized Pareto front via inverse design. Grey and blue markers show the ground-truth drag of training and held-out designs satisfying

C_{l}\leq 0.05

, while the blue staircase denotes the best dataset design satisfying each volume floor

V\geq V_{0}

. The dashed and solid red curves show the CANTO-predicted optima and their CFD validations, respectively. At every volume floor, CANTO identifies valid designs beyond the dataset benchmark front, with predictions closely matching CFD.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: CANTO geometry encoder architecture. NURBS surfaces are represented by their control points, weights, and knot vectors. Three Perceiver Pooling modules independently encode the weighted control points and the two knot vectors. Their outputs are concatenated and projected to produce one fixed-dimensional token per patch. Self-attention across patch tokens incorporates global context from the complete CAD geometry.

Parameter	AhmedML	WindsorML	DrivaerML	HiLiftAeroML
$q^{\mathcal{P}}_{cp}$	$8$	$16$	$4$	$16$
$d^{\mathcal{P}}_{cp}$	$128$	$192$	$128$	$192$
$K^{\mathcal{P}}_{cp}$	$4$	$2$	$4$	$2$
$q^{\mathcal{P}}_{kv}$	$4$	$4$	$4$	$4$
$d^{\mathcal{P}}_{kv}$	$64$	$64$	$64$	$64$
$K^{\mathcal{P}}_{kv}$	$1$	$2$	$1$	$2$

Appendix

Table 3: Geometry encoder hyper-parameters.

Parameter
Hidden dim.	$192$
Attention heads	$3$
Surface decoder depth	$6$
Volume decoder depth	$6$

Appendix

Table 4: Physics and decoder blocks hyper-parameters.

Parameter	AhmedML	DrivAerML	WindsorML	HiLiftAeroML
Learning rate	$5{\times}10^{-5}$	$1{\times}10^{-4}$	$3{\times}10^{-4}$	$2{\times}10^{-4}$
Weight decay	$0.05$	$0.05$	$0.05$	$0.05$
Training steps	$200\text{k}$	$200\text{k}$	$105\text{k}$	$630\text{k}$
Warmup steps	$1,000$	$2,500$	$675$	$31,500$
Volume loss weight	$0.2$	$0.2$	$0.2$	$0.2$

Appendix

Table 5: Training hyper-parameters.

Dataset	All	Geometry	Non-geometry
AhmedML	12.61M	4.82M	7.79M
WindsorML	13.47M	5.46M	8.01M
DrivAerML	17.17M	9.24M	7.93M
HiLiftAeroML	13.47M	5.46M	8.01M

Appendix

Table 6: Model parameter counts across datasets. Parameter counts are separated into geometry-related and non-geometry parameters.

Figure 5: Effect of the NURBS anchor split resolution on predictive accuracy. Error drops sharply as the exact geometric context set is parametrically refined, plateauing at 4 splits, the level used for training.

Method	$\boldsymbol{p}_{s}$	$\boldsymbol{u}$	$\boldsymbol{\tau}$
CANTO-PA	8.04	7.30	7.55
CANTO	6.81	6.69	5.46

Appendix

Table 7: Ablation of CANTO variants on the WindsorML dataset.

Figure 6: Robustness to anchor sparsity for CANTO-PA . Retaining point-based surface anchors causes performance to degrade when the number of surface anchors is reduced (a), even though the global CAD encoding provides strong robustness against volume sparsity (b).

Figure 7: Robustness to anchor sparsity for surface fields. Relative L2 error is reported for pressure (

p_{s}

) and wall shear stress (

\tau

) against varying the number of surface anchors (left two panels) and volume anchors (right two panels). CANTO completely bypasses the need for spatial surface anchors (shown as a flat dashed line). Furthermore, it maintains high accuracy on these surface fields even at lower volume anchor counts where the AB-UPT baseline degrades sharply.

Figure 8: Robustness to anchor sparsity for volume fields. Relative L2 error is reported for volume velocity (

u

) and volume vorticity (

\omega

) against varying surface anchor counts (left two panels) and volume anchor counts (right two panels). CANTO’s performance on volumetric fields is completely decoupled from the surface anchor sampling resolution (represented by the flat dashed line), and exhibits superior robustness to volume sparsity compared to AB-UPT.

Dataset	Hardware	Precision	Epochs	Training time
AhmedML	$8\times$ H100	FP32	1140	17 h
WindsorML	$8\times$ H100	FP32	1326	13 h
DrivAerML	$4\times$ GB200	FP16 (mixed)	500	11 h
HiLiftAeroML	$4\times$ GB200	FP16 (mixed)	500	69 h

Appendix

Table 8: Computational resources and training time per dataset. All runs use a single node; per-step throughput is dominated by attention over the volume-anchor set.

Dataset	Hardware	Peak mem	Surf. pts	Vol. pts	Inference time	Throughput
AhmedML	H100	13 GB	$\sim$ 1M	$\sim$ 22M	$\sim$ 75 s	$\sim$ 0.3M queries/s
WindsorML	H100	55 GB	$\sim$ 5M	$\sim$ 290M	$\sim$ 330 s	$\sim$ 0.9M queries/s
DrivAerML	GB200	40 GB	$\sim$ 9M	$\sim$ 150M	$\sim$ 160 s	$\sim$ 1.0M queries/s
HiLiftAeroML	GB200	42 GB	$\sim$ 30M	$\sim$ 200M	$\sim$ 155 s	$\sim$ 1.5M queries/s

Appendix

Table 9: Inference latency and peak memory.

Chunk size	10k	50k	100k	250k	500k	1M
Eval. runtime	543 s	209 s	169 s	146 s	138 s	131 s
Peak memory	3.6 GB	3.6 GB	4.5 GB	7.6 GB	12.6 GB	22.7 GB

Appendix

Table 10: Effect of query chunk size on inference runtime and peak memory.

Dataset	GPU	Untrim.	Inference	Meshing	CFD	Inf. speedup	Pipeline speedup
AhmedML	H100	0 s	$\sim$ 75 s	30 min	48 h	$\sim$ 2300 $\times$	$\sim$ 2300 $\times$
WindsorML	H100	$\sim$ 5 s	330 s	2 min	28 h	$\sim$ 300 $\times$	$\sim$ 300 $\times$
DrivAerML	GB200	$\sim$ 2800 s	$\sim$ 160 s	–	40 h	$\sim$ 900 $\times$	$\sim$ 50 $\times$
HiLiftAeroML	GB200	$\sim$ 1000 s	$\sim$ 155 s	–	–	–	–

Appendix

Table 11: Inference and preprocessing cost compared with the CFD pipeline. Inference speedup compares model inference with the CFD solve, while pipeline speedup compares untrimming and inference with CFD meshing and solve. “–” indicates that the corresponding information was not reported in the reference paper.

Figure 9: Surface decomposition of an AhmedML geometry into 13 untrimmed NURBS patches. The decomposition consists of nine planar surfaces and four ellipsoidal surfaces near the front, with each patch shown in a different color..

Dataset	Bbox diag. (m)	Max rel. error	Median rel. error
Windsor	$1.19$	$6.40\times 10^{-4}$	$1.73\times 10^{-14}$

Appendix

Table 12: Per-patch surface preservation error of the untrimming pipeline on the Windsor dataset. Per-patch error is the maximum, over an

8\times 8

uniform grid in

[0,1]^{2}

, of the Newton-projected perpendicular distance from the output sample to the original CAD parent surface; we report the maximum and median across patches, normalised by the model’s 3-D bounding-box diagonal.

Dataset	Runtime	Memory	Full preprocessing	Initial surf.	Final surf.
WindsorML	$\sim$ 5 s	0.5 GB	$\sim$ 15 s	42	71
DrivAerML	$\sim$ 2800 s	3.9 GB	$\sim$ 8 h	$\sim$ 8.5k	$\sim$ 45k
HiLiftAeroML	$\sim$ 1000 s	2.3 GB	$\sim$ 10 h	$\sim$ 350	$\sim$ 4.5k

Appendix

Table 13: Untrimming computational cost.

Parameter	Lower	Upper
body length	800.0	1200.0
body height	250.0	315.0
body width	300.0	500.0
front arc diameter	160.0	240.0
slant angle length	75.0	246.2
slant angle height	26.0	216.5

Appendix

Table 14: Design parameters and bounds (mm).

$V_{0}$ [L]	$L_{\mathrm{cfd}}$	$D_{\mathrm{surr}}$	$D_{\mathrm{cfd}}$	error	dataset	gain
70	0.044	0.1566	0.1586	$+1.3\%$	0.1867	$+15.1\%$
80	0.044	0.1566	0.1586	$+1.3\%$	0.1867	$+15.1\%$
90	0.020	0.1649	0.1685	$+2.2\%$	0.1867	$+9.7\%$
100	0.027	0.1811	0.1817	$+0.4\%$	0.2285	$+20.4\%$
110	0.013	0.2004	0.2026	$+1.1\%$	0.2317	$+12.5\%$
120	0.020	0.2196	0.2241	$+2.1\%$	0.2589	$+13.4\%$

Appendix

Table 15: CFD-validated optimized front .

D_{\mathrm{cfd}}

is the protocol-averaged coefficient; “dataset” is the lowest-drag dataset design satisfying the same constraints.

Figure 10: Qualitative AhmedML surface-pressure prediction. Left: CANTO prediction. Center: CFD reference field. Right: pointwise prediction error on the vehicle surface.

Figure 11: Qualitative WindsorML surface-pressure prediction. Left: CANTO prediction. Center: CFD reference field. Right: pointwise prediction error on the vehicle surface.

Explore similar work

CATO: Charted Attention for Neural PDE Operators
May 9, 2026Chun-Wun Cheng, Sifan Wang, Carola-Bibiane Schönlieb +1Neural Operators Neural Partial Differential Equation Solvers
DAMTP University of Cambridge · Institute for Foundations of Data Science Yale University · Yau Mathematical Sciences Center Tsinghua University
CADENA: Stepwise CAD Reverse Engineering
Aug 1, 2026Soslan Kabisov, Gennadiy Savrasov, Maksim Elistratov +9Parametric Computer-Aided Design Programs 3D Models
Lomonosov Moscow State University · DAIMLD · Independent Researcher +1
Cluster Attention Neural Operators for Solving Parametric Partial Differential Equations
Sep 30, 2026Ming Zhong, Antonio Colanera, Gianluigi Rozza +1Neural Operators Partial Differential Equations
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing 100049, China. · State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China. · International School for Advanced Studies (SISSA), Trieste 34136, Italy. +1

May 9, 2026, cs.AIJ/K move · Enter open · S save

CATO: Charted Attention for Neural PDE Operators

Chun-Wun Cheng, Sifan Wang, Carola-Bibiane Schönlieb, Angelica I. Aviles-Rivero

DAMTP University of Cambridge · Institute for Foundations of Data Science Yale University · Yau Mathematical Sciences Center Tsinghua University

Neural Operators Neural Partial Differential Equation Solvers Energy-Based Transformer Low-Rank Structure

Open paper AskPDF arXiv

Neural operators have emerged as powerful data-driven solvers for PDEs, offering substantial acceleration over classical numerical methods. However, existing transformer-based operators still face critical challenges when modeling PDEs on complex geometries: directly processing over massive mesh points is computationally expensive, while operating in raw discretization coordinates may obscure the intrinsic geometry where physical interactions are more naturally expressed. To address these limitations, we introduce the Charted Axial Transformer Operator (CATO), a geometry-adaptive and derivative-aware neural operator for PDEs on general geometries. Instead of applying attention directly in the physical coordinate system, CATO learns a continuous latent chart that maps mesh coordinates into a learned chart space, where chart-conditioned axial attention efficiently captures long-range dependencies with reduced computational cost. In addition, CATO introduces a derivative-aware physics loss for steady-state PDEs that jointly supervises solution values, mesh-consistent gradients, and an auxiliary flux-like field, improving physical fidelity and reducing oversmoothing. We further provide a theoretical approximation result showing that, under a favorable chart, charted axial attention can represent low-rank axial solution operators with controlled error, and that small chart perturbations induce bounded approximation degradation. CATO achieves the best performance across all evaluated datasets, yielding an average improvement of approximately 26.76% over the strongest competing baselines while reducing the number of parameters by 81.98%. These results highlight the effectiveness of learning geometry-adaptive charts and derivative-aware physical supervision for accurate and efficient PDE operator learning.

Abstract

Explore similar work

CATO: Charted Attention for Neural PDE Operators

CADENA: Stepwise CAD Reverse Engineering

Cluster Attention Neural Operators for Solving Parametric Partial Differential Equations