CarBench: A Comprehensive Benchmark for Neural Surrogates on High-Fidelity 3D Car Aerodynamics

Mohamed Elrefaie; Dule Shu; Matt Klenk; Faez Ahmed

Figure 1: Overview of CarBench. The benchmark provides a unified framework for evaluating machine learning models on high-fidelity automotive aerodynamics. It integrates diverse car geometries, CFD-derived surface pressure fields, and automated training pipelines with physics-based and ML metrics. Architectures from multiple model families (GNNs, Geometric Deep Learning, Neural Operators, Transformers, and Triplane networks) are evaluated under a standardized protocol. The framework includes cross-category generalization, uncertainty quantification, computational efficiency analysis, statistical significance evaluation, subsampled vs. full-mesh performance comparison, and wheel aerodynamics prediction. Together, these components enable a comprehensive and domain-specific assessment of aerodynamic surrogate models.

Model	Parameters (M)	Peak Memory (GB)	Mean Latency (ms)	Throughput (samples/s)	MSE (m 4 /s 4 )	MAE (m 2 /s 2 )	RMSE (m 2 /s 2 )	$R^{2}_{\text{test}}$	Rel L2
PointNet [ 39 ]	1.67	0.29	1.54	648.85	3350 $\pm$ 50	31.12 $\pm$ 0.170	57.9 $\pm$ 0.44	0.7639 $\pm$ 0.0027	0.3803 $\pm$ 0.0020
NeuralOperator [ 29 ]	2.10	0.04	2.14	466.36	2130 $\pm$ 47	25.02 $\pm$ 0.140	46.2 $\pm$ 0.51	0.8503 $\pm$ 0.0026	0.3016 $\pm$ 0.0019
PointMAE [ 37 ]	1.67	0.39	3.11	321.68	1716 $\pm$ 39	22.58 $\pm$ 0.110	41.4 $\pm$ 0.47	0.8791 $\pm$ 0.0023	0.2713 $\pm$ 0.0016
PointNetLarge [ 39 ]	32.58	1.50	8.29	120.65	1384 $\pm$ 37	20.51 $\pm$ 0.095	37.2 $\pm$ 0.50	0.9025 $\pm$ 0.0022	0.2436 $\pm$ 0.0013
RegDGCNN [ 17 ]	1.44	27.11	231.98	4.31	957 $\pm$ 50	17.41 $\pm$ 0.090	30.9 $\pm$ 0.81	0.9327 $\pm$ 0.0033	0.2006 $\pm$ 0.0016
PointTransformer [ 56 ]	3.05	6.65	95.68	10.50	917 $\pm$ 82	17.05 $\pm$ 0.085	30.3 $\pm$ 1.3	0.9359 $\pm$ 0.0054	0.1909 $\pm$ 0.0024

Table 1: Quantitative comparison on the unseen test set of 1,154 samples across machine learning models, ranked from worst to best by Relative L2 (lower is better). All models are evaluated with batch size 1 on an NVIDIA A100-SXM4-80GB GPU under identical inference settings. Per ISO GUM (Guide to the expression of uncertainty in measurement), uncertainties are rounded to two significant digits and central values are rounded to the same decimal place; precision is standardized per column using the largest uncertainty in that column (MSE (Mean Squared Error): 10, MAE (Mean Absolute Error): 0.01, RMSE (Root Mean Squared Error): 0.1,

R^{2}_{\text{test}}

:

10^{-4}

, Rel L2 (Relative L2 Error):

10^{-4}

). Here NeuralOperator denotes the voxelized Fourier Neural Operator baseline, i.e. FNO (voxelized), in which the geometry is mapped to a fixed

32^{3}

grid rather than through a learned deformation; see the Baseline Models section of the Supplementary Information for the distinction from Geo-FNO and GINO. An asterisk ( * ) denotes a model evaluated from the checkpoint provided by its original authors rather than trained in this work; the corresponding models trained from random initialization on the DrivAerNet++ training split alone are reported in Supplementary Table 6.

Figure 2: Performance–efficiency trade-offs across model families on CarBench. Left: Relative L2 error versus model size (in millions of parameters, log scale). Right: Relative L2 error versus mean inference time (ms) on an NVIDIA A100 GPU. Transformer-based models together with the implicit-field model (TripNet) achieve the best accuracy–efficiency balance, combining low error with compact parameter counts and fast inference. Point-based baselines suffer from higher errors despite lower complexity, while graph-based networks incur large memory and latency overheads. These plots highlight the accuracy–efficiency trade-off across aerodynamic surrogate models, revealing models that are both accurate and scalable. The filled marker denotes the model evaluated from the checkpoint provided by its original authors (AB-UPT * ); the hollow marker denotes the same architecture retrained from random initialization in this work on the DrivAerNet++ training split alone, joined to its counterpart by a grey line. TripNet appears only as retrained in this work. Because parameter count and inference latency are determined by the architecture rather than by the weights, the two AB-UPT variants differ only along the accuracy axis: retraining shifts AB-UPT from a Relative L2 of

0.1358

to

0.1539

.

Figure 3: Qualitative comparison of surface pressure predictions for design E_S_WW_WM_648 from the unseen test set of the DrivAerNet++ dataset. a , Isometric view. b , Side view. In both panels the top row shows the predicted pressure field for each model and the bottom row the corresponding absolute error against the CFD ground truth. While all models capture the general pressure distribution, transformer-based architectures produce smoother predictions that match the CFD reference more closely around the stagnation region and in areas of high curvature, and TripNet gives the smoothest error map, which follows from its triplane-based volumetric sampling. TransolverLarge and AB-UPT * , together with TripNet, form the top-performing group in both views, maintaining spatial coherence and capturing high-pressure regions more accurately than the other baselines. The panels labelled AB-UPT show the author-provided checkpoint, denoted AB-UPT * in the tables.

Figure 4: Pressure prediction on the front-left wheel compared to the CFD ground truth. Models are shown in approximate order of Relative L2 error (worst to best, left to right); PointNet is omitted for space. Wheels can contribute up to 25% of a car’s total aerodynamic drag, making accurate pressure modeling around rotating components critical. Compared to prior open-source datasets, DrivAerNet++ uniquely provides detailed and smooth wheel and tire geometries, enabling evaluation across both open and closed wheel configurations.

Train categories	Test categories	Rel L2		$\boldsymbol{R^{2}_{\text{test}}}$		Train	Test	Ratio
		Transolver	Transolver++	Transolver	Transolver++	size	size
Fastbacks + Notchbacks	Estatebacks	0.2391	0.2182	0.9088	0.9241	5,388	1,386	3.89
Estatebacks + Fastbacks	Notchbacks	0.1725	0.1729	0.9542	0.9540	5,374	1,403	3.83
Estatebacks + Notchbacks	Fastbacks	0.4894	0.4988	0.6083	0.5933	2,231	5,332	0.42
Estatebacks	Fastbacks + Notchbacks	0.3980	0.4192	0.7446	0.7167	1,108	6,735	0.16
Notchbacks	Estatebacks + Fastbacks	0.4295	0.4483	0.7002	0.6731	1,122	6,718	0.17

Table 2: Cross-category generalization performance of Transolver and Transolver++ across combinations of car design categories. Each row corresponds to a training/test category split using the DrivAerNet++ dataset. Models are evaluated in a zero-shot setting, with no fine-tuning or transfer learning on the test category. We report the Relative L2 error (lower is better) and

R^{2}_{\text{test}}

score (higher is better), together with the training and test set sizes and the corresponding train-to-test ratio. In every experiment the designs of the training categories are split

80/20

into training and validation sets, while all designs of the held-out categories are used for testing. The same category therefore appears with different counts depending on its role: the

5{,}332

Fastback designs contribute

4{,}265

training designs when trained on, and all

5{,}332

are evaluated when held out. These results highlight the models’ ability to extrapolate aerodynamic behavior across distinct vehicle classes under varying data imbalance conditions.

	Relative L2			$\mathbf{R^{2}}$
Model	Subsampled	Interpolated	Patchwise	Subsampled	Interpolated	Patchwise
PointNet	0.3763	0.3937	0.3791	0.7700	0.7485	0.7664
NeuralOperator	0.2962	0.3259	0.2987	0.8575	0.8277	0.8549
PointMAE	0.2619	0.3137	0.2670	0.8888	0.8403	0.8839
PointNetLarge	0.2390	0.3203	0.2423	0.9075	0.8337	0.9044
RegDGCNN	0.1954	0.2803	0.1994	0.9378	0.8724	0.9346

Table 3: Subsampled versus full-resolution evaluation on a stratified subset of

61

geometries drawn from the held-out test split, sampled to preserve the archetype proportions of the full split, with all three protocols applied to the same geometries. Subsampled scores a

10{,}000

-point prediction on those points; Interpolated upscales that same prediction to every node of the full CFD surface mesh (

3.7

–

5.5\times 10^{5}

nodes, mean

455{,}000

) with the radial-basis interpolation; Patchwise instead predicts every node directly using

\lceil N/10^{4}\rceil

disjoint

10{,}000

-point passes (

37

–

55

per geometry), and therefore involves no interpolation. Patchwise evaluation recovers almost all of the apparent degradation, showing that the loss attributed to resolution is dominated by interpolation error. † AB-UPT * and TripNet decode arbitrary query points directly, so their patchwise column evaluates every node from a single geometry encoding, with queries chunked only to fit memory, rather than

\lceil N/10^{4}\rceil

independent passes. They are held to the identical interpolated protocol as every other model, with the prediction restricted to the same

10{,}000

points and upscaled in the same way, so that the column compares evaluation protocols rather than architectures. AB-UPT * is evaluated from the author-provided checkpoint, as in Table 1 .

Model	Median Rel. Error	P50 Error (m 2 /s 2 )	P90 Error (m 2 /s 2 )	P95 Error (m 2 /s 2 )	P99 Error (m 2 /s 2 )
PointNet	0.19	17.57	66.04	101.32	249.43
NeuralOperator	0.15	14.36	53.35	82.93	192.86
PointMAE	0.13	13.17	48.44	72.70	167.14
PointNetLarge	0.12	11.10	46.61	72.06	156.91
RegDGCNN	0.10	9.91	37.03	54.15	111.75
PointTransformer	0.10	9.50	34.77	50.40	103.05

Table 4: Statistical error distribution of surface pressure prediction across models, sorted from worst to best according to Median Relative Error. Median relative error is dimensionless, while percentile errors are reported in absolute pressure units (m 2 /s 2 ). Lower values indicate higher predictive accuracy. Percentiles describe the cumulative distribution of absolute errors across all surface points, capturing both typical and extreme deviations. As in Table 1 , AB-UPT * denotes the checkpoint provided by the original authors, while TripNet refers to our own reimplementation.

Figure 5: Training and validation loss curves for eight models evaluated on the DrivAerNet++ dataset. Each plot shows model convergence behavior over training epochs, with blue indicating training loss and red indicating validation loss. These trends highlight differences in optimization stability and representational efficiency across network architectures for aerodynamic learning.

Figure 6: Surface pressure distributions for representative designs across different car categories in DrivAerNet++. Rows correspond to Estateback, Notchback, and Fastback archetypes under smooth and detailed configurations. The color scale represents kinematic pressure (m 2 /s 2 ), highlighting regions of low pressure over the windshield and roof and high pressure near the front bumper and rear stagnation zones. These examples illustrate the geometric and aerodynamic diversity captured in the dataset.

Figure 7: Comparison of large-scale, high-fidelity, and industry-standard open-source car datasets. DrivAerNet++ (8,150 designs, based on eight parametric models ) provides the broadest coverage, modeling both detailed underbodies representative of internal combustion engine (ICE) cars and smooth underbodies for electric vehicles (EVs). It is also the only dataset to incorporate wheel and tire shape variations, including open and closed wheel designs. DrivAerML (

\sim

484 designs) is derived from a single DrivAer parametric model, while SHIFT-SUV (

\sim

2,000 designs) is based on two AeroSUV baseline models.

Figure 8: PCA projection of dataset distributions across DrivAerNet++, DrivAerML, and SHIFT-SUV. Each point represents a car geometry, with DrivAerNet++ spanning a wider and more continuous design space, while DrivAerML and SHIFT-SUV are confined to narrower regions. Notably, DrivAerML lies largely within the coverage of DrivAerNet++, as it is derived from the DrivAer Notchback baseline, one of the eight parametric archetypes used to construct DrivAerNet++. This highlights the geometric diversity and design-space coverage of DrivAerNet++.

Figure 9: DrivAerNet++ dataset geometric statistics. Left: Distribution of point counts across all meshes, showing a consistent resolution with mean and median values around 450K. Middle: Distribution of cell counts, exhibiting a similar bimodal trend and central tendency to the point distribution. Right: Distribution of total surface area, indicating moderate variation across designs with a mean of 33.7 units 2 . Each subplot reports dataset size (

N=8{,}150

) and standard deviation, providing an overview of mesh density and geometric uniformity across the corpus.

Model	Training Configuration							Model Architecture
	Loss	Epochs	LR	Opt	Batch	WD	Scheduler	Layers	Heads	Embed	Dropout	$k$ -NN	Extra
PointNet	MSE	58 (150)	$10^{-3}\!\to\!10^{-5}$	Adam	12	$10^{-4}$	ReduceLR	–	–	1024	0.3	–	–
NeuralOperator	MSE	100	$2{\times}10^{-3}\!\to\!5{\times}10^{-7}$	AdamW	16	$10^{-4}$	Cosine	2	–	16	–	–	Grid= $32^{3}$ , $M$ =8
PointMAE	SmoothL1	106 (500)	$10^{-3}$	AdamW	128	$10^{-2}$	Cosine ‡	4	–	256	0.3/0.2	–	Efficient
PointNetLarge	MSE	82 (150)	$4{\times}10^{-4}$	AdamW	8	$10^{-4}$	OneCycle ‡	–	–	4096	0.3/0.5	–	Residual
RegDGCNN	MSE	200	$10^{-3}\!\to\!10^{-5}$	Adam	12	$10^{-4}$	ReduceLR	3	–	1024	0.4	40	EdgeConv

Table 5: Training and model configurations for all ML baselines, as executed in the runs reported in Table 1 . Epoch entries give the executed count, with the configured budget in parentheses where early stopping intervened; learning-rate entries give the initial and, where a schedule was active, final values. All models are trained on 10,000 points per surface sample with mean squared error loss except where noted.

Study	Dataset	Simulations	Test Size	Design Categories	# Models Evaluated	Flow Regime	Benchmark Availability	Cross-Category Generalization	Uncertainty Quantification
CarBench (Ours)	DrivAerNet++	8,150	1,154	8	11	Steady RANS (OpenFOAM)	✓	✓	✓
Jacob et al. [ 26 ]	VW Real-World*	343	69	32	2	DES (OpenFOAM)	✗	✗	✗
Li et al. [ 30 ]	ShapeNet-Car	889	100	1	5	Steady RANS (OpenFOAM)	✗	✗	✗
Tangsali et al. [ 46 ]	DrivAerML	484	48	1	3	Hybrid RANS-LES (OpenFOAM)	✓	✗	✗
Li et al. [ 30 ]	Ahmed body	551	51	1	5	Steady RANS (OpenFOAM)	✗	✗	✗
Luo et al. [ 34 ]	DrivAerNet++	200	10	3	13	Steady RANS (OpenFOAM)	✗	✗	✗

Table 1: Comparison of existing CFD-based automotive aerodynamics datasets and benchmarks. Each entry lists the corresponding study and dataset, characterized by simulation scale, test set size, number of design categories, number of evaluated machine learning models, flow regime, and benchmark availability. DrivAerNet++ is the largest and most diverse open dataset, supporting full-field aerodynamic supervision, standardized benchmarking, and cross-category generalization across distinct car archetypes. RANS stands for Reynolds-Averaged Navier–Stokes Simulation, DES stands for Detached Eddy Simulation, and LES for Large Eddy Simulation. Bold values indicate the best entries, while underlined values indicate the second best. *Note that the VW Real-World benchmark evaluates only drag coefficients and does not provide full-field flow supervision.

Model	Attention Type	Position Encoding	Dim	Layers	Params (M)	Complexity
PointTransformer	Local $k$ -NN self-attn	Relative offsets $\mathbf{p}_{i}-\mathbf{p}_{j}$ (MLP)	128	12	3.05	$O(Nkd)$
Transolver	Slice-based physics attention	Features only (6D)	256	5	2.47	$O(NG+G^{2}d)$
Transolver++	Slice-based physics attn (Gumbel)	Coord concat (9D)	256	5	1.81	$O(NG+G^{2}d)$
TransolverLarge	Slice-based attention (scaled-up)	Features only (6D)	384	7	7.58	$O(NG+G^{2}d)$
AB-UPT *	Anchor self-attn + cross-attn	Rotary Positional Embedding (RoPE)	192	11	6.01	$O(N_{a}^{2}+N_{q}N_{a})$

Table 2: Comparison of transformer-based architectures evaluated on DrivAerNet++ for 3D surface-field prediction. The table summarizes the core attention mechanisms, positional encoding strategies, model scales, and computational complexity of each architecture. Complexity notation:

N

= number of surface points;

k

= number of nearest neighbors for local attention;

G

= number of learnable slices/groups in slice-based attention;

d

= hidden feature dimension;

N_{a}

= number of anchor tokens in AB-UPT encoder self-attention;

N_{q}

= number of query tokens in decoder cross-attention. These quantities determine the computational cost of attention operations across different architectural paradigms.

Model	Parameters	MAE (m 2 /s 2 )	RMSE (m 2 /s 2 )	Rel L2	$\mathbf{R^{2}_{\text{test}}}$
TransolverLarge	7.58M	11.61	23.61	0.1557	0.9610
Transolver	2.47M	11.83	23.63	0.1558	0.9609
Transolver++	1.81M	11.78	23.72	0.1564	0.9606
RegDGCNN	1.44M	17.08	30.04	0.1985	0.9364
PointMAE	1.67M	17.15	30.63	0.2024	0.9339
PointNetLarge	32.58M	22.07	34.03	0.2248	0.9184

Table 3: Performance of all models trained using the same optimization configuration: Adam with

\mathrm{lr}=10^{-3}

and no weight decay, MSE loss, a cosine learning-rate schedule,

100

epochs without early stopping, batch size

4

,

10^{4}

sampled surface points, and random seed

42

. Results are reported on the held-out test split and sorted by

R^{2}_{\text{test}}

. MAE and RMSE are reported in kinematic pressure units (m 2 /s 2 ).

Model	Input representation	Features	Tensor shape
PointNet	Surface point cloud	$[x,y,z]$	$(10000,\,3)$
PointNetLarge	Surface point cloud	$[x,y,z]$	$(10000,\,3)$
PointTransformer	Point cloud + local kNN ( $k{=}16$ )	$[x,y,z]$	$(10000,\,3)$
RegDGCNN	Point cloud + dynamic kNN ( $k{=}40$ )	$[x,y,z]$	$(10000,\,3)$
PointMAE	Surface point cloud	$[x,y,z]$	$(10000,\,3)$
NeuralOperator	Voxel grid from $10$ k points	Occupancy ( $1$ channel)	$(1,\,32,32,32)$

Table 4: Input representation, feature format, and tensor shape used by each evaluated model, with the batch dimension omitted. Point-based models sample

10{,}000

surface points from the full mesh. All models predict per-point surface pressure.

Variant	Input representation	MSE	MAE	RMSE	$\mathbf{R^{2}_{\mathrm{test}}}$	$\Delta$ MSE vs. benchmark
Repeat of benchmark config †	$[x,y,z,n_{x},n_{y},n_{z}]$	575.7	13.09	23.99	0.9595	$+2.14\%$ $[+1.31,+2.86]$
Benchmark configuration	$[x,y,z,n_{x},n_{y},n_{z}]$	563.6	12.87	23.74	0.9604	—
$+$ signed distance field	$[x,y,z,\mathrm{sdf},n_{x},n_{y},n_{z}]$	578.5	12.94	24.05	0.9593	$+2.64\%$ $[+1.52,+3.87]$
Best architecture, no SDF	$[x,y,z,n_{x},n_{y},n_{z}]$	566.3	12.87	23.80	0.9602	$+0.47\%$ $[-0.47,+1.31]$
Best architecture $+$ SDF	$[x,y,z,\mathrm{sdf},n_{x},n_{y},n_{z}]$	565.4	12.76	23.78	0.9602	$+0.32\%$ $[-0.51,+1.14]$

Table 5: Effect of the input geometry representation on surface-pressure accuracy, with architecture and training held fixed. All variants use the Transolver backbone trained for

200

epochs on the same training designs. Every variant is scored on the same

5{,}000

measured surface points per design (

5{,}770{,}000

points over the

1{,}154

test geometries), and errors are in kinematic-pressure units. The final column reports the change in MSE relative to the benchmark configuration with a

95\%

confidence interval from a paired bootstrap over test geometries (

B=10{,}000

). † This variant was configured to remove surface normals, but the option had no effect and the model trained on inputs identical to the benchmark configuration; the row therefore constitutes a repeat run of that configuration, and its deviation provides one observation of run-to-run variability. No representational or architectural change is statistically distinguishable from that repeat-run difference.

Figure 1: t-SNE projection of the test geometries only: the official DrivAerNet++ test split (

1{,}154

designs), a

10\%

sample of SHIFT-SUV (

200

designs) and

50

DrivAerML cases. Point clouds of

1{,}024

points per geometry are embedded with the same feature extractor used for Fig. 8, which is projected linearly by PCA; here the same features are projected non-linearly by t-SNE (perplexity

30

, PCA initialization, random seed

42

). Distances and apparent containment in a t-SNE layout are qualitative and should not be read as quantitative measures of design-space coverage. Restricting the embedding to held-out geometries preserves the structure seen over the full datasets, with SHIFT-SUV forming a separate cluster and DrivAerML confined to a compact region inside the DrivAerNet++ distribution. The DrivAerNet++ test split alone is roughly

23

times larger than a comparable DrivAerML test set and nearly six times larger than a

10\%

SHIFT-SUV split, which is what allows the confidence intervals in Table 1 to separate models whose errors differ by less than

0.01

Relative L2.

Model	Training source	MSE	MAE	RMSE	Rel L2	$\mathbf{R^{2}_{\mathrm{test}}}$
AB-UPT *	Original authors	559	10.81	23.6	0.1358	0.9607
AB-UPT	This work	605	12.57	24.6	0.1539	0.9576

Table 6: Comparison between the pretrained AB-UPT checkpoint released by its original authors and the same architecture trained from random initialization in this work. The from-scratch model uses only the official DrivAerNet++ training split and both are evaluated on the same held-out surface-pressure test split containing

1{,}154

geometries. MSE is reported in squared kinematic-pressure units (m 4 /s 4 ), while MAE and RMSE are reported in kinematic-pressure units (m 2 /s 2 ). Throughout the manuscript, AB-UPT * denotes the checkpoint obtained from the original authors, which is the variant reported in Table 1 and Fig. 2, whereas AB-UPT without the asterisk denotes the model trained from random initialization in this work.

Model	10k	25k	50k	100k	200k	500k
Transolver	0.8	1.9	3.9	7.6	15.0	37.1
Transolver++	0.7	1.7	3.3	6.4	12.8	31.9
TransolverLarge	1.8	4.3	8.5	16.6	33.1	OOM
PointNet	0.3	0.8	1.5	3.0	6.0	15.0
PointNetLarge	1.9	4.1	7.7	15.0	36.3	OOM
PointMAE	0.4	1.0	1.9	3.8	7.6	19.0

Table 7: Peak training memory in GB for one forward-and-backward pass with batch size

1

as a function of the number of input surface points. Measurements were performed on a single NVIDIA A100 GPU with 80 GB of memory. OOM denotes an out-of-memory failure, and “–” indicates configurations that were not evaluated after an earlier out-of-memory failure.

CarBench: A Comprehensive Benchmark for Neural Surrogates on High-Fidelity 3D Car Aerodynamics

Abstract

Figures & tables

Automotive Aerodynamics

Faster by Design: Interactive Aerodynamics via Neural Surrogates Trained on Expert-Validated CFD

CarCrashNet: A Large-Scale Dataset and Hierarchical Neural Solver for Data-Driven Structural Crash Simulation

FSAN: Flow State Attention Network for Aerodynamic Prediction

Abstract

Automotive Aerodynamics

Explore similar work

Faster by Design: Interactive Aerodynamics via Neural Surrogates Trained on Expert-Validated CFD

CarCrashNet: A Large-Scale Dataset and Hierarchical Neural Solver for Data-Driven Structural Crash Simulation

FSAN: Flow State Attention Network for Aerodynamic Prediction