Federated Learning of Car Aerodynamics Surrogates with Spectral Optimizers
Authors: Arman Bolatov, Nikolay Spitsyn, Daniil Medyakov, Vladimir Aletov, Dmitry Bylinkin, Martin Takáč, Aleksandr Beznosikov
Organizations: Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · Moscow Independent Research Institute of Artificial Intelligence (MIRAI) · Federated Learning Problems Laboratory · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Innopolis University
Neural surrogates predict the aerodynamic forces on a car in seconds, compared with hours for a CFD simulation, but training them requires thousands of CFD simulations of different car designs, which car manufacturers keep proprietary. Federated learning allows several manufacturers to train a shared surrogate without exchanging data. We study which federated optimizer brings such a surrogate closest to one trained on pooled data. On DrivAerNet++, we train a Transolver surrogate across six clients that each hold cars of a single body type, and we compare three spectral optimizers, which orthogonalize the update of each weight matrix (FedMuon and two methods we propose, FedMARS-M and FedOLion), with three non-spectral ones (FedAvg, SCAFFOLD-Adam and FedLion), all tuned with the same protocol. FedMuon and FedMARS-M reach a drag error within 2% of centralized training with Muon, using two to four local epochs per round. The non-spectral methods perform best with sixteen local epochs, the largest number we tested, and still have a 17 to 37% higher drag error than FedMARS-M. We relate this gap to the low stable rank of the weight gradients, for which an orthogonalized step yields about twice the first-order decrease of a gradient step of the same spectral norm. Code: https://github.com/armanbolatov/fedmars_drivaernet
Figures & tables
Figure 1: Problem setting: (a) the airflow exerts a pressure and a wall shear stress on the surface of the car (colors are schematic), (b) integrated over the surface, the two fields give the drag and lift forces, and (c) each manufacturer trains on its own cars while only model updates are exchanged.
Group
Method
E
cd error
cl error
Spectral
FedMARS-M
2
6.77±0.04
26.3±0.3
FedMuon
4
6.86±0.08
26.4±0.1
FedOLion
8
7.90±0.51
27.7±0.2
Non-spectral
SCAFFOLD-Adam
16
7.94
27.0
FedLion
16
8.94
33.3
FedAvg
16
9.26
37.1
Table 1: Single-class partition after 100 rounds: errors of the drag and lift coefficients in units of 10−3 , each method at its tuned number of local epochs E .
Figure 2: Errors on the four client partitions with one local epoch per round, after 100 rounds. Horizontal lines show the centralized models.
cd error
cl error
Method
single-class
α=0.5
α=10
α=1000
single-class
α=0.5
α=10
α=1000
FedMARS-M
7.02
6.86
6.82
6.78
27.1
27.1
26.8
26.8
FedMuon
6.99
6.98
6.90
6.86
27.2
27.2
27.0
27.0
FedOLion
7.47
7.23
7.13
7.07
28.1
27.6
27.1
27.4
SCAFFOLD-Adam
13.7
14.2
14.4
14.3
51.1
45.9
45.3
45.5
FedLion
16.4
11.2
12.8
13.2
51.2
41.0
43.7
45.2
Table 2: One local epoch per round on the four partitions after 100 rounds, errors in units of 10−3 . Values above 17.5 have the same color as 17.5.
Figure 3: Training loss and force errors of the global model over the rounds on the single-class partition, for the runs of Table 1 .
Figure 4: Errors after 30 rounds as a function of the number of local epochs per round, with drag in the top row and lift in the bottom row. On the Dirichlet partitions, only the spectral methods were run.
cd error
cl error
Method
E
K=1
2
3
6
K=1
2
3
6
FedMARS-M
2
7.21
6.65
6.69
6.73
27.1
26.2
26.3
26.0
FedMuon
4
7.30
7.07
6.85
6.93
34.3
26.1
25.9
26.4
FedOLion
8
8.59
7.47
7.02
8.45
43.0
27.5
27.8
27.7
SCAFFOLD-Adam
16
NaN
NaN
8.01
7.94
NaN
NaN
28.7
27.0
FedLion
16
12.6
8.32
9.27
8.94
44.3
30.9
34.6
33.3
Table 4: Single-class partition with K of the six clients per round, after 100 rounds, errors in units of 10−3 . NaN marks a diverged run.
Figure 5: Partial participation on the single-class partition: training loss and force errors over the rounds for K clients per round.
Figure 6: The runs of Figure 5 as a function of the number of client uploads.
Figure 7: Gradient spectra of the 42 weight matrices on the single-class partition: stable rank, gain g of an orthogonalized step, and relative spectral norm ρ of a sign step.
Movement per round
Kept by averaging (%)
Method
E=1
2
4
8
16
E=1
2
4
8
16
FedMARS-M
12.3
23.4
46.0
74.3
102
60
49
42
37
38
FedMuon
12.2
25.1
46.1
84.0
116
60
49
43
35
36
FedOLion
5.85
10.5
17.7
30.7
52.2
65
55
48
42
37
SCAFFOLD-Adam
0.53
0.88
1.28
1.45
2.08
83
91
90
87
90
FedLion
4.30
6.60
9.72
12.4
16.4
91
89
85
79
76
Table 5: Client drift at round 5 of the runs of Table 3 : the mean Frobenius norm of the client updates over one round (all parameters), and the norm of their average as a fraction of it.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
Data
DrivAerNet++: 5 703 training and 1 154 test cars, about 414 thousand surface points per car (330 to 513 thousand), 125 GB
Input and output
point coordinates and surface normals in, pressure and wall shear stress at every point out
Model
Transolver: 5 blocks, width 256, 8 heads, 32 slices, MLP ratio 1, dropout 0.2, GELU. 1.87M parameters, of which 1.85M in 42 weight matrices
Loss
sum of the mean squared errors of the normalized pressure and wall shear stress
Points per car
50 thousand sampled per training pass, all points at test time
Clients
6, and all of them train in every round unless K is stated
Appendix
Table 6: Training setup shared by all methods.
Partition
Clients
Cars per client
Entropy
single-class
6
917 / 950.5 / 1074
0.00
Dirichlet α=0.5
6
554 / 950.5 / 1954
0.56
Dirichlet α=10
6
700 / 950.5 / 1231
0.81
Dirichlet α=1000
6
943 / 950.5 / 959
0.82
Appendix
Table 7: Client partitions of the 5 703 training cars: the smallest, mean and largest number of cars per client, and the mean normalized entropy of the car categories within a client (0.82 for the whole training set).
Method
E
LR at E=1
LR at tuned E
Adam LR
β1 / β2
Weight decay
FedMuon
4
3e-3
3e-3
3e-5
0.9 / 0.95
0.1 (matrices)
FedMARS-M
2
3e-3
3e-3
1e-5
0.9 / 0.95
0.1 (matrices)
FedOLion
8
1e-3
1e-3
3e-6
0.95 / 0.95
0.1 (matrices)
Centralized Muon
–
1e-3
–
1e-5
0.95 / 0.95
0.1 (matrices)
FedAvg (AdamW)
16
1e-4, 1e-3, 3e-3, 1e-4
3e-4
–
0.7 / 0.999
0.01
SCAFFOLD-Adam
16
3e-5
1e-4
–
0.8 / 0.95
–
Appendix
Table 8: Selected hyperparameters, with learning rates written a e b for a×10b . Rates that differ between partitions are listed for single-class, α=0.5 , 10 and 1000.
Methods
Searched values
Size
Searched
spectral
matrix LR {3e-4, 1e-3, 3e-3, 1e-2, 3e-2}, Adam LR {3e-6, 1e-5, 3e-5, 1e-4, 3e-4}, β1 {0.7, 0.8, 0.9, 0.95, 0.99}
125
19%
FedAvg, centralized AdamW
LR {1e-5, 3e-5, 1e-4, 3e-4, 1e-3}, β1 as above
25
68%, 72%
SCAFFOLD-Adam
LR {1e-6, 3e-6, 1e-5, 3e-5, 1e-4}, β1 as above
25
72%
FedLion
LR {1e-5, 3e-5, 1e-4, 3e-4, 1e-3}, β1 as above, β2 {0.9, 0.95, 0.99, 0.995}
100
24%
FedMARS-M, γ
MARS weight γ {0.01, 0.025, 0.04, 0.1, 1} at FedMARS-M’s selection. The value 0.04 scored 0.7% better than the reference 0.01, a difference within noise, so we kept 0.01
5
100%
Appendix
Table 9: Stage-1 search spaces. Size is the number of configurations in the space, and searched the share of them that was evaluated.
Figure 8: Stage-2 learning-rate walks: cd error after 30 rounds at one local epoch, one line per partition. Circles mark the selected rate, triangles a diverged run.
Figure 9: The comparison of Figure 2 after 30 rounds, each method at its selected rate.
Method
Schedule
E=1
2
4
8
16
FedAvg
constant
21.5 / 60
20.1 / 58
18.4 / 57
14.0 / 65
13.6 / 54
cosine
18.7 / 61
19.1 / 57
18.3 / 56
14.4 / 66
11.6 / 57
SCAFFOLD-Adam
constant
22.8 / 56
22.0 / 59
14.9 / 57
11.9 / 46
14.4 / 45
cosine
17.4 / 61
19.0 / 56
13.9 / 59
11.7 / 56
10.7 / 46
FedLion
constant
31.6 / 85
27.0 / 69
19.0 / 58
15.3 / 56
12.2 / 50
cosine
25.9 / 73
22.4 / 70
16.3 / 61
13.7 / 53
10.3 / 42
Appendix
Table 10: Constant learning rate against the cosine schedule for the non-spectral methods: cd / cl error after 30 rounds on the single-class partition, units of 10−3 . The constant-rate runs used A100 GPUs and the cosine runs A6000 GPUs.
Partition
Method
E
K
A100
RTX A6000
single-class
FedMARS-M
2
6
6.73 / 26.0
6.69 / 26.3
FedMuon
4
6
6.93 / 26.4
6.77 / 26.0
FedAvg
1
6
17.0 / 57.8
15.9 / 59.2
FedAvg
16
6
9.26 / 37.1
10.0 / 38.3
FedAvg
16
3
9.80 / 34.8
9.53 / 36.3
FedAvg
16
2
10.3 / 33.3
8.52 / 32.5
Appendix
Table 11: The same configurations on A100 GPUs (the results of the paper) and on RTX A6000 GPUs after 100 rounds: cd error / cl error in units of 10−3 , with K clients per round.
GPU
Runs
Time per round
Peak memory
A100 40 GB
non-spectral, E=1
130 s
31.4 GB
A100 40 GB
non-spectral, E=16
2 060 s (2.4 days for 100 rounds)
32.0 GB
A100 40 GB
non-spectral, E=16 , K=3
1 050 s
32.0 GB
RTX A6000 48 GB
non-spectral, E=1
233 s
31.4 GB
RTX A6000 48 GB
non-spectral, E=16
3 820 to 4 230 s
31.4 GB
A100 40 GB
spectral, E=1
140 s
31.4 GB
Appendix
Table 12: Wall-clock time per round of one run on one GPU. Times grow linearly with E and with the number of clients per round.
Neural surrogates for external vehicle aerodynamics are fast alternatives to physics-based computational fluid dynamics (CFD) simulations. Their foundational limitation is a lack of zero-shot generalization across vehicle classes, which results in an expensive, custom training campaign for each new vehicle design. Similar to pretrained models in other fields, a pretrained surrogate should adapt to a new vehicle class using only a few simulations and stay accurate on its pretraining domain. No prior study has systematically tested this transfer behavior. We conduct a large-scale, controlled benchmarking study on five DrivAerML-pretrained surrogates across four independent target datasets. These datasets differ from the source in geometry, CFD solver, turbulence model, and flow conditions. We find that fine-tuning these surrogates outperforms training from scratch in fewer than half of the comparisons and lowers the accuracy of the surrogate on its source dataset. We introduce GeoReplay, a geometry- and gradient-aligned source replay that keeps source simulations resembling the target in training and applies their updates only when they agree with the target gradient. A risk bound separates geometric coverage from differences in the simulated physics, and we prove that every sufficiently small GeoReplay step decreases the target training objective. GeoReplay lowers the source error of fine-tuning in 77 of 80 settings and matches its target accuracy.
Favour Nerrise, Julian Busch, Gurpreet Singh Hora +3
Siemens · Department of Electrical Engineering, Stanford University, USA
Deploying Scientific Machine Learning surrogates in industrial CFD workflows requires adapting pretrained models to new vehicle families without large datasets; yet whether geometric representations learned by a geometry encoder transfer to topologically distinct shapes remains unvalidated. We address this through leave-one-family-out experiments on a 61.47M-parameter Transformer surrogate (AB-UPT) pretrained on four vehicle families (411 external aerodynamics cases) and adapted to the held-out fifth with only 20 samples. Three strategies are compared: Full Fine-Tuning (FFT), Lightweight Fine-Tuning (LFT), and Low-Rank Adaptation (LoRA). The central finding is that pretrained geometry encoders learn transferable representations, but the adaptation mechanism determines whether they can be exploited. FFT destabilizes as 61.47M unconstrained parameters overfit to 20 samples (R^2=0.40); LFT fails because the frozen encoder cannot represent unseen shapes (R^2<0). LoRA resolves both: rank-constrained adapters injected into all layers regularize the loss landscape while preserving pretrained features, achieving R^2=0.85+/-0.02 across all five families with 50% lower force RMSE than FFT and 28% lower pointwise field errors. LoRA also outperforms from-scratch training using 3x more target-family data, eliminating the need for large per-family datasets. These results recast LoRA from a memory-saving convenience into a convergence enabler for geometry transfer: a shared backbone paired with lightweight per-family adapters trainable in hours from minimal data.
Computational Fluid Dynamics (CFD) is central to race-car aerodynamic development, yet its cost -- tens of thousands of core-hours per high-fidelity evaluation -- severely limits the design space exploration feasible within realistic budgets. AI-based surrogate models promise to alleviate this bottleneck, but progress has been constrained by the limited complexity of public datasets, which are dominated by smoothed passenger-car shapes that fail to exercise surrogates on the thin, complex, highly loaded components governing motorsport performance. This work presents three primary contributions. First, we introduce a high-fidelity RANS dataset built on a parametric LMP2-class CAD model and spanning six operating conditions (map points) covering straight-line and cornering regimes, generated and validated by aerodynamics experts at Dallara to preserve features relevant to industrial motorsport. Second, we present the Gauge-Invariant Spectral Transformer (GIST), a graph-based neural operator whose spectral embeddings encode mesh connectivity to enhance predictions on tightly packed, complex geometries. GIST guarantees discretization invariance and scales linearly with mesh size, achieving state-of-the-art accuracy on both public benchmarks and the proposed race-car dataset. Third, we demonstrate that GIST achieves a level of predictive accuracy suitable for early-stage aerodynamic design, providing a first validation of the concept of interactive design-space exploration -- where engineers query a surrogate in place of the CFD solver -- within industrial motorsport workflows.
Nicholas Thumiger, Andrea Bartezzaghi, Mattia Rigotti +5