Physics and Data Driven Transformer-Mamba Framework for Flow Field
Authors: Zhuo Zhang, Shun Zou, Canqun Yang, Xi Yang
Organizations: College of Computer Science and Technology, National University of Defense Technology, Changsha, China · College of Artificial Intelligence, Nanjing Agricultural University, NanJing, China · National SuperComputer Center in Tianjin, Binhai New Area, Tianjin, China · National Key Laboratory of Parallel and Distributed Computing, National University of Defense Technology, Changsha, China
While deep learning accelerates expensive partial differential equation solving in computational fluid dynamics (CFD), existing methods like PINNs and FNOs often struggle with generalization, noise robustness, and physical consistency. We introduce the Transformer-Mamba for Flow Field (TM4FF) framework, a physics-constrained operator learning model with three key innovations: a Residual Wavelet Mamba (RWM) layer for feature denoising, a Transformer-based attention mechanism for enhanced feature fusion, and a physics-informed loss using Fourier derivatives to enforce the Navier-Stokes equations. Experiments on four CFD datasets show TM4FF achieves high accuracy and robust generalization across varying flow conditions.
Figures & tables
Figure 1: Overall evaluation metrics for each method in flow field prediction.
Figure 2: Overview of Transformer-Mamba for Flow Field (TM4FF). RWM: Residual Wavelet Mamba, DWConv: Depthwise Convolution, LN: LayerNorm.
Figure 3: Overview of the Residual Wavelet Mamba (RWM) layer, containing the core Vision State-Space Module (VSSM).
Case
Model
Source
Param (K)
MSE (u)
RMSE (u)
MAE (u)
NMSE (u)
MSE (v)
RMSE (v)
MAE (v)
NMSE (v)
Cavity
FFN
JCP-2019
72
7.5073394
2.4209118
1.1220862
0.1134040
113.7456860
9.3089929
7.1327300
1.4379886
Deeponet
NMI-2019
143
1.9921126
1.2580220
0.7409260
0.0416701
0.6104441
1.1387197
0.5477043
0.0400961
Auto_Deeponet
NMI-2019
552
0.0639269
0.1530177
0.0587990
0.0013988
0.0662839
0.1114433
0.0417463
0.0030466
Auto_Edeeponet
Arxiv-2022
623
0.0571324
0.1476592
0.0504779
0.0011877
0.0456277
0.1088518
0.0788765
0.0022441
Auto_FFN
JCP-2019
1,102
0.0639877
0.1531136
0.0581093
0.0014016
0.0733171
0.1063512
0.0792814
0.0033839
UNet
MICCAI-2015
1,095
0.0404604
0.1178060
0.0894393
0.0013797
0.0436799
0.1080112
0.0475035
0.0014662
Table 1: Comparative Study of Fluid Prediction Methods and SOTA Models (Including Autoregressive and Non-Autoregressive Models). The bold black values represent the best results, while the underlined values indicate the second-best results.
Figure 4: The wavelet decomposition and reconstruction of strip features in neural networks (DWT aggregates strip noise into Sll and Shl ).
Figure 5: Overview of Bi-level Routing Attention.
Components
Cavity
Tube
Dam
Cylinder
Backbone only
0.0404604
0.0014302
0.0000777
0.0027909
+ Case Params
0.0389877
0.0012312
0.0014455
0.0027534
+ Mamba
0.0397174
0.0013258
0.0007616
0.0027721
+ BiS
0.0392813
0.0012937
0.0011103
0.0027408
+ Params + BiS
0.0382510
0.0010783
0.0010610
0.0026881
+ Mamba + BiS
0.0390156
0.0011826
0.0007944
0.0027615
Table 2: Ablation Study of Various Strategies
RWM
Cavity
Tube
Dam
Cylinder
✗
0.0226239
0.0005742
0.0000667
0.0025262
✓
0.0194232
0.0004528
0.0000548
0.0020187
Table 3: Ablation Study of Residual Wavelet Mamba on Model Performance
Dept. of Computer Engineering Middle East Technical University Ankara, Turkey · Dept. of Mechanical Engineering Middle East Technical University Ankara, Turkey