Latent attention on masked patches for flow reconstruction
Authors: Ben Eze, Luca Magri, Andrea Nóvoa
Organizations: Aeronautics Dept., Imperial College London, Exhibition Rd, London SW7 2BX, UK · DIMEAS, Politecnico di Torino, Corso Duca degli Abruzzi, 24, Torino, 10129, Italy · I-X, Imperial College London, 84 Wood Lane, London W12 0BZ, UK
Vision transformers have shown outstanding performance in image generation, yet their adoption in fluid dynamics remains limited. We introduce the Latent Attention on Masked Patches (LAMP) model, an interpretable regression-based modified vision transformer designed for masked flow reconstruction. LAMP follows a three-fold strategy: (i) partition of each flow snapshot into patches, (ii) patch-wise dimensionality reduction via proper orthogonal decomposition, and (iii) reconstruction of the full field from a masked input using a single-layer transformer trained via closed-form linear regression. We test the method on two canonical 2D unsteady wakes: a laminar wake past a bluff body, and a chaotic wake past two cylinders. On the laminar case, LAMP accurately reconstructs the full flow field from a 90%-masked and noisy input, across signal-to-noise ratios between 10 and 30dB. Further, the learned attention matrix yields interpretable multi-fidelity optimal sensor-placement maps. LAMP's performance on the chaotic wake is limited, but outperforms other regression methods such as gappy POD. The modularity of the framework, however, naturally accommodates nonlinear compression and deep attention blocks, thereby providing an efficient baseline for nonlinear, high-dimensional masked flow reconstruction.
Figures & tables
Figure 1: Pictorial illustration of the patch-wise latent attention model (LAMP). A linear autoencoder embeds the input in a lower-dimensional latent space, where predictions can be made efficiently. The predictions are pair-wise (i.e. each patch predicts every other patch) and weighted with a confidence score.
Figure 2: Patch-wise POD reconstruction performance for varying patch size P . Loss LAE versus (a) latent dimension Ne and (b) compression factor D/Ne .
Figure 3: Reconstruction error Lpred for varying noise levels, patch size P and latent dimension Ne , all with 10% patches unmasked. The input is noisy masked data with SNR = (a) ∞ (noise-free), (b) 30 dB, (c) 20 dB, (d) 10 dB. The loss is computed against the noise-free target Xtest and the model is trained on noise-free data. Horizontal dashed lines show the noise variance. Lpred is calculated for 25 random patch arrangements and the median is plotted. An example reconstruction for the noise-free case with P=16 , Ne=8 is shown.
Figure 4: Predictive-power maps for varying P and Ne . Patches with higher predictive power predict the rest of the patches with lower average loss.
Figure 5: Masked reconstruction of the chaotic wake from 25% unmasked patches, patches placed at the locations of highest predictive power. The reconstruction of one snapshot with the best model ( P=48,Ne=250 ) is shown on the right.
Division of Future Vehicle Korea Advanced Institute of Science and Technology Daejeon, 34051, Republic of Korea · Department of Mechanical Engineering Hanyang University Ansan, 15588, Republic of Korea · Cho Chun Shik Graduate School of Mobility Korea Advanced Institute of Science and Technology Narnia Labs Daejeon, 34051, Republic of Korea