Sparse Autoencoders as a Steering Basis for Phase Synchronization in Graph-Based CFD Surrogates

Yeping Hu; Ruben Glatt; Shusen Liu

cs.CEMar 28, 2026

Sparse Autoencoders as a Steering Basis for Phase Synchronization in Graph-Based CFD Surrogates

Authors: Yeping Hu, Ruben Glatt, Shusen Liu

Organizations: Computational Engineering Division Lawrence Livermore National Laboratory Livermore, CA 94550 · Center for Applied Scientific Computing Lawrence Livermore National Laboratory Livermore, CA 94550

Abstract

Graph-based surrogate models provide fast alternatives to high-fidelity CFD solvers, but their opaque latent spaces and limited controllability restrict use in safety-critical settings. A key failure mode in oscillatory flows is phase drift, where predictions remain qualitatively correct but gradually lose temporal alignment with observations, limiting use in digital twins and closed-loop control. Correcting this through retraining is expensive and impractical during deployment. We ask whether phase drift can instead be corrected post hoc by manipulating the latent space of a frozen surrogate. We propose a phase-steering framework for pretrained graph-based CFD models that combines the right representation with the right intervention mechanism. To obtain disentangled representation for effective steering, we use sparse autoencoders (SAEs) on frozen MeshGraphNet embeddings. To steer dynamics, we move beyond static per-feature interventions such as scaling or clamping, and introduce a temporally coherent, phase-aware method. Specifically, we identify oscillatory feature pairs with Hilbert analysis, project spatial fields into low-rank temporal coefficients via SVD, and apply smooth time-varying rotations to advance or delay periodic modes while preserving amplitude-phase structure. Using a representation-agnostic setup, we compare SAE-based steering with PCA and raw embedding spaces under the same intervention pipeline. Results show that sparse, disentangled representations outperform dense or entangled ones, while static interventions fail in this dynamical setting. Overall, this work shows that latent-space steering can be extended from semantic domains to time-dependent physical systems when interventions respect the underlying dynamics, and that the same sparse features used for interpretability can also serve as physically meaningful control axes.

Figures & tables

Figure 1: Overview of the phase-steering framework. A pretrained, frozen MeshGraphNet (MGN) produces node embeddings that are passed through a representation map $g$ , which can be a sparse autoencoder (SAE), PCA projection, the identity (raw embedding), or any other suitable basis. Given a detected phase mismatch between surrogate predictions and target observations (step 1), oscillatory feature pairs exhibiting near-quadrature relationships are identified via Hilbert transform analysis (step 2). Each selected feature field undergoes spatial mode decomposition via SVD to obtain low-dimensional coefficient trajectories (step 3). A smooth, time-varying phase offset $\Delta\phi(t)$ , parameterized by a linear trend and low-frequency cosine basis, is optimized over the steering horizon (step 4). The learned offset is applied by rotating the coefficient pairs, advancing or retarding the oscillation phase without altering its amplitude or spatial structure (step 5). The steered representation is decoded through the inverse map $g^{-1}$ and rolled out through the frozen MGN to produce phase-corrected flow predictions. Only the low-dimensional steering parameters $\{a_{k},b_{k},\mathbf{w}_{k}\}$ are optimized. The same pipeline is applied identically in all three representation spaces; only the choice of $g$ differs.

	Rotation-based steering (ours)			Static interventions (baselines)
Metric	SAE	PCA	Raw	Scale	Additive	Clamp
$\mathrm{frac\%}\,(v_{x})\;\uparrow$	$\mathbf{+26.1\%}$	$+16.0\%$	$+4.1\%$	$-118.6\%$	$+0.0\%$	$-494.1\%$
$\mathrm{ROI\%}\,(v_{x})\;\uparrow$	$\mathbf{+35.0\%}$	$+21.0\%$	$+7.5\%$	$-81.4\%$	$+0.0\%$	$-149.6\%$
$\mathrm{nRMSE}\,(v_{x})\;\downarrow$	$\mathbf{0.9926}$	$1.0582$	$1.1305$	$1.7067$	$1.1542$	$2.8138$
$\mathrm{Corr}\,(v_{x}v_{y})\;\uparrow$	$\mathbf{0.4681}$	$0.3666$	$0.3587$	$0.1333$	$0.3384$	$0.0560$

Table 1: Phase steering performance comparison. Rotation-based steering applies the physics-guided rotation pipeline across three embedding spaces (SAE, PCA, Raw). Static interventions apply standard per-feature manipulations (Scale, Additive, Clamp) in the SAE latent space. All metrics are computed on the cylinder-flow test trajectory. Best value in each row is bolded .

Figure 2: Velocity correction $\Delta v_{x}=v_{x}^{\mathrm{steer}}-v_{x}^{\mathrm{orig}}$ at a selected time frame for the three rotation-based methods. The green dashed rectangle marks the wake ROI. SAE produces the strongest, most spatially coherent correction in the near-wake region.

Figure 3: Spatial distribution of per-node

\mathrm{frac\%}(v_{x})

. Left column: Rotation-based steering across three embedding spaces. Right column: Static SAE interventions. Blue indicates improvement; red indicates degradation. Dashed green rectangles mark the wake ROI. Rotation-based steering produces structured, wake-localized improvement, with SAE showing the strongest and most spatially extensive effect. Static interventions produce either no discernible effect (Additive) or widespread degradation (Scale, Clamp).

Figure 4: Distribution of per-node $\mathrm{frac\%}(v_{x})$ within the wake ROI. SAE shifts the entire distribution rightward, indicating broad-based improvement rather than localized artifacts.

Figure 5: Spatial footprint of three individual salient SAE dimensions selected via the mean-absolute metric. Red markers show the $\eta$ most active nodes ( $\eta=100$ ) at four representative time steps. Each dimension localizes to a distinct flow feature.

Figure 6: Characterization of oscillatory SAE feature pairs selected for phase-rotation steering within the steering horizon ( $H=120$ ). (a) Node-averaged, normalized activation time series of a near-quadrature pair, exhibiting a phase lag of approximately $89^{\circ}$ at the vortex-shedding frequency. (b) Instantaneous Hilbert-transform phase difference for the same pair, showing stable fluctuations around the median ( $0.53\pi$ ) with coherence 0.810; the reference line at $0.5\pi$ indicates ideal quadrature. (c, d) Spatial energy footprints of a complementary pair, computed as the variance-weighted sum of the leading six SVD spatial modes. The gray circle marks the cylinder; the dashed green rectangle delineates the wake ROI. (e) Phase-space orbit of leading SVD coefficients for a selected pair, forming a smooth elliptical trajectory characteristic of coupled periodic oscillation. The pairwise rotation advances or retards the trajectory along this ellipse to implement phase correction. Green and red markers indicate the initial and final time steps, respectively.

Figure 7: Pareto frontier of $\mathrm{frac\%}(v_{x})$ vs. $\mathrm{Corr}(v_{x}v_{y})$ across all swept $(P,\lambda_{\mathrm{mag}})$ configurations. Each small marker is one configuration; starred markers denote the best configuration per representation. SAE configurations consistently occupy the upper-right region, dominating both PCA and Raw.

Computational Fluid Dynamics

Computational fluid dynamics numerically models fluid motion, turbulence, heat and mass transport, and fluid–structure interactions.

See all 275 papers

Sep 8, 2026

ONE CYLinder: A Benchmark for Graph-Based Surrogate Modeling of Unsteady Bluff-Body Flows

Graph-based surrogate models offer a promising route to accelerate computational fluid dynamics (CFD) simulations on unstructured meshes. However, their development is limited by the scarcity of benchmark datasets spanning multiple flow regimes and standardized protocols for long-horizon autoregressive prediction. We introduce ONECYL (ONE CYLinder), a new benchmark for unsteady flow past a circular cylinder across laminar, transitional, and high-Reynolds-number regimes. The benchmark comprises 450 high-fidelity Variational Multiscale finite-element simulations (270,000 flow snapshots) with randomized cylinder geometries, providing time-resolved velocity and pressure fields together with mesh connectivity, geometric descriptors, Reynolds numbers, and integrated aerodynamic quantities. Beyond the dataset, ONECYL establishes a unified evaluation framework combining full-field rollout errors, virtual probes, and drag and lift predictions to assess numerical accuracy and physical fidelity. To accompany the benchmark, we develop a Graph Transformer as a reference baseline predicting velocity and pressure fields autoregressively on unstructured meshes. Using ONECYL, we investigate geometric representations and physics-based regularization across the three Reynolds-number regimes. The results show that explicitly encoding the cylinder geometry through a level-set representation consistently improves long-horizon prediction accuracy and generalization to unseen geometries, while divergence-based regularization becomes increasingly beneficial as flow complexity increases. The ONECYL benchmark and its Graph Transformer baseline provide a reproducible framework for evaluating graph-based surrogate models and establish a foundation for future research on long-horizon prediction of unsteady bluff-body flows.

Théodore Michel, Antoine Campos, Alban Dujardin +3

Mines Paris PSL University Centre for Material Forming (CEMEF) UMR CNRS, Sophia Antipolis, 06904, France

Surrogate Modeling PDE Surrogate Modeling Physics-Informed ML Graph Transformer Computational Fluid Dynamics

Ask PDF

May 2, 2026

Mesh Based Simulations with Spatial and Temporal awareness

Machine Learning surrogates for Computational Fluid Dynamics (CFD), particularly Graph Neural Networks (GNNs) and Transformers, have become a new important approach for accelerating physics simulations. However, we identify a critical bottleneck in the field: while architectures have advanced significantly, the common underlying training paradigms remain bound to naive assumptions, such as node-wise supervision and explicit Euler time-stepping. These legacy choices ignore the stiff dynamics and local flux continuity inherent to numerous partial differential equations resolution methods, such as Finite Element, Difference, or Volume (FEM). In this work, we propose a unified framework to bridge the gap between geometric deep learning and rigorous numerical analysis. We introduce three key innovations: (1) Multi Node Prediction, a stencil-level objective that predicts field values for a node's full local topology, enforcing spatial derivative consistency; (2) Temporal Correction, replacing unstable explicit schemes with a predictor-corrector via temporal Cross-Attention; and (3) Geometric Inductive Biases, leveraging 3D Rotary Positional Embeddings (RoPE) to robustly capture rotational symmetries in unstructured meshes. We evaluate this framework across three architectures (MeshGraphNet, Transolver, and a Transformer) on diverse physics datasets. Our approach yields consistent improvements in accuracy and stability, particularly in long-horizon rollouts, while producing latent representations that generalize to unseen subtasks such as Wall Shear Stress or Pressure prediction. Code is available at https://github.com/DonsetPG/graph-physics.

Paul Garnier, Vincent Lannelongue, Elie Hachem

1CEMEF - Mines Paris PSL

Graph Neural Networks Neural PDE Solvers PDE Surrogate Modeling Computational Fluid Dynamics

Ask PDF

Jul 27, 2026

The balance between compactness and forecast accuracy of data-driven latent-space reduced-order models in controlled wake flows

Model-based active flow control requires predictive models that are accurate, stable, and fast enough for real-time optimisation. In controlled wake flows, this is often achieved through Reduced-Order Models (ROMs) that first compress high-dimensional velocity snapshots into a latent space and then learn a time- stepping predictor for the dynamics in the latent space. Here, we study how the choice of the spatial encoder affects the predictability of the resulting latent coordinates for wake flows under control inputs. Using two actuated 2D wake configurations, a simplified truck wake and the fluidic pinball, we compare Proper Orthogonal Decomposition (POD) against nonlinear Convolutional Autoencoders (CAEs) and two types of variational autoencoders for compression, and evaluate several temporal predictors based on Long Short-Term Memory networks. CAEs achieve higher compression efficiency and sharper short-term reconstructions, but they produce latent dynamics that are more irregular and with broadband spectral content. As a consequence, long-horizon forecasts degrade faster and show a higher probability of catastrophic divergence than POD-based models. POD yields smoother latent trajectories that are easier to learn and extrapolate, leading to more reliable predictions beyond the short- term regime. These results reveal a clear trade-off between compactness and forecast accuracy, and suggest that the stability of the latent dynamics prediction can outweigh maximal compression. This is particularly relevant for control strategies rooted in forecasts of the dynamics, such as model predictive control and reinforcement learning. The findings provide practical guidance for designing actuation-aware, hardware-feasible predictive ROMs for real-time flow control.

Alberto Solera-Rico, Patricia García-Caspueñas, Carlos Sanmiguel Vila +1

Sub-directorate general of aeronautical systems, Spanish National Institute for Aerospace Technology (INTA), ctra. M-301, km. 10.5, 28330 San Mart´ın de la Vega, Madrid, Spain. · Departamento de Ingenier´ıa Aeroespacial, Universidad Carlos III de Madrid, Avenida de la Universidad 30, 28911 Legan´es, Madrid, Spain. · Department of Mechanical Engineering, University of Washington, 1410 NE Campus Parkway, Seattle, WA 98195, USA.

Variational Autoencoders Latent Dynamics Modeling Autoencoders Proper Orthogonal Decomposition Convolutional Autoencoder Feedback Control+2

Ask PDF

Abstract

Computational Fluid Dynamics

Explore similar work

ONE CYLinder: A Benchmark for Graph-Based Surrogate Modeling of Unsteady Bluff-Body Flows

Mesh Based Simulations with Spatial and Temporal awareness

The balance between compactness and forecast accuracy of data-driven latent-space reduced-order models in controlled wake flows