RheOPT: Few-Shot Non-Newtonian Flow Prediction via Newtonian Pretraining of RheOFormer

Maedeh Saberi; Amir Barati Farimani; Safa Jamali

cs.LGOct 22, 2026

RheOPT: Few-Shot Non-Newtonian Flow Prediction via Newtonian Pretraining of RheOFormer

Authors: Maedeh Saberi, Amir Barati Farimani, Safa Jamali

Organizations: Department of Mechanical and Industrial Engineering, Northeastern University, Boston, Massachusetts 02115, USA · Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, USA

Abstract

Despite advances in numerical methods, non-Newtonian flow simulations remain computationally expensive because velocity and stress must be advanced together through rate- and history-dependent constitutive dynamics. Neural-operator surrogates reduce inference cost after training, but their training data require the same expensive simulations they aim to replace, making data-efficient learning essential when only a few non-Newtonian simulations can be afforded. Here, we introduce the Rheological Operator Pretrained Transformer (RheOPT), which pretrains a transformer-based neural operator on inexpensive Newtonian simulations and fine-tunes it on a small number of non-Newtonian samples. Rather than transferring the Newtonian constitutive response itself, RheOPT reuses geometry-conditioned transport structure that remains useful after the stress model changes. We evaluate RheOPT in two geometry-matched settings: a $4{:}1$ contraction channel for Giesekus, Oldroyd-B, and Bingham fluids, and the oscillatory wake behind a triangular obstacle for Oldroyd-B flow. In the contraction, Newtonian pretraining substantially improves few-shot prediction, and a model fine-tuned on four target samples can match or exceed a from-scratch model trained on four times as many. This benefit is reliable under reversible instance normalization (RevIN) but dataset dependent under Global normalization, a difference we trace to the dynamic range of each fluid's velocity and stress fields. In the wake, where relative $L_2$ error alone cannot verify that vortex shedding is recovered, phase-insensitive diagnostics show that pretraining recovers the shedding frequency and wake fluctuation amplitude that are suppressed by the from-scratch model in the few-shot regime. Overall, RheOPT establishes Newtonian simulation as a physically grounded pretraining source for data-efficient non-Newtonian operator learning.

Figures & tables

Figure 1: Overview of RheOPT. (1) RheOFormer is trained from a random initialization, $\theta_{\mathrm{rand}}$ , on Newtonian simulations of the target geometry, yielding the pretrained weights $\theta_{\mathrm{Newt}}$ . (2) The same few-shot non-Newtonian target data are used to fine-tune the pretrained model (RheOPT, initialized with $\theta_{\mathrm{Newt}}$ ) and to train an otherwise identical RheOFormer from scratch (initialized with $\theta_{\mathrm{rand}}$ ). The fields illustrate the triangular-obstacle wake: a Newtonian snapshot used for pretraining (gray), an Oldroyd-B target snapshot (left), and the fine-tuned and from-scratch predictions (right).

Fluid model	Geometry	Varied target controls	Fine-tuning samples	Input window	Prediction horizon
Giesekus	$4{:}1$ contraction	Inlet velocity	$4,8,12,16$	$t\in[0,0.2]$ s	$t\in[0.4,5]$ s
Oldroyd-B	$4{:}1$ contraction	Inlet velocity	$4,8,12,16$	$t\in[0,0.2]$ s	$t\in[0.4,5]$ s
Bingham	$4{:}1$ contraction	Inlet velocity & yield stress	$4,8,12,16$	$t\in[0,0.2]$ s	$t\in[0.4,5]$ s
Oldroyd-B	Triangular obstacle	Inlet velocity	$4,8,12,16,24,32$	$t\in[0,1.8]$ s	$t\in[2,10]$ s

Table 1: Summary of the RheOPT experimental design.

Figure 2: Representative contraction-flow velocity fields. Velocity magnitude, $|\mathbf{v}|$ , at $t=5~\mathrm{s}$ for Giesekus ( $Re=15.4$ , $Wi=1.54$ ), Oldroyd-B ( $Re=15.6$ , $Wi=0.156$ ), and Bingham ( $Re=16$ , $Bi=0.25$ ) fluids in the $4{:}1$ contraction, predicted with $N=4$ fine-tuning samples under RevIN. Rows show the Newtonian pretraining simulation at the same $Re$ , the ground truth, the fine-tuned (pretrained) prediction, and the from-scratch prediction; all fields of a column share one color scale. The bottom row shows the corresponding profiles across the downstream channel at $x=1$ , marked by the dashed lines, one downstream half-width past the contraction plane. The corresponding error maps and the shear-stress fields are shown in Figs. S2 and S3 of the Supplementary Information.

Figure 3: Newtonian pretraining improves data-efficient prediction of non-Newtonian contraction flows. Real-space relative $L_{2}$ velocity (top row) and stress (bottom row) error as a function of the number of non-Newtonian fine-tuning samples, $N$ , for Giesekus, Oldroyd-B, and Bingham fluids in the $4{:}1$ contraction geometry. Global Normalization and RevIN are distinguished by color, while pretrained models and from-scratch models are shown by solid and dashed curves, respectively.

Fluid	Reynolds number	Additional parameter
Newtonian	$Re\in[0.4,80]$	$Bi=0,Wi=0$
Giesekus	$Re\in[1,42]$	$Wi=0.1\,Re$
Oldroyd-B	$Re\in[0.4,80]$	$Wi=0.01\,Re$
Bingham	$Re\in[0.4,60]$	$Bi\in[0,5]$

Table 2: Datasets for the contraction-flow experiments.

Figure 4: Prediction error in the triangular-obstacle flow. Real-space relative $L_{2}$ error as a function of the number of fine-tuning samples, $N$ , under Global normalization. Solid curves denote Newtonian-pretrained models after fine-tuning, and dashed curves denote models trained from scratch using the same target samples and training protocol. Red and blue curves correspond to stress and velocity errors, respectively, and error bars denote the standard deviation across validation samples.

Figure 5: Representative wake fields. Newtonian simulation at the same Reynolds number, ground truth, pretrained (fine-tuned) prediction, and from-scratch prediction of the streamwise velocity field, $v_{x}$ , for a representative triangular-obstacle Oldroyd-B case at $Re=94$ and $Wi=0.94$ using $N=12$ training samples under Global Normalization.

Figure 6: Pretraining recovers the shedding-frequency trend. Dominant temporal frequency in the wake as a function of Reynolds number for the triangular-obstacle Oldroyd-B flow under Global normalization. Ground-truth frequencies are shown as reference markers, and model predictions are compared for pretrained and from-scratch models across fine-tuning budgets. The pretrained models track the increasing frequency–

Re

trend, whereas the from-scratch models trained with few samples show a weaker or nearly flat frequency response.

Figure 7: Pretraining recovers the wake-fluctuation amplitude. RMS fluctuation amplitude of the transverse velocity,

v_{y}

, measured in the downstream wake for a representative triangular-obstacle case at

Re=94

and

Wi=0.94

under Global normalization, as a function of the streamwise coordinate measured from the channel inlet. Colors denote the number of non-Newtonian fine-tuning samples,

N

, solid curves denote Newtonian-pretrained models after fine-tuning, dashed curves denote models trained from scratch, and black markers denote the ground truth. The purple dash-dotted curve shows the Newtonian simulation at the same Reynolds number. The RMS envelope compares the strength and spatial persistence of the oscillatory wake in the ground truth, pretrained, and from-scratch prediction across different training budgets.

Neural Operators

Neural operators learn mappings between functions, enabling fast surrogates for PDEs and other continuous physical systems.

See all 310 papers

Oct 1, 2025

RheOFormer: A generative transformer model for simulation of complex fluids and flows

The ability to model mechanics of soft materials under flowing conditions is key in designing and engineering processes and materials with targeted properties. This generally requires solution of internal stress tensor, related to the deformation tensor through nonlinear and history-dependent constitutive models. Traditional numerical methods for non-Newtonian fluid dynamics often suffer from prohibitive computational demands and poor scalability to new problem instances. Developments in data-driven methods have mitigated some limitations but still require retraining across varied physical conditions. In this work, we introduce Rheological Operator Transformer (RheOFormer), a generative operator learning method leveraging self-attention to efficiently learn different spatial interactions and features of complex fluid flows. We benchmark RheOFormer across a range of different viscometric and non-viscometric flows with different types of viscoelastic and elastoviscoplastic mechanics in complex domains against ground truth solutions. Our results demonstrate that RheOFormer can accurately learn both scalar and tensorial nonlinear mechanics of different complex fluids and predict the spatio-temporal evolution of their flows, even when trained on limited datasets. Its strong generalization capabilities and computational efficiency establish RheOFormer as a robust neural surrogate for accelerating predictive complex fluid simulations, advancing data-driven experimentation, and enabling real-time process optimization across a wide range of applications.

Maedeh Saberi, Amir Barati Farimani, Safa Jamali

Department of Mechanical and Industrial Engineering, Northeastern University, Boston, MA 02115, USA · Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA

Neural Surrogate Modeling Transformer PDE Operator Learning Computational Fluid Dynamics

Ask PDF

May 8, 2026

Accelerated and data-efficient flow prediction in stirred tanks via physics-informed learning

The simulation of fluid flows is computationally expensive due to the complexity of its governing partial differential equations. Machine learning models offer a potential surrogate, enabling learning from simulations and significantly faster predictions of flow fields. However, these models require large training datasets, which introduces a trade-off between dataset generation cost and predictive accuracy. In this work, we investigate the relationship between the size of the training-set and accuracy of the prediction when learning steady flow fields in an industrial-scale stirred vessel. A data set of steady flows is generated using Reynolds Averaged Navier Stokes (RANS) simulations in a range of realistic operating conditions, including impeller speeds and liquid heights. We train implicit neural representations of flow fields and compare purely data-driven and constrained variants. Model performance is evaluated using global mean squared error (MSE), qualitative spatial comparisons of predicted and reference flow fields, and tracer transport simulations. We find that the prediction error decreases monotonically with increasing training data, but also that it exhibits clear diminishing returns beyond moderate dataset sizes. Physics-based constraints significantly improve accuracy and reduce variability across training runs in low-data regimes, and they lead to more stable tracer-transport behavior. Furthermore, reasonable interpolation can be achieved over different impeller speeds and liquid heights. However, these benefits come with an increase in the complexity of training, and their relative advantage diminishes as the training set grows.

Mahdi Naderibeni, Liang Wu, David M. J. Tax

aPattern Recognition and Bio-informatics Group, Delft University of Technology, Delft, the Netherlands · bScience & Research and Innovation, dsm-firmenich, Delft, the Netherlands

PDE Surrogate Modeling Physics-Informed ML Data-Efficient Learning Computational Fluid Dynamics

Ask PDF

Oct 20, 2026

Neural Operator Splitting for Fluid Dynamics in Complex Geometry

Solving dynamical systems in complex and diverse geometries remains challenging for both numerical solvers and learning-based surrogate models. We propose Neural Operator Splitting, a framework for learning fluid dynamics while preserving the physical structure of fractional-step numerical schemes. Instead of learning the full-step evolution monolithically, we decompose each time step into physically interpretable steps, and learn each operator separately using intermediate states from the underlying numerical solver. Guided by the linearity properties of the fractional steps, we introduce a conditional-linear Fourier neural operator (CL-FNO) whose linear state mappings are modulated by nonlinear conditioning on inputs such as boundary geometry. We further introduce two complementary strategies, symmetry sampling and corrective cycling, to mitigate error accumulation during autoregressive rollout. Experiments on 2D and 3D fluid flows with varying geometries and initial conditions show that our full model reduces mean relative

L^2

rollout error by 68% in 2D and 40% in 3D compared with a single full-step neural operator. At comparable error levels, neural-operator inference is approximately

10^3

times faster per step than the Monte Carlo solver.

Jianxiang Li, Xi Deng, Hong Chul Nam +1

New York University

PDE Operator Learning Neural Operators Computational Fluid Dynamics Fourier Neural Operator

Ask PDF

Abstract

Neural Operators

Explore similar work

RheOFormer: A generative transformer model for simulation of complex fluids and flows

Accelerated and data-efficient flow prediction in stirred tanks via physics-informed learning

Neural Operator Splitting for Fluid Dynamics in Complex Geometry