An improved periodic activation for PINNs reconstructing convective flows

Michael Mommert; Marie-Christine Volk; Christian Bauer

physics.flu-dynSep 18, 2026

An improved periodic activation for PINNs reconstructing convective flows

Authors: Michael Mommert, Marie-Christine Volk, Christian Bauer

Organizations: German Aerospace Center (DLR), Institute of Aerodynamics and Flow Technology, Department Ground Vehicles, Bunsenstr. 10, 37073 Göttingen, Germany · Technische Universität Ilmenau, Institute of Thermodynamics and Fluid Mechanics, 98693 Ilmenau, Germany

Abstract

Architectures with periodic activation functions have already been shown to be beneficial in comparison to monotonic counterparts for a wide range of applications of physics-informed neural networks. Here, we investigate a network architecture which uses the complex exponential function, generating pairs of sine and cosine outputs as activation functions. Testing it against comparable, sine-activated multi-layer perceptrons for the task of temperature reconstruction from sparse velocity data for cubic Rayleigh-Bénard convection reveals significant improvements in the reconstruction quality without a substantial increase in computational cost per training step. Vice versa, the improved architecture enables reaching similar reconstruction qualities for a fraction of the expense. Analyzing the mathematical structure of these networks points to the improvements being rooted in the property of passing both a sine and cosine function forward. This way, the subsequent layer is able to adapt the phase of the provided latent periodic functions, and doing it individually for each of its neurons.

Figures & tables

Name	$N_{\mathrm{L}}$	$N_{\mathrm{N}}$	total parameter count $N_{\theta}$	wall time per epoch	$\mathrm{MAE}_{T}$	$\mathrm{PCC}_{T}$	$\mathrm{R2}_{T}$
eq_net	$8$	$256$	$397318$	$\sim 0.7\,\mathrm{s}^{*}$	$0.0291$	$0.9351$	$0.8706$
eq_par	$8$	$362$	$792062$	$\sim 1\,\mathrm{s}$	$0.0258$	$0.9518$	$0.8991$
new	$8$	$256$	$792070$	$\sim 1\,\mathrm{s}$	$\mathbb{0.0168}$	$\mathbb{0.9775}$	$\mathbb{0.9547}$

Table 1: Configuration and metrics of the test cases. (* Referring to an Nvidia RTX 4090, while the remining cases refer to an Nvidia RTX 6000 Ada using the same AD102 die.)

Panel of Figure 1 — Figure 1: Ground truth and PINN reconstruction for the three test cases in a central vertical section and the central time instance.

Figure 2: Evolution of the R2 score for the temperature field over the course of the training.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

initialization	sampling	$\mathrm{MAE}_{T}$	$\mathrm{PCC}_{T}$	$\mathrm{R2}_{T}$
zeros	A	$0.0168$	$0.9775$	$0.9547$
same	A	$0.0216$	$0.9732$	$0.9387$
zeros	B	$0.0161$	$0.9842$	$0.9670$
same	B	$0.0208$	$0.9720$	$0.9420$

Appendix

Table 2: Metrics achieved for the architecture based on complex exponential activation for two different initialization strategies of the real part of the weight matrices, namely "zeros" and "same" for to different random samplings of the data of the test case. (* marks the case presented in section 3.3 )

Figure 3: Evolution of the R2 score for the temperature field over the course of the training for different initialization strategies of the real part of the weight matrices and data samplings.

Neural Network Activation Functions

Neural network activation functions shape how models represent nonlinear patterns, learn, remain stable, and map efficiently to hardware.

See all 78 papers

LRX-PINN: A Layer-Resolving XNet Physics-Informed Neural Network with Integrated Cauchy Activations for Convection-Dominated Problems
Jul 4, 2026Zihao Guo, Xin Li, Zhihong XiaNeural PDE Solvers PDE Solving
Institute for Advanced Research, Great Bay University, Dongguan, China · College of Computer Science and Technology, Dongguan University of Technology, Dongguan, China · Department of Mathematics, Northwestern University, Evanston, USA
When Does Frequency Decomposition Benefit Physics-Informed Neural Networks? A Preliminary Ablation Study
Aug 24, 2026Shubham RaiNeural PDE Solvers Physics-Informed ML
Complementary, Not Cumulative: Interaction Effects in Physics-Informed Neural Networks for Navier-Stokes Vortex Shedding
Aug 20, 2026Devesh ShahSinusoidal Representation Networks Physics-Informed ML
Independent Researcher

Jul 4, 2026J/K move · Enter open · S save

LRX-PINN: A Layer-Resolving XNet Physics-Informed Neural Network with Integrated Cauchy Activations for Convection-Dominated Problems

Zihao Guo, Xin Li, Zhihong Xia

Institute for Advanced Research, Great Bay University, Dongguan, China · College of Computer Science and Technology, Dongguan University of Technology, Dongguan, China · Department of Mathematics, Northwestern University, Evanston, USA

Neural PDE Solvers PDE Solving Neural Network Activation Functions Neural Network Approximation Theory Physics-Informed ML

Open paper AskPDF arXiv

Convection-dominated convection-diffusion problems often develop thin layers, where the solution has sharp transition profiles and its derivatives are highly localized. This creates a structural mismatch for standard physics-informed neural networks (PINNs), whose trial spaces are not designed to match the value--derivative structure of such layers. We propose a Layer-Resolving XNet Physics-Informed Neural Network (LRX-PINN) based on integrated Cauchy activations. The proposed basis is transition-type at the solution level, while its derivative recovers a localized Cauchy kernel. We show that this structure matches the scaling of convection-dominated layers, inherits the Cauchy approximation mechanism at the derivative-profile level, and identifies

d/\|w\|

as the effective physical width of a ridge neuron. For analytic layer profiles, this yields derivative-stable exponential approximation in the stretched coordinate and a layer-scaled estimate for the strong residual of the singularly perturbed operator. Numerical experiments on several convection-dominated benchmarks show that LRX-PINN achieves higher accuracy than PIKAN and Fourier-feature PINNs while using less than

30\%

of their trainable parameters. On more challenging benchmarks, embedding the proposed representation into hp-VPINN-based frameworks further improves the best results obtained by existing hp-VPINN-based baselines without changing their original loss functionals or stabilization strategies. These results show that neural representations aligned with layer structure provide a compact and effective approach for convection-dominated problems.

Abstract

Neural Network Activation Functions

Explore similar work

LRX-PINN: A Layer-Resolving XNet Physics-Informed Neural Network with Integrated Cauchy Activations for Convection-Dominated Problems

When Does Frequency Decomposition Benefit Physics-Informed Neural Networks? A Preliminary Ablation Study

Complementary, Not Cumulative: Interaction Effects in Physics-Informed Neural Networks for Navier-Stokes Vortex Shedding