Spectral-Informed Neural Networks Outperform Spectral Methods in High-dimensional PDEs

Solving high-frequency partial differential equations (PDEs) with neural networks is notoriously difficult due to the spectral bias of conventional architectures. We propose the Multi-Scale Separable Fourier Neural Network (MS-SFNN), a framework designed to overcome this limitation by explicitly encoding multi-scale Fourier features within a separable representation. The network factorizes the solution into $d$ single-coordinate subnetworks with fixed, randomly initialized weights; these subnetworks are combined via element-wise products to form a rich set of basis functions. This separable construction scales linearly with the problem dimension, thus inherently alleviating the curse of dimensionality. Crucially, each subnetwork is equipped with trainable scaling factors coupled with cosine activations, providing an adaptive mechanism for multi-scale frequency selection that endows the model with strong spectral approximation capability. The PDE solution is expressed as a linear combination of these learned basis functions, and the combination coefficients are determined by solving a large-scale least-squares system. To resolve the memory bottleneck in high-frequency or three-dimensional settings, we replace automatic differentiation (AD) with analytical derivatives of the basis functions and use a memory-efficient batched QR decomposition for solving the least-squares systems efficiently. Extensive numerical experiments demonstrate that MS-SFNN achieves superior accuracy and substantially outperforms state-of-the-art methods, including Physics-Informed Neural Network (PINN) and the Separated-Variable Spectral Neural Network (SV-SNN).

Time-dependent high-dimensional partial differential equations (PDEs) with spatially localised and dynamically evolving solutions pose a fundamental challenge for physics-informed neural networks (PINNs), as uniform collocation sampling becomes increasingly ineffective in high-dimensional spatiotemporal domains. In this work, a deep adaptive sampling framework for PINNs is extended to the time-dependent setting by treating space and time as a unified domain without any explicit time marching. A normalising flow neural network model effectively learns the distribution induced by the PDE residual and generates new collocation points concentrated in regions where the solution is most difficult to learn. Unlike conventional adaptive strategies that require explicit time stepping or moving meshes, high-residual regions are automatically identified and tracked across both space and time, driven purely by the PDE residual distribution. The effectiveness of the proposed strategy is assessed on a range of benchmark problems, from sharp and moving features in two spatial dimensions to localised structures in up to eight spatial dimensions.

Physics-Informed Neural Networks (PINNs) provide a meshless approach for solving partial differential equations (PDEs), but suffer severe degradation in stiff and shock-dominated problems, where small PDE residuals can correspond to globally inaccurate solutions. We show these failures are multi-causal, arising from the concurrent interplay of (i) spectral bias against sharp features, (ii) imbalanced multi-term optimization and loss-weight collapse, (iii) violation of temporal causality, and (iv) under-resolved collocation. We present SPARC-Net, a unified architecture and training framework that jointly addresses all four pathologies. SPARC-Net leverages an adaptive multi-scale spectral encoder with a learnable spectral gate, a gated residual backbone, adaptive activations, and a hard-constraint output ansatz that exactly enforces initial and boundary conditions, structurally eliminating loss-weight collapse. Training employs stabilized gradient-norm loss balancing, floored causality-respecting residual weighting, and residual-based adaptive collocation (RAD). Validated against exact analytic and high-order spectral reference solutions across four canonical benchmarks -- viscous Burgers', Allen-Cahn, convection (beta=30), and reaction -- SPARC-Net yields substantial improvements over vanilla PINNs: relative L2 error drops from 1.47e-1 to 1.14e-1 on Burgers' (22% reduction), 9.93e-1 to 5.78e-2 on Allen-Cahn (94% reduction), and 9.82e-1 to 3.54e-3 on reaction (100% reduction). A characteristic-coordinate encoder for hyperbolic transport further reduces convection error from 5.14e-1 to 9.88e-5 (100% reduction). We report five-seed mean +/- standard deviation errors, Wilcoxon significance tests, full ablation studies, hyperparameter sensitivities, an extension to the 2D heat equation, and comparisons against parameter-matched baselines.