Physics-Informed ML

ML: Machine Learning

Ask about this topic

Momentum

93 papers in the last four weeks, up 26% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 642

All topics
CardsList
Aug 29, 2025cs.LG

Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation

Physics informed neural networks (PINNs) represent a very popular class of neural solvers for partial differential equations. In practice, one often employs stochastic gradient descent type algorithms to train the neural network. Therefore, the convergence guarantee of stochastic gradient descent is of fundamental importance. In this work, we establish the linear convergence of stochastic gradient descent / flow in training over-parameterized two layer PINNs with a general class of activation functions for solving one model second-order elliptic problem, i.e., the Poisson equation, in the sense of high probability. These results extend the existing result [20] in which gradient descent was analyzed. The challenge of the analysis lies in handling the dynamic randomness introduced by stochastic optimization methods. The key of the analysis lies in ensuring the positive definiteness of suitable Gram matrices during the training. The analysis sheds insight into the dynamics of the optimization process, and provides guarantees on physics informed neural networks trained by stochastic algorithms.

Bangti Jin, Longjun Wu

Department of Mathematics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong, P.R. China

Neural PDE SolversPDE SolvingStochastic Optimization ConvergencePhysics-Informed MLStochastic Gradient Descent
AskPDF
Aug 25, 2025cs.CV

InSituRes: A Physics-Informed Same-Grid Model for Enhanced Dynamic X-ray Micro-CT Reconstructions

X-ray micro-computed tomography (micro-CT) provides non-destructive three-dimensional (3D) imaging of porous material microstructures. In situ experiments, including mechanical loading and reactive transport, increasingly require dynamic four-dimensional (4D) imaging with volumes repeatedly acquired during experiments. However, rapid acquisition typically requires fewer projections, shorter exposures, or reduced fields of view, producing reconstructions with noise, blur, and artifacts that obscure pores, microcracks, and interfaces. To address this challenge, this study introduces InSituRes, a physics-informed same-grid volumetric enhancement framework for fast dynamic X-ray micro-CT imaging of temporally evolving materials. InSituRes maps fast-acquisition volumes to higher-quality long-acquisition reconstructions using paired scans of the same specimens. The model integrates 3D convolutional feature extraction with slice-wise transformer attention to capture local and broader in-plane context. A learnable forward degradation model approximates rapid acquisition effects, including spatial blurring, intensity scaling differences, and signal-dependent noise. During training, reconstructed volumes should match high-quality reference scans and reproduce observed fast acquisition data after propagation through the forward model, imposing a physics-guided consistency constraint. Experiments on unseen micro-CT datasets demonstrate improved reconstruction fidelity and enhanced visibility of fine microstructural features relative to conventional interpolation and learning-based enhancement approaches. The framework supports quantitative interpretation of fast 4D X-ray micro-CT scans of evolving materials.

Qinyi Tian, Andrea Bisciotti, Soniya Tiwari +2

Department of Civil & Environmental Engineering, Duke University, Durham, North Carolina, 27708, USA · Department of Physics and Earth Sciences, University of Ferrara, Via Saragat 1, Ferrara 44141, Italy · Department of Mechanical Engineering & Materials Science, Duke University, Durham, North Carolina, 27708, USA

Image RestorationPhysics-Informed MLComputed Tomography
AskPDF
Aug 5, 2025cs.LG

Physics-Constrained Fine-Tuning of Flow-Matching Models for Generation and Inverse Problems

We present a framework for fine-tuning flow-matching generative models to enforce physical constraints and solve inverse problems in scientific systems. Starting from a model trained on low-fidelity or observational data, we apply a differentiable post-training procedure that minimizes weak-form residuals of governing partial differential equations (PDEs), promoting physical consistency and adherence to boundary conditions without distorting the underlying learned distribution. To infer unknown physical inputs, such as source terms, material parameters, or boundary data, we augment the generative process with a learnable latent parameter predictor and propose a joint optimization strategy. The resulting model produces physically valid field solutions alongside plausible estimates of hidden parameters, effectively addressing ill-posed inverse problems in a data-driven yet physicsaware manner. We validate our method on canonical PDE benchmarks, demonstrating improved satisfaction of PDE constraints and accurate recovery of latent coefficients. Our approach bridges generative modelling and scientific inference, opening new avenues for simulation-augmented discovery and data-efficient modelling of physical systems.

Jan Tauberschmidt, Sophie Fellenz, Sebastian J. Vollmer +1

DSA, German Research Center for Artificial Intelligence (DFKI) · Department of Computer Science, University of Kaiserslautern–Landau (RPTU) · Department of Mathematics, Imperial College London

Physics-Informed Generative ModelingFlow MatchingGenerative ModelingPhysics-Informed MLPDE Inverse Problems
AskPDF
Jun 23, 2025math.NA

DPG loss functions for learning parameter-to-solution maps by neural networks

We develop, analyze, and experimentally explore residual-based loss functions for machine learning of parameter-to-solution maps in the context of parameter-dependent families of partial differential equations (PDEs). Our primary concern is on rigorous accuracy certification to enhance the prediction capability of the resulting deep neural network reduced models. This is achieved by the use of variationally correct loss functions. Through one specific example of an elliptic PDE, details for establishing the variational correctness of a loss function from an ultraweak Discontinuous Petrov Galerkin (DPG) discretization are worked out. Despite the focus on the example, the proposed concepts apply to a much wider scope of problems, namely problems for which stable DPG formulations are available. The issue of high-contrast diffusion fields and ensuing difficulties with degrading ellipticity are discussed. Both numerical results and theoretical arguments illustrate that for high-contrast diffusion parameters the proposed DPG loss functions deliver much more robust performance than simpler least-squares losses.

Pablo Cortés Castillo, Wolfgang Dahmen, Jay Gopalakrishnan

PDE SolvingPDE Surrogate ModelingRobust Loss FunctionsPDE Operator LearningPhysics-Informed ML
AskPDF
Jun 23, 2025cs.LG

GeNeRT: A Physics-Informed Approach to Intelligent Wireless Channel Modeling via Generalizable Neural Ray Tracing

Neural ray tracing (RT) has emerged as a promising paradigm for channel modeling by integrating physical propagation principles with neural networks. However, existing neural RT methods remain limited by strong spatial dependence and weak adherence to electromagnetic laws. We propose GeNeRT, a generalizable neural RT framework that improves generalization and accuracy through relative geometric features, scatterer semantics, and a Fresnel-inspired polarization-driven architecture. GeNeRT is trained through a three-stage strategy: polarization-specific module-wise pre-training captures general ray-surface interaction behavior; system-wise end-to-end training uses only receiver-side channel impulse responses to learn site-specific propagation characteristics; and measurement-based fine-tuning employs sparse measured multipath components (MPCs) to adapt polarization-related modules to real-world environments. Extensive outdoor simulations demonstrate robust intra-scenario transferability and inter-scenario zero-shot generalization. In an unseen scenario, GeNeRT achieves an overall error of −35.36-35.36 dB and an average-delay error of 4.91 ns, compared with −10.85-10.85 dB and 32.38 ns for the best baseline. With only 75 measured reflected MPCs, fine-tuning further reduces the overall error from −14.48-14.48 to −22.90-22.90 dB and the average-delay error from 6.28 to 3.58 ns. Ablation studies confirm the effectiveness of the proposed architecture and training strategy.

Kejia Bian, Meixia Tao, Shu Sun +2

School of Information Science and Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China · Shanghai Innovation Institute, Shanghai 200231, China

Neural Network GeneralizationWireless CommunicationsRadio Propagation ModelingPhysics-Informed ML
AskPDF
Jun 18, 2025cs.LG

Interpretability and Generalization Bounds for Learning Spatial Physics

While there are many applications of ML to scientific problems that look promising, visuals can be deceiving. Using numerical analysis techniques, we rigorously quantify the accuracy, convergence rates, and generalization bounds of certain ML models applied to linear differential equations for parameter discovery or solution finding. Beyond the quantity and discretization of data, we identify that the function space of the data is critical to the generalization of the model. A similar lack of generalization is empirically demonstrated for commonly used models, including physics-specific techniques. Counterintuitively, we find that different classes of models can exhibit opposing generalization behaviors. Based on our theoretical analysis, we also introduce a new mechanistic interpretability lens on scientific models whereby Green's function representations can be extracted from the weights of black-box models. Our results inform a new cross-validation technique for measuring generalization in physical systems, which can serve as a benchmark.

Alejandro Francisco Queiruga, Theo Gutman-Solo, Shuai Jiang

OpenAI, San Francisco, CA, USA · Google, Mountain View, CA, USA · Sandia National Laboratories, Albuquerque, NM, USA

Neural Network GeneralizationMechanistic InterpretabilityScientific MLPhysics-Informed MLGeneralization Bounds
AskPDF
May 28, 2025cs.LG

Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation

Modeling physical systems in a generative manner offers several advantages, including the ability to handle partial observations, generate diverse solutions, and address both forward and inverse problems. Recently, diffusion models have gained increasing attention in the modeling of physical systems, particularly those governed by partial differential equations (PDEs). However, diffusion models only access noisy data xt\boldsymbol{x}_t at intermediate steps, making it infeasible to directly enforce constraints on the clean sample x0\boldsymbol{x}_0 at each noisy level. As a workaround, constraints are typically applied to the expectation of clean samples E[x0∣xt]\mathbb{E}[\boldsymbol{x}_0|\boldsymbol{x}_t], which is estimated using the learned score network. However, imposing PDE constraints on the expectation does not strictly represent the one on the true clean data, known as Jensen's Gap. This gap creates a trade-off: enforcing PDE constraints may come at the cost of reduced accuracy in generative modeling. To address this, we propose a simple yet effective post-hoc distillation approach, where PDE constraints are not injected directly into the diffusion process, but instead enforced during a post-hoc distillation stage. We term our method as Physics-Informed Distillation of Diffusion Models (PIDDM). This distillation not only facilitates single-step generation with improved PDE satisfaction, but also support both forward and inverse problem solving and reconstruction from randomly partial observation. Extensive experiments across various PDE benchmarks demonstrate that PIDDM significantly improves PDE satisfaction over several recent and competitive baselines, such as PIDM, DiffusionPDE, and ECI-sampling, with less computation overhead. Our approach can shed light on more efficient and effective strategies for incorporating physical constraints into diffusion models.

Yi Zhang, Peng Wang, Difan Zou

Institute of Data Science · Department of Computer and Information Science · Institute of Data Science & School of Computing and Data Science

Physics-Informed Diffusion ModelsDiffusion Model DistillationGenerative ModelingPhysics-Informed MLPDE Inverse Problems
AskPDF
May 16, 2025math.NA

Accelerating Natural Gradient Descent for PINNs with Randomized Numerical Linear Algebra

Natural Gradient Descent (NGD) has emerged as a promising optimization algorithm for training neural network-based solvers for partial differential equations (PDEs), such as Physics-Informed Neural Networks (PINNs). However, its practical use is often limited by the high computational cost of solving linear systems involving the Gramian matrix. While matrix-free NGD methods based on the conjugate gradient (CG) method avoid explicit matrix inversion, the ill-conditioning of the Gramian significantly slows the convergence of CG. In this work, we extend matrix-free NGD to broader classes of problems than previously considered and propose the use of Randomized Numerical Linear Algebra (RandNLA) techniques for efficient preconditioning of the inner CG solver. The resulting algorithms demonstrate substantial performance improvements over existing NGD-based methods on a range of PDE problems discretized using neural networks, and offer competitive results compared to other state-of-the-art optimizers.

Ivan Bioli, Carlo Marcati, Giancarlo Sangalli

Department of Mathematics, University of Pavia, Via A. Ferrata 5, 27100 Pavia, Italy · Department of Civil Engineering and Architecture, University of Pavia, Via A. Ferrata 3, 27100 Pavia, Italy · Istituto di Matematica Applicata e Tecnologie Informatiche “E. Magenes”, CNR, Via A. Ferrata 1, 27100 Pavia, Italy

Neural PDE SolversGradient DescentPhysics-Informed ML
AskPDF
May 6, 2025stat.ML

Physics-Informed Sylvester Normalizing Flows for Bayesian Inference in Magnetic Resonance Spectroscopy

Magnetic resonance spectroscopy (MRS) is a non-invasive technique to measure the metabolic composition of tissues, offering valuable insights into neurological disorders, tumor detection, and other metabolic dysfunctions. However, accurate metabolite quantification is hindered by challenges such as spectral overlap, low signal-to-noise ratio, and various artifacts. Traditional methods like linear-combination modeling are susceptible to ambiguities and commonly only provide a theoretical lower bound on estimation accuracy in the form of the Cramér-Rao bound. This work introduces a Bayesian inference framework using Sylvester normalizing flows (SNFs) to approximate posterior distributions over metabolite concentrations, enhancing quantification reliability. A physics-based decoder incorporates prior knowledge of MRS signal formation, ensuring realistic distribution representations. We validate the method on simulated 7T proton MRS data, demonstrating accurate metabolite quantification, well-calibrated uncertainties, and insights into parameter correlations and multi-modal distributions.

Julian P. Merkofer, Dennis M. J. van de Sande, Alex A. Bhogal +1

Eindhoven University of Technology, Eindhoven, The Netherlands · University Medical Center Utrecht, Utrecht, The Netherlands

Normalizing FlowsBayesian InferencePhysics-Informed ML
AskPDF
Mar 24, 2025cs.LG

Residuals Are Not Enough: Limits of Physics-Informed Pre-Training for Scientific Foundation Models

Scientific foundation models (SciFMs) aim to learn generalizable representations of physical systems governed by partial differential equations (PDEs), enabling transfer across tasks and domains. While physics-informed methods, which leverage PDE residuals as supervisory signals, have shown promise in scientific machine learning (SciML) for improving accuracy and reducing data requirements, their potential in the context of SciFMs remains relatively unexplored. In this evaluation study, we investigate whether (and how) physics-informed pre-training improves the generalization, robustness, and data efficiency of SciFMs. We conduct systematic experiments across a diverse set of PDEs, ranging from simple problems with periodic boundary conditions to more challenging systems such as the Navier-Stokes equations and non-periodic geometries. Our results show that physics-informed pre-training provides clear benefits in nice,'' e.g., structured, well-aligned settings: it enhances generalization and reduces data dependence, compared to data-only pre-training. However, these advantages diminish significantly as the downstream tasks become harder,'' e.g., as they involve discontinuities or deviate from the pre-training distribution. In complex or structurally different problems, such as those involving new boundary conditions or PDE operators, physics-informed models may perform only on par with---or even worse---than data-driven baselines. While residual-based pre-training helps in idealized regimes, realizing broadly transferable SciFMs will likely require subtler spatiotemporal inductive biases and more principled integration of physical knowledge into model architectures.

Serge Kotchourko, Amin Totounferoush, Michael W. Mahoney +1

University of Stuttgart, Germany · ICSI, LBNL, and University of California, Berkeley, USA

PDE Surrogate ModelingPhysics-Informed ML
AskPDF
Mar 3, 2025eess.IV

Hyperspectral Image Restoration and Super-resolution with Physics-Aware Deep Learning for Biomedical Applications

Hyperspectral imaging is a powerful bioimaging tool which can uncover novel insights, thanks to its sensitivity to the intrinsic properties of materials. However, this enhanced contrast comes at the cost of system complexity, constrained by an inherent trade-off between spatial, spectral, and temporal resolution. To overcome this limitation, we present a self-supervised deep learning-based approach that restores and enhances pixel resolution post-acquisition without requiring external training data beyond the images to be restored. Fine-tuned using metrics aligned with the imaging model, our physics-aware method achieves a 16×\times pixel super-resolution enhancement and a 12×\times imaging speedup without the need of additional training data for transfer learning. Applied to both synthetic and experimental data from five different sample types, including healthy and diseased tissues, we demonstrate that the model preserves biological integrity, as we did not detect systematic loss of biological features or biologically consequential hallucinations in tested datasets. We also concretely demonstrate the model's ability to reveal disease-associated metabolic changes that would otherwise remain undetectable. Furthermore, we provide physical insights into the model's inner workings, paving the way for future refinements that could potentially reveal novel high resolution features in an explainable manner. All methods are available as open-source software on GitHub.

Yuchen Xiang, Zhaolu Liu, Monica Emili Garcia-Segura +12

Department of Metabolism, Digestion and Reproduction, Imperial College London, London, UK · Department of Mathematics, Imperial College London, London, UK · Department of Clinical Neurosciences and NIHR Biomedical Research Centre, University of Cambridge, Cambridge, UK +4

Self-Supervised LearningPhysics-Guided Image RestorationHyperspectral Image Super-ResolutionHyperspectral ImagingImage RestorationPhysics-Informed ML
AskPDF
Feb 16, 2025quant-ph

Physics-Informed Support Vector Kernels via Green-Function Analogies and Jackson-Chebyshev Spectral Design

Kernel selection for regression of physical observables is often heuristic. We investigate a physics-informed strategy in which functional forms and spectral structures associated with Green's functions motivate kernel selection without requiring an exact identification between a machine-learning kernel and a physical propagator. The principal construction is a Jackson-damped Chebyshev kernel inspired by the kernel polynomial method (KPM); its explicit feature map yields a positive-semidefinite Gram matrix by construction and provides an inspectable spectral prior for structured observables. We evaluate standard and custom SVR models on copper-conductivity proxies, local Dirac-like band dispersion, quartic-oscillator energy levels, photonic-crystal transmission, and Fibonacci-chain transmission using repeated nested validation, learning curves, random-forest and multilayer-perceptron baselines, and low-rank Nyström tests where relevant. The framework is intended for finite-data regression of precomputed observables while boundary conditions remain part of the physical model that generates those observables.

Nan-Hong Kuo, Renata Wong

Department of Physics, National Taiwan University, Taipei, Taiwan · Department of Artificial Intelligence, Chang Gung University, Taoyuan, Taiwan · Department of Neurology, Chang Gung Memorial Hospital, Keelung, Taiwan +1

Support Vector MachinesKernel MethodsPhysics-Informed MLSpectral Methods
AskPDF