math.STOct 10, 2026

Energy-Guided Learning for Heat Equation with Noisy Source: Optimal Estimation and Generalization Bounds

Authors: Zixiang Zhou, Xiaohui Chen, Shanghua Teng

Organizations: Department of Mathematics, University of Southern California · Thomas Lord Department of Computer Science, University of Southern California · Computing and Mathematical Sciences, California Institute of Technology

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Data and the governing partial differential equations (PDEs) of continuous-time dynamical systems provide rich structure for statistical learning. A central challenge is how to integrate data with PDE constraints in a principled way to improve generalization. In this paper, we characterize the statistical estimation and generalization properties of learning solutions to the heat equation with a noisy source function from finite, randomly sampled data. We first derive a novel variational formulation, which we characterize as the minimizer of loss function for an energy regularized and ellipticity debiased Sobolev regression. By formulating the learning task as a supervised problem over space-time domains under physically meaningful norms, our approach goes beyond the traditional supervised learning framework by effectively leveraging auxiliary data---samples of the source function of the heat equation---to design a physics-informed energy regularization. Our analysis reveals that learning in linear parabolic PDEs exhibits intrinsic anisotropy between spatial and temporal regularity, in sharp contrast to the static elliptic problems, leading to fundamentally different estimation and generalization behaviors. Specifically, we derive non-asymptotically rate-optimal estimation errors, as well as generalization error, which explicitly capture the interplay among temporal sampling, spatial resolution, and solution smoothness. Our numerical experiments demonstrate substantial orders of magnitude of reduction in testing errors against various physics-informed neural network methods on various parabolic and hyperbolic PDEs.

Figures & tables

Figure 1: Comparison of solutions from our method and PINN for a 1-D heat equation.
Figure 2: Top row: Near t≈0t\approx 0 snapshot comparison of solutions from our method and a PINN for a 1-D solution. Bottom row: zoom-in snapshot comparison of solutions from our method and a PINN different times.
Figure 3: 3D heat equation example. Left plot is 3D solution projection onto two coordinates at t=0t=0 . Middle plot is 3D solution projection onto two coordinates at t=0.3t=0.3 . Right plot is 3D solution projection onto two coordinates at t=0.9t=0.9 .
Figure 4: 3D Fokker-Planck equation example. Left plot is ground-truth solution projection onto x,y coordinates over time evolution. Right plot is our solution projection onto x,y coordinates over time evolution.
Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

BenchmarkPDE typeDimension
Heat equationLinear parabolicd=1,3d=1,3
Sine–Gordon equationNonlinear hyperbolicd=1,3d=1,3
LP1–LP3Variable-coefficient parabolicd=3d=3
Viscous HJ equationNonlinear parabolicd=1,3,10,20,50d=1,3,10,20,50
Appendix
Table 1: Overview of the additional PDEs experiments .
Figure 5: Error plot.
Figure 6: Learned solutions for PINN variants for 1d heat equation.
Figure 8: Comparison with other methods on LP1. Points show mean test errors and bars indicate one sample standard deviation over five seeds.
Figure 9: Comparison with other methods on LP2. Points show mean test errors and bars indicate one sample standard deviation over five seeds.
Figure 10: Comparison with other methods on LP3. Points show mean test errors and bars indicate one sample standard deviation over five seeds.
Figure 16: MSE reduction from regularization for parabolic equations at n,m=128n,m=128 and 5%5\% noise.
Equationρu\rho_{u}ρf=0\rho_{f}=0ρf=0.05\rho_{f}=0.05ρf=0.10\rho_{f}=0.10
LP1001.616×10−41.616\times 10^{-4}1.609×10−41.609\times 10^{-4}1.608×10−41.608\times 10^{-4}
LP10.050.051.644×10−41.644\times 10^{-4}1.641×10−41.641\times 10^{-4}1.636×10−41.636\times 10^{-4}
LP10.10.12.070×10−42.070\times 10^{-4}2.073×10−42.073\times 10^{-4}2.072×10−42.072\times 10^{-4}
LP2001.604×10−41.604\times 10^{-4}1.593×10−41.593\times 10^{-4}1.597×10−41.597\times 10^{-4}
LP20.050.051.659×10−41.659\times 10^{-4}1.657×10−41.657\times 10^{-4}1.654×10−41.654\times 10^{-4}
LP20.10.12.091×10−42.091\times 10^{-4}2.074×10−42.074\times 10^{-4}2.067×10−42.067\times 10^{-4}
Appendix
Table 2: Median test relative L2L^{2} error over ten paired seeds for the heatmap-selected mixed- H1H^{1} estimator under additive solution and forcing noise. Derivative labels and validation/test values are exact.
Equation(0,0)(0,0)(0.05,0.05)(0.05,0.05)(0.10,0.10)(0.10,0.10)Δ0.05\Delta_{0.05}Δ0.10\Delta_{0.10}
LP11.616×10−41.616\times 10^{-4}1.641×10−41.641\times 10^{-4}2.072×10−42.072\times 10^{-4}+1.50%+1.50\%+28.17%+28.17\%
LP21.604×10−41.604\times 10^{-4}1.657×10−41.657\times 10^{-4}2.067×10−42.067\times 10^{-4}+3.28%+3.28\%+28.85%+28.85\%
LP31.593×10−41.593\times 10^{-4}1.666×10−41.666\times 10^{-4}2.090×10−42.090\times 10^{-4}+4.59%+4.59\%+31.22%+31.22\%
HJ5.569×10−35.569\times 10^{-3}6.075×10−36.075\times 10^{-3}9.574×10−39.574\times 10^{-3}+9.09%+9.09\%+71.90%+71.90\%
Appendix
Table 3: Fresh reruns of the clean and jointly noisy conditions with ten paired seeds. These values are the corresponding diagonal entries of Table 2 ; percentage changes use unrounded medians.
Figure 19: LP1: relative L2L^{2} error (first heatmap) and test MSE (second heatmap). Within each heatmap, the upper row has noisy ff and clean uu ; the lower row has clean ff and noisy uu . Orange outlines mark the validation-selected β\beta for each fixed (ϵ,N)(\epsilon,N) .
Figure 20: LP1: signed value-MSE generalization gap, G=Etest−EtrainG=E_{\mathrm{test}}-E_{\mathrm{train}} (first heatmap), and test-MSE ratio to the paired mixed- H1H^{1} fitting baseline (second heatmap). Gap cells are means of signed gaps across seeds; ratios compare mean test MSEs. Ratios below one indicate improvement. Noise rows and orange outlines follow Figure 19 .
Figure 21: LP2: relative L2L^{2} error (first heatmap) and test MSE (second heatmap). Within each heatmap, the upper row has noisy ff and clean uu ; the lower row has clean ff and noisy uu . Orange outlines mark the validation-selected β\beta for each fixed (ϵ,N)(\epsilon,N) .
Figure 22: LP2: signed value-MSE generalization gap, G=Rtest−RtrainG=R_{\mathrm{test}}-R_{\mathrm{train}} (first heatmap), and test-MSE ratio to the paired mixed- H1H^{1} fitting baseline (second heatmap). Gap cells are means of signed gaps across seeds; ratios compare mean test MSEs. Ratios below one indicate improvement. Noise rows and orange outlines follow Figure 21 .
Figure 23: LP3: relative L2L^{2} error (first heatmap) and test MSE (second heatmap). Within each heatmap, the upper row has noisy ff and clean uu ; the lower row has clean ff and noisy uu . Orange outlines mark the validation-selected β\beta for each fixed (ϵ,N)(\epsilon,N) .
Figure 24: LP3: signed value-MSE generalization gap, G=Rtest−RtrainG=R_{\mathrm{test}}-R_{\mathrm{train}} (first heatmap), and test-MSE ratio to the paired mixed- H1H^{1} fitting baseline (second heatmap). Gap cells are means of signed gaps across seeds; ratios compare mean test MSEs. Ratios below one indicate improvement. Noise rows and orange outlines follow Figure 23 .
Figure 25: HJB-3D: relative L2L^{2} error (first heatmap) and test MSE (second heatmap). Within each heatmap, the upper row has noisy ff and clean uu ; the lower row has clean ff and noisy uu . Orange outlines mark the validation-selected β\beta for each fixed (ϵ,N)(\epsilon,N) .
Figure 26: HJB-3D: signed value-MSE generalization gap, G=Rtest−RtrainG=R_{\mathrm{test}}-R_{\mathrm{train}} (first heatmap), and test-MSE ratio to the paired mixed- H1H^{1} fitting baseline (second heatmap). Gap cells are means of signed gaps across seeds; ratios compare mean test MSEs. Ratios below one indicate improvement. Noise rows and orange outlines follow Figure 25 .
Figure 27: Learned solution from our method and the exact traveling-wave solution for the one-dimensional Sine–Gordon equation.
Figure 28: Error comparison for the one-dimensional Sine–Gordon equation: vanilla PINN, the weighted-energy objective without solution labels, and our complete energy-guided estimator.
Figure 29: Relative L2L^{2} error over training for the three objectives on the one-dimensional Sine–Gordon equation with 5,000 training samples.
Figure 30: Batch-size and learning-rate ablations for the one-dimensional Sine–Gordon equation. Left: batch-size comparison with learning rate 10−310^{-3} . Right: training with the reduced learning rate 10−410^{-4} .
Figure 31: Method comparison for the one-dimensional Sine–Gordon equation with learning rate 10−410^{-4} .
Figure 32: Representative slices for the three-dimensional Sine–Gordon equation. The ground-truth slices are shown together with the corresponding errors of the learned solutions.
Figure 28
More in this topic

Physics-Informed ML

Physics-informed machine learning combines data-driven models with physical laws to solve, predict, and infer scientific systems.

See all 836 papers

Explore similar work

CardsList

  1. Generalization Error Bounds for Picard-Type Operator Learning in Nonlinear Parabolic PDEs

    May 11, 2026Koichi Taniguchi, Sho SonodaPDE SolvingPDE Operator Learning

    Department of Mathematical and Systems Engineering, Faculty of Engineering Shizuoka University 3-5-1 Johoku, Chuo-ku, Hamamatsu, 432-8561, Japan · RIKEN AIP / CyberAgent 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan

  2. A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learning

    May 25, 2026Thien V. Nguyen, Amaury Habrard, Benjamin GuedjPAC-Bayesian Generalization BoundsPhysics-Informed ML

    Université Jean Monnet Saint-Étienne, CNRS, Institut d’Optique Graduate School, Laboratoire Hubert Curien UMR 5516, F-42023, Saint-Étienne, France · Université Jean Monnet Saint-Étienne, CNRS, Institut d’Optique Graduate School, Laboratoire Hubert Curien UMR 5516, Inria, F-42023, Saint-Étienne, France · Institut Universitaire de France +1

  3. Fast Learning Rates for Physics-Informed Kernel Methods

    Sep 16, 2026Luc Brogat-Motte, Joachim Bona-Pellissier, Giacomo Meanti +1Kernel RegressionPhysics-Informed ML

    SAIL Unit, Istituto Italiano di Tecnologia, Genoa, Italy · MaLGa Center, DIBRIS, Università degli Studi di Genova, Genoa, Italy