Parameter-Minimal Neural DE Solvers via Horner Polynomials

T. Matulić; D. Seršić

cs.LGFeb 16, 2026

Parameter-Minimal Neural DE Solvers via Horner Polynomials

Authors: T. Matulić, D. Seršić

Organizations: Dept. of Electronic Systems and Information Processing, Faculty of Electrical Engineering and Computing, University of Zagreb, Zagreb, Croatia

Abstract

We propose a parameter-minimal neural architecture for solving differential equations by restricting the hypothesis class to Horner-factorized polynomials, yielding an implicit, differentiable trial solution with only a small set of learnable coefficients. Initial conditions are enforced exactly by construction by fixing the low-order polynomial degrees of freedom, so training focuses solely on matching the differential-equation residual at collocation points. To reduce approximation error without abandoning the low-parameter regime, we introduce a piecewise ("spline-like") extension that trains multiple small Horner models on subintervals while enforcing continuity (and first-derivative continuity) at segment boundaries. On illustrative ODE benchmarks and a heat-equation example, Horner networks with tens (or fewer) parameters accurately match the solution and its derivatives and outperform small MLP and sinusoidal-representation baselines under the same training settings, demonstrating a practical accuracy-parameter trade-off for resource-efficient scientific modeling.

Figures & tables

Type	ODE	Initial conditions	Exact solution
A	$\displaystyle x^{\prime}(t)+2x(t)=1$	$\displaystyle x(0)=1$	$\displaystyle x(t)=\tfrac{1}{2}\bigl(1+e^{-2t}\bigr)$
B	$\displaystyle x^{\prime}(t)\,x(t)=t$	$\displaystyle x(0)=1$	$\displaystyle x(t)=\sqrt{t^{2}+1}$
C	$\displaystyle x^{\prime\prime}(t)+4x^{\prime}(t)+13x(t)=2$	$\displaystyle x(0)=0,\;\;x^{\prime}(0)=1$	$\displaystyle x(t)=\frac{2}{13}+e^{-2t}\!\left(\frac{3}{13}\sin(3t)-\frac{2}{13}\cos(3t)\right)$

TABLE I: Benchmark ODEs used throughout the paper.

Panel of Figure 2 — Fig. 1 : Type A benchmark: baseline network predictions for the solution and its derivatives. Rows correspond to architectures (MLP with Leaky ReLU, MLP with sigmoid, and SIREN), while columns show $x(t)$ , $x^{\prime}(t)$ , and $x^{\prime\prime}(t)$ . Each panel compares the learned output against the analytical reference.

Panel of Figure 3 — Fig. 2 : Type A: polynomial regression (degree $m=15$ ) versus the analytical solution on $I=[0,4]$ .

Panel of Figure 4 — Fig. 3 : Matched forcing: polynomial regression (degree $m=15$ ) versus the analytical solution on $I=[0,4]$ .

Panel of Figure 5 — Fig. 4 : Type C: polynomial regression (degree $m=15$ ) versus the analytical solution on $I=[0,3]$ .

Panel of Figure 6 — Fig. 5 : Horner-based architecture used as a parameter-minimal neural solver.

		LeakyReLU	Sigmoid	SIREN	Horner net
Type A	$\mathrm{RMSE}(N,x)$	$2.6\cdot 10^{-3}$	$8.5\cdot 10^{-5}$	$2.7\cdot 10^{-5}$	$\mathbf{4.6\cdot 10^{-6}}$
	$\mathrm{RMSE}(N^{\prime},x^{\prime})$	$2.0\cdot 10^{-2}$	$5.6\cdot 10^{-4}$	$5.2\cdot 10^{-4}$	$\mathbf{3.3\cdot 10^{-5}}$
	$\mathrm{RMSE}(N^{\prime\prime},x^{\prime\prime})$	$1$	$2.3\cdot 10^{-2}$	$2.7\cdot 10^{-2}$	$\mathbf{5.6\cdot 10^{-4}}$
Type B	$\mathrm{RMSE}(N,x)$	$2.5\cdot 10^{-3}$	$2.7\cdot 10^{-4}$	$6.9\cdot 10^{-4}$	$\mathbf{6.6\cdot 10^{-6}}$
	$\mathrm{RMSE}(N^{\prime},x^{\prime})$	$1.8\cdot 10^{-2}$	$1.5\cdot 10^{-3}$	$1\cdot 10^{-3}$	$\mathbf{7.8\cdot 10^{-5}}$
	$\mathrm{RMSE}(N^{\prime\prime},x^{\prime\prime})$	$7.7\cdot 10^{-1}$	$1.4\cdot 10^{-2}$	$1.6\cdot 10^{-2}$	$\mathbf{2.3\cdot 10^{-3}}$

TABLE II : RMSE on a dense uniform evaluation grid (

N_{\mathrm{eval}}=100\,000

points on

I

) for the solution and its first two derivatives. Lower is better; the best value in each row is shown in bold.

Panel of Figure 8 — Fig. 6 : Horner network predictions for the three benchmark ODEs (Types A–C): solution and derivatives compared to analytical references.

Panel of Figure 9 — Fig. 7 : Comparison of the solution obtained from the multidimensional Horner network with the exact solution of the heat equation. The error is minimal, even with only 45 learnable parameters.

Fig. 8 : Spline-like model architecture. The input value

x

determines which Horner network

HN_{i}

is selected based on the interval it falls into, ensuring a piecewise smooth representation of the solution.

Panel of Figure 11 — Fig. 9 : Comparison of the solution and its derivatives predicted by the spline-like Horner network with the exact solutions. The results demonstrate improved accuracy and smoothness across the subinterval boundaries.

Neural PDE Solvers

Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.

See all 261 papers

Approximation of solutions of parameter-dependent problems by residual neural networks
Jul 15, 2026Ana CarpioNeural Surrogate Modeling Neural Network Optimization
Universidad Complutense de Madrid, Spain
DeSyR: A Decoupled Symbolic Recovery Framework with PINN-Guided Structure Search and Physics-Informed Coefficient Refinement
Sep 1, 2026Pancheng Niu, Jun Guo, Qiaolin He +2Parameter Estimation Physics-Informed ML
Chengdu University of Information Technology, College of Applied Mathematics, Chengdu, 610225, China · Sichuan University, School of Mathematics, Chengdu, 610065, China · Hong Kong Polytechnic University, Department of Computing, Hong Kong, 999077, China +1
Differentiable Chemistry in PINNs for Solving Parameterized and Stiff Reaction Systems
May 6, 2026Miloš Babić, Franz M. Rohrhofer, Stefan PoschDynamical Systems Differentiable Physics
CD Laboratory for Physics-driven Machine Learning in Industrial Applications, Graz, Austria · The Institute of Thermodynamics and Sustainable Propulsion Systems, Graz University of Technology, Graz, Austria · Know Center Research GmbH, Graz, Austria

Jul 15, 2026J/K move · Enter open · S save

Approximation of solutions of parameter-dependent problems by residual neural networks

Ana Carpio

Universidad Complutense de Madrid, Spain

Neural Surrogate Modeling Neural Network Optimization Neural Network Approximation Theory

Open paper AskPDF arXiv

We develop a convergent scheme to train neural networks involving analytic activation functions based on gradient flows. Convergence properties are guaranteed by Lojasiewicz theory. The main advantage of this approach is its simplicity of implementation. The coefficients of the network are approximated by solving a system of ordinary differential equations. We test the method by constructing residual neural network approximations of solutions of parametric problems. The dependence of the solutions of simple ordinary differential equations on a few parameters is correctly reproduced. The solutions of inverse problems involving wave constraints which depend on a few parameters can be reasonably approximated, even in regions in which the problem is severely ill posed.

Abstract

Neural PDE Solvers

Explore similar work

Approximation of solutions of parameter-dependent problems by residual neural networks

DeSyR: A Decoupled Symbolic Recovery Framework with PINN-Guided Structure Search and Physics-Informed Coefficient Refinement

Differentiable Chemistry in PINNs for Solving Parameterized and Stiff Reaction Systems