Hyperparameter selection for equation learning with biologically-informed neural networks

William Lavery; Jodie A. Cochrane; John T. Nardini; Sara Hamis

cs.LGOct 2, 2026

Hyperparameter selection for equation learning with biologically-informed neural networks

Authors: William Lavery, Jodie A. Cochrane, John T. Nardini, Sara Hamis

Organizations: Department of Information Technology, Uppsala University, Sweden · Department of Mathematics and Statistics, The College of New Jersey, USA

Abstract

Biologically-informed neural networks (BINNs) have emerged as a flexible subclass of physics-informed neural networks (PINNs) for learning terms in partial differential equations from data. BINNs are particularly suited for biological systems, where the governing equations are highly nonlinear and only partially known a priori, and where data observations are often sparse, noisy, and incomplete. However, applying BINNs effectively in practice depends critically on hyperparameter selection, which remains a central challenge in equation-learning frameworks. Hyperparameters are often chosen heuristically and only cursorily documented, which limits the reproducibility of results and the transferability of methods. We present a diagnostic workflow for hyperparameter selection that can be used when the ground-truth equations are not known. The workflow is guided by three main questions: (1) Are the benefits of greater network capacity worth the cost? (2) Do more training epochs keep reducing the validation loss? (3) Do the learned terms stop changing as network capacity and training increase? We apply our workflow to synthetic systems of varying complexity with known ground truth, spanning diffusion and growth right-hand side terms and data ranging from 1D+t to 2D+t. We demonstrate that the validation loss generally follows the true error in the learned terms and distil practical rules of thumb for selecting hyperparameters in the BINN architecture. By providing a structured workflow, practical guidelines, and suggested starting values for hyperparameter selection, this work lowers the barrier to BINN-based equation learning.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of the biologically-informed neural network (BINN) architecture. Schematic overviews of the network flow ( left ) and the corresponding network architectures ( right ) are shown for the diffusion equation and the reaction–diffusion equation. The ground truth is the diffusion function $D$ for the former, and the diffusion function $D$ together with the growth function $G$ for the latter. Discretised state data $u_{\text{data}}$ and their spatiotemporal coordinates $(\mathbf{x},t)$ are input to the state multilayer perceptron (MLP) $\mathrm{NN}_{u}$ , which outputs the predicted state $\hat{u}$ (blue). The right-hand side MLPs are trained jointly to recover the underlying biological mechanisms: the diffusion MLP ( $\mathrm{NN}_{D}$ , orange) learns the diffusion function $\hat{D}$ and the growth MLP ( $\mathrm{NN}_{G}$ , green) learns the growth function $\hat{G}$ . Training minimises a total loss comprising a data fitting term $\mathcal{L}_{1}(\hat{u})$ and a physics-informed term $\mathcal{L}_{2}(\hat{u},\hat{D},\hat{G})$ , the latter enforcing the governing partial differential equation and optionally incorporating user-specified biological constraints.

Panel of Figure 2 — Figure 2: Data points versus collocation points in $\bm{1\mathrm{D}{+}t}$ . Left : data grid for $u_{\mathrm{data}}$ , comprising $38\times 5$ points. Right : collocation points sampled over $1$ and $10$ training minibatches, coloured by minibatch. For $1$ minibatch, a single set $\mathcal{C}_{1}=\{(x_{k}^{(c)},t_{k}^{(c)})\}_{k=1}^{n_{c}}$ is sampled; for $10$ minibatches, the sets $\mathcal{C}_{1},\ldots,\mathcal{C}_{10}$ illustrate how repeated resampling of random collocation points progressively fills the spatiotemporal domain.

Panel of Figure 3 — Figure 3: Training and validation sets under different training–validation (TV) splits and initial conditions. Panels are generated using $1\text{D}{+}t$ data sampled at 38 spatial and 5 temporal points with an 80/20 TV split. Top row shows spatial density profiles at 3 time points. Bottom row shows the corresponding training and validation distributions over density. The two boxed panels use a cosine initial condition and differ only in the random seed defining the TV split, while the stand-alone panel uses an inverted cosine initial condition with the same seed as TV split 1.

Panel of Figure 4 — Figure 4: The hyperparameter-selection workflow. A semi-sequential procedure: the hyperparameters are decided in an order determined by the biologically-informed neural network architecture. The capacity—network width and/or depth—of the state multilayer perceptron (MLP), which must learn the data, is proposed first, and is confirmed and selected once it is robust to the downstream MLP capacities and early stopping, and otherwise re-proposed (dashed loop). The capacities of the RHS MLPs are then selected, followed by the early-stopping patience; if the RHS capacities prove not robust to that patience they are reselected (second dashed loop).

PDE term	Function	Expression	Parameters (units)
Diffusion	$D_{1}$	$d_{1,1}$	$d_{1,1}=0.02472~\mathrm{mm}^{2}\,\mathrm{days}^{-1}$
	$D_{2}$	$d_{2,1}+d_{2,2}\,\tilde{u}$	$(d_{2,1},d_{2,2})=(0.015,\,0.06)~\mathrm{mm}^{2}\,\mathrm{days}^{-1}$
	$D_{3}$	$d_{3,1}+d_{3,2}\,\tilde{u}^{2}$	$(d_{3,1},d_{3,2})=(0.01,\,0.044)~\mathrm{mm}^{2}\,\mathrm{days}^{-1}$
	$D_{4}$	$d_{4,1}+d_{4,2}\!\left(1-e^{-d_{4,3}\tilde{u}}\right)$	$\begin{aligned} (d_{4,1},d_{4,2})&=(0.003,\,0.095)~\mathrm{mm}^{2}\,\mathrm{days}^{-1},~d_{4,3}=2.5\end{aligned}$
Growth	$G_{1}$	$g_{1,1}$	$g_{1,1}=0.65~\mathrm{days}^{-1}$
	$G_{2}$	$g_{2,1}+g_{2,2}\,\tilde{u}$	$(g_{2,1},g_{2,2})=(1.2,\,-1.5)~\mathrm{days}^{-1}$

Table 1: Ground-truth diffusion and growth functions used to generate the simulated data. Functions are defined on the dimensionless density

\tilde{u}=u/K

;

K=1700\text{ cells mm}^{-2}

	$\bm{1\mathrm{D}{+}t}$	$\bm{2\mathrm{D}{+}t}$
Spatial grid points	$38$	$11\times 11$
Spatial domain	$\mathrm{L}=1.85\,\mathrm{mm}$	$\mathrm{L}_{1}=\mathrm{L}_{2}=1.85\,\mathrm{mm}$
Time points	$5$	$9$
Duration	$2\,\mathrm{days}$	$2\,\mathrm{days}$
Noise (MPE, $\gamma$ )	—	$(5\%,~1)$
Initial condition	$\tfrac{1}{2}\left[1+\cos(2\pi\xi)\right]$	$\tfrac{1}{4}\cos^{2}\!\bigl(\pi(\xi_{1}-\tfrac{1}{2})\bigr)\cos^{2}\!\bigl(\pi(\xi_{2}-\tfrac{1}{2})\bigr)$

Table 2: Data-generation settings. Normalised spatial coordinates are defined as

\xi=(x-x_{\min})/\mathrm{L}

1\mathrm{D}{+}t

and

\xi_{i}=(x_{i}-x_{i,\min})/\mathrm{L}_{i}

2\mathrm{D}{+}t

. MPE denotes mean percentage error.

Panel of Figure 5 — Figure 5: Diagnostics of the hyperparameter-selection workflow. Exemplified on data generated from a simple PDE with one term—constant diffusion—on the RHS. (a) Trade-off between validation loss and runtime across varying widths ( $W_{u}$ ) of the state network and the early-stopping patience ( $ES$ ), with the RHS network width fixed (here at $W_{D}=2$ ). Bars and whiskers respectively show mean values and min–max ranges across $3$ repeats with different training–validation (TV) splits. (b) Validation-loss trajectories over training epochs, with the stopping points for the investigated early-stopping values marked. (c) The same trade-off as in (a) across varying values for $W_{u}$ and $W_{D}$ , with early stopping fixed (here at $ES=2000$ ). (d) The learned function $\hat{D}(u)$ at successive early-stopping values (for $W_{D}=2,W_{u}=64$ ). Regions outside the central $90\%$ of the density support are shown in grey. The dashed ground-truth line and $\pm 0.5\%$ bands play no role in the selection workflow and are shown only for reference. In (b) and (d) filled bands show min–max ranges across $3$ repeats with different TV splits.

Panel of Figure 6 — Figure 6: Impact of MLP depth on the fit and the recovered function. Exemplified on the Case I problem, with data generated from a simple PDE with one term—constant diffusion—on the right-hand side (RHS). $ES=2000$ is fixed. (a) Trade-off between validation loss and runtime across the width and depth of the state network $\mathrm{NN}_{u}$ . (b) The same as (a) for the diffusion network $\mathrm{NN}_{D}$ . (c) Consistency of the learned diffusion function $\hat{D}(u)$ across the depth of $\mathrm{NN}_{u}$ with width $W_{u}=64$ . $\mathrm{NN}_{D}$ ( $W_{D}=2$ , $L_{D}=3$ ) is fixed. (d) The same as (c) across the depth of $\mathrm{NN}_{D}$ with width $W_{D}=2$ . $\mathrm{NN}_{u}$ ( $W_{u}=64$ , $L_{u}=3$ ) is fixed. The dashed ground-truth line and $\pm 0.5\%$ bands play no role in the selection workflow and are shown only for reference.

Panel of Figure 7 — Figure 7: Ground-truth verification of the learned function in the synthetic Case I study. Data generated from a simple PDE with one term—constant diffusion—on the right-hand side (RHS). Mean-squared error (MSE) of the learned diffusion coefficient function $\hat{D}(u)$ against the ground-truth $D_{1}$ . (a) MSE over training epochs, for different $W_{u}$ and early-stopping patiences $ES$ with $W_{D}=2$ . Filled bands show min–max ranges across $3$ repeats with different training–validation (TV) splits. (b) MSE across varying widths ( $W_{u}$ ) of the state network and the early-stopping patience ( $ES$ ), with the RHS network width fixed (here at $W_{D}=2$ ). (c) The same MSE across varying values for $W_{u}$ and $W_{D}$ , with fixed $ES=2000$ . In (b) and (c) , the bars and whiskers respectively show mean values and min–max ranges across the different TV splits, and mean percentage error (MPE) levels of $0.25\%$ , $0.5\%$ , and $1\%$ are indicated by horizontal lines.

Panel of Figure 8 — Figure 8: Selecting $W_{D}$ for non-constant diffusion. Data are generated from three diffusion functions of increasing complexity: linear, quadratic, and exponential. (a) Trade-off between validation loss and runtime across varying widths ( $W_{D}$ ) of the diffusion network and the early-stopping patience ( $ES$ ), with the state network width fixed (here at $W_{u}=64$ ). Bars and whiskers respectively show mean values and min–max ranges across $3$ repeats with different training–validation (TV) splits. (b) The learned diffusion function $\hat{D}(u)$ across these widths, shown at $ES=2000$ , $3000$ and $5000$ . Filled bands show min–max ranges across the $3$ TV splits. Regions outside the central $90\%$ of the density support are shown in grey. The dashed ground-truth line and $\pm 5\%$ bands play no role in the selection workflow and are shown only for reference.

Panel of Figure 9 — Figure 9: Selecting hyperparameters under coupled diffusion and growth. Data are generated from constant diffusion ( $D_{1}$ ) paired with four growth forms—constant ( $G_{1}$ ), linear ( $G_{2}$ ), quadratic ( $G_{3}$ ), and exponential ( $G_{4}$ ). Consistency of the learned diffusion and growth functions across early-stopping patiences $ES\in\{2000,3000,5000\}$ with fixed $W_{u}=64$ and $W_{D}=W_{G}=2$ is shown. Filled bands show min–max ranges across $3$ repeats with different training–validation splits. Regions outside the central $90\%$ of the density support are shown in grey. The dashed ground-truth line and $\pm 5\%$ bands play no role in the selection workflow and are shown only for reference.

Panel of Figure 10 — Figure 10: Selecting the early-stopping patience on noisy $\bm{2\mathrm{D}{+}t}$ data. Data generated from constant diffusion ( $D_{1}$ ) and linear growth ( $G_{2}$ ), with $5\%$ MPE noise added, at fixed widths ( $W_{u}=64$ , $W_{D}=W_{G}=4$ ). (a) Cost-performance trade-off between validation loss and runtime across the early-stopping patience ( $ES$ ). Bars and whiskers respectively show mean values and min–max ranges across $3$ repeats with different training–validation (TV) splits. (b) Consistency of the learned diffusion and growth functions, $\hat{D}(u)$ and $\hat{G}(u)$ , across varying $ES$ . Regions outside the central $90\%$ of the density support are shown in grey. The dashed ground-truth line and $\pm 5\%$ bands play no role in the selection workflow and are shown only for reference.

Panel of Figure 11 — Figure 11: A user guide to hyperparameter selection. Top . Which hyperparameters a researcher is recommended to reuse from the table below, and how the remaining ones are obtained, depend on how closely their problem matches the reaction–diffusion setting. Bottom . Hyperparameters categorised by groups and problem; grey cells mark hyperparameters that must be re-derived on the user’s own data.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 14 — Figure A.1 : Schematic overview of diffusion and reaction–diffusion models for synthetic dataset generation. Top . The diffusion-only BINN architecture handles Case I (constant diffusion) and Case II (linear, quadratic, and exponential diffusion). Bottom . The reaction–diffusion BINN architecture handles Cases III and IV ( $1\text{D}{+}t$ and $2\text{D}{+}t$ , respectively). Cases I–III are $1\text{D}{+}t$ and use a cosine initial condition with noise-free cell-density data. Case IV is $2\text{D}{+}t$ and uses a cosine-squared initial condition with $5\%$ additive Gaussian noise ( $\gamma=1$ ), illustrated by a representative constant- $x_{2}$ cross-section.

Panel of Figure 15 — Figure A.2 : Ground-truth verification for Case II. Mean-squared error (MSE) of the learned diffusion function $\hat{D}(u)$ with respect to the known forms, across the diffusion-network width ( $W_{D}$ ) and the early-stopping patience ( $ES$ ), one panel per form: (a) linear ( $D_{2}$ ), (b) quadratic ( $D_{3}$ ), (c) exponential ( $D_{4}$ ).

Panel of Figure 16 — Figure A.3 : Cost-performance and function-error diagnostics for Case III across right-hand side (RHS) network widths. Constant diffusion ( $D_{1}$ ) paired with four growth forms—constant ( $G_{1}$ ), linear ( $G_{2}$ ), quadratic ( $G_{3}$ ), and exponential ( $G_{4}$ ). Diffusion and growth network widths are always equal ( $W_{D}=W_{G}$ ). Early-stopping patience is fixed at $3000$ epochs. Panel (a) shows the validation loss across the four ground-truth growth functions and the RHS network widths, (b) the corresponding runtimes, (c) the diffusion-function error, and (d) the growth-function error.

Panel of Figure 17 — Figure A.4 : Cost-performance and function-error diagnostics for Case III across early-stopping patiences. Constant diffusion ( $D_{1}$ ) paired with four growth forms—constant ( $G_{1}$ ), linear ( $G_{2}$ ), quadratic ( $G_{3}$ ), and exponential ( $G_{4}$ ). Early-stopping patience across $ES\in\{2000,3000,5000\}$ , and right-hand side network widths $W_{D}=W_{G}=2$ . (a) shows the validation loss across the four ground-truth growth functions and early-stopping patience, (b) the corresponding runtimes, (c) the diffusion-function error, and (d) the growth-function error.

Panel of Figure 18 — Figure A.5 : Performance for Case IV across right-hand side (RHS) widths. Constant diffusion ( $D_{1}$ ) paired with linear growth ( $G_{2}$ ). Panel (a) shows the validation loss across four joint RHS MLP widths $W_{D}=W_{G}\in\{2,4,8,16\}$ and four early-stopping patience values $ES\in\{500,1000,2000,3000\}$ . Panel (b) shows the corresponding diffusion and growth predictions for $ES=3000$ .

Panel of Figure 19 — Figure A.6 : Ground-truth verification for Case IV with $W_{D}=W_{G}=4$ . Mean-squared error (MSE) of learned functions against the known $D_{1}(u)$ and $G_{2}(u)$ , across the early-stopping patiences ( $ES$ ), for the noisy $2\text{D}{+}t$ data. (a) Diffusion-function error. (b) Growth-function error.

Panel of Figure 20 — Figure S.1 : Performance across PDE-loss weights $\lambda_{\text{PDE}}$ for synthetic Case I with 1% mean percentage error noise. Ground truth: $D_{1}(u)$ . Hyperparameters: $\lambda_{\text{data}}=1$ , $W_{u}=64$ , $W_{D}=2$ , early-stopping patiences $ES\in\{1000,2000\}$ , and $\lambda_{\text{PDE}}\in\{1,2,10,100\}$ . (a) , (b) consist of $3$ training–validation (TV) splits: markers show the minimum for each $ES$ , and shading shows the corresponding min–max range. (c) – (i) consist of the first TV split at $ES=2000$ . (a) running minimum validation loss. (b) running minimum diffusion-function mean-squared error (MSE). (c) running normalised mean validation data loss smoothed over the preceding 25 epochs. (d) running normalised mean validation PDE loss smoothed over the preceding 25 epochs. (e) mean difference between the density network and the data as a function of density (bin size of $100\text{ cells mm}^{-2}$ ). Blocks (f) – (i) show diffusion predictions sampled every 100 epochs with light-to-dark shading corresponding to increasing epochs, with the final best model indicated by a solid black line. The $\pm 5\%$ mean percentage error (MPE) bands are shown for reference.

Panel of Figure 21 — Figure S.2 : Performance when using large weights in the biological loss for synthetic Case I. Ground truth: $D_{1}(u)$ . Hyperparameters: $W_{u}\in\{16,64\}$ , $W_{D}=2$ , and early-stopping patience $ES=1000$ . (a) Running minimum validation loss across $3$ training–validation (TV) splits: markers show the minimum for $ES=1000$ , and shading shows the corresponding min–max range. (b) Diffusion predictions for $W_{u}=16$ sampled every $100$ epochs for the first TV split. Light-to-dark shading corresponds to increasing epochs, with the best model indicated by a solid black line. The $\pm 5\%$ mean percentage error (MPE) bands are shown for reference. (c) Same as (b) for $W_{u}=64$ . (d) – (f) as in (a) – (c) for $\lambda_{D_{\lim}}=\lambda_{\partial D}=0$ .

Panel of Figure 22 — Figure S.3 : Performance across monotonicity weights $\lambda_{\partial D}$ for synthetic Case I. Ground truth: $D_{1}(u)$ . Hyperparameters: $W_{u}\in\{16,64\}$ , $W_{D}=2$ , and early-stopping patience $ES=1000$ . All plots consist of $3$ training–validation splits: markers show the minimum for $ES=1000$ , and shading shows the corresponding min–max range. Within each block (a) – (d) , from left to right: running minimum validation loss, running minimum diffusion-function mean-squared error (MSE), and running mean monotonicity loss.

Panel of Figure 23 — Figure S.4 : Diffusion predictions across monotonicity weights $\lambda_{\partial D}$ for synthetic Case I. Ground truth: $D_{1}(u)$ . Hyperparameters: $W_{u}\in\{16,64\}$ , $W_{D}=2$ , and early-stopping patience $ES=1000$ . All plots correspond to the first of the $3$ training–validation splits. Blocks (a) – (d) : columns show predictions for $W_{u}=16$ and $64$ . Light-to-dark shading corresponds to increasing epochs, with the final best model indicated by a solid black line. The $\pm 5\%$ mean percentage error (MPE) bands are shown for reference.

Panel of Figure 24 — Figure S.5 : Performance across collocation points $n_{c}$ for synthetic Case III. Ground truth: $D_{1}(u)$ and $G_{2}(u)$ . Hyperparameters: $W_{u}=64$ , $W_{D}=W_{G}=4$ , early-stopping patiences $ES\in\{1000,3000\}$ and $n_{c}\in\{10,100,1000\}$ . (a) – (e) consist of $3$ training–validation (TV) splits: markers show the minimum for each $ES$ , and shading in (a) , (d) and (e) shows the corresponding min–max range. (a) running minimum validation loss. (b) runtimes. (c) running minimum PDE and data loss components. (d) and (e) running minimum mean-squared error (MSE) for diffusion and growth predictions. Blocks (f) – (h) show $n_{c}=10$ , $100$ and $1000$ for $ES=3000$ , where predictions correspond to the second TV split. Within each block, from left to right: diffusion predictions over a wide interval, diffusion predictions on a narrow interval, and growth predictions over a wide interval. Light-to-dark shading corresponds to increasing epochs, with the final best model indicated by a solid black line. The $\pm 5\%$ mean percentage error (MPE) bands are shown for reference.

Panel of Figure 25 — Figure S.6 : Performance across activation functions SiLU , tanh and sigmoid for synthetic Case I. Hyperparameters: $W_{u}\in\{8,16,32,64,128\}$ , $W_{D}=2$ , early-stopping patiences $ES\in\{1000,2000,3000\}$ and activation functions $\{\texttt{SiLU},\texttt{tanh},\texttt{sigmoid}\}$ . For all running-minimum plots across training–validation (TV) splits: markers show the minimum for each $ES$ , and shading shows the corresponding min–max range. For each of (a) – (c) top to bottom: validation loss, running minimum validation loss, running minimum diffusion mean-squared error (MSE), and diffusion predictions across epochs for the first TV split of $W_{u}=64,W_{D}=2$ . Light-to-dark shading corresponds to increasing epochs, with the final best model indicated by a solid black line. The $\pm 5\%$ mean percentage error (MPE) bands are shown for reference.

Panel of Figure 26 — Figure S.7 : Performance of generalised least squares (GLS) loss versus least squares (LS) loss for synthetic Case I with 1% mean percentage error noise. Ground truth: $D_{1}(u)$ . Hyperparameters: $W_{u}=64$ , $W_{D}=2$ , early-stopping patiences $ES\in\{500,2000\}$ . LS and GLS correspond to $\mathcal{L}_{\text{data}}$ and $\mathcal{L}_{\text{GLS}}$ objectives, respectively. (a) – (f) consist of $3$ training–validation (TV) splits: markers show the minimum for each $ES$ , and shading shows the corresponding min–max range. (a) running minimum validation loss. (b) running minimum validation data and PDE loss components. (c) running minimum diffusion-function MSE. (d) ratio $\mathcal{L}_{\text{GLS}}/\mathcal{L}_{\text{data}}$ across epochs. (e) effective sample size $N_{\text{eff}}$ as a fraction of total sample size $N$ , computed from the weights $w_{k}^{\text{LS}}=(\hat{u}_{k}^{\text{LS}})^{-2}$ and $w_{k}^{\text{GLS}}=(\hat{u}_{k}^{\text{GLS}})^{-2}$ (data-generating $\gamma=1$ ) for the LS- and GLS-trained models, respectively. (f) fraction of the total data objective formed by the largest 1% of pointwise contributions. Blocks (g) – (h) show, for the first TV split, diffusion predictions sampled every 100 epochs with light-to-dark shading corresponding to increasing epochs, with the final best model indicated by a solid black line. The $\pm 5\%$ mean percentage error (MPE) bands are shown for reference.

Hyperparameter Optimization

Hyperparameter optimization selects model and training settings to improve predictive performance, efficiency, or reliability under limited evaluation budgets.

See all 161 papers

Apr 20, 2026

Physics-Informed Neural Networks for Biological $2\mathrm{D}{+}t$ Reaction-Diffusion Systems

Physics-informed neural networks (PINNs) provide a powerful framework for learning governing equations of dynamical systems from data. Biologically-informed neural networks (BINNs) are a variant of PINNs that preserve the known differential operator structure (e.g., reaction-diffusion) while learning constitutive terms via trainable neural subnetworks, enforced through soft residual penalties. Existing BINN studies are limited to

1\mathrm{D}{+}t

reaction-diffusion systems and focus on forward prediction, using the governing partial differential equation as a regulariser rather than an explicit identification target. Here, we extend BINNs to

2\mathrm{D}{+}t

systems within a PINN framework that combines data preprocessing, BINN-based equation learning, and symbolic regression post-processing for closed-form equation discovery. We demonstrate the framework's real-world applicability by learning the governing equations of lung cancer cell population dynamics from time-lapse microscopy data, recovering

2\mathrm{D}{+}t

reaction-diffusion models from experimental observations. The proposed framework is readily applicable to other spatio-temporal systems, providing a practical and interpretable tool for fast analytic equation discovery from data.

William Lavery, Jodie A. Cochrane, Christian Olesen +3

Dept. of Information Technology Uppsala University Uppsala, Sweden · Dept. of Medical Genetics Oslo University Hospital Oslo, Norway · Dept. of Translational Hematology and Oncology Research, Cleveland Clinic, Cleveland, USA +1

Dynamical Systems Reaction-Diffusion Systems PDE Discovery Physics-Informed ML

Ask PDF

Jul 8, 2026

Reliable mechanistic operator recovery with biologically-informed neural networks: principles for architecture and optimisation design

Many biological processes are governed by complex dynamical mechanisms that remain incompletely understood despite increasing volumes of experimental data. Biologically-informed neural networks (BINNs) seek to address this challenge by embedding mechanistic differential equations into neural network training, enabling interpretable constitutive operators to be recovered directly from sparse and noisy observations. However, reliable operator recovery depends sensitively on network architecture, optimisation strategy, and data informativeness. Here, we present a systematic empirical study of how these factors influence mechanistic inference using BINNs applied to canonical one-dimensional advection-diffusion-reaction partial differential equation models. Across a suite of benchmark problems, we investigate how network expressivity, learning rate, loss weighting, and batch size influence optimisation behaviour and operator recovery. We show that successful mechanistic inference depends on balancing competing objectives rather than maximising any single aspect of the model or optimisation. Moderately expressive architectures outperform overly complex networks, intermediate learning rates improve optimisation stability, balanced data and PDE losses are essential for accurate operator recovery, and intermediate batch sizes provide the best compromise between computational efficiency and reproducibility. We further identify practical diagnostics for recognising common failure modes, including over-fitting, unstable optimisation, and poor mechanistic recovery when the ground truth is unavailable. Together, these findings provide evidence-based guidelines for deploying BINNs as credible tools for biological model discovery.

Rebecca M. Crossley, Yuan Yin, Sarah L. Waters +1

Mathematical Institute, University of Oxford, Oxford, United Kingdom, OX2 6GG

Neural Network Optimization Mechanistic Interpretability PDE Operator Learning

Ask PDF

Jan 1, 2026

Deep Neural Networks as Discrete Dynamical Systems: Implications for Physics-Informed Learning

We revisit the analogy between feed-forward deep neural networks (DNNs) and discrete dynamical systems derived from neural integral equations and their corresponding partial differential equation (PDE) forms. A comparative analysis between the numerical/exact solutions of the Burgers' and Eikonal equations, and the same obtained via PINNs is presented. We show that PINN learning provides a different computational pathway compared to standard numerical discretization in approximating essentially the same underlying dynamics of the system. Within this framework, DNNs can be interpreted as discrete dynamical systems whose layer-wise evolution approaches attractors, and multiple parameter configurations may yield comparable solutions, reflecting the degeneracy of the inverse mapping. In contrast to the structured operators associated with finite-difference (FD) procedures, PINNs learn dense parameter representations that are not directly associated with classical discretization stencils. This distributed representation generally involves a larger number of parameters, leading to reduced interpretability and increased computational cost. However, the additional flexibility of such representations may offer advantages in high-dimensional settings where classical grid-based methods become impractical.

Abhisek Ganguly, Santosh Ansumali, Sauro Succi

Engineering Mechanics Unit, Jawaharlal Nehru Centre for Advanced Scientific Research, Jakkur, Bangalore, 560064, Karnataka, India · Italian Institute of Technology, Viale Regina Elena 291, 00161 Rome, Italy · University of Roma Tre, Italy +2

Dynamical Systems Neural PDE Solvers Physics-Informed ML Neural Network Training Dynamics

Ask PDF

Abstract

Hyperparameter Optimization

Explore similar work

Physics-Informed Neural Networks for Biological 2D+t2\mathrm{D}{+}t2D+t Reaction-Diffusion Systems

Reliable mechanistic operator recovery with biologically-informed neural networks: principles for architecture and optimisation design

Deep Neural Networks as Discrete Dynamical Systems: Implications for Physics-Informed Learning

Physics-Informed Neural Networks for Biological $2\mathrm{D}{+}t$ Reaction-Diffusion Systems