Adaptive Randomized Neural Networks for PDEs: Structure-Aware Feature Redistribution and Residual-Driven Block Enrichment

You Yang; Fei Wang

math.NAOct 15, 2026

Adaptive Randomized Neural Networks for PDEs: Structure-Aware Feature Redistribution and Residual-Driven Block Enrichment

Authors: You Yang, Fei Wang

Organizations: School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China. · School of Mathematics and Statistics & State Key Laboratory of Multiphase Flow in Power Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China.

Abstract

Randomized neural networks retain an output-linear least-squares solve, but globally sampled random features may fail to capture the spatial and scale structure of PDE solutions. We propose SFRBE--RaNN, a two-stage adaptive trial-space construction. SFR redistributes a fixed feature budget according to regional indicator masses and adapts regional weight scales through low-dimensional residual-based optimization. RBE constructs nonredundant enrichments by screening multiple activation families, removing represented response directions, and selecting the smallest admissible prefix. For solution-dependent operators, candidates are screened with a frozen operator and accepted only after nonlinear residual verification. For fixed linear problems, we establish an exact discrete residual-reduction identity and a conditional solution-error estimate. Building on this identity, we combine safeguarded SFR with RBE capture factors to derive a finite-horizon error bound involving baseline best approximation and empirical consistency, together with a conditional random-search success-probability bound. Numerical experiments on localized, multiscale, and nonlinear space--time problems demonstrate the method's effectiveness.

Figures & tables

Figure 1: Standard randomized neural network followed by the two adaptive modules. SFR diagnoses the residual structure and refreshes the existing feature budget through regional anchors and scales. RBE screens candidate activation families, removes represented directions, and appends only an accepted residual-aligned block.

Family	Representative feature	Residual structure typically captured
tanh	$\tanh(\bm{w}^{\mathsf{T}}\bm{x}+b)$	smooth transitions and anisotropic layers
ridge_gauss	$\exp[-(\bm{w}^{\mathsf{T}}\bm{x}+b)^{2}]$	localized ridge-like defects
sin	$\sin(\pi(\bm{w}^{\mathsf{T}}\bm{x}+b))$	oscillatory or high-frequency residuals
radial_gauss	$\exp[-\sum_{\ell}r_{\ell}^{2}(x_{\ell}-a_{\ell})^{2}]$	isolated peaks and compact defects
mexican_hat	$(d-s)\exp(-s/2),\hskip 9.24994pts=\sum_{\ell=1}^{d}r_{\ell}^{2}(x_{\ell}-a_{\ell})^{2}$	sign-changing localized residuals

Table 1: Candidate activation families screened by RBE. The family is chosen from the residual-capture score after residualization; no single activation is prescribed for all PDEs.

Stage	Features (avg.)	Time (s, avg.)	Rel. $L^{2}$ (avg.)	Rel. $L^{2}$ (best)	Rel. $L^{2}$ (worst)
AD–RaNN	$3500$	$8.477$	$2.068\mathrm{e}{-02}\pm 1.950\mathrm{e}{-03}$	$1.829\mathrm{e}{-02}$	$2.327\mathrm{e}{-02}$
SFR	$3500$	$24.555$	$4.866\mathrm{e}{-03}\pm 1.171\mathrm{e}{-03}$	$3.609\mathrm{e}{-03}$	$6.974\mathrm{e}{-03}$
RBE	$6450$	$93.250$	$1.164\mathrm{e}{-06}\pm 2.114\mathrm{e}{-06}$	$8.588\mathrm{e}{-08}$	$7.096\mathrm{e}{-06}$

Table 2: Numerical results for the two-peak Poisson equation ( 6.3 ) (mean / best / worst over seeds

0,\ldots,9

). “Features” denotes the cumulative raw randomized-feature count reported at each stage; SFR keeps a fixed-width trial space. The values after

\pm

denote one standard deviation over the ten seeds. Times are mean stage wall-clock times.

Family	tanh	ridge_gauss	sin	radial_gauss	mexican_hat
Mean accepted features	380	370	830	480	890

Table 3: Mean accepted RBE features by activation family over Poisson seeds

0,\ldots,9

Panel of Figure 2 — Figure 2: Two-peak Poisson equation. Exact solution (left), final numerical prediction after RBE enrichment (middle), and SFR feature-anchor distribution (right). In the feature-anchor plot, different colors distinguish the detected structure regions and the background region.

Panel of Figure 3 — Figure 3: Pointwise absolute error maps for the two-peak Poisson equation. From left to right: initial AD–RaNN, after SFR, and after RBE.

(a) Comparison with external baselines
Method	DOFs / features	Time (s)	Abs. $L^{2}$	Rel. $L^{2}$	Abs. $H^{1}$	Rel. $H^{1}$
Expert-guided PINN	$462593$ params.	$5062.382$	$1.366\mathrm{e}{-02}$	$2.321\mathrm{e}{-01}$	$7.375\mathrm{e}{-02}$	$3.168\mathrm{e}{-02}$
HyResPINN	$256711$ params.	$6806.590$	$1.220\mathrm{e}{-03}$	$2.073\mathrm{e}{-02}$	$1.564\mathrm{e}{-02}$	$6.717\mathrm{e}{-03}$
Adaptive P1 FEM	$7247310$ DOFs	$279.380$	$2.029\mathrm{e}{-07}$	$3.447\mathrm{e}{-06}$	$2.670\mathrm{e}{-03}$	$1.147\mathrm{e}{-03}$
Adaptive P2 FEM	$5724817$ DOFs	$195.110$	$1.878\mathrm{e}{-08}$	$3.190\mathrm{e}{-07}$	$2.695\mathrm{e}{-05}$	$1.158\mathrm{e}{-05}$
AG–RaNN–1	$1200$ features	$3.742$	$4.462\mathrm{e}{-04}$	$7.581\mathrm{e}{-03}$	$4.755\mathrm{e}{-02}$	$2.043\mathrm{e}{-02}$

Table 4: Comparison and mechanism ablation on the two-peak Poisson equation. The upper panel compares SFRBE–RaNN with representative external baselines. The lower panel isolates the effects of global initialization, fixed-budget feature redistribution, and residual-driven enrichment, with the wider SFR–RaNN row serving as a capacity control. Times and errors for the AD–RaNN, SFR–RaNN, RBE–RaNN (no SFR), and SFRBE–RaNN rows are means over seeds

0,\ldots,9

; the remaining rows are representative comparison or capacity-control results. The columns report absolute and relative

L^{2}

errors, together with the corresponding errors measured in the full

H^{1}

norm.

Panel of Figure 4 — Figure 4: Adaptive finite element results for the two-peak Poisson equation ( 6.3 ). From left to right: adaptive P1 mesh, absolute and relative $L^{2}$ and full $H^{1}$ error decay for adaptive P1 FEM, adaptive P2 mesh, and the corresponding error decay for adaptive P2 FEM. Both meshes are shown at the 20th AFEM iteration.

Stage	$\rho_{L}$	$\beta_{L}$	$e_{L}$	$B_{L}$	$B_{L}/e_{L}$	$\varrho_{L}$	$s_{L}^{(0)}$	$q_{L}^{\rm obs}$	$q_{L}^{\rm pred}$
	Poisson residual-to-error diagnostics					RBE projection diagnostics
Initial	$3.7801\mathrm{e}{+01}$	$2.0260\mathrm{e}{-04}$	$1.0768\mathrm{e}{-03}$	$1.9152\mathrm{e}{+00}$	$1.78\mathrm{e}{+03}$	–	–	–	–
SFR	$1.0008\mathrm{e}{+01}$	$1.1999\mathrm{e}{-04}$	$2.4459\mathrm{e}{-04}$	$5.0710\mathrm{e}{-01}$	$2.07\mathrm{e}{+03}$	$5.5174\mathrm{e}{-02}$	–	–	–
RBE 1	$1.1027\mathrm{e}{-01}$	$9.0593\mathrm{e}{-07}$	$2.2659\mathrm{e}{-06}$	$5.5870\mathrm{e}{-03}$	$2.47\mathrm{e}{+03}$	$6.0005\mathrm{e}{-04}$	0.99988	0.010876	0.010954
RBE 2	$3.6376\mathrm{e}{-02}$	$3.7292\mathrm{e}{-07}$	$1.2297\mathrm{e}{-06}$	$1.8431\mathrm{e}{-03}$	$1.50\mathrm{e}{+03}$	$1.9923\mathrm{e}{-04}$	0.88976	0.332022	0.332024
RBE 3	$1.7380\mathrm{e}{-02}$	$1.4947\mathrm{e}{-07}$	$4.1892\mathrm{e}{-07}$	$8.8059\mathrm{e}{-04}$	$2.10\mathrm{e}{+03}$	$9.4681\mathrm{e}{-05}$	0.77416	0.475235	0.475226

Table 5: Residual-to-error and RBE projection diagnostics for the detailed seed-4 Poisson run.

Panel of Figure 5 — Figure 5: Numerical results for the multiscale Helmholtz equation ( 6.5 ). From left to right: exact solution, final numerical prediction after RBE enrichment, initial pointwise absolute error, and pointwise absolute error after RBE enrichment.

Stage	Family	Features	Total features	Scale $\bm{r}$	Time (s)	Rel. $L^{2}$
Initial AD–RaNN	tanh	3500	3500	[63.667,66.557]	15.303	$2.069\mathrm{e}{-01}$
SFR	Not triggered	–	3500	–	10.247	$2.069\mathrm{e}{-01}$
RBE	sin / tanh / ridge_gauss / mexican_hat	5600 (10 blocks)	9100	–	501.654	$1.647\mathrm{e}{-05}$

Table 6: Stage-wise results for the multiscale Helmholtz equation ( 6.5 ). The SFR row records the inactive gate and unchanged 3500-feature allocation. “Total features” is cumulative, and the reported times are stage wall-clock values.

Stage	Family	Features	Total	Scale $\bm{r}$	Time (s)	Rel. $L^{2}$
Initial AD–RaNN	tanh	3000	3000	[18.530, 18.510]	63.924	$1.239\mathrm{e}{-01}$
SFR	tanh	$m_{S}=1482$	3000	$\mathcal{S}_{1}:[34.875,\,35.573]$ $\mathcal{S}_{2}:[35.572,\,36.671]$	24.560	$8.188\mathrm{e}{-02}$
SFR	tanh	$m_{B}=1518$	3000	[29.131, 22.165]	24.560	$8.188\mathrm{e}{-02}$
RBE	mexican_hat / ridge_gauss / tanh	4900 (7 blocks)	7900	–	141.411	$8.362\mathrm{e}{-03}$

Table 7: Stage-wise results for the Multiscale Darcy problem ( 6.7 ). The two SFR rows report the structure- and background-region neuron counts

m_{S}

and

m_{B}

; “Total” is the cumulative feature count. Times are stage wall-clock times, and the detailed RBE blocks are listed in Appendix D . The run uses seed

4

and times exclude figure generation.

Panel of Figure 6 — Figure 6: Multiscale Darcy results for ( 6.7 ). The first row shows the reference solution, the final numerical solution after RBE enrichment, and the SFR feature-anchor distribution. The second row shows the initial pointwise absolute error, the pointwise absolute error after SFR, and the pointwise absolute error after RBE.

Panel of Figure 7 — Figure 7: Generalized KdV equation ( 6.8 ). Exact solution (left), final numerical prediction after RBE enrichment (middle), and SFR feature-anchor distribution (right). The SFR feature anchors are concentrated around the moving localized wave structure.

Stage	Family	Features	Total	Scale $\bm{r}$	Time (s)	Rel. $L^{2}$
Initial AD–RaNN	tanh	2000	2000	[10.199,10.194]	34.005	$6.778\mathrm{e}{-03}$
SFR	tanh	$m_{S}=995$	2000	$\mathcal{S}_{1}:[15.521,13.112]$ $\mathcal{S}_{2}:[13.113,11.290]$	20.495	$1.039\mathrm{e}{-05}$
SFR	tanh	$m_{B}=1005$	2000	$[13.482,9.377]$	20.495	$1.039\mathrm{e}{-05}$
RBE	tanh / ridge_gauss	400 (2 blocks)	2400	–	70.834	$7.423\mathrm{e}{-07}$

Table 8: Stage-wise results for the fixed-operator generalized KdV equation ( 6.8 ). The SFR rows report two structure components and the background, with

m_{S}

denoting the total features allocated to the structure components. “Total” is the cumulative feature count. Times are stage wall-clock times; detailed RBE blocks are listed in Appendix D .

Panel of Figure 8 — Figure 8: Periodic KdV two-soliton interaction problem ( 6.12 ). Left: high-accuracy Fourier spectral reference solution. Middle: final numerical prediction after RBE enrichment. Right: SFR feature-anchor distribution.

Stage	Features (avg.)	Time (s, avg.)	Rel. $L^{2}$ (avg.)	Rel. $L^{2}$ (best)	Rel. $L^{2}$ (worst)
Initial AD–RaNN	$2000$	$87.203$	$2.976\mathrm{e}{-01}\pm 6.354\mathrm{e}{-02}$	$1.788\mathrm{e}{-01}$	$3.822\mathrm{e}{-01}$
SFR	$2000\;(m_{S}=995.4,\ m_{B}=1004.6)$	$96.761$	$4.225\mathrm{e}{-03}\pm 4.807\mathrm{e}{-03}$	$2.219\mathrm{e}{-04}$	$1.435\mathrm{e}{-02}$
RBE	$3130\;(1130\ \text{added},\ 3.7\ \text{blocks})$	$579.581$	$9.061\mathrm{e}{-05}\pm 2.333\mathrm{e}{-05}$	$6.648\mathrm{e}{-05}$	$1.262\mathrm{e}{-04}$

Table 9: Stage-wise numerical results for the periodic KdV two-soliton problem ( 6.12 ) over seeds

0,\ldots,9

. The SFR entry also reports the mean allocations to the structure and background regions; the RBE entry reports the mean cumulative feature count, with the mean number of added features and accepted blocks given in parentheses. Times are mean stage wall-clock times, and the values after

\pm

denote one standard deviation over the ten seeds.

Panel of Figure 9 — Figure 9: Space–time visualization for the Burgers problem ( 6.15 ): exact solution, final prediction after RBE enrichment, pointwise absolute error, and the SFR feature-anchor distribution.

Stage	Family	Features	Total	Scale $\bm{r}$	Time (s)	Rel. $L^{2}$
Initial AD–RaNN	tanh	3000	3000	[10.864,9.549,7.862]	182.302	$8.202\mathrm{e}{-03}$
SFR	tanh	$m_{S}=1499$	3000	$[12.948,12.071,11.089]$ $[8.500,8.986,8.639]$	160.578	$1.622\mathrm{e}{-04}$
SFR	tanh	$m_{B}=1501$	3000	$[12.948,12.071,11.089]$ $[8.500,8.986,8.639]$	160.578	$1.622\mathrm{e}{-04}$
RBE	tanh / ridge_gauss	1000 (4 blocks)	4000	–	564.769	$2.271\mathrm{e}{-05}$

Table 10: Stage-wise results for the Burgers problem ( 6.15 ). The two SFR rows report the structure- and background-region neuron counts

m_{S}

and

m_{B}

; “Total” is the cumulative feature count. Times are stage wall-clock values and exclude figure generation.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Problem	Points	Initial features	Full block size $m_{\rm full}$	Weights $(\sqrt{\eta_{b}},\sqrt{\eta_{0}})$
Two-peak Poisson	$N_{\rm pde}=22500,\ N_{b}=800$	$3500$	$1000$	$(10^{5},\text{--})$
Multiscale Helmholtz	$N_{\rm pde}=62500,\ N_{b}=800$	$3500$	$2000$	$(10^{5},\text{--})$
Multiscale Darcy	$N_{\rm pde}=22500,\ N_{b}=800$	$3000$	$1000$	$(100,\text{--})$
Generalized KdV	$N_{\rm pde}=22500,\ N_{b}=400,\ N_{0}=200$	$2000$	$1000$	$(100,100)$
Periodic KdV	$N_{\rm pde}=32400,\ N_{b}=400,\ N_{0}=200$	$2000$	$1000$	$(100,100)$
Burgers	$N_{\rm pde}=37026,\ N_{b}=4760,\ N_{0}=1225$	$3000$	$1000$	$(300,300)$

Appendix

Table 11: Problem-dependent discretization, features, and constraint-weight settings.

Module	Parameters	Values
SFR	$\vartheta,\tau_{\rm eff},\tau_{\rm conn},\tau_{\rm cover}$	$0.50,0.35,0.01,0.35$
RBE screening	$\tau_{\rm svd},\tau_{\rm cap}$	$10^{-10},10^{-4}$
RBE screening size	$m_{\rm scr}$	$\min\{500,\lceil 0.3m_{\rm full}\rceil\}$
RBE prefix/stop	$\theta_{\rm pref},\tau_{\rm stop}$	$0.80,0.25$
Stabilization	$\varepsilon_{n}$	$10^{-12}$
SFR parameter refinement	$\mathrm{lr}_{S},T_{S}$	$(0.7,30)$

Appendix

Table 12: Shared SFR/RBE and numerical implementation controls.

Round	Family	$m$	Scale/shape	Rel. residual
Two-peak Poisson (Seed 4)
1	radial_gauss	200	[47.872,50.505]	$6.001\mathrm{e}{-04}$
2	mexican_hat	200	[55.822,50.104]	$1.992\mathrm{e}{-04}$
3	ridge_gauss	400	[69.134,66.557]	$9.468\mathrm{e}{-05}$
4	mexican_hat	500	[48.317,57.077]	$5.354\mathrm{e}{-05}$
5	sin	600	[48.802,50.577]	$3.093\mathrm{e}{-05}$

Appendix

Table 13: Activation-family choices and residual decrease during RBE enrichment. Each row gives the post-acceptance weighted relative residual for the indicated block; the two panels are a compact continuation of one table.

Neural PDE Solvers

Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.

See all 269 papers

Adaptive-Distribution Randomized Neural Networks for PDEs: A Low-Dimensional Distribution-Learning Framework
Apr 27, 2026You Yang, Fei WangNeural PDE Solvers PDE Operator Learning
School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China. · School of Mathematics and Statistics & State Key Laboratory of Multiphase Flow in Power Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China.
A Structure-Adaptive Random Feature Method for High-Dimensional Elliptic PDEs
Jul 22, 2026Jiale Linghu, Hao Dong, Yangshuai WangPDE Solving Random Feature Methods
School of Mathematics and Statistics, Xidian University, Xi’an 710071, China. · Department of Mathematics, National University of Singapore, 10 Lower Kent Ridge Road, 119076, Singapore
Physics Informed Random Feature Neural Networks for Solving PDEs
Sep 14, 2026Chi-An Chen, Chunyang Liao, Ming ZhongPDE Solving Random Feature Methods
Department of Applied Mathematics, Mathematics, Illinois Institute of Technology, Chicago, IL, 60616 · Department of Mathematical Sciences, University of Arkansas, Fayetteville, AR, 72701 · Department of Mathematics, University of Houston, Houston, TX, 77204

Apr 27, 2026J/K move · Enter open · S save

Adaptive-Distribution Randomized Neural Networks for PDEs: A Low-Dimensional Distribution-Learning Framework

You Yang, Fei Wang

School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China. · School of Mathematics and Statistics & State Key Laboratory of Multiphase Flow in Power Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China.

Neural PDE Solvers PDE Operator Learning

Open paper AskPDF arXiv

Randomized neural networks (RaNNs) are attractive for partial differential equations (PDEs) because they replace expensive end-to-end training with a linear least-squares solve over randomized hidden features. Their practical performance, however, depends strongly on the sampling distribution of the hidden-layer parameters, which is usually chosen heuristically and problem by problem. This distribution sensitivity is a central bottleneck in randomized neural PDE solvers. In this work, we propose Adaptive-Distribution Randomized Neural Networks (AD-RaNN), a framework that promotes randomized feature generation from a fixed heuristic choice to a low-dimensional adaptive optimization problem. Instead of training all hidden weights and biases, AD-RaNN parameterizes the hidden-feature sampling distribution by a low-dimensional vector p and optimizes only p, thereby preserving the least-squares structure of RaNNs while reducing manual distribution tuning. The method uses a two-stage strategy: ridge-regularized reduced training for stable distribution-parameter optimization, followed by an unregularized least-squares refit for final solution recovery. We develop two adaptive mechanisms, PDE-Driven Adaptive Distribution (PDAD) and Data-Driven Adaptive Distribution (DDAD), and deploy them in space-time solvers, discrete-time solvers, and operator-learning models. We also incorporate an adaptive layer-growth enhancement for localized structures. For the reduced optimization problem, we establish well-posedness of the reduced objectives, consistency of ridge-regularized minimizers, an efficient gradient formula, and a practical lower-bound estimate for the ridge parameter. Numerical experiments on benchmark problems show that AD-RaNN provides an effective distribution-level adaptation mechanism, reduces reliance on hand-crafted hidden-feature distributions, and achieves strong empirical accuracy.

Abstract

Neural PDE Solvers

Explore similar work

Adaptive-Distribution Randomized Neural Networks for PDEs: A Low-Dimensional Distribution-Learning Framework

A Structure-Adaptive Random Feature Method for High-Dimensional Elliptic PDEs

Physics Informed Random Feature Neural Networks for Solving PDEs