Neural Approximation and Its Applications

Wei-Hao Wu; Ting-Zhu Huang; Xi-Le Zhao; Yisi Luo; Deyu Meng

cs.LGMar 4, 2026

Neural Approximation and Its Applications

Authors: Wei-Hao Wu, Ting-Zhu Huang, Xi-Le Zhao, Yisi Luo, Deyu Meng

Organizations: School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, 611731, Sichuan, China. · School of Mathematical and Statistics, Xi’an Jiaotong University, Xi’an, 710049, Shaanxi, China. · Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, Taiyuan, 030006, Shanxi, China.

Abstract

Multivariate function approximation is a fundamental problem in machine learning. Classic multivariate function approximations rely on hand-crafted basis functions (e.g., polynomial basis and Fourier basis), which limits their approximation ability and data adaptation ability, resulting in unsatisfactory performance. To address these challenges, we introduce the neural basis function by leveraging an untrained neural network as the basis function. Equipped with the proposed neural basis function, we suggest the neural approximation (NeuApprox) paradigm for multivariate function approximation. Specifically, the underlying multivariate function behind the multi-dimensional data is decomposed into a sum of block terms. The clear physically-interpreted block term is the product of expressive neural basis functions and their corresponding learnable coefficients, which allows us to faithfully capture distinct components of the underlying data and also flexibly adapt to new data by readily fine-tuning the neural basis functions. Attributed to the elaborately designed block terms, the suggested NeuApprox enjoys strong approximation ability and flexible data adaptation ability over the hand-crafted basis function-based methods. We also theoretically prove that NeuApprox can approximate any multivariate continuous function to arbitrary accuracy. Extensive experiments on diverse multi-dimensional datasets (including multispectral images, light field data, videos, traffic data, and point cloud data) demonstrate the promising performance of NeuApprox in terms of both approximation capability and adaptability.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison of multivariate function approximation behind multi-dimensional data (i.e., MSI Painting and MSI Toy with SR = 0.1) based on different basis function. For different data, the hand-crafted basis functions (e.g., polynomial, Fourier, and Gaussian basis functions) are predefined, while the proposed neural basis functions are data-adaptive. The enlarged patch and corresponding residuals are displayed under the recovered results.

Panel of Figure 2 — Figure 2: Different block terms of the proposed NeuApprox and their corresponding Fourier spectrum images (bottom) on MSI Toy with SR = 0.1. (a)-(c) first, second, and third block terms of the proposed NeuApprox with corresponding Fourier spectrum images.

Figure 3: Illustration of the proposed NeuApprox for a three-variable function $f(x,y,z)$ . Leveraging multiple block terms to approximate the target function behind the multi-dimensional data. Each block term, which can inherently capture different components of the target function, is formed by the product of a neural basis function and its corresponding coefficient. From top to bottom: multi-dimensional data, cross-section of the three-variable functions behind the multi-dimensional data, and the flowchart of our proposed NeuApprox. Here, $(x,y,z)$ is the coordinate; $f_{\theta_{1}^{j}}(\cdot)$ , $f_{\theta_{2}^{j}}(\cdot)$ , and $f_{\theta_{3}^{j}}(\cdot)$ are the suggested neural basis functions for block term index $j=1,2,\cdots,T$ .

\begin{split}&\mathcal{X}(i_{1},i_{2},\cdots,i_{n})\\ &=\sum\limits_{j=1}^{T}\mathcal{C}^{j}\times_{1}f_{\theta_{1}^{j}}(i_{1})\times_{2}f_{\theta_{2}^{j}}(i_{2})\dots\times_{n}f_{\theta_{n}^{j}}(i_{n});\end{split}

Algorithm 1 NeuApprox-based Multi-Dimensional Data Completion

Sampling rate		0.1			0.15			0.2			0.25			0.3
Data	Method	PSNR	SSIM	NRMSE	PSNR	SSIM	NRMSE	PSNR	SSIM	NRMSE	PSNR	SSIM	NRMSE	PSNR	SSIM	NRMSE
MSIs Toys Painting (256 $\times$ 256 $\times$ 31)	Observed	12.30	0.315	0.247	12.54	0.349	0.240	12.81	0.382	0.232	13.09	0.413	0.225	13.38	0.444	0.218
	TRLRF	30.08	0.919	0.031	31.58	0.939	0.026	32.79	0.953	0.023	33.73	0.961	0.021	34.54	0.967	0.019
	TNN	29.20	0.928	0.035	31.64	0.956	0.026	33.73	0.972	0.021	35.71	0.981	0.016	37.62	0.988	0.013
	FCTN	32.08	0.940	0.025	33.74	0.956	0.021	34.25	0.962	0.019	35.63	0.970	0.017	36.12	0.973	0.016
	FSA	21.54	0.664	0.084	21.55	0.669	0.084	21.75	0.675	0.082	21.84	0.680	0.081	21.78	0.687	0.082

Table 1: The average quantitative results by different methods for multi-dimensional image inpainting. The best and second-best values are highlighted. (PSNR

\uparrow

, SSIM

\uparrow

, and NRMSE

\downarrow

)

Panel of Figure 4 — Figure 4: The results of multi-dimensional image inpainting with SR=0.15 by different methods on MSIs (i.e., Toys and Painting ), videos (i.e., Foreman and Carphone ), and light field data (i.e., Greek and Origami ).

Missing rate		0.1		0.15		0.2		0.25		0.3
Data	Method	RMSE	MAPE	RMSE	MAPE	RMSE	MAPE	RMSE	MAPE	RMSE	MAPE
Guangzhou	HaLRTC	40.09	1.000	40.97	1.000	39.70	1.000	39.55	1.000	39.23	1.000
	LRTC-TNN	40.09	1.000	40.97	1.000	39.70	1.000	39.55	1.000	39.23	1.000
	LATC	40.09	1.000	40.97	1.000	39.70	1.000	39.55	1.000	39.23	1.000
	FSA	10.25	0.241	9.71	0.227	9.48	0.220	9.45	0.220	10.2111	0.237
	LRTFR	9.55	0.215	9.44	0.213	9.31	0.215	9.69	0.217	9.8618	0.225

Table 2: The quantitative results by different methods for traffic data completion. The best and second-best values are highlighted. (RMSE

\downarrow

and MAPE

\downarrow

)

Panel of Figure 5 — Figure 5: The results of traffic data completion by different methods. From top to bottom: the 58-th slice of PEMS07 , 40-th slice of Seattle , and 148-th slice of Seattle .

Data	Doll		Duck		Frog		Mario		Rabbit		Average
Methods	NRMSE	R-Square	NRMSE	R-Square	NRMSE	R-Square	NRMSE	R-Square	NRMSE	R-Square	NRMSE	R-Square
DT	0.2032	0.9074	0.1793	0.6991	0.1510	0.9140	0.2658	0.7573	0.2145	0.5996	0.2028	0.7747
KNR	0.1426	0.9544	0.1355	0.8282	0.1287	0.9375	0.2112	0.8469	0.1621	0.7715	0.1561	0.8677
RF	0.1698	0.9353	0.1375	0.8231	0.1252	0.9199	0.2022	0.8596	0.1649	0.7635	0.1651	0.7461
FSA	0.2061	0.9048	0.2085	0.5932	0.2291	0.8021	0.3296	0.6270	0.2689	0.3710	0.2485	0.7849
LRTFR	0.1433	0.9588	0.1224	0.8387	0.1252	0.9214	0.2139	0.8515	0.1636	0.7729	0.1536	0.8687

Table 3: The quantitative results by different methods for point cloud data completion. The best and second-best values are highlighted. (NRMSE

\downarrow

and SR-Square

\downarrow

)

Panel of Figure 6 — Figure 6: The results of point cloud data completion by different methods. From top to bottom: the visual results of Doll , Duck , Frog , Mario , and Rabbit by different methods.

Sampling rate	0.1			0.15			0.2			0.25			0.3
Basis function	PSNR	SSIM	NRMSE	PSNR	SSIM	NRMSE	PSNR	SSIM	NRMSE	PSNR	SSIM	NRMSE	PSNR	SSIM	NRMSE
Poly	22.15	0.510	0.078	22.29	0.520	0.077	23.87	0.520	0.064	25.86	0.610	0.051	29.99	0.875	0.032
Fourier	26.27	0.696	0.049	27.77	0.769	0.041	28.43	0.804	0.038	29.66	0.825	0.033	30.30	0.827	0.031
Gaussian	30.07	0.827	0.031	30.43	0.836	0.030	31.30	0.867	0.027	31.66	0.876	0.026	32.22	0.892	0.024
Neural	33.28	0.864	0.022	35.82	0.919	0.016	37.45	0.942	0.013	38.66	0.949	0.012	39.49	0.968	0.011

Table 4: The average quantitative results of the proposed NeuApprox with different neural basis functions on MSI Toy and Painting with different SRs.

Data	Sampling rate	0.1			0.15			0.2			0.25			0.3
Data	Method	PSNR	SSIM	Time	PSNR	SSIM	Time	PSNR	SSIM	Time	PSNR	SSIM	Time	PSNR	SSIM	Time
Painting	Pretraining	29.25	0.759	2.35	30.97	0.813	3.67	33.52	0.891	2.98	34.46	0.910	3.75	35.78	0.930	3.48
	Fine-Tuning	30.92	0.802	4.07	32.65	0.855	7.15	35.19	0.915	8.23	36.69	0.939	9.74	37.88	0.952	7.14
	Training from Scratch	31.32	0.839	28.55	33.96	0.903	30.62	35.35	0.928	31.72	36.83	0.946	41.09	37.78	0.980	40.53
Toys	Pretraining	29.59	0.758	2.14	30.83	0.799	2.55	33.32	0.889	2.77	34.82	0.913	3.49	35.70	0.932	3.74
	Fine-Tuning	32.04	0.820	4.18	35.19	0.905	6.47	38.05	0.953	7.62	39.50	0.967	9.56	40.44	0.974	10.53

Table 5: Inpainting results of different training strategies on MSI Toys and Painting with different SRs. The neural basis functions are pretrained on out-of-distribution datasets.

Figure 7: The PSNR curves of the recovered MSI Toy and Painting by NeuApprox with different numbers of layers. From left to right: The PSNR curves with respect to number of layers of neural networks used in basis functions on MSI Toy and Painting with SR=0.1, 0.15, 0.2, 0.25, and 0.3.

Figure 8: The PSNR values of the recovered by the proposed NeuApprox with single block term representation and multiple block terms on MSI Toy with respect to the number of parameters. From left to right: The PSNR curves with respect to number of parameters in NeuApprox on MSI Toy and Painting with SR=0.1, 0.15, 0.2, 0.25, and 0.3.

Panel of Figure 9 — Figure 9: Different block terms (top) of the proposed NeuApprox and their Fourier spectrum images (bottom) on MSI Toy with SR=0.1. (a)-(c) first, second, and third block terms of the proposed NeuApprox with corresponding Fourier spectrum images.

Figure 10: The PSNR curves of the recovered MSI Toy and Painting by NeuApprox with different coefficient sizes. From top to bottom: The PSNR curves with respect to

R_{1}

values and the PSNR curves with respect to

R_{2}

values. From left to right: The PSNR curves of the recovered MSI Toy and Painting with SR=0.1, 0.15, 0.2, 0.25, and 0.3.

Neural Network Approximation Theory

Neural network approximation theory studies how network size, depth, activations, and structure determine the accuracy of functions and operators.

See all 236 papers

Do Neural Networks Really Beat the Curse of Dimensionality? A Bit-Complexity View
Aug 2, 2026Tong Mao, Jinchao XuCurse of Dimensionality Neural Network Approximation Theory
King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia
Data-Driven Variational Basis Learning Beyond Neural Networks: A Non-Neural Framework for Adaptive Basis Discovery
Apr 17, 2026Andrew KirulutaRepresentation Learning Dictionary Learning
UC Berkeley, CA
Efficient Approximation to Analytic and $L^p$ functions by Height-Augmented ReLU Networks
Mar 11, 2026ZeYu Li, FengLei Fan, TieYong ZengNeural Network Approximation Theory ReLU Neural Networks
Chinese University of Hong Kong, Department of Mathematics, Hong Kong, 999077, NT, Hong Kong SAR, China · City University of Hong Kong, Department of Data Science, Hong Kong, 999077, Kowloon, Hong Kong SAR, China · Guangzhou Nanfang College, School of Mathematics and Statistics, Guangzhou, 510970, Guangdong, China +1

Aug 2, 2026J/K move · Enter open · S save

Do Neural Networks Really Beat the Curse of Dimensionality? A Bit-Complexity View

Tong Mao, Jinchao Xu

King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia

Curse of Dimensionality Neural Network Approximation Theory Function Approximation

Open paper AskPDF arXiv

Traditional approximation theory measures convergence rates in terms of the number of parameters or degrees of freedom. However, practical computation operates under finite precision: parameters must be encoded using a finite number of bits. Therefore, approximation efficiency should be evaluated in terms of computational bit complexity, which is intrinsically connected to the metric entropy of the underlying function class. In this work, we develop a unified approximation framework based on binary encoding and metric entropy. We analyze classical methods (including polynomial approximation, sparse grids, and finite elements) as well as shallow and deep neural networks, and compare their approximation rates for function classes with comparable metric entropy. We observe that, when evaluated in terms of bits, most classical methods are in general suboptimal relative to the intrinsic limits dictated by metric entropy, while neural network methods may exhibit different behaviors. We show that when complexity is measured in bits rather than parameters, no method fundamentally exceeds the approximation order achieved by classical approaches. Our results also indicate that many seeming advantages of neural networks, including dimension-independent rates and superconvergence phenomena, stem from differences in function class complexity rather than intrinsic architectural superiority. In this sense, the traditional curse of dimensionality can be misleading; the fundamental limitation is instead a curse of bit complexity, governed by metric entropy.

Abstract

Neural Network Approximation Theory

Explore similar work

Do Neural Networks Really Beat the Curse of Dimensionality? A Bit-Complexity View

Data-Driven Variational Basis Learning Beyond Neural Networks: A Non-Neural Framework for Adaptive Basis Discovery

Efficient Approximation to Analytic and LpL^pLp functions by Height-Augmented ReLU Networks

Efficient Approximation to Analytic and $L^p$ functions by Height-Augmented ReLU Networks