Reproducing Kernel (Quasi-)Banach Spaces with the $\ell^p$ Coefficient Norms

Tianfang Wang; Guohui Song; Haizhang Zhang

cs.LGNov 5, 2026

Reproducing Kernel (Quasi-)Banach Spaces with the $\ell^p$ Coefficient Norms

Authors: Tianfang Wang, Guohui Song, Haizhang Zhang

Organizations: School of Mathematics, Sun Yat-sen University, Zhuhai, P. R. China. · Department of Mathematics and Statistics, Old Dominion University, Norfolk, VA 23529, USA.

Abstract

This paper advances the theory of reproducing kernel Banach spaces (RKBSs) by investigating the general $\ell^p$ regularization framework, with a particular focus on the non-convex regime $0<p<1$ . While our preceding work established the foundation for RKBSs with the $\ell^1$ norm, the pursuit of stronger sparsity---closer to the $\ell^0$ pseudo-norm---naturally motivates this exploration into $\ell^p$ quasi-norms. For $0<p<1$ , we construct reproducing kernel quasi-Banach spaces (RKQBSs) with $\ell^p$ quasi-norms and establish existence and sparsity results for interpolation and regularized learning under appropriate assumptions. We characterize the linear and relaxed representer theorems in terms of a generalized Lebesgue constant and derive bounds under kernel decay, sample spacing, and stability assumptions. For $p>1$ , we construct RKBSs using a fixed countable dictionary, establish minimizer existence, and identify the optimal relaxed representer factor through an off-sample coefficient operator. These results reveal fundamental differences between the regimes $0<p<1$ , $p=1$ , and $p>1$ . Numerical experiments compare the accuracy and sparsity of solutions for these three cases.

Figures & tables

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	7.38E-4	13 (16)	8.18E-4	13.08 (15)	9.00E-4	13.1 (16)
$\ell^{\frac{2}{3}}$ -Regularized Model	6.90E-4	13.66 (16)	7.77E-4	13.66 (16)	8.63E-4	13.52 (16)
$\ell^{1}$ -Regularized Model	6.32E-4	23.04 (33)	7.19E-4	23.14(40)	8.20E-4	21.92(28)
$\ell^{\frac{3}{2}}$ -Regularized Model	6.21E-4	199.98(200)	7.11E-4	200 (200)	8.23E-4	200 (200)
$\ell^{2}$ -Regularized Model	6.52E-4	200 (200)	7.43E-4	200 (200)	8.64E-4	200 (200)

Table 6.1 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the Gaussian Kernel for the GMO function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	6.88E-04	11.3 (14)	7.52E-04	11.5 (14)	8.27E-04	11.38 (14)
$\ell^{\frac{2}{3}}$ -Regularized Model	5.73E-04	12.32 (15)	6.37E-04	12.04 (14)	7.12E-04	12.26 (15)
$\ell^{1}$ -Regularized Model	5.51E-04	20.42 (25)	6.20E-04	20.04 (24)	7.10E-04	20.18 (24)
$\ell^{\frac{3}{2}}$ -Regularized Model	5.86E-04	200 (200)	6.64E-04	200 (200)	7.72E-04	200 (200)
$\ell^{2}$ -Regularized Model	6.37E-04	200 (200)	7.20E-04	200 (200)	8.39E-04	200 (200)

Table 6.2 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the Gaussian Kernel for the HRG function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	6.71E-04	11.84 (14)	7.40E-04	12.32 (14)	8.58E-04	11.86 (14)
$\ell^{\frac{2}{3}}$ -Regularized Model	6.35E-04	12.36 (15)	7.30E-04	12.4 (16)	8.12E-04	12.44 (14)
$\ell^{1}$ -Regularized Model	6.08E-04	20.52 (25)	6.97E-04	20.4 (30)	8.06E-04	20.5 (24)
$\ell^{\frac{3}{2}}$ -Regularized Model	6.49E-04	199.98 (200)	7.46E-04	200 (200)	8.60E-04	200 (200)
$\ell^{2}$ -Regularized Model	6.84E-04	200 (200)	7.81E-04	200 (200)	9.05E-04	200 (200)

Table 6.3 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the Gaussian Kernel for the SPF function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	6.99E-04	12.02 (15)	7.33E-04	12 (14)	8.52E-04	12.16 (14)
$\ell^{\frac{2}{3}}$ -Regularized Model	6.66E-04	12.38 (15)	7.24E-04	12.44 (15)	8.50E-04	12.66 (15)
$\ell^{1}$ -Regularized Model	6.38E-04	21.46 (29)	7.18E-04	20.96 (26)	8.29E-04	20.9 (27)
$\ell^{\frac{3}{2}}$ -Regularized Model	6.59E-04	200 (200)	7.47E-04	199.98 (200)	8.67E-04	199.98 (200)
$\ell^{2}$ -Regularized Model	6.92E-04	200 (200)	7.80E-04	200 (200)	9.09E-04	200 (200)

Table 6.4 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the Gaussian Kernel for the MTH function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	1.38E-03	18.9 (22)	1.49E-03	18.94 (23)	1.71E-03	18.94 (24)
$\ell^{\frac{2}{3}}$ -Regularized Model	1.32E-03	19.98 (23)	1.46E-03	19.92 (22)	1.66E-03	19.62 (23)
$\ell^{1}$ -Regularized Model	1.12E-03	62.36 (93)	1.25E-03	58.16 (81)	1.43E-03	59.36 (78)
$\ell^{\frac{3}{2}}$ -Regularized Model	9.92E-04	200 (200)	1.11E-03	200 (200)	1.26E-03	200 (200)
$\ell^{2}$ -Regularized Model	9.92E-04	200 (200)	1.10E-03	200 (200)	1.26E-03	200 (200)

Table 6.5 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the Laplacian Kernel for the GMO function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	5.33E-04	9.24 (11)	5.78E-04	9.64 (12)	6.61E-04	9.52 (12)
$\ell^{\frac{2}{3}}$ -Regularized Model	4.99E-04	10.24 (12)	5.57E-04	10.36 (14)	6.45E-04	10.24 (13)
$\ell^{1}$ -Regularized Model	4.75E-04	25.78 (46)	5.42E-04	25.6 (37)	6.56E-04	24.9 (38)
$\ell^{\frac{3}{2}}$ -Regularized Model	5.62E-04	200 (200)	6.40E-04	200 (200)	7.66E-04	200 (200)
$\ell^{2}$ -Regularized Model	6.74E-04	200 (200)	7.68E-04	200 (200)	9.05E-04	200 (200)

Table 6.6 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the Laplacian Kernel for the HRG function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	3.43E-04	4.02 (5)	4.32E-04	4.04 (5)	4.78E-04	4.04 (5)
$\ell^{\frac{2}{3}}$ -Regularized Model	3.43E-04	4.24 (6)	4.08E-04	4.14 (5)	4.82E-04	4.18 (6)
$\ell^{1}$ -Regularized Model	2.69E-04	11.42 (19)	3.05E-04	11.16 (20)	3.61E-04	11.02 (17)
$\ell^{\frac{3}{2}}$ -Regularized Model	3.30E-04	200 (200)	3.63E-04	200 (200)	4.33E-04	200 (200)
$\ell^{2}$ -Regularized Model	4.15E-04	200 (200)	4.67E-04	200 (200)	5.48E-04	200 (200)

Table 6.7 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the Laplacian Kernel for the SPF function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	1.09E-03	15.36 (18)	1.20E-03	15.54 (21)	1.37E-03	15.5 (19)
$\ell^{\frac{2}{3}}$ -Regularized Model	1.04E-03	16.16 (19)	1.14E-03	16.42 (21)	1.28E-03	16.32 (20)
$\ell^{1}$ -Regularized Model	8.69E-04	50.7 (85)	9.63E-04	48.36 (71)	1.11E-03	47.6 (75)
$\ell^{\frac{3}{2}}$ -Regularized Model	8.40E-04	200 (200)	9.15E-04	200 (200)	1.06E-03	200 (200)
$\ell^{2}$ -Regularized Model	9.15E-04	200 (200)	9.98E-04	200 (200)	1.15E-03	200 (200)

Table 6.8 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the Laplacian Kernel for the MTH function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	5.32E-04	13.04 (17)	5.88E-04	13.14 (16)	6.94E-04	13.24 (17)
$\ell^{\frac{2}{3}}$ -Regularized Model	5.15E-04	18.62 (25)	5.82E-04	18.94 (23)	6.76E-04	18.46 (23)
$\ell^{1}$ -Regularized Model	5.16E-04	129.48 (156)	5.84E-04	126.34 (150)	6.81E-04	125.78 (149)
$\ell^{\frac{3}{2}}$ -Regularized Model	5.57E-04	200 (200)	6.34E-04	200 (200)	7.37E-04	200 (200)
$\ell^{2}$ -Regularized Model	6.00E-04	200 (200)	6.87E-04	200 (200)	7.98E-04	200 (200)

Table 6.9 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the IMQ Kernel for the GMO function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	3.80E-04	9.22 (12)	4.19E-04	9.36 (11)	4.95E-04	9.54 (12)
$\ell^{\frac{2}{3}}$ -Regularized Model	3.90E-04	10.28 (12)	4.40E-04	10.46 (13)	5.21E-04	10.62 (13)
$\ell^{1}$ -Regularized Model	4.53E-04	66.54 (109)	5.02E-04	70.14 (118)	5.97E-04	75.22 (112)
$\ell^{\frac{3}{2}}$ -Regularized Model	5.34E-04	200 (200)	6.00E-04	200 (200)	7.00E-04	200 (200)
$\ell^{2}$ -Regularized Model	5.89E-04	200 (200)	6.68E-04	200 (200)	7.77E-04	200 (200)

Table 6.10 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the IMQ Kernel for the HRG function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	4.01E-04	6.34 (8)	4.19E-04	6.22 (8)	5.15E-04	6.16 (8)
$\ell^{\frac{2}{3}}$ -Regularized Model	3.83E-04	6.22 (8)	3.94E-04	6.26 (7)	4.65E-04	6.22 (8)
$\ell^{1}$ -Regularized Model	3.50E-04	14.02 (23)	3.85E-04	14.38 (22)	4.59E-04	14.92 (29)
$\ell^{\frac{3}{2}}$ -Regularized Model	4.27E-04	200 (200)	4.82E-04	200 (200)	5.71E-04	200 (200)
$\ell^{2}$ -Regularized Model	5.03E-04	200 (200)	5.62E-04	200 (200)	6.56E-04	200 (200)

Table 6.11 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the IMQ Kernel for the SPF function.

	Gaussian noise		Uniform noise		Pepper sauce noise
	Error	Sparsity (Max)	Error	Sparsity (Max)	Error	Sparsity (Max)
$\ell^{\frac{1}{2}}$ -Regularized Model	4.93E-04	11.04 (14)	5.58E-04	10.82 (13)	6.53E-04	10.7 (13)
$\ell^{\frac{2}{3}}$ -Regularized Model	4.86E-04	12.92 (15)	5.57E-04	12.46 (15)	6.47E-04	12.42 (14)
$\ell^{1}$ -Regularized Model	4.91E-04	104.22 (140)	5.57E-04	102.1 (131)	6.51E-04	101.7 (131)
$\ell^{\frac{3}{2}}$ -Regularized Model	5.49E-04	200 (200)	6.24E-04	200 (200)	7.25E-04	200 (200)
$\ell^{2}$ -Regularized Model	5.96E-04	200 (200)	6.83E-04	200 (200)	7.93E-04	200 (200)

Table 6.12 : Comparison of Least-Squares Regularization in RKHS and with

\ell^{p}

-Regularized Models Using the IMQ Kernel for the MTH function.

Kernel Methods

Kernel methods use similarity functions and reproducing kernel spaces to model nonlinear relationships, quantify uncertainty, and compare data.

See all 248 papers

Oct 14, 2024

Which Spaces can be Embedded in $L_p$ -type Reproducing Kernel Banach Space? A Characterization via Metric Entropy

In this paper, we establish a novel connection between the metric entropy growth and the embeddability of function spaces into reproducing kernel Hilbert/Banach spaces. Metric entropy characterizes the information complexity of function spaces and has implications for their approximability and learnability. Classical results show that embedding a function space into a reproducing kernel Hilbert space (RKHS) implies a bound on its metric entropy growth. Surprisingly, we prove a converse: a bound on the metric entropy growth of a function space allows its embedding to a

L_p-

type Reproducing Kernel Banach Space (RKBS). This shows that the

{L}_p-

type RKBS provides a broad modeling framework for learnable function classes with controlled metric entropies. Our results shed new light on the power and limitations of kernel methods for learning complex function spaces.

Yiping Lu, Daozhe Lin, Qiang Du

Industrial Engineering & Management Sciences, Northwestern University · Department of Applied Physics & Applied Mathematics, Columbia University

Reproducing Kernel Hilbert Spaces Kernel Methods Statistical Learning Theory

Ask PDF

Apr 25, 2026

A Unified Fractional Regularization Framework for Sparse Recovery

We propose a unified fractional regularization framework for sparse signal recovery based on the

\ell_1/\ell_p^q

model. This model generalizes several widely used sparsity-promoting regularizers and provides additional flexibility through the parameters

p

and

q

. Our main theoretical contribution is the characterization of the equivalence between the first-order stationary points of the

\ell_1/\ell_p^q

formulation and the subtractive

\ell_1-α\ell_p

model, thereby offering a unified perspective on these nonconvex regularizers. In addition, we establish a new sufficient recovery condition under the Restricted Isometry Property (RIP), which shows that the proposed framework can provide relaxed recovery guarantees and improved robustness. To solve the resulting nonconvex problem, we develop a majorization--minimization (MM) algorithm and prove its convergence by using the Kurdyka--Łojasiewicz (KL) property. Numerical experiments on sparse recovery problems with different sensing matrices and MRI reconstruction demonstrate that the proposed approach outperforms existing methods in recovery accuracy.

Yinhao Zhao, Yaohua Hu, Hao Wang

School of Information Science and Technology, ShanghaiTech University, 201210

Sparse Recovery Nonconvex Optimization

Ask PDF

Jun 22, 2026

Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence

We develop a comprehensive theory for regularized M-estimation in reproducing kernel Hilbert spaces. Under mild conditions on the loss we establish existence and measurability of the estimator, covering a wide range of convex and non-convex losses, including bounded robust losses. We further prove sharp rates of convergence with an explicit bias-variance decomposition governed by a novel complexity measure. We show that the variance is independent of misspecification, while the bias depends on a source condition parameter known in the learning literature. For tensor product Sobolev spaces we obtain new rates that connect to spaces of functions with dominating mixed smoothness, substantially extending existing results and explaining why these estimators circumvent the curse of dimensionality. Our methodology, combining elements from both functional analysis and empirical process theory, allows for an asymptotic linearisation of the objective function that avoids both closed-form solutions and global Lipschitz assumptions, and may be of independent interest. The estimators are implemented in C++ and theory is supported by numerical experiments.

Ioannis Kalogridis

School of Mathematics and Statistics, University of Glasgow, United Kingdom

Nonparametric Regression Reproducing Kernel Hilbert Spaces Kernel Methods Statistical Learning Theory

Ask PDF

Abstract

Kernel Methods

Explore similar work

Which Spaces can be Embedded in LpL_pLp​-type Reproducing Kernel Banach Space? A Characterization via Metric Entropy

A Unified Fractional Regularization Framework for Sparse Recovery

Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence

Which Spaces can be Embedded in $L_p$ -type Reproducing Kernel Banach Space? A Characterization via Metric Entropy