pESTO: Iterative p-greedy kernel regression with risk-driven Early STOpping
Organizations: Dipartimento di Scienze Matematiche Giuseppe Luigi Lagrange, Politecnico di Torino, Italy · Gruppo Nazionale per il Calcolo Scientifico INδAM, Rome, Italy · Institute for Data Science, University of Applied Sciences and Arts Northwestern Switzerland, Switzerland
Abstract
Kernel-based greedy algorithms select suitable approximation nodes by minimizing point-wise error indicators, such as the power function; in this context, they are usually referred to as p-greedy approaches. In the presence of noisy data, deterministic criteria may fail, as the discrepancy between the true solution and its estimate is influenced by both approximation error and noise amplification, and the balance between the two is unknown. This study explicitly separates these two contributions, which are governed by the power function and a Lebesgue-type term, respectively. These bounds motivate the use of the Stein's unbiased risk estimator (SURE) as a proxy of the mean squared error, which in turn provides a principled and computable stopping criterion for the number of nodes to select in the p-greedy procedure. Numerical experiments demonstrate that this approach effectively balances deterministic accuracy and statistical noise contributions.
Figures & tables
| Kernel | Nodes | Num. | SSIM | ||||||
|---|---|---|---|---|---|---|---|---|---|
| ESTO | All | Opt. | ESTO | All | Opt. | ||||
| Gaussian | Halton | 0.05 | 0.8 | 0.7 | 0.2 | 0.04 | 0.03 | 0.02 | |
| 0.1 | 1.8 | 2.6 | 1.6 | 0.06 | 0.03 | 0.04 | |||
| 0.05 | 0.6 | 2.4 | 0.4 | 0.03 | 0.04 | 0.02 | |||
| 0.1 | 1.3 | 10.0 | 1.2 | 0.03 | 0.04 | 0.03 | |||
| 0.05 | 0.3 | 6.0 | 0.2 | 0.02 | 0.03 | 0.02 | |||
| Kernel | Nodes | Num. | SSIM | ||||||
|---|---|---|---|---|---|---|---|---|---|
| ESTO | All | Opt. | ESTO | All | Opt. | ||||
| Gaussian | Halton | 0.05 | 0.9 | 0.8 | 0.4 | 0.05 | 0.03 | 0.01 | |
| 0.1 | 2.6 | 3.1 | 1.4 | 0.07 | 0.03 | 0.04 | |||
| 0.05 | 0.6 | 2.6 | 0.4 | 0.04 | 0.04 | 0.02 | |||
| 0.1 | 1.8 | 10.4 | 1.7 | 0.04 | 0.04 | 0.03 | |||
| 0.05 | 0.4 | 6.1 | 0.3 | 0.02 | 0.03 | 0.02 | |||
| Kernel | Nodes | Num. | Selected greedy iteration | ||||
|---|---|---|---|---|---|---|---|
| Franke | Product function | ||||||
| ESTO | Opt. | ESTO | Opt. | ||||
| Gaussian | Halton | 0.05 | 25.9 | 10.4 | 35.6 | 20.0 | |
| 0.1 | 9.6 | 7.4 | 21.5 | 9.6 | |||
| 0.05 | 27.4 | 12.6 | 34.1 | 21.5 | |||
| 0.1 | 9.6 | 10.4 | 13.3 | 8.2 | |||
Kernel Regression
Kernel regression uses similarity functions to predict outputs from data, with theory linking kernel choice, regularization, and generalization.
See all 36 papers