Functional linear regression from sparse to dense designs: a pooling-ridge method and minimax optimality

Shunxing Yan; Fang Yao

stat.MEAug 26, 2026

Functional linear regression from sparse to dense designs: a pooling-ridge method and minimax optimality

Authors: Shunxing Yan, Fang Yao

Organizations: Department of Statistics and Data Science, Tsinghua University · School of Mathematical Sciences, Center for Statistical Science, Peking University

Abstract

Functional data analysis is an important statistical field that treats data as random functions. In practice, the random functions are often not fully observed but instead measured at discrete times. While simpler problems, such as mean and covariance estimation, have been widely studied for discretely observed data, optimal estimation of linear regression for this data type has remained unsolved for over two decades. To tackle this fundamental challenge, we propose a novel approach, referred to as pooling ridge estimation, which combines the advantages of pooling strategy and RKHS-based method by incorporating the unbiased estimation of operators based on discretely observed measurements from all subjects. This unified estimation framework enables us to achieve minimax optimality in prediction risk in arbitrary sampling schemes ranging from sparse to dense designs, for both scalar-on-function and function-on-function regression models. Such methodological and theoretical advances are obtained for the first time and accurately reveal the influence of discrete sampling. For scalar-on-function regression, the phase transition occurs once, separating the convergence behavior into two distinct regimes. Remarkably, for function-on-function regression, up to three phase transitions may occur, determined by the sampling frequencies of the predictor/response functions. Finally, simulation experiments and two real data examples provide empirical support for the proposed methods.

Figures & tables

Fig 1: The intuitive implications of each term in minimax optimal rate of function-on-function regression in terms of norm induced by prediction, under the aligned case. The term $n^{-(2r_{bx}+2r_{c})/(2r_{bx}+2r_{c}+1)}$ is determined by $n$ pairs functions; while $(nm_{x})^{-(2r_{bx}+2r_{c})/(2r_{bx}+4r_{c}+1)}$ and $(nm_{y})^{-2r_{by}/(2r_{by}+1)}$ are introduced by the discrete measurements of the predictor and response functions, respectively.

Panel of Figure 2 — Fig 2: Phase diagram of function-on-function regression with discretely observed functional data. The subfigure (a)-(d) correspond to the four cases: $r_{bx}+r_{c}\leq r_{by}$ , $r_{by}<r_{bx}+r_{c}<r_{by}(1+2r_{c})$ , $r_{bx}+r_{c}=r_{by}(1+2r_{c})$ and $r_{bx}+r_{c}>r_{by}(1+2r_{c})$ respectively. The horizontal (vertical) coordinate represents the order of the predictor (response) sampling frequency with respect to $n$ , increasing further to the right (up). The boundaries $p_{1}$ , $p_{2}$ , $p_{3}$ indicate phase transition positions in ( 1 ), respectively. They divide the convergence into several sampling regimes; where red indicates the dense regime, while green/blue parts are the predictor/response-sparse regime.

	Sparse	Phase transition	Dense
Proposed	${(nm_{x})^{-\frac{2r_{b}+2r_{c}}{2r_{b}+4r_{c}+1}}}$	${n^{\frac{2r_{c}}{2r_{b}+2r_{c}+1}}}$	${n^{-\frac{2r_{b}+2r_{c}}{2r_{b}+2r_{c}+1}}}$
Existing	——	$n^{\frac{4r_{c}+2}{2r_{b}+2r_{c}+1}}$ $\vee$ $n^{\frac{4r_{b}+2r_{c}+1}{4r_{b}+4r_{c}+2}}$	${{n^{-\frac{2r_{b}+2r_{c}}{2r_{b}+2r_{c}+1}}}}$

Table 1: A table comparing newly established rates with existing results for scalar-on-function regression.

	Phase position	Convergence rate
Dense regime	$m_{x}\gtrsim n^{\frac{2s}{2(1-\alpha)r_{x}+1}},\ m_{y}\gtrsim n^{\frac{r_{x}-r_{y}}{(2(1-\alpha)r_{x}+1)r_{y}}}$	$n^{-\frac{2(1-\alpha)r_{x}}{2(1-\alpha)r_{x}+1}}$
Predictor-sparse regime	$m_{x}\lesssim n^{\frac{2s}{2(1-\alpha)r_{x}+1}},m_{y}\gtrsim n^{\frac{r_{x}-r_{y}(2s+1)}{r_{y}(2(1-\alpha)r_{x}+2s+1)}}m_{x}^{\frac{r_{x}(2(1-\alpha)r_{y}+1)}{r_{y}(2(1-\alpha)r_{x}+2s+1)}}$	$(nm_{x})^{-\frac{2(1-\alpha)r_{x}}{2(1-\alpha)r_{x}+2s+1}}$
Response-sparse regime	$\ m_{y}\lesssim n^{\frac{r_{x}-r_{y}}{(2(1-\alpha)r_{x}+1)r_{y}}},m_{y}\lesssim n^{\frac{r_{x}-r_{y}(2s+1)}{r_{y}(2(1-\alpha)r_{x}+2s+1)}}m_{x}^{\frac{r_{x}(2(1-\alpha)r_{y}+1)}{r_{y}(2(1-\alpha)r_{x}+2s+1)}}$	$(nm_{y})^{-\frac{2(1-\alpha)r_{y}}{2(1-\alpha)r_{y}+1}}$

Table 2: A summary of the three convergence rate phases in function-on-function regression. All orders in this table are optimal and first established in this work for discretely observed functional data.

$n$	$m$	Proposed	FullyRKHS	Plug-in	IN	PACE	MC
$50$	$3$	0.5788 (0.4117)	1.2675 (0.6028)	0.9492 (1.1531)	1.7499 (1.5332)	1.3767 (2.4667)	1.4872 (0.8066)
$50$	$6$	0.2392 (0.1751)	0.5440 (0.2946)	0.5170 (0.4968)	0.6847 (0.4745)	0.5239 (1.4602)	0.6361 (0.3854)
$50$	$10$	0.1672 (0.1483)	0.2742 (0.1741)	0.2772 (0.2098)	0.4099 (0.2765)	0.2683 (0.4861)	0.2771 (0.1573)
$50$	$20$	0.0807 (0.0714)	0.1244 (0.0805)	0.1837 (0.1465)	0.1750 (0.1211)	0.1087 (0.0931)	0.1346 (0.0887)
$100$	$3$	0.4936 (0.2782)	1.2094 (0.3722)	0.5715 (0.6122)	1.6057 (1.1057)	1.0182 (2.6082)	1.2742 (0.4419)
$100$	$6$	0.1640 (0.1372)	0.4371 (0.2129)	0.3271 (0.2389)	0.5920 (0.3519)	0.2528 (0.1944)	0.4480 (0.1952)

Table 3: The Monte Carlo averages and standard deviation (in parentheses) of the error

\mathcal{E}_{\beta_{0}}

of the proposed method and the other five benchmark methods, for scalar-on-function regression with cosine design.

Fig 3: Logarithmic error under different sampling frequencies. The left panel shows the scalar-on-function regression setting and the right panel shows the function-on-function regression in cosine design setting. The horizontal axis is the sampling frequency, taking values

3,5,10,15,20,30,40,50

; for function-on-function regression, we set

m_{x}=m_{y}

. Each line represents a different sample size.

$n$	$m_{x}$	$m_{y}$	Proposed	OPFFR-S	IN	PACE	MC	RKHSPCA
50	3	3	1.2120 (2.6020)	12.4922 (39.4993)	2.7684 (1.1817)	2.5760 (3.1839)	2.1556 (0.9035)	1.4128 (1.7811)
50	6	6	0.7593 (2.7422)	7.5348 (18.5352)	1.1771 (0.4902)	2.0945 (4.2141)	1.0853 (0.4371)	0.8459 (0.5838)
50	10	10	0.3940 (0.1957)	1.2879 (1.6185)	0.7344 (0.2530)	0.6376 (0.4747)	0.7243 (0.2362)	0.6871 (0.4528)
50	20	20	0.2846 (0.1850)	0.4236 (0.2920)	0.4337 (0.1934)	0.2868 (0.1461)	0.4788 (0.2190)	0.5096 (0.2135)
100	3	3	0.5314 (0.3044)	4.4701 (6.4807)	2.3611 (0.8503)	1.5384 (2.5360)	1.6284 (0.5184)	0.8282 (0.5696)
100	6	6	0.4795 (0.8887)	2.0291 (2.4097)	0.9392 (0.3231)	0.6803 (0.6065)	0.7580 (0.2132)	0.4932 (0.2038)

Table 4: The Monte Carlo averages and standard deviation (in parentheses) of the error

\mathcal{E}_{\beta_{0}}

of the proposed method and the other five benchmark methods, for function-on-function regression with cosine design.

$m$	Proposed	FullyRKHS	Plug-in	PACE	IN	MC
$5$	0.8029 (0.3514)	0.8261 (0.3514)	0.9024 (0.3777)	0.9218 (0.3737)	0.8701 (0.3792)	0.8231 (0.3435)
$10$	0.6761 (0.2905)	0.7491 (0.3128)	0.8793 (0.3909)	0.8764 (0.3733)	0.7955 (0.3566)	0.7190 (0.3048)
$20$	0.5920 (0.2562)	0.6696 (0.2758)	0.7351 (0.3782)	0.6715 (0.3165)	0.7123 (0.3318)	0.6271 (0.2513)

Table 5: The Monte Carlo averages and standard deviation (in parentheses) of the prediction error of the proposed method and the other five benchmarks, for the scalar-on-function regression on the wheat dataset at different sparsity.

Fig 4: Longitudinal observation of Z-score for BMI as a function of day from birth. Each colored thin line corresponds to an individual. The red vertical dotted line separates the past and future, which serve as predictors and responses in this study.

Method	Mean	Median	Sd	firstQu	thirdQu
Proposed	0.3536	0.2754	0.1972	0.1868	0.4899
OPFFR-S	0.4992	0.3782	0.3430	0.2458	0.6890
IN	0.4271	0.3517	0.2362	0.2324	0.5802
PACE	0.4024	0.3072	0.2258	0.2093	0.5926
MC	0.3823	0.3036	0.2171	0.1916	0.6020
RKHSPCA	0.5258	0.4113	0.2842	0.2834	0.7647

Table 6: The mean, standard deviation, and three quartiles of the empirical error of the proposed method and five alternative approaches, for 100 repeated experiments on the training/testing sets partition. The best result of each column is in boldface.

Explore similar work

Sep 8, 2026

Optimal estimation for Functional Linear Regression with Noisy Discretized Data

In this paper, we consider the scalar-on-function linear regression model under a realistic sampling scheme in which the functional covariates are observed on a regular grid and contaminated by additive noise. We propose a two-step estimation procedure: first, the underlying curves are reconstructed from the discrete noisy observations using a Fourier-based projection method; second, the slope function is estimated by a penalized least-squares criterion over finite-dimensional trigonometric spaces, with data-driven selection of the model dimension. We establish oracle-type inequalities for the prediction error, both with respect to the reconstructed curves and to the true latent curves. Under regularity assumptions on the slope function and polynomial decay of the eigenvalues of the covariate, we derive convergence rates for the prediction error and show that our estimator attains the minimax rate when the number of grid points is sufficiently large. Finally, the proposed method is illustrated on simulated data and on a real meteorological dataset.

Sixtine Sphabmixay

Université Paris Cité, CNRS, MAP5, F-75006 Paris, France

Nonparametric Regression Minimax Estimation Linear Regression

Ask PDF

Sep 5, 2026

Functional Attentive Interpretable Regression

In function-on-function regression, the coefficient surface

β(s,t)

may exhibit complex support structure---from localized patches to global patterns such as disconnected regions, bands, or rings---where effect similarity does not align with Euclidean proximity. Projection-based methods that rely on fixed basis expansions can obscure such structure, while direct smoothing approaches risk oversmoothing the surface and its boundaries. We propose Functional Attentive Interpretable Regression (FAIR), which represents

β(s,t)

directly through coordinate features and uses self-attention to learn effect-adaptive neighborhoods, enabling information sharing at both local and global scales. A scalar compression network maps these learned representations to the coefficient surface. Sparsity and smoothness penalties applied over these neighborhoods promote localized support with coherent boundaries. We establish a sieve equivalence to tensor-product spline spaces and derive convergence rates. Simulations and applications to oceanographic and hydrological data demonstrate that FAIR recovers support geometry more accurately than existing methods while achieving superior prediction, particularly under sparse sampling.

Haixu Wang, Tianyu Guan, Jiguo Cao

Department of Mathematics and Statistics, University of Calgary · Department of Mathematics and Statistics, York University · Department of Statistics and Actuarial Science, Simon Fraser University

Nonparametric Regression

Ask PDF

Aug 7, 2026

High-dimensional ridgeless least squares interpolation under spiked covariance structures

This paper investigates the asymptotic behavior of the out-of-sample prediction risk of the high-dimensional ridgeless least-squares estimator when the feature dimension

p

and the sample size

n

grow proportionally. We consider a generalized spiked population covariance model with multiple latent factors, where the number of spiked eigenvalues may remain finite or increase with

n

, and the spiked eigenvalues may be bounded or diverge at arbitrary rates. Beyond characterizing the impact of covariance spectra, we reveal a new mechanism underlying benign overfitting: the prediction behavior of ridgeless interpolation is fundamentally governed by the alignment between the regression coefficient

\boldsymbolβ

and the spiked eigenspaces of the population covariance matrix. In particular, we show that the signal energy distributed along latent spike directions determines whether interpolation leads to benign, tempered, or catastrophic overfitting. Our theoretical framework establishes sharp prediction risk limits under minimal moment conditions, requiring only finite fourth moments rather than Gaussianity. We characterize how the number, strength, and geometric structure of the spikes jointly influence the double-descent phenomenon. These results provide a unified understanding of when latent covariance structures facilitate or hinder generalization in overparameterized regression.

Zhijun Liu, Dandan Jiang

College of Sciences, Northeastern University, China. · School of Mathematics and Statistics, Xi’an Jiao Tong University, China.

Double Descent Benign Overfitting Statistical Learning Theory

Ask PDF