Functional linear regression from sparse to dense designs: a pooling-ridge method and minimax optimality

Shunxing Yan; Fang Yao

stat.MEAug 26, 2026

Functional linear regression from sparse to dense designs: a pooling-ridge method and minimax optimality

Authors: Shunxing Yan, Fang Yao

Organizations: Department of Statistics and Data Science, Tsinghua University · School of Mathematical Sciences, Center for Statistical Science, Peking University

Abstract

Functional data analysis is an important statistical field that treats data as random functions. In practice, the random functions are often not fully observed but instead measured at discrete times. While simpler problems, such as mean and covariance estimation, have been widely studied for discretely observed data, optimal estimation of linear regression for this data type has remained unsolved for over two decades. To tackle this fundamental challenge, we propose a novel approach, referred to as pooling ridge estimation, which combines the advantages of pooling strategy and RKHS-based method by incorporating the unbiased estimation of operators based on discretely observed measurements from all subjects. This unified estimation framework enables us to achieve minimax optimality in prediction risk in arbitrary sampling schemes ranging from sparse to dense designs, for both scalar-on-function and function-on-function regression models. Such methodological and theoretical advances are obtained for the first time and accurately reveal the influence of discrete sampling. For scalar-on-function regression, the phase transition occurs once, separating the convergence behavior into two distinct regimes. Remarkably, for function-on-function regression, up to three phase transitions may occur, determined by the sampling frequencies of the predictor/response functions. Finally, simulation experiments and two real data examples provide empirical support for the proposed methods.

Figures & tables

Fig 1: The intuitive implications of each term in minimax optimal rate of function-on-function regression in terms of norm induced by prediction, under the aligned case. The term $n^{-(2r_{bx}+2r_{c})/(2r_{bx}+2r_{c}+1)}$ is determined by $n$ pairs functions; while $(nm_{x})^{-(2r_{bx}+2r_{c})/(2r_{bx}+4r_{c}+1)}$ and $(nm_{y})^{-2r_{by}/(2r_{by}+1)}$ are introduced by the discrete measurements of the predictor and response functions, respectively.

Panel of Figure 2 — Fig 2: Phase diagram of function-on-function regression with discretely observed functional data. The subfigure (a)-(d) correspond to the four cases: $r_{bx}+r_{c}\leq r_{by}$ , $r_{by}<r_{bx}+r_{c}<r_{by}(1+2r_{c})$ , $r_{bx}+r_{c}=r_{by}(1+2r_{c})$ and $r_{bx}+r_{c}>r_{by}(1+2r_{c})$ respectively. The horizontal (vertical) coordinate represents the order of the predictor (response) sampling frequency with respect to $n$ , increasing further to the right (up). The boundaries $p_{1}$ , $p_{2}$ , $p_{3}$ indicate phase transition positions in ( 1 ), respectively. They divide the convergence into several sampling regimes; where red indicates the dense regime, while green/blue parts are the predictor/response-sparse regime.

	Sparse	Phase transition	Dense
Proposed	${(nm_{x})^{-\frac{2r_{b}+2r_{c}}{2r_{b}+4r_{c}+1}}}$	${n^{\frac{2r_{c}}{2r_{b}+2r_{c}+1}}}$	${n^{-\frac{2r_{b}+2r_{c}}{2r_{b}+2r_{c}+1}}}$
Existing	——	$n^{\frac{4r_{c}+2}{2r_{b}+2r_{c}+1}}$ $\vee$ $n^{\frac{4r_{b}+2r_{c}+1}{4r_{b}+4r_{c}+2}}$	${{n^{-\frac{2r_{b}+2r_{c}}{2r_{b}+2r_{c}+1}}}}$

Table 1: A table comparing newly established rates with existing results for scalar-on-function regression.

	Phase position	Convergence rate
Dense regime	$m_{x}\gtrsim n^{\frac{2s}{2(1-\alpha)r_{x}+1}},\ m_{y}\gtrsim n^{\frac{r_{x}-r_{y}}{(2(1-\alpha)r_{x}+1)r_{y}}}$	$n^{-\frac{2(1-\alpha)r_{x}}{2(1-\alpha)r_{x}+1}}$
Predictor-sparse regime	$m_{x}\lesssim n^{\frac{2s}{2(1-\alpha)r_{x}+1}},m_{y}\gtrsim n^{\frac{r_{x}-r_{y}(2s+1)}{r_{y}(2(1-\alpha)r_{x}+2s+1)}}m_{x}^{\frac{r_{x}(2(1-\alpha)r_{y}+1)}{r_{y}(2(1-\alpha)r_{x}+2s+1)}}$	$(nm_{x})^{-\frac{2(1-\alpha)r_{x}}{2(1-\alpha)r_{x}+2s+1}}$
Response-sparse regime	$\ m_{y}\lesssim n^{\frac{r_{x}-r_{y}}{(2(1-\alpha)r_{x}+1)r_{y}}},m_{y}\lesssim n^{\frac{r_{x}-r_{y}(2s+1)}{r_{y}(2(1-\alpha)r_{x}+2s+1)}}m_{x}^{\frac{r_{x}(2(1-\alpha)r_{y}+1)}{r_{y}(2(1-\alpha)r_{x}+2s+1)}}$	$(nm_{y})^{-\frac{2(1-\alpha)r_{y}}{2(1-\alpha)r_{y}+1}}$

Table 2: A summary of the three convergence rate phases in function-on-function regression. All orders in this table are optimal and first established in this work for discretely observed functional data.

$n$	$m$	Proposed	FullyRKHS	Plug-in	IN	PACE	MC
$50$	$3$	0.5788 (0.4117)	1.2675 (0.6028)	0.9492 (1.1531)	1.7499 (1.5332)	1.3767 (2.4667)	1.4872 (0.8066)
$50$	$6$	0.2392 (0.1751)	0.5440 (0.2946)	0.5170 (0.4968)	0.6847 (0.4745)	0.5239 (1.4602)	0.6361 (0.3854)
$50$	$10$	0.1672 (0.1483)	0.2742 (0.1741)	0.2772 (0.2098)	0.4099 (0.2765)	0.2683 (0.4861)	0.2771 (0.1573)
$50$	$20$	0.0807 (0.0714)	0.1244 (0.0805)	0.1837 (0.1465)	0.1750 (0.1211)	0.1087 (0.0931)	0.1346 (0.0887)
$100$	$3$	0.4936 (0.2782)	1.2094 (0.3722)	0.5715 (0.6122)	1.6057 (1.1057)	1.0182 (2.6082)	1.2742 (0.4419)
$100$	$6$	0.1640 (0.1372)	0.4371 (0.2129)	0.3271 (0.2389)	0.5920 (0.3519)	0.2528 (0.1944)	0.4480 (0.1952)

Table 3: The Monte Carlo averages and standard deviation (in parentheses) of the error

\mathcal{E}_{\beta_{0}}

of the proposed method and the other five benchmark methods, for scalar-on-function regression with cosine design.

Fig 3: Logarithmic error under different sampling frequencies. The left panel shows the scalar-on-function regression setting and the right panel shows the function-on-function regression in cosine design setting. The horizontal axis is the sampling frequency, taking values

3,5,10,15,20,30,40,50

; for function-on-function regression, we set

m_{x}=m_{y}

. Each line represents a different sample size.

$n$	$m_{x}$	$m_{y}$	Proposed	OPFFR-S	IN	PACE	MC	RKHSPCA
50	3	3	1.2120 (2.6020)	12.4922 (39.4993)	2.7684 (1.1817)	2.5760 (3.1839)	2.1556 (0.9035)	1.4128 (1.7811)
50	6	6	0.7593 (2.7422)	7.5348 (18.5352)	1.1771 (0.4902)	2.0945 (4.2141)	1.0853 (0.4371)	0.8459 (0.5838)
50	10	10	0.3940 (0.1957)	1.2879 (1.6185)	0.7344 (0.2530)	0.6376 (0.4747)	0.7243 (0.2362)	0.6871 (0.4528)
50	20	20	0.2846 (0.1850)	0.4236 (0.2920)	0.4337 (0.1934)	0.2868 (0.1461)	0.4788 (0.2190)	0.5096 (0.2135)
100	3	3	0.5314 (0.3044)	4.4701 (6.4807)	2.3611 (0.8503)	1.5384 (2.5360)	1.6284 (0.5184)	0.8282 (0.5696)
100	6	6	0.4795 (0.8887)	2.0291 (2.4097)	0.9392 (0.3231)	0.6803 (0.6065)	0.7580 (0.2132)	0.4932 (0.2038)

Table 4: The Monte Carlo averages and standard deviation (in parentheses) of the error

\mathcal{E}_{\beta_{0}}

of the proposed method and the other five benchmark methods, for function-on-function regression with cosine design.

$m$	Proposed	FullyRKHS	Plug-in	PACE	IN	MC
$5$	0.8029 (0.3514)	0.8261 (0.3514)	0.9024 (0.3777)	0.9218 (0.3737)	0.8701 (0.3792)	0.8231 (0.3435)
$10$	0.6761 (0.2905)	0.7491 (0.3128)	0.8793 (0.3909)	0.8764 (0.3733)	0.7955 (0.3566)	0.7190 (0.3048)
$20$	0.5920 (0.2562)	0.6696 (0.2758)	0.7351 (0.3782)	0.6715 (0.3165)	0.7123 (0.3318)	0.6271 (0.2513)

Table 5: The Monte Carlo averages and standard deviation (in parentheses) of the prediction error of the proposed method and the other five benchmarks, for the scalar-on-function regression on the wheat dataset at different sparsity.

Fig 4: Longitudinal observation of Z-score for BMI as a function of day from birth. Each colored thin line corresponds to an individual. The red vertical dotted line separates the past and future, which serve as predictors and responses in this study.

Method	Mean	Median	Sd	firstQu	thirdQu
Proposed	0.3536	0.2754	0.1972	0.1868	0.4899
OPFFR-S	0.4992	0.3782	0.3430	0.2458	0.6890
IN	0.4271	0.3517	0.2362	0.2324	0.5802
PACE	0.4024	0.3072	0.2258	0.2093	0.5926
MC	0.3823	0.3036	0.2171	0.1916	0.6020
RKHSPCA	0.5258	0.4113	0.2842	0.2834	0.7647

Table 6: The mean, standard deviation, and three quartiles of the empirical error of the proposed method and five alternative approaches, for 100 repeated experiments on the training/testing sets partition. The best result of each column is in boldface.

Explore similar work

Optimal estimation for Functional Linear Regression with Noisy Discretized Data
Sep 8, 2026Sixtine SphabmixayNonparametric Regression Minimax Estimation
Université Paris Cité, CNRS, MAP5, F-75006 Paris, France
Functional Attentive Interpretable Regression
Sep 5, 2026Haixu Wang, Tianyu Guan, Jiguo CaoNonparametric Regression
Department of Mathematics and Statistics, University of Calgary · Department of Mathematics and Statistics, York University · Department of Statistics and Actuarial Science, Simon Fraser University
High-dimensional ridgeless least squares interpolation under spiked covariance structures
Aug 7, 2026Zhijun Liu, Dandan JiangDouble Descent Benign Overfitting
College of Sciences, Northeastern University, China. · School of Mathematics and Statistics, Xi’an Jiao Tong University, China.

Sep 8, 2026J/K move · Enter open · S save

Optimal estimation for Functional Linear Regression with Noisy Discretized Data

Sixtine Sphabmixay

Université Paris Cité, CNRS, MAP5, F-75006 Paris, France

Nonparametric Regression Minimax Estimation Linear Regression

Open paper AskPDF arXiv

In this paper, we consider the scalar-on-function linear regression model under a realistic sampling scheme in which the functional covariates are observed on a regular grid and contaminated by additive noise. We propose a two-step estimation procedure: first, the underlying curves are reconstructed from the discrete noisy observations using a Fourier-based projection method; second, the slope function is estimated by a penalized least-squares criterion over finite-dimensional trigonometric spaces, with data-driven selection of the model dimension. We establish oracle-type inequalities for the prediction error, both with respect to the reconstructed curves and to the true latent curves. Under regularity assumptions on the slope function and polynomial decay of the eigenvalues of the covariate, we derive convergence rates for the prediction error and show that our estimator attains the minimax rate when the number of grid points is sufficiently large. Finally, the proposed method is illustrated on simulated data and on a real meteorological dataset.

Abstract

Explore similar work

Optimal estimation for Functional Linear Regression with Noisy Discretized Data

Functional Attentive Interpretable Regression

High-dimensional ridgeless least squares interpolation under spiked covariance structures