Nonparametric Regression

Follow this topic Ask about this topic

Nonparametric regression studies how to estimate an unknown relationship between inputs and outcomes while allowing its shape to be determined by data and assumptions such as smoothness or additivity. Common approaches include kernel and spline smoothing, Gaussian processes and reproducing-kernel methods, tree ensembles, and neural networks; researchers also develop online, functional, and distributional variants for specialized data. A central theoretical goal is to match minimax rates, which quantify the best possible prediction accuracy under a given smoothness class, while adapting to unknown structure and handling dependence, missingness, or distribution shift. The main tension is that flexible models can achieve strong guarantees under suitable conditions, but high dimension, computational cost, and reliable uncertainty estimates remain difficult.

Latest papers

45 papers7 papers in the last four weeks, down 30% on the four weeks before. 0.0% of all new papers.

ParaRNN: An Interpretable and Parallelizable Recurrent Neural Network for Time-Dependent Data
May 4, 2026Yuxi Cai, Lan Li, Feiqing Huang +1Interpretable ML Recurrent Neural Networks
Department of Statistics and Actuarial Science, University of Hong Kong
Adversarial Robustness of NTK Neural Networks
Apr 28, 2026Yuxuan HouNeural Network Robustness Adversarial Robustness
1Qiuzhen College, Tsinghua University
SMART Fine-tuning Factor Augmented Neural Lasso
Apr 14, 2026Jinhang Chai, Jianqing Fan, Cheng Gao +1Fine-Tuning Transfer Learning
Department of Operations Research and Financial Engineering, Princeton University
Online Quantile Regression for Nonparametric Additive Models
Apr 10, 2026Haoran ZhanQuantile Regression Nonparametric Regression
The Theory and Practice of Highly Scalable Gaussian Process Regression with Nearest Neighbours
Apr 8, 2026Robert Allison, Tomasz Maciazek, Anthony StephensonGaussian Process Regression Nonparametric Regression
School of Mathematics University of Bristol Bristol, BS8 1UG, UK
Nonparametric Regression Discontinuity Designs with Survival Outcomes
Apr 3, 2026Maximilian Schuessler, Erik Sverdrup, Robert Tibshirani +1Causal Effect Estimation Survival Analysis
Gradient Descent with Projection Finds Over-Parameterized Neural Networks for Learning Low-Degree Polynomials with Nearly Minimax Optimal Rate
Mar 22, 2026Yingzhen Yang, Ping LiLow-Degree Polynomial Method Neural Network Generalization
School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ 85281, USA · VecML Inc., Bellevue, WA 98004, USA
Consistency of the $k$ -Nearest Neighbor Regressor under Complex Survey Designs
Mar 18, 2026Caren Haslerk-Nearest Neighbors Nonparametric Regression
Deep regression learning from dependent observations with minimum error entropy principle
Mar 11, 2026William Kengne, Modou WadeNonparametric Regression Minimax Estimation
Institut Camille Jordan, Université Jean Monnet, 23 Rue Dr Paul Michelon 42023 Saint-Etienne Cedex 2, France · THEMA, CY Cergy Paris Université, 33 Boulevard du Port, 95011 Cergy-Pontoise Cedex, France
Standard Transformers Achieve the Minimax Rate in Nonparametric Regression with $C^{s,λ}$ Targets
Feb 24, 2026Yanming Lai, Defeng SunTransformer Expressivity Neural Network Approximation Theory
Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Hong Kong, China
When is Fine-Tuning Scalable? Certified Recovery from Noisy Data and Linear-Cost Fine-Tuning on Structured Task Shifts
Aug 31, 2025Anastasis Kratsios, Tin Sum Cheng, Bum Jun Kim +1Fine-Tuning OOD Generalization
Department of Mathematics, McMaster University, Canada · Department of Mathematics and Computer Science, University of Basel, Switzerland · Department of Statistical Sciences, University of Toronto, Canada +1
Identifiable Convex-Concave Regression via Sub-gradient Regularised Least Squares
Jun 22, 2025William ChungParameter Identifiability Nonparametric Regression
Department of Decision Analytics and Operations, City University of Hong Kong, Hong Kong
Deep learning with missing data
Apr 21, 2025Tianyi Ma, Tengyao Wang, Richard J. SamworthNeural Network Generalization Learning with Missing Data
Statistical Laboratory, University of Cambridge · Department of Statistics, London School of Economics
Dense ReLU Neural Networks for Temporal-spatial Model
Nov 15, 2024Carlos Misael Madrid Padilla, Zhi Zhang, Xiaokai Luo +2Nonparametric Regression ReLU Neural Networks
Department of Statistics and Data Science, Washington University in St. Louis. · Department of Statistics and Data Science, University of California, Los Angeles · Department of Statistics, University of Notre Dame
Statistical Properties of Deep Neural Networks with Dependent Data
Oct 14, 2024Chad BrownNonparametric Regression Neural Network Approximation Theory
Department of Economics, University of Colorado Boulder, 80309, USA.

May 4, 2026J/K move · Enter open · S save

ParaRNN: An Interpretable and Parallelizable Recurrent Neural Network for Time-Dependent Data

Yuxi Cai, Lan Li, Feiqing Huang, Guodong Li

Department of Statistics and Actuarial Science, University of Hong Kong

Interpretable ML Recurrent Neural Networks Nonparametric Regression Neural Network Approximation Theory

Open paper AskPDF arXiv

The proliferation of large-scale and structurally complex data has spurred the integration of machine learning methods into statistical modeling. Recurrent neural networks (RNNs), a foundational class of models for time-dependent data, can be viewed as nonlinear extensions of classical autoregressive moving average models. Despite their flexibility and empirical success in machine learning, RNNs often suffer from limited interpretability and slow training, which hinders their use in statistics. This paper proposes the Parallelized RNN (ParaRNN), a novel model composed of multiple small recurrent units. ParaRNN admits an additive representation that decouples recurrent dynamics into interpretable components, whose behavior can be characterized through recurrence features. This interpretability enables its applications in nonparametric regression for time-dependent data, while the design also allows efficient parallelization. The approximation capacity and non-asymptotic prediction error bounds in a nonparametric regression setting are established for ParaRNN. Empirical results on three sequential modeling tasks further demonstrate that ParaRNN achieves performance comparable to vanilla RNNs while offering improved interpretability and efficiency.