Deep regression learning from dependent observations with minimum error entropy principle

William Kengne; Modou Wade

stat.MLMar 11, 2026

Deep regression learning from dependent observations with minimum error entropy principle

Authors: William Kengne, Modou Wade

Organizations: Institut Camille Jordan, Université Jean Monnet, 23 Rue Dr Paul Michelon 42023 Saint-Etienne Cedex 2, France · THEMA, CY Cergy Paris Université, 33 Boulevard du Port, 95011 Cergy-Pontoise Cedex, France

Abstract

This paper considers nonparametric regression from strongly mixing observations. The proposed approach is based on deep neural networks with minimum error entropy (MEE) principle. We study two estimators: the non-penalized deep neural network (NPDNN) and the sparse-penalized deep neural network (SPDNN) predictors. Upper bounds of the expected excess risk are established for both estimators over the classes of Hölder and composition Hölder functions. For the models with Gaussian error, the rates of the upper bound obtained match (up to a logarithmic factor) with the lower bounds established in \cite{schmidt2020nonparametric}, showing that both the MEE-based NPDNN and SPDNN estimators from strongly mixing data can achieve the minimax optimal convergence rate.

Nonparametric Regression

Nonparametric regression estimates flexible relationships between predictors and outcomes without fixing a finite-dimensional model in advance.

See all 45 papers

Statistical Properties of Deep Neural Networks with Dependent Data
Oct 14, 2024Chad BrownNonparametric Regression Neural Network Approximation Theory
Department of Economics, University of Colorado Boulder, 80309, USA.
Deep learning with missing data
Apr 21, 2025Tianyi Ma, Tengyao Wang, Richard J. SamworthNeural Network Generalization Learning with Missing Data
Statistical Laboratory, University of Cambridge · Department of Statistics, London School of Economics
Adaptive deep nonparametric regression from dependent data under covariate shift
Jul 22, 2026William Kengne, Ehud Mossa OckegnaQuantile Regression Covariate Shift
Université Jean Monnet, ICJ UMR5208, CNRS, Ecole Centrale de Lyon, INSA Lyon, Universite Claude Bernard Lyon 1, 42000 Saint-Étienne, France

Oct 14, 2024J/K move · Enter open · S save

Statistical Properties of Deep Neural Networks with Dependent Data

Chad Brown

Department of Economics, University of Colorado Boulder, 80309, USA.

Nonparametric Regression Neural Network Approximation Theory Statistical Learning Theory

Open paper AskPDF arXiv

This paper develops theory for deep neural network (DNN) estimators under dependent data. To provide theory applicable to a variety of DNN-based estimators, I first establish nonasymptotic probability bounds on the theoretical and empirical

\mathcal{L}^{2}

-errors of nonparametric sieve estimators for a general class of estimation problems under possibly nonstationary

β

-mixing data taking values in unbounded sets. I then apply the theory to fully connected and convolutional DNN estimators without bounds or sparsity restrictions on the DNN weights. For both DNN classes, I derive general results when the function to be estimated is Hölder smooth and the data are nonstationary, subgaussian, and

β

-mixing with either exponential or polynomial decay. I then specialize these to nonparametric regression, logistic regression, and quantile regression settings. Under exponential

β

-mixing, the resulting estimators attain the nonparametric minimax rate of Stone (1982) up to logarithmic factors.

Abstract

Nonparametric Regression

Explore similar work

Statistical Properties of Deep Neural Networks with Dependent Data

Deep learning with missing data

Adaptive deep nonparametric regression from dependent data under covariate shift