Neural Networks as Linear Regression: An Introduction for Statisticians

Abigail Loe; Susan Murray; Zhenke Wu

stat.MLJun 22, 2026

Neural Networks as Linear Regression: An Introduction for Statisticians

Authors: Abigail Loe, Susan Murray, Zhenke Wu

Organizations: Department of Mathematics, Statistics and Computer Science, Macalester College, Saint Paul, Minnesota, U.S.A. · Department of Biostatistics, University of Michigan, Ann Arbor, Michigan, U.S.A.

Abstract

Neural networks are a commonly used prediction tool in computer science and statistics. However, the barrier to entry of this interesting field remains high, particularly for classical statisticians trained in a frequentist perspective. In this letter, we demystify neural networks by describing networks that approximate a linear regression and describe common customizations that provide a foundation for further study.

Explore similar work

Neural Statistical Functions
May 11, 2026Daniel Xu, Yuxin Xie, Minghao Guo +2Statistics Regression
Steering Neural Network Training through Interpretable Constraints Based on Partial Dependence
Jul 9, 2026Yann Claes, Pierre Geurts, Vân Anh Huynh-ThuExplanations Machine Learning
Lecture Notes on Statistical Physics and Neural Networks
May 7, 2026Olaf HohmTwo-Dimensional Ising Model Mean-Field Limit

May 11, 2026 · cs.LGJ/K move · Enter open · S save

Neural Statistical Functions

Daniel Xu, Yuxin Xie, Minghao Guo, Haixu Wu+1

Columbia University · MIT CSAIL

Classical deep learning typically operates on individual cases. Despite its success, real-world usage often requires repeated inference to estimate statistical quantities for complex decision-making tasks involving uncertainty or extreme-value analysis, resulting in substantial latency. We introduce neural statistical functions, a new family of models learned from pre-trained single-sample predictors and scattered data samples, which can directly infer statistics over continuous operating condition ranges without explicit sampling. By introducing the notion of prefix statistics, we transform and unify diverse statistical functions (e.g., integrals, quantiles, and maxima) into an interval-conditional framework, in which a principled identity between the prefix statistics and the individual-case regression serves as the learning objective. Neural statistical functions achieve strong performance in estimating essential statistics of complex physical processes, including accumulated energy in dynamical systems, quantiles of aerodynamic responses, and maximum stress in crash processes, while achieving up to a 100

\times

reduction in model evaluations.

Statistics Regression Heavy-Tailed Noise

Open paper Ask about it PDF

Abstract

Explore similar work

Neural Statistical Functions

Steering Neural Network Training through Interpretable Constraints Based on Partial Dependence

Lecture Notes on Statistical Physics and Neural Networks