Linear Regression

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 48

All topics

Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data
Oct 6, 2026Kaito Takanami, Takashi Takahashi, Yoshiyuki KabashimaStatistical Physics of Learning Linear Regression
Graduate School of Science, The University of Tokyo · Institute for Physics of Intelligence, The University of Tokyo · Trans-Scale Quantum Science Institute, The University of Tokyo
LinearPFN: Amortized Variable Selection for Linear Models with Interactions
Oct 5, 2026Louis Schiekiera, Max Zimmer, Christophe Roux +3Model Selection Prior-Data Fitted Networks
Institute of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany · Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany · Department for AI in Society, Science, and Technology, Zuse Institute Berlin, Germany +1
Distributionally robust linear regression through the lens of adversarial training
Sep 30, 2026Elis Stefansson, David Vävinggren, Antônio H. RibeiroRobust Regression Distributionally Robust Optimization
Uppsala University, Sweden · Science for Life Laboratory, Sweden
Principal Component Regression Dominates all Monotone Spectral Filters for Linear Regression
Sep 30, 2026Juno Kim, Hengyu Fu, Peter Bartlett +2Spectral Regularization Linear Regression
University of California, Berkeley · Google DeepMind
Statistical Benefits of Fine-Tuning from Pretrained Initialization in Diagonal Linear Networks
Sep 28, 2026Alexandre Declèves, Etienne Boursier, Nicolas FlammarionFine-Tuning Deep Linear Networks
TML, EPFL, Lausanne, Switzerland · INRIA, LMO, Université Paris-Saclay, Orsay, France
Sparse Regression Distilled from a Single Robust Fit
Sep 20, 2026Wooyoung Shin, Seunghwan ParkRobust Regression Linear Regression
Department of Information Statistics, Kangwon National University, 1, Kangwondaehak-gil, Chuncheon-si, Gangwon-do, 24341, Republic of Korea.
Learning with Synthetic Data via SGD in High-Dimensional Linear Regression
Sep 10, 2026Jichu li, Difan ZouSynthetic Data Pretraining Model Collapse
Renmin University of China · The University of Hong Kong
Optimal estimation for Functional Linear Regression with Noisy Discretized Data
Sep 8, 2026Sixtine SphabmixayNonparametric Regression Minimax Estimation
Université Paris Cité, CNRS, MAP5, F-75006 Paris, France
Exact Recovery Thresholds for Weighted Data Selection in Vector-Valued Linear Regression
Aug 31, 2026Guangjian ZhangData Selection Training Data Selection
Contact Geometry and Covariance Deficits in Volume-Sampled Least Squares
Aug 27, 2026Kihun Rhee, Hanjoon Byun, Junpyo SeoRandomized Sketching Linear Regression
Algorithms for adaptive and heteroskedastic linear regression at the computational threshold
Aug 19, 2026Spencer Compton, Tselil SchrammMinimax Estimation Heteroscedastic Regression
Stanford University
Feature Priming in Online Linear Regression: Sparse-Regret Lower Bounds and Tight Coordinatewise Rates
Aug 18, 2026Huibo Xu, Shi Fu, Qixin Zhang +1Linear Regression Statistical Learning Theory
Nanyang Technological University, Singapore
The Fourth Quadrant: A Stylized View of Benign Misfitting
Aug 2, 2026Gireeja Ranade, Anant SahaiBenign Overfitting Linear Regression
University of California, Berkeley
The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression
Jul 27, 2026Kevin Han Huang, Haoyu Ye, Somak Laha +1Double Descent Linear Regression
University of Warwick · Harvard University
Beyond Negative-Ridge Endpoints: Mixed-Sign Spectral Regularization via Negative-Shifted Gradient Descent
Jul 24, 2026Peng ZhaoSpectral Regularization Early Stopping
Department of Applied Economics and Statistics · University of Delaware
De-floored Principal Component Regression: When Rank Selection Alone Is Insufficient for Prediction
Jul 18, 2026Peng ZhaoPrincipal Component Analysis Linear Regression
Department of Applied Economics and Statistics, University of Delaware
In-context learning of closed form solution to simple linear regression task using transformer with linear self-attention
Jul 17, 2026Katsuyuki HagiwaraIn-Context Learning Linear Regression
Faculty of Education, Mie University · Kurima-Machiya-cho, Tsu, 514-8507, Japan
Partial Identification with Multiple Nonlinear Measurements of a Latent Regressor
Jul 13, 2026Burhan Ogut, Michelle YinPartial Identification Linear Regression
American Institutes for Research. · Northwestern University, School of Education and Social Policy.
Distributed Sketching on Data Partitions for OLS Regression
Jul 8, 2026Luyuan Yang, Brayden Garner, Shayan Shafaei +1Randomized Sketching Linear Regression
a School of Computer Science, University of Oklahoma, U.S.
The Approximation Ratio for the Risk of Myopic Bayesian Active Learning for Linear Regression
Jul 7, 2026Stephen MussmannActive Learning Linear Regression
School of Computer Science Georgia Institute of Technology
Distributionally Robust Linear Regression With Block Lewis Weights
Jun 30, 2026Naren Sarayu Manoj, Kumar Kshitij PatelRobust Regression Distributionally Robust Optimization
TTIC · Institute for Foundations of Data Science, Yale University
Target-Aware Linear Regression Under Distribution Shift
Jun 22, 2026Zhewen Hou, Tian ZhengDistribution Shift Robustness Distribution Matching
Department of Statistics Columbia University New York, NY
Alternate loss functions and regression models that achieve robustness to outliers by modulating the learning rate
Jun 20, 2026Mathew Mithra Noel, Arindam Banerjee, Yug D. Oswal +2Robust Regression Robust Loss Functions
aSchool of Electrical Engineering, Vellore Institute of Technology, Vellore, 632 014, Tamil Nadu, India · bErnst and Young GDS, Godrej Waterside, Ring Rd, DP Block, Sector V, Bidhannagar, Kolkata, 700 091, West Bengal, India · cSchool of Computer Science and Engineering, Vellore Institute of Technology, Vellore, 632 014, Tamil Nadu, India
Measurement noise limits the advantage of nonlinear models over linear models in biomedical prediction
Jun 16, 2026Marc-Andre Schulz, Kerstin RitterLinear Regression Clinical Prediction
Hertie Institute for AI in Brain Health, University of Tübingen, Tübingen, Germany · Tübingen AI Center, University of Tübingen, Tübingen, Germany · Department of Psychiatry and Neurosciences, Charité – Universitätsmedizin Berlin, Berlin, Germany +2
The limits of interpretability in multiple linear regression
Jun 14, 2026Anand Sharma, Chen Liu, Daniele Coslovich +1Interpretable ML Linear Regression
Indian Institute of Science Education and Research, Dr. Homi Bhabha Road, Pashan, Pune 411008, India · Innovation and Research Division, Ge-Room Inc., 93160 Noisy le Grand, France · Dipartimento di Fisica, Universit`a di Trieste, Strada Costiera 11, 34151, Trieste, Italy∗ +1
Geometric Domain Adaptation via Optimal Transport for Linear Regression in R^2
Jun 12, 2026Brian Britos, Mathias BourelDomain Adaptation Linear Regression
IESTA, Facultad de Ciencias Económicas y de Administración, Universidad de la República, Montevideo, Uruguay · IRL-2030, Instituto Franco-Uruguayo de Matemática e Interacciones (IFUMI)
Bidirectional Random Projections
Jun 9, 2026Chao Lan, Luyuan YangRandomized Sketching Linear Regression
a School of Computer Science, University of Oklahoma, U.S.
Ridge Regression from Poisson Resetting: A Renewal Perspective on Spectral Regularization
May 28, 2026Petar JolakoskiSpectral Regularization Linear Regression
Research Center for Computer Science and Information Technologies, Macedonian Academy of Sciences and Arts, Bul. Krste Misirkov 2, 1000 Skopje, Macedonia
Joint Model and Data Sparsification via the Marginal Likelihood
May 28, 2026Alexander Timans, Thomas Möllenhoff, Christian A. Naesseth +2Dataset Pruning Robust Regression
1UvA-Bosch Delta Lab, University of Amsterdam · 2RIKEN Center for AI Project, Tokyo, Japan · Department of Computer Science, Johns Hopkins University.
Optimal ridge regularization revisited
May 27, 2026Jack Timmermans, Sergio A. AlvarezLinear Regression Statistical Learning Theory
Department of Computer Science, Boston College, Chestnut Hill, MA 02467 USA

Oct 6, 2026, cond-mat.dis-nnJ/K move · Enter open · S save

Asymptotic Analysis of Empirical Risk Minimization on Entry-wise i.i.d. Heavy-Tailed Data

Kaito Takanami, Takashi Takahashi, Yoshiyuki Kabashima

Graduate School of Science, The University of Tokyo · Institute for Physics of Intelligence, The University of Tokyo · Trans-Scale Quantum Science Institute, The University of Tokyo

Statistical Physics of Learning Linear Regression Heavy-Tailed Distributions Empirical Risk Minimization

Open paper AskPDF arXiv

Many real-world datasets exhibit unusually large values far more frequently than predicted by Gaussian models. Heavy-tailed distributions capture this behavior, yet evaluating learning performance under them remains challenging because rare, large feature entries retain non-vanishing effects even in high dimensions. Even in the canonical setting of empirical risk minimization for linear regression with entry-wise i.i.d. symmetric

α

-stable data, a precise asymptotic characterization of prediction has been lacking. In this work, we introduce a functional order parameter that describes the random effective problem associated with each coefficient. Using the replica method, we fully characterize the generalization error in the proportional high-dimensional limit where the sample size and feature dimension diverge at a fixed ratio. Additionally, this analysis establishes a heavy-tail universality law, scaling laws relating typical errors to prediction reliability, and the Bayes-optimal prediction error. In addition to characterizing the effects of extreme entries on the learning process, our method applies broadly to other systems with persistent local heterogeneity.

Momentum

Related topics

Latest papers 48