High-Dimensional Linear Regression

Latest papers

19 papers4 papers in the last four weeks, down 20% on the four weeks before. 0.0% of all new papers.

On the Limits of Sparse Linear Model with Shuffled Differential Privacy
Oct 23, 2026Yan Zhang, Jiong Tuo, Meng Ding +2High-Dimensional Linear Regression Differential Privacy
Provable Responsible AI and Data Analytics Lab · Nankai University · King Abdullah University of Science and Technology (KAUST) +3
Score-Preserving Continual Lasso under Task Shift
Oct 23, 2026Jingyi Guo, Runze Li, Lingzhou XueContinual Learning Regularization-Based Continual Learning
Pennsylvania State University
Minimax-optimal Scaling Laws in Ridge Regression via Transfer Learning
Oct 16, 2026Yalda Shabanzadeh, Diyuan Wu, Marco MondelliTransfer Learning Ridge Regression
Institute of Science and Technology Austria (ISTA).
Optimal Explicit and Implicit Regularization Strengths in High-Dimensional Continual Linear Regression
Oct 14, 2026Gilad Karpel, Daniel SoudryHigh-Dimensional Linear Regression Regularization-Based Continual Learning
Department of Data and Decisions Sciences, Technion · Department of Electrical and Computer Engineering, Technion
When do data mixtures improve scaling laws? Insights from high-dimensional regression
Sep 29, 2026Diyuan Wu, Lehan Chen, Theodor Misiakiewicz +1High-Dimensional Linear Regression Data Mixture Optimization
Institute of Science and Technology Austria (ISTA). · Department of Applied and Computational Mathematics, Yale University. · Department of Statistics and Data Science, Yale University.
Benign Overfitting for General Norms and Distributions
Sep 27, 2026Daniel Barzilai, Ohad ShamirBenign Overfitting Statistical Learning Theory
Weizmann Institute of Science · University of Toronto & Vector Institute
A Bayesian Vertical Federated Learning Framework for Multivariate Reduced-Rank High-Dimensional Regression
Sep 18, 2026Brigham Halverson, Sharmistha Guha, Jessica Bernard +1High-Dimensional Linear Regression Federated Learning
Department of Statistics Texas A&M University · Department of Psychological & Brain Sciences Texas A&M University
Structured Features Overfit Where Random Features Grok
Sep 14, 2026Chon-Fai Kam, Miloud Bessafi, Frederic CadetNeural Network Generalization Ridge Regression
University Paris City & University of Reunion, Paris, France; Dipartimento di Fisica e Chimica · Emilio Segr`e, Universit`a degli Studi di Palermo, Italy · EnergyLab, University of Reunion, Saint-Denis, France +2
Learning with Synthetic Data via SGD in High-Dimensional Linear Regression
Sep 9, 2026Jichu li, Difan ZouSynthetic Data Pretraining High-Dimensional Linear Regression
Renmin University of China · The University of Hong Kong
High-dimensional ridgeless least squares interpolation under spiked covariance structures
Aug 7, 2026Zhijun Liu, Dandan JiangDouble Descent Benign Overfitting
College of Sciences, Northeastern University, China. · School of Mathematics and Statistics, Xi’an Jiao Tong University, China.
The Fourth Quadrant: A Stylized View of Benign Misfitting
Aug 2, 2026Gireeja Ranade, Anant SahaiBenign Overfitting Linear Regression
University of California, Berkeley
The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression
Jul 27, 2026Kevin Han Huang, Haoyu Ye, Somak Laha +1Double Descent Linear Regression
University of Warwick · Harvard University
De-floored Principal Component Regression: When Rank Selection Alone Is Insufficient for Prediction
Jul 18, 2026Peng ZhaoPrincipal Component Analysis High-Dimensional Linear Regression
Department of Applied Economics and Statistics, University of Delaware
What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regression
May 31, 2026Wendao Wu, Fangqing Zhang, Haihan Zhang +1High-Dimensional Linear Regression Weak-to-Strong Generalization
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University, China · School of Mathematical Sciences, Peking University, China
High dimensional theory of two-phase optimizers
Mar 27, 2026Atish AgarwalaMomentum Methods Distributed Optimization
Google DeepMind
How Does the ReLU Activation Affect the Implicit Bias of Gradient Descent on High-dimensional Neural Network Regression?
Mar 5, 2026Kuo-Wei Lai, Guanghui Wang, Molei Tao +1High-Dimensional Linear Regression Random Feature Methods
School of Electrical and Computer Engineering, Georgia Institute of Technology · College of Computing, Georgia Institute of Technology · School of Mathematics, Georgia Institute of Technology +1
Scaling Laws for Precision in High-Dimensional Linear Regression
Feb 22, 2026Dechen Zhang, Xuan Tang, Yingyu Liang +1High-Dimensional Linear Regression Scaling Laws
Institute of Data Science, The University of Hong Kong · School of Computing & Data Science, The University of Hong Kong · Institute of Data Science and School of Computing & Data Science, The University of Hong Kong +1
Optimal L2 Regularization in High-dimensional Continual Linear Regression
Jan 20, 2026Gilad Karpel, Edward Moroshko, Ran Levinstein +3Regularization-Based Continual Learning High-Dimensional Linear Regression
Department of Data and Decisions Sciences, Technion · School of Engineering, University of Edinburgh · Department of Computer Science, Technion +2
Efficient Cross-Validation for Sparse Linear Regression
Jun 26, 2023Ryan Cory-Wright, Andrés GómezModel Selection Cross-Validation
Department of Analytics, Marketing and Operations, Imperial Business School, London, UK · Department of Industrial and Systems Engineering, Viterbi School of Engineering, University of Southern California, CA

Oct 23, 2026J/K move · Enter open · S save

On the Limits of Sparse Linear Model with Shuffled Differential Privacy

Yan Zhang, Jiong Tuo, Meng Ding, Yuheng Ma+1

Provable Responsible AI and Data Analytics Lab · Nankai University · King Abdullah University of Science and Technology (KAUST)+3

High-Dimensional Linear Regression Differential Privacy Linear Regression

Open paper AskPDF arXiv

Sparse linear regression is a fundamental problem in high-dimensional statistics, and its private counterparts leave a large gap between the central and the local model of differential privacy. Shuffled differential privacy (SDP) sits between the two, offering accuracy closer to the central model without the trust requirement of a central curator. This paper studies sparse linear regression under both non-interactive and interactive SDP protocols. In the non-interactive setting, any non-interactive

(ε,δ)

-SDP protocol is shown to incur an

\ell_2

-norm estimation error of

Ω( \frac{\sqrt{dk}}{nε} )

with dimension

d

, sparsity

k

, and

n

users, revealing an inherent dimensionality barrier. In addition, a non-interactive algorithm is proposed that achieves an

\ell_2

-norm upper bound of

\tilde{\mathcal{O}}( \sqrt{\frac{k}{n}} + \frac{d\sqrt{k}}{nε})

. To overcome this barrier, we turn to the interactive user-level setting, where each user holds

m

samples. A two-round interactive algorithm is developed that first recovers the support through a heavy hitter step and then estimates the parameter, achieving an error of

\tilde{\mathcal{O}}( \sqrt{\frac{k}{nm}} + \frac{k}{n\sqrt{m}ε} )

with no polynomial dependence on

d

. Furthermore, by extending the score attack framework to user-level data, a new lower bound

Ω( \sqrt{\frac{k\log(d/k)}{nm}} + \frac{k\log(d/k)}{n\sqrt{m}ε} )

is established for any user-level

(ε,δ)

-DP, showing that the proposed algorithm is near-optimal with only a gap of the factor

\log(d/k)