Ridge Regression

Follow this topic Ask about this topic

Ridge regression studies how shrinking least-squares coefficients with an L2 penalty can reduce prediction error and improve stability when features are correlated or the number of features is large. Work examines how to choose the penalty using cross-validation, SURE, and high-dimensional risk analysis, and how early stopping in gradient descent can act like ridge or other spectral regularization. Kernel ridge regression extends the approach to nonlinear prediction, with Nyström approximations and sketching used to reduce computational and memory costs. A central challenge is balancing variance reduction against bias: the best regularization depends on data geometry, signal and noise, while scalable methods must preserve predictive accuracy.

Latest papers

31 papers6 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Local and Global Recursive Ridge Regression for Online Residual Correction of Frozen Multivariate Forecasters under Label Delay
Oct 21, 2026Takumi Fujimoto, Hiroaki NishiMultivariate Time Series Forecasting Online Time Series Forecasting
School of Science for Open and Environmental Systems, Graduate School of Science and Technology, Keio University, Japan · Department of System Design, Faculty of Science and Technology, Keio University, Japan
Minimax-optimal Scaling Laws in Ridge Regression via Transfer Learning
Oct 16, 2026Yalda Shabanzadeh, Diyuan Wu, Marco MondelliTransfer Learning Ridge Regression
Institute of Science and Technology Austria (ISTA).
Optimal Explicit and Implicit Regularization Strengths in High-Dimensional Continual Linear Regression
Oct 14, 2026Gilad Karpel, Daniel SoudryHigh-Dimensional Linear Regression Regularization-Based Continual Learning
Department of Data and Decisions Sciences, Technion · Department of Electrical and Computer Engineering, Technion
Gauss-Newton Accuracy and Indefinite Hessians: Uniform Coexistence in Low-Cost Sets
Oct 7, 2026Kihun Rhee, Hanjoon ByunGauss-Newton Optimization Second-Order Optimization
Exact Calibration and Sharp Risk Geometry for Volume-Sampled Ridge Regression
Oct 6, 2026Kihun RheeRidge Regression
The Advantages of Fresh Sketching for Ridge Regression
Sep 29, 2026Linkai Ma, Qilin Li, Petros DrineasImportance Sampling Randomized Sketching
Purdue University · University of Wisconsin-Madison
Stochastic Reconfiguration as Statistical Filtering for Overparameterized Neural Quantum States
Sep 20, 2026Tak HurOverparameterized Neural Network Neural Quantum States
Department of Applied Mathematics, University of Waterloo 200 University Avenue West, Waterloo, Ontario N2L 3G1, Canada
Structured Features Overfit Where Random Features Grok
Sep 14, 2026Chon-Fai Kam, Miloud Bessafi, Frederic CadetNeural Network Generalization Ridge Regression
University Paris City & University of Reunion, Paris, France; Dipartimento di Fisica e Chimica · Emilio Segr`e, Universit`a degli Studi di Palermo, Italy · EnergyLab, University of Reunion, Saint-Denis, France +2
(Mis)Understanding Benign Overfitting in Equity Return Prediction
Aug 24, 2026Hui Guo, Jiawei Huang, Runze Li +1Benign Overfitting Quantitative Finance
Carl H. Lindner College of Business, University of Cincinnati · Católica Lisbon School of Business and Economics, Universidade Católica Portuguesa · Department of Statistics, Pennsylvania State University
Regularised Iterative Generalised Least Squares with Optimal Selection of the Hyper-Parameter for Identifying Nonlinear Phenomenological Models
Aug 19, 2026Mark Cary, Charles BokorNonlinear System Identification Ridge Regression
Department of Aeronautical and Automotive Engineering, Loughborough University, Loughborough, LE11 3TU, UK · School of Engineering, Computing and Mathematics, Oxford Brookes University, Oxford OX33 1HX, UK
Causal Discovery in Equal Variance Linear Gaussian DAGs via SURE-Tuned Ridge Regression
Aug 17, 2026Sambit Mishra, Urbashi MitraCausal Discovery Causal Structure Learning
Department of Electrical and Computer Engineering University of Southern California
A Simple Approximation to the Distribution of the Ridge Regression Estimator
Aug 3, 2026José Luis Montiel Olea, Ryan Strong, Amilcar Velez +2Ridge Regression Hyperparameter Optimization
Department of Economics at Cornell University
Beyond Negative-Ridge Endpoints: Mixed-Sign Spectral Regularization via Negative-Shifted Gradient Descent
Jul 24, 2026Peng ZhaoSpectral Regularization Early Stopping
Department of Applied Economics and Statistics · University of Delaware
Regularized Machine Learning for System Identification of Ship Free-Running Manoeuvres from CFD-Based Synthetic Data: A Comparative Study
Jun 15, 2026R. F. Suárez, J. C. Berndt, M. Abdel-MaksoudNonlinear System Identification System Identification
Hamburg University of Technology (TUHH), Institute for Fluid Dynamics and Ship Theory (M8), Am Schwarzenberg-Campus 4 (C), 21073, Hamburg, Germany
IR3DE: A Linear Router for Large Language Models
Jun 4, 2026Eros Fanì, Oğuzhan ErsoyLLM Inference LLM Routing
Gensyn
When New Generators Arrive: Lifelong Machine-Generated Text Attribution via Ridge Feature Transfer
Jun 4, 2026Zhen Sun, Yifan Liao, Zhicong Huang +4Class-Incremental Learning AI-Generated Text Detection
Wuhan University · Ant Group · The Hong Kong University of Science and Technology (Guangzhou) +1
GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs
May 29, 2026Junjie Peng, You Wu, Haoyi Wu +4KV Caching Long-Context Language Modeling
School of Computer Science and Engineering, Sun Yat-sen University · School of Information Science and Technology, ShanghaiTech University · Shanghai Engineering Research Center of Intelligent Vision and Imaging +2
Ridge Regression from Poisson Resetting: A Renewal Perspective on Spectral Regularization
May 28, 2026Petar JolakoskiSpectral Regularization Linear Regression
Research Center for Computer Science and Information Technologies, Macedonian Academy of Sciences and Arts, Bul. Krste Misirkov 2, 1000 Skopje, Macedonia
Optimal ridge regularization revisited
May 27, 2026Jack Timmermans, Sergio A. AlvarezLinear Regression Statistical Learning Theory
Department of Computer Science, Boston College, Chestnut Hill, MA 02467 USA
Conditional KRR: Injecting Unpenalized Features into Kernel Methods with Applications to Kernel Thresholding
May 25, 2026Rustem Takhanov, Zhenisbek AssylbekovKernel Ridge Regression Kernel Regression
Department of Mathematics, Nazarbayev University, Astana, Kazakhstan · Nazarbayev University Research Administration, Astana, Kazakhstan · Purdue University Fort Wayne, Indiana, USA.
Canonical Regularisation of Wide Feature-Learning Neural Networks
May 18, 2026George Whittle, Pranav Vaidhyanathan, Juliusz Ziomek +2Kernel Ridge Regression Representation Learning
Department of Engineering Science University of Oxford
Large Dimensional Kernel Ridge Regression: Extending to Product Kernels
May 14, 2026Yang Zhou, Yicheng Li, Yuqian Cheng +1Generalization Bounds Kernel Ridge Regression
Department of Statistics and Data Science Tsinghua University Beijing, 100084, China · Department of Mathematical Science Tsinghua University Beijing, 100084, China
Reframing preprocessing selection as model-internal calibration in near-infrared spectroscopy: A large-scale benchmark of operator-adaptive PLS and Ridge models
May 13, 2026Gregory Beurier, Robin Reiter, Camille Noûs +2Linear Regression Spectral Methods
CIRAD, UMR AGAP Institut, Montpellier, France · UMR AGAP Institut, Univ Montpellier, CIRAD, INRAE, Institut Agro, Montpellier, France · CIRAD, UMR AGAP Institut, F-34398 Montpellier, France +1
Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regression
May 8, 2026Mingsong Yan, Dongyang Li, Charles Kulick +1Softmax Attention Transformer Interpretability
Department of Mathematics, University of California, Santa Barbara, CA
Differentiable Kernel Ridge Regression for Deep Learning Pipelines
May 4, 2026Jean-Marc Mercier, Gabriele SantinKernel Ridge Regression Kernel Regression
1MPG-Partners, Paris, France · Department of Environmental Sciences, Informatics and Statistics, Ca’ Foscari University of Venice, Italy
Pack only the essentials: Adaptive dictionary learning for kernel ridge regression
Apr 24, 2026Daniele Calandriello, Alessandro Lazaric, Michal ValkoKernel Ridge Regression Kernel Regression
SequeL team, INRIA Lille - Nord Europe, France
Analysis of Nystrom method with sequential ridge leverage scores
Apr 22, 2026Daniele Calandriello, Alessandro Lazaric, Michal ValkoKernel Ridge Regression Kernel Regression
SequeL team INRIA Lille - Nord Europe
Choosing the Right Regularizer for Applied ML: Simulation Benchmarks of Popular Scikit-learn Regularization Frameworks
Apr 4, 2026Benjamin S. Knight, Ahsaas BajajRidge Regression Model Selection
Optimal Unconstrained Self-Distillation in Ridge Regression: Strict Improvements, Precise Asymptotics, and One-Shot Tuning
Feb 19, 2026Hien Dang, Pratik Patil, Alessandro RinaldoSelf-Distillation Statistical Learning Theory
Department of Statistics and Data Sciences, University of Texas, Austin, TX 78712, USA.
To Grok Grokking: Provable Grokking in Ridge Regression
Jan 27, 2026Mingyue Xu, Gal Vardi, Itay SafranNeural Network Generalization Ridge Regression
Department of Computer Science, Purdue University, West Lafayette, IN, USA · Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Israel · Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Israel

Oct 21, 2026J/K move · Enter open · S save

Local and Global Recursive Ridge Regression for Online Residual Correction of Frozen Multivariate Forecasters under Label Delay

Takumi Fujimoto, Hiroaki Nishi

School of Science for Open and Environmental Systems, Graduate School of Science and Technology, Keio University, Japan · Department of System Design, Faculty of Science and Technology, Keio University, Japan

Multivariate Time Series Forecasting Online Time Series Forecasting Residual Learning Ridge Regression

Open paper AskPDF arXiv

A frozen multi-horizon forecaster can be corrected online from its own residuals. However, under label delay the target of a block arrives only after the whole horizon has passed, and the corrector's retained state grows with the number of channels. Aiming to combine a small retained state with a large error reduction, we propose DRiC (DCT Ridge Corrector), which fits a ridge regression of the leading discrete cosine transform (DCT) coefficients of the next residual block on those of the frozen forecast and of the latest input block, with sufficient statistics updated recursively. Its local variant, DRiC-L, keeps one such regression per channel; its global variant, DRiC-G, pools all channels into one regression, whose statistics do not depend on the channel count. A guard tracks a running scale of the frozen forecast's residual for each channel; it skips updates on blocks far outside that scale and clips the issued correction. We evaluate on seven datasets (four ETT series, Weather with its missing-value codes filled, Electricity, and Traffic) with DLinear, PatchTST, and iTransformer backbones, against 10 naive, gradient-trained, and closed-form methods. DRiC-L with eight DCT coefficients (DRiC-L8) reduces the mean squared error (MSE) by 14.55% on average over 126 conditions, against 12.98% for ELF, the most accurate comparator, while retaining 15-17 times less state, although ELF is more accurate on Traffic. DRiC-G retains 14.6 kB on the 321-channel Electricity data and 18.9 kB on the 862-channel Traffic data and is the most accurate method at or below its state on both. However, 5 to 7 of the 10 comparators, all with at least 10 times its state, are more accurate there. On the unfilled Weather file, DRiC-L8 without the guard has on average 8.0 times the frozen forecast's MSE, whereas with the guard it reduces the MSE by 7.94%. Code: https://github.com/keiotakmin/dric.