Ridge Regression

Ridge regression studies how shrinking least-squares coefficients with an L2 penalty can reduce prediction error and improve stability when features are correlated or the number of features is large. Work examines how to choose the penalty using cross-validation, SURE, and high-dimensional risk analysis, and how early stopping in gradient descent can act like ridge or other spectral regularization. Kernel ridge regression extends the approach to nonlinear prediction, with Nyström approximations and sketching used to reduce computational and memory costs. A central challenge is balancing variance reduction against bias: the best regularization depends on data geometry, signal and noise, while scalable methods must preserve predictive accuracy.