Exact Calibration and Sharp Risk Geometry for Volume-Sampled Ridge Regression

Kihun Rhee

math.STOct 6, 2026

Exact Calibration and Sharp Risk Geometry for Volume-Sampled Ridge Regression

Authors: Kihun Rhee

Abstract

We study ridge regression from exactly $s$ distinct rows of a fixed design. Responses are fixed, and only the subset is random. The determinant law and selected ridge fit share one positive definite penalty. Established mean identities and exponential-family duality give the unique penalty that matches a prescribed full-data ridge fit in expectation. It exists exactly when $s$ exceeds the target's effective dimension. Our main result concerns centered covariance risk normalized by full-data penalized loss. For balanced signed coordinate replicas, a strict sector inequality gives the sharp risk and all maximizing responses at every budget from the dimension to one below the row count. This holds for any nonzero positive semidefinite query. With the target and query fixed, the maximizing response space is unchanged across these budgets. For general designs, we characterize attainment of a leave-one-out envelope. For existing real equiangular tight frames, flat row query energy characterizes when every nonzero residual response maximizes at two deletions. At three deletions, we give the sharp risk and complete maximizing space for isotropic queries, using unequal triangle weights. The balanced geometry yields a same-sample unbiased ridge--Horvitz--Thompson mixture with lower sharp risk and an exact mean-share improvement boundary. Under full recalibration after feature changes, we prove quadratic regret from searching the complete old maximizing space and a query-uniform bound on the mixture's risk gain. The strongest sector inequalities have exact computer-assisted proofs.

Figures & tables

Design and budget	Target and query	Conclusion
General full rank; $s=m-1$	Any SPD target; any nonzero PSD query	Envelope and exact attainment test. The sharp value is smaller when the test fails.
Balanced replicas; $d\leq s<nd$	Positive scalar target; any nonzero PSD query	Sharp value and complete maximizing space at every stated budget.
Existing real ETF; $s=m-2$	Positive scalar target; any nonzero PSD query	Flat row energy iff every residual maximizes. Exact value and space under flatness.
Existing real ETF; $s=m-3$	Positive scalar target; isotropic query; $d,m-d\geq 3$	Sharp value and complete residual maximizing space under unequal triangle weights.

Table 1: Exact response geometry under the calibrated law. The target and physical query are fixed while the response varies.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

$(n,d,s)$	$r$	$\lambda_{0}$	$T$	$R$	$\Phi(R)$ upper bound
$(2,2,2)$	$2/3$	$5/3$	$1089/3890$	$1/1000$	$291/100000$
$(2,2,3)$	$5/4$	$55/31$	$28665/123952$	$1/1000$	$111/100000$
$(2,3,3)$	$3/2$	$363/101$	$98972519/449051012$	$1/1000$	$209/100000$

Appendix

Table 2: Exact sufficient neighborhoods. The penalty

rI

is the matched base penalty, and

\lambda_{0}I

is the fixed physical Full target. The entries in the last column are strict rational upper bounds for

\Phi(R)

, not rounded decimal estimates.

Weight	Median	$\geq 1\%$	$\geq 5\%$
Theorem $T$	1.24%	80	16
Common $T$	0.98%	70	12
Optimal mixture	4.50%	121	62

Appendix

Table 3: Sharp-risk reduction from mixing on 140 structured target-size points. The thresholds summarize this fixed grid.

Weight	Query	Same-law HT	Uniform HT	Quota
$T$	Identity	82/0/58	93/0/47	116/0/24
	Coordinate	82/0/58	93/0/47	30/0/110
	Flat	140/0/0	140/0/0	116/0/24
	Tilted	98/0/42	106/0/34	35/0/105
Optimal	Identity	85/0/55	94/0/46	130/0/10
	Coordinate	85/0/55	94/0/46	44/0/96

Appendix

Table 4: Mixture wins/ties/losses against strong controls. Each row has 140 paired target-size cases. Quota sampling uses the query and changes the law.

Explore similar work

Aug 27, 2026cs.LG

Contact Geometry and Covariance Deficits in Volume-Sampled Least Squares

We classify when ordinary fixed-size volume sampling followed by unweighted least squares attains its sharp coefficient-covariance ceiling on a fixed design. For a real whitened design without coloops and a fixed positive-loss residual, the contact space is unchanged at every strict-interior sample size. Its possible nonzero values form a finite orthogonal family: each maximal parallel class of normalized Naimark-complement rows determines a deletion nullspace of dimension one less than the class size. A single residual attains an entire query precisely when the query range lies in one class space. The proof starts from two-sided Loewner comparison of every normalized covariance deficit with an explicit leave-one-out operator, using supported omission moments and reverse deletion. Residual augmentation provides resolvent and second-moment upper bounds, while complement geometry yields query-specific margins, angular concentration, local alignment, and a multi-output energy obstruction. Exact families give closed-form margins and covariances, exhibit support-boundary jumps, and approach the ceiling despite a uniformly positive geometric margin. Finally, the same moment identities give upper and lower bounds on expected fixed-query squared-loss excess. The subset draw is the only randomness; all support and endpoint restrictions are explicit.

Kihun Rhee, Hanjoon Byun, Junpyo Seo

Covariance Readout Sharpness-Aware Minimization

Ask PDF

May 27, 2026cs.LG

Optimal ridge regularization revisited

We consider

L^2

-regularized linear (ridge) regression over a finite data sample

X

with bounded covariance and linear prediction targets

y

with additive isotropic noise of finite variance. We present an iterative procedure to compute the optimal regularization strength numerically from the generative parameters in the fixed-

X

setting and prove its convergence at limited noise levels. Our experimental evaluation over synthetic data shows that the proposed procedure combined with sample-based parameter estimates attains near-optimal random-

X

generalization across a wide range of sample sizes, aspect ratios, and noise levels, at an added computational cost equivalent to one preliminary ridge regression in the underparameterized regime and two in the overparameterized case.

Jack Timmermans, Sergio A. Alvarez

Department of Computer Science, Boston College, Chestnut Hill, MA 02467 USA

Kernel Ridge Regression Regularization Generalization Bounds Covariance Ridge

Ask PDF

Aug 3, 2026econ.EM

A Simple Approximation to the Distribution of the Ridge Regression Estimator

We present a simple Gaussian approximation to the finite-sample distribution of the classical ridge regression estimator. Our approximation captures the fact that, in finite samples, the ridge regression estimator trades off bias and variance to reduce estimation and prediction error. Our approximation is based on nonstandard asymptotics where

i)

we let the estimator's regularization parameter grow proportionally to the sample size; and

ii)

we treat the population regression coefficients as \emph{local} to the reference vector that defines the estimator's direction of shrinkage. In contrast to other asymptotic approximations in the literature, we allow for general forms of heteroskedasticity and autocorrelation in the data generating process (at the cost of considering a low-dimensional model where the number of covariates is not allowed to grow with the sample size). We use our simple Gaussian approximation to propose two new strategies to select the regularization parameter for the ridge regression estimator. The suggested strategies select the regularization parameter to minimize either average or worst-case excess prediction risk, where risk is computed using our suggested Gaussian approximation.

José Luis Montiel Olea, Ryan Strong, Amilcar Velez +2

Department of Economics at Cornell University

Kernel Ridge Regression Heteroskedasticity Finite-Sample Approximation Gaussian Primitives Distributions+1

Ask PDF