Estimating Population-Risk Curves Along Nonconvex Gradient Flows from the Training Sample

Mingzhi Song

stat.MLAug 31, 2026

Estimating Population-Risk Curves Along Nonconvex Gradient Flows from the Training Sample

Authors: Mingzhi Song

Organizations: Department of Mathematics, The University of Hong Kong

Abstract

We estimate the conditional population-risk curve of a realized smooth nonconvex gradient flow from the training sample. Flow approximate leave-one-out (Flow-ALO) propagates a deletion response and evaluates omitted observations at approximate deleted paths. The risk-curve error decomposes into response approximation, exact-LOO fluctuation, and deletion-to-full risk transfer. On each fixed finite horizon, bounded centered training-loss gradients, a one-sided Hessian lower bound, locally Lipschitz Hessians, and a strict tube-closure condition yield an explicit $(n-1)^{-2}$ bound for the deletion-response error. Bounded evaluation-loss gradients transfer the deletion-response bound to the score without requiring the Hessian to be invertible. Direct first-order jackknife cancellation and exact-LOO concentration control deletion-to-full risk transfer and fluctuation, respectively, completing recovery of the conditional population-risk curve. For bounded smooth two-layer mean-field networks training both layers, the score-error bound is uniform in width.

Explore similar work

Jun 17, 2026stat.ML

On Local Population-Risk Certificates

We develop finite-sample certificates for local population-risk increments

Pδ_v=R(θ_0+v)-R(θ_0)

v\in\mathcal D

. The primitive object is an expected-valid upper endpoint

\widehat{\mathsf U}_{\mathcal D}

satisfying

\mathbb E\sup_{v\in\mathcal D} \{Pδ_v-\widehat{\mathsf U}_{\mathcal D}(v)\}\le0

. This uniform criterion certifies any measurable update selected from the same sample and allows penalties to depend on empirical geometry. The main construction is a cross-fitted ridge calibration for linear feature classes. A pilot fold learns the ridge metric, the complementary fold calibrates the squared mean error in that metric, and complete split averaging recovers the full empirical covariance in the directional quadratic form

\widehat q_{X,λ}

. The optimized diagnostic scale is

\{\widehat q_{X,λ}(h) \widehat r_{X,n_{\rm p},λ}^{\rm cf}/n\}^{1/2}

, and the calibrated trace factor

\widehat r_{X,n_{\rm p},λ}^{\rm cf}

is compared with the ordinary ridge effective dimension

\widehat r_{X,λ}

. For nonsmooth losses, an exact fixed-mask decomposition

δ_v=J_v^0+R_v^\circ+C_v

separates frozen Taylor fluctuations, good-path remainders, and interface crossings. Applying the linear and composite certificates componentwise yields endpoints for same-sample expected local search and concentrated release rules.

Mingzhi Song

Finite-Sample Certificates Finite-Sample Decision Thresholds Covariance Estimation

Ask PDF

May 7, 2026math.ST

Optimal Confidence Band for Kernel Gradient Flow Estimator

In this paper, we investigate the supremum-norm generalization error and the uniform inference for a specific class of kernel regression methods, namely the kernel gradient flows. Under the widely adopted capacity-source condition framework in the kernel regression literature, we first establish convergence rates for the supremum norm generalization error of both continuous and discrete kernel gradient flows under the source condition

s>α_0

, where

α_0\in(0,1)

denotes the embedding index of the kernel function. Moreover, we show that these rates match the minimax optimal rates. Building on this result, we then construct simultaneous confidence bands for both continuous and discrete kernel gradient flows. Notably, the widths of the proposed confidence bands are also optimal, in the sense that their shrinkage rates are greater than, while can be arbitrarily close to, the minimax optimal rates.

Yuqian Cheng, Zhuo Chen, Qian Lin

Confidence Intervals Wasserstein Gradient Flows Spectral Norm Gradient Capacity

Ask PDF

Jul 30, 2026stat.ML

Error Analysis of Neural-Network-Based Engression

Engression (Shen and Meinshausen, 2024) learns a conditional distribution by fitting a generative model

Y = f(X,\varepsilon)

under the energy score, a strictly proper scoring rule. We provide a theoretical error analysis of engression implemented with deep neural networks. We decompose the excess risk into three components: the approximation error, the stochastic error, and the Monte Carlo error. Based on this decomposition, we establish convergence rates under the assumption that the target conditional generator admits a compositional smoothness structure.

Juntong Chen, Zijian Guo, Xinwei Shen

Neural-Network Approximation Equal Error Rate Conditional Distribution Optimization Convergence Analysis Errors Stochastic+1

Ask PDF