Generalization Bounds
Overview
Generalization bounds estimate how well a model trained on finite data will perform on new examples, helping explain when learning is reliable. Common tools include VC and Rademacher complexity, covering numbers, PAC-Bayes and information-theoretic bounds, and algorithmic stability. For deep networks and modern settings, analyses increasingly use learned feature geometry, optimization dynamics, and structure in data, while extending guarantees to distribution shift, generative models, and dependent or distributed samples. A central tension is that broad worst-case guarantees can be too loose to be useful, whereas tighter, non-vacuous bounds often rely on assumptions about the algorithm, data geometry, or shift between training and test distributions.
Latest papers
172 papers12 papers in the last four weeks, down 50% on the four weeks before. 0.1% of all new papers.
Large Dimensional Kernel Ridge Regression: Extending to Product Kernels
On Kernel Eigen-alignments of KRR: Reconstruction and Generalization
Scale-Sensitive Shattering: Learnability and Evaluability at Optimal Scale
Rethinking Generalization in Graph Neural Networks: A Structural Complexity Perspective
Unified generalization analysis for physics informed neural networks
A Survey on Data-Dependent Worst-Case Generalization Bounds
On the Generalization of Knowledge Distillation: An Information-Theoretic View
Information-Theoretic Generalization Bounds for Sequential Decision Making
Adaptive Context Matters: Towards Provable Multi-Modality Guidance for Super-Resolution
Generalization Error Bounds for Picard-Type Operator Learning in Nonlinear Parabolic PDEs
Unveiling High-Probability Generalization in Decentralized SGD
When and How to Canonize: A Generalization Perspective
Tight Generalization Bounds for Noiseless Inverse Optimization
Bounded Fitting for Expressive Description Logics
Spectrum-Adaptive Generalization Bounds for Trained Deep Transformers
A Fine-Grained Understanding of Uniform Convergence for Halfspaces
Optimal Confidence Band for Kernel Gradient Flow Estimator
Information-theoretic Limits of Learning and Estimation
Expert Routing for Communication-Efficient MoE via Finite Expert Banks
A Hierarchical Sampling Framework for bounding the Generalization Error of Federated Learning
Stability and Generalization for Decentralized Markov SGD
Quantifying Multimodal Capabilities: Formal Generalization Guarantees in Pairwise Metric Learning
Information-Theoretic Generalization Bounds for Stochastic Gradient Descent with Predictable Virtual Noise
Generalization Bounds of Spiking Neural Networks via Rademacher Complexity
On the Stability and Generalization of First-order Bilevel Minimax Optimization
Generalization at the Edge of Stability
Separating Geometry from Probability in the Analysis of Generalization
in-sample'' performance) or some entirely new $S'$ (in which case we speak of out-of-sample'' performance). Traditional analysis of generalization assumes that both in- and out-of-sample data are i.i.d.\ draws from an infinite population. However, these probabilistic assumptions cannot be verified even in principle. This paper presents an alternative view of generalization through the lens of sensitivity analysis of solutions of optimization problems to perturbations in the problem data. Under this framework, generalization bounds are obtained by purely deterministic means and take the form of variational principles that relate in-sample and out-of-sample evaluations through an error term that quantifies how close out-of-sample data are to in-sample data. Statistical assumptions can then be used ex post to characterize the situations when this error term is small (either on average or with high probability).