Cross-Validation

Follow this topic Ask about this topic

Latest papers

16 papers4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Sharp Bounds on the Winner's Bias under Independent and Nonnegatively Correlated Noise
Oct 22, 2026Ivan Viakhirev, Kirill Borodin, Grach MkrtchianCross-Validation Selection Bias
lab260 (lab260.ru) · BitmanagerAI (bitmanager.ai)
EasyClassifier: Honest, Reproducible Machine-Learning Classification for Researchers Who Do Not Program
Oct 3, 2026Ahmad B. A. Hassanat, Ghada A. AltarawnehML Reproducibility Classification
Faculty of Information Technology, Mutah University, Karak, Jordan · Department of Accounting, Mutah University, Karak, Jordan
When Is Accuracy Evidence? A Unified Theory of Generalisation, Validation, and Information Fusion
Oct 2, 2026JM GorrizCross-Validation Generalization Bounds
Data Science and Computational Intelligence Institute University of Granada Granada, Spain · Department of Psychiatry University of Cambridge Cambridge, UK
Model validation in machine learning: A scenario-based guide from hold-out splits to nested group cross-validation in biomedical and applied research
Oct 1, 2026Mehmet Baygin, Sengul Dogan, Turker TuncerData Leakage Cross-Validation
Department of Computer Engineering, Faculty of Engineering and Architecture, Erzurum Technical University, Erzurum, Türkiye · Department of Digital Forensics Engineering, Technology Faculty, Firat University, Elazig, Türkiye
The Impossible Trinity of Time-Series Validation: A Conservation Law among Training Sufficiency, Test Coverage, and Temporal Causality
Sep 2, 2026Jiayu LiCross-Validation Temporal Data Leakage
On Cross-Validation for Hyperparameter Optimization of Deep Learning Image Classifiers
Aug 10, 2026Ljubomir ButurovicHyperparameter Optimization Cross-Validation
East Palo Alto, United States · East Palo Alto, California
Crossing the Validation Crisis: Cross-Validation Reduces Benchmarking Variance Surprisingly Well
Jun 10, 2026Célestin Eve, Gaël Varoquaux, Thomas MoreauCross-Validation Benchmark Validity
MIND Team, Université Paris-Saclay, Inria, CEA, Palaiseau, France · SODA Team, Inria, Palaiseau, France · Probabl
RobustModelMaker: Coupling Bootstrap Stability Selection with Leakage-Safe Nested Cross-Validation for Scientific Machine Learning
Jun 1, 2026Amanda S BarnardData Leakage Scientific ML
School of Computing Australian National University Acton, ACT 2601
Minimax Limits of k-Fold Cross-Validation via Majority
May 25, 2026Ido Nachum, Rüdiger Urbanke, Thomas WeinbergerMinimax Estimation Statistical Learning Theory
University of Haifa, Haifa, Israel. · EPFL, Lausanne, Switzerland.
Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation
May 18, 2026Tristan Kirscher, Markus Bujotzek, Yannick Kirchhoff +5Deep Ensembles Medical Image Segmentation
ICube, Institut Strauss, DKFZ · ICube Laboratory, CNRS UMR-7357, University of Strasbourg, Strasbourg, France · CLCC Institut-Strauss, Strasbourg, France +6
fastml: Guarded Resampling Workflows for Safer Automated Machine Learning in R
Apr 6, 2026Selcuk Korkmaz, Dincer Goksuluk, Eda KaraismailogluCross-Validation Automated Machine Learning
Faculty of Medicine, Department of Biostatistics, Trakya University, Edirne 22030, Türkiye · Faculty of Medicine, Department of Biostatistics, Sakarya University, Sakarya 54290, Türkiye · Faculty of Medicine, Department of Biostatistics, University of Health Sciences, Ankara 06010, Türkiye
Beyond the Fold: Quantifying Split-Level Noise and the Case for Leave-One-Dataset-Out AU Evaluation
Apr 2, 2026Saurabh Hinduja, Gurmeet Kaur, Maneesh Bilalpur +2Cross-Validation Facial Action Unit Detection
CGI Technologies and Solutions Inc · University of Pittsburgh · University of South Florida
From Cross-Validation to SURE: Asymptotic Risk of Tuned Regularized Estimators
Mar 20, 2026Karun Adusumilli, Maximilian Kasy, Ashia WilsonCross-Validation Shrinkage Estimation
Department of Economics, University of Pennsylvania. · Department of Economics, University of Oxford. · Department of Electrical Engineering and Computer Science, MIT.
Efficient Cross-Validation for Sparse Linear Regression
Jun 26, 2023Ryan Cory-Wright, Andrés GómezModel Selection Cross-Validation
Department of Analytics, Marketing and Operations, Imperial Business School, London, UK · Department of Industrial and Systems Engineering, Viterbi School of Engineering, University of Southern California, CA
Distribution-free Deviation Bounds and The Role of Domain Knowledge in Learning via Model Selection with Cross-validation Risk Estimation
Mar 15, 2023Diego Marcondes, Cláudia PeixotoModel Selection Cross-Validation
Concentration inequalities for leave-one-out cross validation
Nov 4, 2022Benny Avelin, Lauri ViitasaariCross-Validation

Oct 22, 2026J/K move · Enter open · S save

Sharp Bounds on the Winner's Bias under Independent and Nonnegatively Correlated Noise

Ivan Viakhirev, Kirill Borodin, Grach Mkrtchian

lab260 (lab260.ru) · BitmanagerAI (bitmanager.ai)

Cross-Validation Selection Bias Statistical Learning Theory

Open paper AskPDF arXiv

Reporting the best of

M

noisy, unbiased scores overstates the winner's value by up to

c\sqrt{V}

, where

V

is the total noise variance and the sharp constant

c

depends on how the noise is dependent. It was known only at the two extremes, arbitrary dependence (

\sqrt{(M-1)/M}

) and i.i.d. noise. We determine it for two classes in between: for independent, not identically distributed noise it is

\sqrt{(M-1)/(2M)}

, and for pairwise nonnegatively correlated noise it is

(M-1)/M

; for uncorrelated noise this answers the maximum version of an open question about the expected range. Explicit finite constructions attain both. As a corollary, the bias of the cross-validation estimator is bounded sharply within each class. All theorems are formally verified in Lean 4. Across 448 language-model and classifier selection pipelines, designs with per-candidate items realize a median 0.75 of the independent-class ceiling. At a fixed item budget, scoring all candidates on shared items is markedly less optimistic than giving each candidate its own items.