Statistical Hypothesis Testing

Follow this topic Ask about this topic

Statistical hypothesis testing asks whether observed data are sufficiently inconsistent with a null claim, while balancing false positives, missed effects, and sample cost. Methods range from likelihood and rank tests to permutation and bootstrap calibration, kernel and classifier-based tests for distributional or conditional differences, and sequential e-value or betting procedures for ongoing monitoring. Multiple-testing correction and selective inference address many comparisons and data-dependent analyses, while robust and semi-supervised methods handle contamination, covariates, and limited labels; applications include causal effects, model evaluation, and scientific discovery. A central challenge is preserving reliable error control in small samples, high dimensions, adaptive workflows, and complex data without sacrificing power or computational efficiency.

Latest papers

101 papers14 papers in the last four weeks, up 56% on the four weeks before. 0.1% of all new papers.

Valid Feature-Level Inference for Tabular Foundation Models via the Conditional Randomization Test
Feb 19, 2026Mohamed SalemStatistical Hypothesis Testing Feature Selection
Falsifying Discriminant Validity of Predictive Algorithms
Jan 23, 2026Amanda CostonAlgorithmic Auditing Psychometric Validation
Statistics Department University of California, Berkeley Berkeley, CA, USA
Computationally efficient goodness-of-fit tests through kernelized Stein discrepancy
Dec 23, 2025Zhihan Huang, Ziang NiuKernel Methods Statistical Hypothesis Testing
Department of Statistics and Data Science University of Pennsylvania
Maximum Mean Discrepancy with Unequal Sample Sizes via Generalized U-Statistics
Dec 16, 2025Aaron Wei, Milad Jalali, Danica J. SutherlandTwo-Sample Testing Maximum Mean Discrepancy
University of British Columbia · Independent researcher, Vancouver, Canada · Alberta Machine Intelligence Institute
Reliable Selection of Heterogeneous Treatment Effect Estimators
Nov 23, 2025Jiayi Guo, Zijun GaoCausal Effect Estimation Conditional Average Treatment Effect Estimation
Peking University · Marshall School of Business, University of Southern California
Structural Enforcement of Statistical Rigor in AI-Driven Discovery: A Functional Architecture
Nov 10, 2025Karen SargsyanFDR Control Formal Verification
aInstitute of Chemistry, Academia Sinica, Taipei, Taiwan
Making Interpretable Discoveries from Unstructured Data: A High-Dimensional Multiple Hypothesis Testing Approach
Nov 3, 2025Jacob CarlsonSelective Inference Interpretable ML
Harvard University∗
Forecasting Generative Amplification
Sep 9, 2025Henning Bahl, Sascha Diefenbacher, Nina Elmer +2Generative Modeling High-Energy Physics
Institut für Theoretische Physik, Universität Heidelberg, Germany · Physics Division, Lawrence Berkeley National Laboratory, Berkeley, USA · Interdisciplinary Center for Scientific Computing (IWR), Universität Heidelberg, Germany
Diffusion-Based Hypothesis Testing and Change-Point Detection
Jun 19, 2025Sean Moushegian, Taposh Banerjee, Vahid TarokhChange-Point Detection Statistical Hypothesis Testing
Department of Electrical and Computer Engineering, Duke University, Durham, North Carolina 27708. · Department of Industrial Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15213.
Second Maximum of a Gaussian Random Field and Exact (t-)Spacing test
Jun 26, 2024Jean-Marc Azaïs, Federico Dalmao, Yohann De CastroStatistical Hypothesis Testing Sparse Recovery
Institut de Mathématiques de Toulouse Université Paul Sabatier, France · DMEL, CENUR Litoral Norte Universidad de la República, Salto, Uruguay · Institut Camille Jordan, CNRS UMR 5208 École Centrale Lyon, France +1
Combinatorial Inference on the Optimal Assortment in Multinomial Logit Models
Jan 28, 2023Shuting Shen, Xi Chen, Ethan X. Fang +1Statistical Hypothesis Testing Assortment Optimization
Department of Biostatistics, Harvard T.H. Chan School of Public Health · Leonard N. Stern School of Business, New York University · Department of Biostatistics & Bioinformatics, Duke University

Feb 19, 2026J/K move · Enter open · S save

Valid Feature-Level Inference for Tabular Foundation Models via the Conditional Randomization Test

Mohamed Salem

Statistical Hypothesis Testing Feature Selection Conditional Independence Testing

Open paper AskPDF arXiv

Modern machine learning models are highly expressive but notoriously difficult to analyze statistically. In particular, while black-box predictors can achieve strong empirical performance, they rarely provide valid hypothesis tests or p-values for assessing whether individual features contain information about a target variable. This article presents a practical approach to feature-level hypothesis testing that combines the Conditional Randomization Test (CRT) with TabPFN, a probabilistic foundation model for tabular data. The resulting procedure yields finite-sample valid p-values for conditional feature relevance, even in nonlinear and correlated settings, without requiring model retraining or parametric assumptions.