Statistical Hypothesis Testing

Ask about this topic

Overview

Statistical hypothesis testing asks whether observed data are sufficiently inconsistent with a null claim, while balancing false positives, missed effects, and sample cost. Methods range from likelihood and rank tests to permutation and bootstrap calibration, kernel and classifier-based tests for distributional or conditional differences, and sequential e-value or betting procedures for ongoing monitoring. Multiple-testing correction and selective inference address many comparisons and data-dependent analyses, while robust and semi-supervised methods handle contamination, covariates, and limited labels; applications include causal effects, model evaluation, and scientific discovery. A central challenge is preserving reliable error control in small samples, high dimensions, adaptive workflows, and complex data without sacrificing power or computational efficiency.

Latest papers

CardsList

101 papers14 papers in the last four weeks, up 56% on the four weeks before. 0.1% of all new papers.

  1. Detecting Temporal Concepts in Video Transformers via a No-False-Positives Test

    Oct 23, 2026Jianghuai Li, Andrew Ren, Greg ShakhnarovichSpatiotemporal TransformersStatistical Hypothesis Testing

    University of Chicago · Toyota Technological Institute at Chicago

  2. Asymmetric Likelihood-Free Hypothesis Testing

    Oct 21, 2026Gunjan Kumar, Yash Pote, Jonathan ScarlettStatistical Hypothesis TestingSample Complexity

    IIT Kanpur · National University of Singapore

  3. Equivalence testing and run-count design for stochastic machine-learning experiments

    Oct 16, 2026Dmytro Knopov, Nadiya ShvaiStatistical Hypothesis TestingStatistical Power Analysis

    Faculty of Computer Sciences, Department of Mathematics, National University of Kyiv-Mohyla Academy, Kyiv, Ukraine. · Cyclope.ai, Paris, France.

  4. Strategic Multi-Agent Resource Allocation via Statistical Contracts

    Oct 13, 2026Ben Chugg, Michael I. Jordan, Aaditya RamdasMechanism DesignStatistical Hypothesis Testing

    Carnegie Mellon University · UC Berkeley, ÉNS, & Inria · Stanford University

  5. Selective Inference for Clustering Under Unknown Covariance

    Oct 12, 2026Yiqun T. Chen, Lucy L. GaoSelective InferenceStatistical Hypothesis Testing

    Departments of Biostatistics and Computer Science, Johns Hopkins University · Department of Statistics and Actuarial Science, University of Waterloo

  6. Testing Central Structure for Complex Networks

    Oct 11, 2026Qilong Wu, Yuxiao Li, Simeng Zhang +3Statistical Hypothesis Testing

    School of Software, Dalian University of Technology, China · College of Electrical Engineering, Sichuan University, China · School of Software Engineering, Dalian University, China

  7. From Imbalance to Evidence: Deep Nonparametric Testing for Complex Data under Severe Sample Imbalance

    Oct 9, 2026Guanhao Zhou, Xiufan YuTwo-Sample TestingStatistical Hypothesis Testing

    Department of Applied and Computational Mathematics and Statistics University of Notre Dame

  8. Conditional Kernel Stein Discrepancy

    Oct 8, 2026Federico Matteucci, Florian KalinkeKernel MethodsStatistical Hypothesis Testing

    Karlsruhe Institute of Technology

  9. What Can a Gaussian Process Design Test

    Oct 7, 2026Ivan De Boi, Marnix Van SoomGaussian ProcessesStatistical Hypothesis Testing

    Department of Electromechanical Engineering University of Antwerp Campus Groenenborger – Z373 Groenenborgerlaan 171, 2020 Antwerp, Belgium · Artificial Intelligence Lab Vrije Universiteit Brussel Pleinlaan 2, 1050 Brussels, Belgium

  10. Two-Sample Testing via Generative Processes

    Oct 6, 2026Eshant English, Kenji Fukumizu, Taiji SuzukiTwo-Sample TestingStatistical Hypothesis Testing

    The University of Tokyo, Tokyo, Japan · Hasso Plattner Institute for Digital Engineering, Germany · Institute of Statistical Mathematics, Japan +1

  11. Finding Gaussian Structure in Bosonic States

    Oct 5, 2026Alvan Arulandu, Sitan Chen, Ziyun Chen +2Quantum State TomographyStatistical Hypothesis Testing

    Harvard University · University of Washington · Stanford University

  12. Two-Sample Testing via Path-based Inference

    Oct 5, 2026Eshant English, Wei-Cheng Lai, Yanfeng Yang +3Two-Sample TestingPermutation Testing

    The University of Tokyo, Tokyo, Japan · Hasso Plattner Institute for Digital Engineering, Germany · Institute of Statistical Mathematics, Japan +3

  13. Routing Probes Can Improve Without New Information: An Exact-Null Audit of Uncertainty Beyond Model Outputs

    Sep 30, 2026Wenhao Liang, Lin Yue, Wei Emma Zhang +3Vision TransformerStatistical Hypothesis Testing

  14. Cheap and Powerful Tests for Supervised Subspaces: Per-Component Inference for PLS

    Sep 28, 2026Paweł Lenartowicz, Hubert PlisieckiPermutation TestingStatistical Hypothesis Testing

    Society for Open Science (Stowarzyszenie na Rzecz Otwartej Nauki) Centre for Brain Research, Jagiellonian University · Society for Open Science (Stowarzyszenie na Rzecz Otwartej Nauki) IDEAS Research Institute

  15. When 10,000 Windows Are Not 10,000 Tests: Auditing Statistical Confidence in Sliding-Window Time-Series Classification

    Sep 25, 2026Xinze Shi, Litian Zhang, Binrui ShiTime Series ClassificationBenchmark Validity

    Beijing University of Posts and Telecommunications Beijing, China

  16. Robust Detection of LLM-Generated Text under Contamination

    Sep 24, 2026Jiaxun Li, Saptarshi Chakraborty, Ambuj TewariAI-Generated Text DetectionLanguage Model Robustness

    Department of Statistics, University of Michigan

  17. Human-AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation

    Sep 24, 2026Dae Woong Ham, Xuejun Zhao, Stefanus Jasin +1Cost-Aware InferenceHuman-in-the-Loop Evaluation

  18. Selective Inference for Deep Clustering in Latent Spaces

    Sep 23, 2026Eina Mizui, Tomohiro Shiraishi, Shunichi Nishino +1Selective InferenceClustering

    Nagoya University · RIKEN

  19. How to Run Statistics over LLM Judges and Trust the Results: Calibrated Inference for Small-Sample AI Evaluation with evalstats

    Sep 20, 2026Ian ArawjoStatistical Hypothesis TestingLLM Evaluation

    Department of Computer Science and Operations Research (DIRO) Université de Montréal Montréal, Quebec, Canada

  20. Certifying Model Upgrades with Slice-Wise Non-Regression and Incumbent Fallback

    Sep 12, 2026Shengwei Zhang, Tao Wu, Fei QianModel SelectionAutomated Evaluation

    University of Pennsylvania · Alibaba International Digital Commerce

  21. When Validation Stops Learning: Auditing Update Admission for Continual Embodied Agents

    Sep 9, 2026Qinzhen Ma, Ruihai WuContinual Robot LearningSafe RL

    Rice University · University of California, Berkeley

  22. Likelihood-free inference with nuisance parameters through normalizing flows

    Sep 9, 2026Phil AsshetonSimulation-Based InferenceNormalizing Flows

    statsadvice.com, Berlin, Germany.

  23. Embedded Conditional Independence Tests for Large Language Model Generated Text with an Application to German Parliament Speeches

    Sep 1, 2026Marco Simnacher, Georg Keilbar, Benjamin König +2LLM EvaluationPrivacy Leakage in Language Models

    Chair of Statistics Humboldt-Universität zu Berlin Spandauer Str. 1, 10178 Berlin, Germany · Chair Digital Health & Machine Learning Hasso-Plattner-Institut for Digital Engineering Prof.-Dr.-Helmert-Straße 2-3, 14482 Potsdam, Germany

  24. SteerCheck: Attribution Specificity and Alignment Leakage in Activation-Steering Audits

    Aug 25, 2026Daming Luo, Christy Liang, Junyu XuanActivation SteeringLLM Auditing

    University of Technology Sydney

  25. Generative Neural Networks for Sinkhorn Distributionally Robust Hypothesis Testing

    Aug 24, 2026Fenglin Zhang, Teyan Liu, Jie WangStatistical Hypothesis TestingDistributionally Robust Optimization

    School of Artificial Intelligence and School of Data Science, The Chinese University of Hong Kong, Shenzhen · Department of Decision Analytics and Operations, College of Business, City University of Hong Kong

  26. The Type-II Error of Test Supermartingales: e-Power versus the Chernoff-Stein Exponent

    Aug 19, 2026Patrick ForréStatistical Hypothesis TestingSequential Hypothesis Testing

    AI4Science Lab, Korteweg-de Vries Institute for Mathematics, University of Amsterdam

  27. Certifying Compressed Language Models: An Audit and a Statistical Toolkit

    Aug 15, 2026Amogh SinghLLM EvaluationLLM Auditing

    Georgia Institute of Technology

  28. Certifying What Helps Customer-Return Timing: A Screen-and-Confirm Test for Conditioning Signals, and Why Decay Is Nearly Enough

    Aug 12, 2026Sang Su Lee, Vineeth Loganathan, Shishir Dash +1Event Time PredictionTemporal Point Processes

    Thumbtack, Inc. San Francisco, CA, USA

  29. Conditional Independence Tests for Constraint-Based Causal Discovery: A Survey

    Aug 11, 2026Pavel Averin, Theodoros Moysiadis, Ioannis KatakisCausal DiscoveryConstraint-Based Causal Discovery

    Department of Computer Science School of Sciences and Engineering University of Nicosia 2417, Nicosia, Cyprus

  30. RCShift: Certifying When Partial Linkage Suffices for Finite-Sample Decisions

    Aug 11, 2026Shuheng Cao, Ruiqi Chen, Zhenhao Zhang +5Entity ResolutionStatistical Hypothesis Testing

    University of California, San Diego · University of Michigan, Ann Arbor · ShanghaiTech University +4