Anytime-Valid Inference

Follow this topic Ask about this topic

Anytime-valid inference studies how to make trustworthy statistical decisions when data are inspected repeatedly and the time to stop is chosen based on what has been observed. It replaces fixed-horizon tests with tools such as e-values and e-processes, betting martingales, and confidence sequences that control false alarms or maintain coverage at every time. These methods are used for sequential audits, model and leaderboard evaluation, selective prediction, streaming decisions, and performance certification, often aiming to save samples or computation through early stopping. A central challenge is balancing power and efficiency with guarantees whose assumptions—such as exchangeability, predictable sampling, or valid dependence models—may not hold in real adaptive systems.

Latest papers

35 papers4 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

Anytime Valid Confidence Bands for Distribution Functions
Oct 26, 2026Bror Hultberg, Dave Zachariah, Antônio H. RibeiroConfidence Estimation Confidence Region Estimation
Uppsala University, Sweden · Uppsala University & SciLifeLab, Sweden
Anytime-Valid Runtime Monitoring of Learned Barrier Certificates
Oct 20, 2026Zhipeng Cao, Taoran Wu, Weiru Cui +2Runtime Verification Control Barrier Functions
Institute of Software, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · University of Chinese Academy of Sciences +1
Certified Corruption Budgets: Anytime-Valid Leaderboard Claims under Adaptive Rigging
Oct 6, 2026Hamed Khosravi, Xiaoming HuoCorruption Robustness Anytime-Valid Inference
H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology, Atlanta, GA, USA
Evidence-Gated Research: Statistically Controlled Model Adoption in Adaptive Search
Oct 1, 2026Yifan GuoFDR Control Anytime-Valid Inference
Department of Applied Mathematics and Statistics Stony Brook University
Prequential E-Values for Selected-GP Near-Optimality Certificates
Sep 30, 2026Ami Tavory, Noa CohenBlack-Box Optimization Anytime-Valid Inference
Meta Platforms
The Price of Peeking: Anytime-Valid Leakage Detection on ML-KEM EM Traces
Sep 27, 2026Georgios Feretzakis, Alexandros PapaspyridisAnytime-Valid Inference Sequential Hypothesis Testing
Anytime-Valid LLM Leaderboards via Benchmark-weighted and Block-Factorized e-Processes
Sep 26, 2026Hongfu Gao, Songxin Zhang, Zejian Xie +3LLM Evaluation Anytime-Valid Inference
National University of Singapore · Southern University of Science and Technology · The Chinese University of Hong Kong, Shenzhen +1
Rank Confidence Sequences:Anytime-valid Leaderboards
Sep 26, 2026Hamed Khosravi, Xiaoming HuoAnytime-Valid Inference Benchmark Validity
H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology, Atlanta, GA 30332, USA
Propose, Don't Judge: An Anytime-Valid Referee for LLM Agents That Mine Investment Factors
Sep 22, 2026Bo Qu, Mingguang Chen, Licheng WangQuantitative Finance Finance
DeepGrounding · AlphaAvatar
Human-LLM Deliberation as Interactive Proof: Conditions for Verifiability Without Transparency
Sep 21, 2026Baotong Zhang, Dean Foster, João SedocLLM Answer Verification Anytime-Valid Inference
Stern School of Business, New York University, New York, NY, USA · Amazon, New York, NY, USA
When the Martingale Never Stops Firing: Anytime-Valid Gating on Real Forecast Streams
Aug 31, 2026Weijia Han, Lisha QuAnytime-Valid Inference Conformal Prediction
University of Washington · Seattle, USA
Betting for Sim-to-Real Performance Certificates
Aug 21, 2026Yujia Chen, Bowen WengRobotic Policy Evaluation Sim-to-Real Transfer
Department of Computer Science, Iowa State University, Ames, IA 50011, USA
Information on trajectories: martingales and random times
Aug 20, 2026Akshay BalsubramaniAnytime-Valid Inference
Evidence Before Expansion: Reuse, Spawn, or Defer in Lifelong Expert Pools
Aug 20, 2026Kentaro OdaOnline Continual Learning Sequential Hypothesis Testing
Center for Management of Information Technologies, Kagoshima University
Pricing the Risk of Runtime Compression: Anytime-Valid Admission and a Served-Output Law for Compressed Serving State
Aug 16, 2026Fanzhe Wei, Li LiuRuntime Assurance Anytime-Valid Inference
Metask Lab
Private Anytime Selective-Risk Certification for Federated Retrieval-Augmented Generation: Guarantees and Empirical Limits
Aug 8, 2026Sanjeda Akter, Ibne Farabi Shihab, Anuj SharmaAnytime-Valid Inference RAG Evaluation
Department of Computer Science, Iowa State University · Department of Civil, Construction & Environmental Engineering, Iowa State University
AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games
Aug 6, 2026Boning Li, Yu Chen, Longbo HuangEarly Stopping Imperfect-Information Games
IIIS, Tsinghua University
On Exploring Input Resolution Scaling For Anytime LiDAR Object Detection
Jul 9, 2026Ahmet Soyyigit, Shuochao Yao, Heechul YunAdaptive Inference Lidar-Based 3D Object Detection
The National Defense University, Istanbul, Turkiye · George Mason University, Fairfax, VA, USA · The University of Kansas, Lawrence, KS, USA
Halt Fast! Early Stopping for Certified Robustness
Jun 26, 2026Andrew C. Cullen, Paul Montague, Benjamin I. P. RubinsteinEfficient Neural Network Inference Early Stopping
University of Melbourne · DST Group, Adelaide
Sim-to-Real Betting on the E-Process: Bringing "simulators" to anytime-valid confidence sequences
Jun 23, 2026Yujia Chen, Bowen WengAnytime-Valid Inference
Department of Computer Science, Iowa State University
CELEUS: Certifiable and Efficient LLM Evaluation via E-Processes
Jun 18, 2026Zhijian Zhou, Zesheng Ye, Zhaorun Chen +2LLM Evaluation Anytime-Valid Inference
The University of Melbourne · Department of Computer Science, University of Chicago · Department of Computer Science, University of Illinois Urbana-Champaign
Who Drifted: the System or the Judge? Anytime-Valid Attribution in LLM Evaluation Pipelines
Jun 13, 2026Yitao LiLLM Evaluation LLM-as-a-Judge
Anytime-Valid Confirmation of Label-Shift Corrections
Jun 12, 2026Seungjin ChoiLabel Shift Anytime-Valid Inference
CROID Research and aSSIST University, Seoul, Korea.
Correcting Split Selection in Online Decision Trees via Anytime-Valid Inference
May 29, 2026Salim I. Amoukou, Saumitra Mishra, Manuela VelosoAnytime-Valid Inference Decision Tree Learning
J.P. Morgan AI Research.
Anytime-Valid Federated Conformal RAG for LLM Swarms
May 27, 2026Prasanjit Dubey, Xiaoming HuoRetrieval-Augmented Generation Federated Conformal Prediction
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332, U.S.A.
Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained LLMs
May 18, 2026Hamed Khosravi, Xiaoming HuoSelective Prediction Anytime-Valid Inference
Georgia Institute of Technology
When Should an AI Workflow Release? Always-Valid Inference for Black-Box Generate-Verify Systems
May 13, 2026Young Hyun Cho, Will Wei SunAnytime-Valid Inference Sequential Hypothesis Testing
Department of Statistics, Purdue University. · Mitch Daniels School of Business, Purdue University.
Asymptotically Log-Optimal Bayes-Assisted Confidence Sequences for Bounded Means
May 8, 2026Valentin Kilian, Stefano Cortinovis, François CaronAnytime-Valid Inference Bayesian Inference
Department of Statistics, University of Oxford
Adaptive auditing of AI systems with anytime-valid guarantees
May 7, 2026Siyu Zhou, Patrick Vossler, Venkatesh Sivaraman +2Algorithmic Auditing Anytime-Valid Inference
University of California, San Francisco · Stanford University
CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency
May 7, 2026Hirofumi Ota, Naoto Iwase, Yuki Ichihara +2LLM Evaluation Anytime-Valid Inference
arXiv:2605.05873v1

Oct 26, 2026J/K move · Enter open · S save

Anytime Valid Confidence Bands for Distribution Functions

Bror Hultberg, Dave Zachariah, Antônio H. Ribeiro

Uppsala University, Sweden · Uppsala University & SciLifeLab, Sweden

Confidence Estimation Confidence Region Estimation Anytime-Valid Inference Conditional Distribution Estimation

Open paper AskPDF arXiv

Nonparametric inference often requires characterizing an unknown function, such as a cumulative distribution function (CDF), across its entire domain rather than a specific point. We develop confidence bands for distribution functions of a data stream that are valid simultaneously over all times and all points, even when each observation may depend arbitrarily on the past. We begin by deriving such bands for the time-averaged conditional CDF and show how their widths are determined by the depth of a binary search tree. We then show that this result is an instance of a framework through which one can construct valid confidence bands uniformly over time and over an arbitrary index set for any indexed family of time-averaged distribution functions. We illustrate the framework on importance-weighted and right-censored CDFs, as well as multivariate CDFs.