Anytime-Valid Empirical Likelihood Inference from LLM Annotations with Adaptive Human Auditing
Authors: Hongfu Gao, Jiang Hu, Pengfei Liu, Wang Zhou
Organizations: Department of Statistics and Data Science, National University of Singapore · School of Mathematics and Statistics, Northeast Normal University · School of Mathematics and Statistics, Jiangsu Normal University
Large language models (LLMs) are increasingly used to annotate unlabeled data as a fast, low-cost alternative to human annotation. However, their outputs can be biased and miscalibrated relative to human labels, potentially leading to misleading analysis and inference. We propose AAELI (Adaptive Human Auditing and Anytime-Valid Empirical Likelihood Inference), an online framework for estimating the population mean of human labels under a limited auditing budget. AAELI uses LLM outputs to guide adaptive human auditing and combines them with audited labels through predictable empirical-likelihood e-processes to construct anytime-valid confidence sequences. Our main theoretical result shows that a predictably learned efficient projection preserves the first-order profile empirical-likelihood statistic under adaptive auditing, without requiring a prescribed convergence rate for the projection estimator. This yields an exactly anytime-valid betting construction whose continuous-mixture form incurs a scalar rather than vector-dimensional mixing penalty. Supporting results establish adaptive empirical-likelihood asymptotics and show that a learned audit rule attains the oracle variance within the stated class of augmented estimating-equation designs under the same audit-probability floor and long-run expected budget. Experiments on synthetic data and three real-world datasets demonstrate that AAELI maintains empirical anytime coverage while requiring fewer human audits than the strongest competing baseline to reach a target precision.
Figures & tables
Figure 1: Anytime-valid inference with n=10,000 items and budget ρ=0.2 , averaged over 500 replications. Left: Average confidence sequence bounds. The dotted line marks the true mean μ=0.5 . Middle: Percentage of replications whose confidence sequence covers the true mean at every item up to the current one. Right: Average interval width vs. the number of human labels.
ρ
Metric
PPI
ASI
RASI
FTRL
BAI
AAELI
0.05
Coverage
0.946 ±0.010
0.942 ±0.010
0.946 ±0.010
0.948 ±0.010
0.950 ±0.010
0.946 ±0.011
Width ↓
0.074 ±0.002
0.074 ±0.003
0.076 ±0.003
0.077 ±0.002
0.080 ±0.004
0.070 ±0.003
0.1
Coverage
0.952 ±0.010
0.954 ±0.009
0.942 ±0.010
0.940 ±0.011
0.940 ±0.011
0.956 ±0.009
Width ↓
0.052 ±0.001
0.051 ±0.001
0.051 ±0.001
0.053 ±0.001
0.056 ±0.002
0.049 ±0.002
0.15
Coverage
0.938 ±0.011
0.944 ±0.010
0.942 ±0.010
0.946 ±0.010
0.950 ±0.010
0.948 ±0.011
Width ↓
0.042 ±0.001
0.041 ±0.001
0.041 ±0.001
0.043 ±0.001
0.045 ±0.002
0.039 ±0.001
Table 1: Fixed-budget inference on synthetic data with n=10,000 items and human audit budgets ρ∈{0.05,0.10,0.15,0.20} , averaged over 500 replications. Each method uses its own audit rule and constructs a fixed-sample 95% confidence interval once, at n=10,000 . Coverage is the proportion of replications whose interval contains μ0=0.5 , and Width is the average interval width. Subscripts give Monte Carlo standard errors. The smallest width for each budget is shown in bold.
Figure 2: Ablation on the human audit policy. Uniform, Uncertainty, Disagreement, and Oracle use the same AAELI inference procedure as AAELI, but replace its learned audit rule with uniform auditing, auditing by the uncertainty of the LLM judges, auditing by the disagreement among the judges, and the oracle rule with known σ(Z) , respectively. ASI is the strongest baseline.
Figure 3: Ablation on the empirical likelihood inference procedure. PPI, ABPPI++, and AAELI use the same uniform auditing, but construct confidence sequences by betting on the PPI estimator ( Waudby-Smith et al., 2024a ; Zrnic and Candès, 2024 ) , by anytime-valid Bayes-assisted prediction-powered inference with the PPI++ estimator ( Kilian et al., 2026 ; Angelopoulos et al., 2026 ) , and by the AAELI inference procedure.
Figure 4: Ablation on the learned moment combination with unmodified random-forest predictions.
Figure 5: Ablation on the learned moment combination under affine miscalibration of random-forest predictions.
Figure 6: Robustness to biased LLM judges.
Figure 7: Robustness to highly correlated LLM judges.
Figure 8: Robustness to shared judge failure.
Figure 9: Anytime-valid inference on Chatbot Arena with human budget ρ=0.2 .
Figure 10: Anytime-valid inference on AllSides with human budget ρ=0.2 .
Figure 11: Anytime-valid inference on Misinfo with human budget ρ=0.2 .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Ablation on the human audit budget: ρ=0.05 , corresponding to 500 expected human labels.
Figure S2: Ablation on the human audit budget: ρ=0.1 , corresponding to 1,000 expected human labels.
Figure S3: Ablation on the human audit budget: ρ=0.15 , corresponding to 1,500 expected human labels.
Figure S4: Robustness to the nuisance learner: m and σ are fitted by a multilayer perceptron (MLP) with two hidden layers.
Figure S5: Robustness to the nuisance learner: m and σ are fitted by support vector regression (SVR) with an RBF kernel.
Figure S6: Anytime-valid inference on Chatbot Arena with a single LLM judge (GPT-6-Luna) and human budget ρ=0.2 .
Figure S7: Anytime-valid inference on AllSides with a single LLM judge (GPT-6-Luna) and human budget ρ=0.2 .
Figure S8: Anytime-valid inference on Misinfo with a single LLM judge (GPT-6-Luna) and human budget ρ=0.2 .
Anytime-valid inference provides statistical tests and confidence bounds that remain valid under repeated monitoring, adaptive sampling, and data-dependent stopping.
For large language models (LLMs) and multimodal large language models (MLLMs), model-reported self-confidence is often miscalibrated, yet it may still be useful for efficient evaluation. At the same time, LLM-as-a-judge offers a scalable alternative to direct human auditing, but its practical benefit depends on calibration quality and inference cost. This paper develops a unified statistical framework for both evaluation routes under alternative sampling strategies. For direct auditing, confidence-stratified sampling is used to recover population accuracy from partially verified outputs without requiring calibrated confidence. Finite-sample uncertainty is characterized, and the analysis shows how highly uneven confidence strata affect audit efficiency and label allocation. For judge-assisted evaluation, the judge is modeled through sensitivity and false-positive rate, yielding an error-corrected estimator with calibration uncertainty. The two routes are then compared through human-label and total-cost efficiency boundaries. Experiments on MMLU-Pro and MMMU-Pro show that miscalibrated self-confidence can still support statistically valid stratified auditing. Under highly imbalanced confidence distributions, proportional allocation substantially reduces estimation error relative to equal-per-stratum allocation. For judge-assisted evaluation, only a small subset of target--verifier pairs satisfies the idealized human-label efficiency condition, and automated inference costs can further limit potential savings. These findings clarify the statistical and economic conditions under which confidence-guided auditing and automated verification are useful.
Harine Choi, Chenchen Kuai, Zihao Li +5
Texas A&M Transportation Institute, Texas A&M University System · Zachry Department of Civil and Environmental Engineering, Texas A&M University · Department of Civil, Construction and Environmental Engineering, Marquette University +2
Large Language Models (LLMs) are increasingly used as judges for scalable evaluation, yet such LLM--as--a--Judge systems exhibit systematic biases that are decoupled from semantic quality, most notably verbosity bias. Meanwhile, human supervision is costly and typically selective, yielding reliable positive judgments but leaving most outputs unlabelled and potentially mixed in quality. We formulate LLM evaluation under selective human supervision as a positive--unlabelled learning problem and propose a geometric auditing framework based on Partial Optimal Transport. By aligning a small set of human--verified positives with a reliable subset of unlabelled outputs in a fixed embedding space, our method identifies human--consistent preferences and corrects biased judges without retraining. Experiments demonstrate improved alignment with human preferences, increased robustness to presentation biases, and interpretable confidence estimates, offering a scalable and statistically grounded alternative to existing LLM--as--a--judge pipelines.
Zilong Zhang, Yi-Ting Hung, Lei Ding +1
Department of Mathematics and Statistics Georgia State University · Department of Statistics University of Manitoba
Large language models (LLMs) are increasingly used as judges for open-ended generation, as large-scale human evaluation is often expensive and difficult to scale, yet their preferences remain imperfect proxies for human judgment. Existing auditing pipelines often assume that a reliable subset of examples or clean supervision signals are available beforehand, for example from human annotation, heuristic filtering, or the outputs of strong judges. In LLM evaluation, this assumption is fragile: the initial split may inherit judge bias, while human verification is typically too scarce to define stable groups at scale. We propose AURA, an adaptive uncertainty--aware refinement framework for auditing pairwise LLM--as--a--judge decisions under selected human verification. AURA iteratively learns a human-consistency signal, propagates reliable evidence, and prioritizes uncertain comparisons for human review. The key idea is to treat trust in a judge as a latent quantity that is progressively refined as evidence accumulates. We provide a compact formulation, a stable refinement procedure, and a comprehensive evaluation on both synthetic and real pairwise LLM-answer data.
Zilong Zhang, Yi-Ting Hung, Weiyi He +3
Department of Mathematics and Statistics Georgia State University · Department of Probability and Statistics Department of Computer Science and Engineering Michigan State University · Department of Mathematics and Statistics Concordia University +1