Best-of-N Sampling

Follow this topic Ask about this topic

Best-of-N sampling spends inference-time compute to generate several answers, then chooses one, aiming to find stronger outputs without retraining the model. Selection methods include reward-model or verifier scores, majority voting and consensus, confidence-based ranking, learned scorers, and regularized or noise-based variants. The approach is used for language-model reasoning and alignment, with evaluations on tasks such as MATH, GSM8K, coding, and instruction following, as well as speech and medical report generation. More samples can improve the chance of finding a good answer, but unreliable judges, reward hacking, tail effects, and rising costs can erase gains or make selections worse; how to evaluate and control these risks remains a central challenge.

Latest papers

33 papers5 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Whether or Where? Tail Calibration Signals Finite-Budget Reward Hacking in Best-of-N Selection
Oct 25, 2026Cheng Gui, Yihua Shao, Lei Li +2Best-of-N Sampling Reward Over-Optimization
City University of Hong Kong · University of Science and Technology Beijing · University of Washington +1
Tail Competition Explains Scaling in Best-of-N Sampling for Verifiable Problems
Oct 14, 2026Taro Yano, Yoichi Ishibashi, Masafumi OyamadaBest-of-N Sampling Inference-Time Scaling
NEC Corporation
Efficient Best-of-N policy evaluation for inference-time alignment
Oct 7, 2026Jonas Schweisthal, Yuxin Wang, Athiya Deviyani +2Best-of-N Sampling Off-Policy Evaluation
LMU Munich & MCML · Carnegie Mellon University · Meta (work in personal capacity)
Holdout Best-of-N: Unbiased Evaluation and Its Cost
Oct 6, 2026Shrey Shah, Yinheng LiBest-of-N Sampling Policy Evaluation
Audit the Scaffold, Not the Checkpoint: A Stationarity Dichotomy for Recursive Self-Improvement in Agentic Coding
Sep 28, 2026Sebastian Bobadilla-Suarez, Bob Suh, Ryan FortinSoftware Engineering Agents AI Agent Auditing
OnCorps
Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidance
Sep 21, 2026Hengyuan Zhang, Chenming Shang, Zunhai Su +10Best-of-N Sampling Inference-Time Guidance
The University of Hong Kong · University of California, Los Angeles · Arizona State University +3
ExpBoN: Exponential-Noise Best-of- $n$ for Efficient Test-Time LLM Alignment
Sep 18, 2026Yanxiao Liu, Sicheng Wan, Deniz GündüzBest-of-N Sampling Inference-Time Language Model Alignment
Imperial College London · University of Washington
Privacy Without Regret: Differentially Private Inference-Time Alignment
Aug 26, 2026Ishi Jain, Nandini Bhattad, Sayak Ray ChowdhuryInference-Time Language Model Alignment Best-of-N Sampling
Indian Institute of Technology Kanpur
Safety Hacking in Constrained Best-of- $N$ Inference-time Scaling
Aug 24, 2026Akifumi Wachi, Takumi Tanabe, Youhei AkimotoLLM Safety Inference-Time Scaling
LY Corporation · University of Tsukuba · RIKEN AIP
Evaluator Ensembles Under Reward Hacking: Covariance Geometry and Finite-Search Guarantees
Aug 8, 2026Fariya Afrin, Ibne Farabi ShihabReward Hacking LLM Evaluation
Department of Computer Science, Kalinga Institute of Industrial Technology · Department of Computer Science, Iowa State University
Stabilized Best-of- $K$ Training for Neural Combinatorial Optimization
Jul 31, 2026Melveena Jolly, Midhun XavierTraveling Salesperson Problem Best-of-N Sampling
1Independent Researcher
Best-of-Evidence: Best-of-N Selection under Partial Verification
Jul 23, 2026Cenwei Zhang, Teng Fang, Yuxia Wang +3Model Selection Evidence-Grounded Reasoning
IQuest Research · INSAIT · Technical University of Denmark
Rank-Conditioned Sample Reuse for the Plackett--Luce Best-of- $K$ Objective
Jul 13, 2026Melveena Jolly, Midhun XavierInverse Probability Weighting Best-of-N Sampling
Independent Researcher
Best-of- $N$ TTS Evaluation is Confounded by ASR Family Alignment
Jul 9, 2026Taehyung Yu, Seongjae KangSpeech Generation Evaluation TTS Evaluation
KAIST, Daejeon, South Korea.
Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Models
Jul 5, 2026Ruchit Rawal, Reza Shirkavand, Sayak Paul +5Inference-Time Scaling Best-of-N Sampling
University of Maryland, College Park, MD, USA · Hugging Face, USA
Best-of-Better- $N$ : Generating Pre-Aligned Responses with In-Context Learning
Jul 3, 2026Eric Lei, Hsiang Hsu, Chun-Fu ChenLLM Alignment Best-of-N Sampling
JPMorganChase Global Technology Applied Research
When More Sampling Hurts: The Modal Ceiling and Correlation Ceiling of Test-Time Scaling
Jun 27, 2026Yong Yi Bay, Kathleen A. YearickLLM Evaluation Self-Consistency Decoding
University of Illinois at Urbana-Champaign
Transition-Aware best-of-N sampling for Longitudinal Chest X-ray Reports
Jun 23, 2026Halil Ibrahim Gulluk, Max Van Puyvelde, Wim Van Criekinge +1Radiology Report Generation Radiology VLMs
Department of Electrical Engineering, Stanford University, Stanford, CA, USA · Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA · Department of Mathematical Modelling, Statistics & Bioinformatics, Ghent University, Ghent, Belgium
Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics
Jun 2, 2026Luyang Zhang, Jingyan LiLLM Evaluation Language Model Performance Prediction
Carnegie Mellon University · Johns Hopkins University
Reward Learning from Best-of- $N$ Preference Data: Targets, Tradeoffs, and Design Principles
May 28, 2026Rattana Pukdee, Maria-Florina Balcan, Pradeep RavikumarPairwise Preference Learning Reward Modeling
Machine Learning Department Carnegie Mellon University
CCS: Clinical Consensus Selection for Radiology Report Generation
May 28, 2026Xi Zhang, Yingshu Li, Zaiqiao Meng +2Radiology Report Generation Radiology VLMs
School of Computing Science, University of Glasgow · School of Electrical and Computer Engineering, University of Sydney · Language Technology Lab, University of Cambridge
What should post-training optimize? A test-time scaling law perspective
May 11, 2026Muheng Li, Jian Qian, Wenlong MouTest-Time Scaling Best-of-N Sampling
Department of Statistical Sciences, University of Toronto† · Department of AI and Data Science, The University of Hong Kong ⋄
On-Policy Distillation with Best-of-N Teacher Rollout Selection
May 10, 2026Ke Zhang, Yunjie Tian, Dongdi Zhao +4CoT Distillation Best-of-N Sampling
Johns Hopkins University · TikTok · University of California, San Diego +1
Flexible Empowerment at Reasoning with Extended Best-of-N Sampling
Apr 17, 2026Taisuke KobayashiReinforcement Learning Best-of-N Sampling
National Institute of Informatics and The Graduate University for Advanced Studies, SOKENDAI Japan
SCATR: Simple Calibrated Test-Time Ranking
Apr 16, 2026Divya Shyamal, Marta Knežević, Lan Tran +3Test-Time Scaling Best-of-N Sampling
EPFL · MIT
Efficient Best-Of-N with Radial Consensus Score
Apr 14, 2026Manh Nguyen, Sunil Gupta, Hung LeBest-of-N Sampling LLM Reranking
Applied Artificial Intelligence Initiative, Deakin University, Australia
From Curiosity to Caution: Mitigating Reward Hacking for Best-of-N with Pessimism
Apr 6, 2026Zhuohao Yu, Zhiwei Steven Wu, Adam BlockReward Hacking Reward Over-Optimization
Carnegie Mellon University · Columbia University
When LLM Judge Scores Look Good but Best-of-N Decisions Fail
Mar 12, 2026Eddie LandesbergLLM Evaluation LLM-as-a-Judge
Best-of-Tails: Bridging Optimism and Pessimism in Inference-Time Alignment
Mar 6, 2026Hsiang Hsu, Eric Lei, Chun-Fu ChenReward Over-Optimization Inference-Time Language Model Alignment
JPMorganChase Global Technology Applied Research
Revisiting the (Sub)Optimality of Best-of-N for Inference-Time Alignment
Mar 5, 2026Ved Sriraman, Adam BlockBest-of-N Sampling Inference-Time Language Model Alignment
Department of Computer Science, Columbia University · Department of Electrical Engineering, Columbia University

Oct 25, 2026J/K move · Enter open · S save

Whether or Where? Tail Calibration Signals Finite-Budget Reward Hacking in Best-of-N Selection

Cheng Gui, Yihua Shao, Lei Li, Jingcai Guo+1

City University of Hong Kong · University of Science and Technology Beijing · University of Washington+1

Best-of-N Sampling Reward Over-Optimization Model Calibration Probability Calibration

Open paper AskPDF arXiv

Best-of-N (BoN) inference returns the candidate with the highest proxy reward, so a larger

N

tests the proxy's extreme tail more than its average ranking quality. We show that the exact BoN value is a Beta-weighted integral of conditional true reward over within-prompt score quantiles. It follows that BoN depends on scores only through their ranks. It also follows that BoN can decline only if the calibration curve in the tail turns downward, and the budget where the decline appears depends on where that turn lies. Under a bounded additive model, the error tail sets the limit: Gaussian errors keep improving, Laplace errors reach a plateau, and heavy-tailed errors return to the prior mean. When error variance is largest at low true reward, even Gaussian noise can push BoN below random selection. Synthetic phase maps agree with this taxonomy. On real candidate pools spanning several verifiable domains, base policies, and learned reward models, tail failures are rare and all occur in code generation. A held-out tail calibration statistic reliably predicts where a cross-fitted budget cap helps. It does about as well as rank accuracy, and the cap is competitive with a Soft-BoN baseline. Tail calibration therefore does not replace ranking metrics. It is an audit that is unchanged by any monotone rescaling of the reward scores and shows which budgets are at risk.