Best-of-N Sampling

Best-of-N sampling spends inference-time compute to generate several answers, then chooses one, aiming to find stronger outputs without retraining the model. Selection methods include reward-model or verifier scores, majority voting and consensus, confidence-based ranking, learned scorers, and regularized or noise-based variants. The approach is used for language-model reasoning and alignment, with evaluations on tasks such as MATH, GSM8K, coding, and instruction following, as well as speech and medical report generation. More samples can improve the chance of finding a good answer, but unreliable judges, reward hacking, tail effects, and rising costs can erase gains or make selections worse; how to evaluate and control these risks remains a central challenge.