Efficient Best-Of-N with Radial Consensus Score

Manh Nguyen; Sunil Gupta; Hung Le

cs.CLApr 14, 2026

Efficient Best-Of-N with Radial Consensus Score

Authors: Manh Nguyen, Sunil Gupta, Hung Le

Organizations: Applied Artificial Intelligence Initiative, Deakin University, Australia

Abstract

Solving complex problems with large language models (LLMs) often requires selecting the best answer from multiple candidate responses (best-of- $N$ ), yet identifying the most reliable one remains challenging, particularly when correctness does not align with majority agreement. Existing approaches based on voting or probability scoring rely on surface-level signals (e.g., frequency or token-level likelihood), often missing high-quality rare answers and ignoring the geometric structure of candidate representations, where correctness may emerge as semantic consistency rather than surface agreement. To address these limitations, we introduce Radial Consensus Score (RCS), a simple, efficient, and training-free method for best-of- $N$ selection. RCS models semantic consensus by computing a weighted Frechet mean (semantic center) of answer embeddings and ranking candidates by their radial distance to this center. This enables it to recover semantically coherent correct answers despite lexical fragmentation, even when no single surface form holds a plurality under exact-match voting. Importantly, RCS offers a flexible framework that supports multiple weighting schemes, enabling the integration of arbitrary signals, such as semantic agreement or model confidence, with the position of the center adapting accordingly. Extensive experiments across five open-weight models and various benchmarks demonstrate that RCS matches or outperforms strong baselines at every sampling budget, with the largest gains when correct answers are lexically fragmented. Moreover, RCS serves as an effective drop-in replacement for majority voting in multi-agent debate and remains effective in black-box scenarios. These results suggest that geometric consensus can serve as a scalable and reliable alternative to voting-based methods for answer selection. Code is publicly available at https://github.com/manhitv/RCS.

Figures & tables

Figure 1: (a) Majority voting ignores semantically similar answers by treating surface forms independently. (b) It also fails to select high-confidence but minority answers. RCS addresses the lexical fragmentation issue by mapping answers into an embedding space via Encoder

\mathbf{E}

and estimating a centroid

\mathbf{c}

, either uniformly or probability-weighted over answer embeddings; the latter can also recover high-confidence minority answers. Additional qualitative examples are provided in Section 5.3 .

Variant	Distribution $P$	Supported LLMs
Uniform	$p_{i}=\frac{1}{N}$	Black-box
Frequency-weighted	$p_{i}=\frac{f_{i}}{\sum_{j}f_{j}}$	Black-box
Cosine-weighted	$p_{i}\propto\frac{1}{N}\sum_{j=1}^{N}\cos(\mathbf{u}_{i},\mathbf{u}_{j})$	Black-box
Probability-weighted	$p_{i}\propto\Pr(y_{i}\|\mathbf{x})$	White-box

Table 1: Different instantiations of the distribution

P

in RCS, where

f_{i}

and

\Pr(y_{i}|\mathbf{x})

denote the empirical frequency and model probability of candidate

y_{i}

, respectively. For the cosine-similarity-weighted variant,

\cos(\mathbf{u}_{i},\mathbf{u}_{j})

denotes the cosine similarity between

\ell_{2}

-normalized embeddings.

Model	Method	BBF	SciQ	GPQA	Arithmetics	GSM8K	Form.Log.	Average
Qwen2.5-3B	Greedy	✓	59.3	20.9	89.0	65.0	34.9	53.8
	Oracle	✓	80.8 $\pm$ 0.7	51.0 $\pm$ 2.9	99.5 $\pm$ 0.5	93.1 $\pm$ 0.8	73.5 $\pm$ 5.7	79.6
	NLL	✗	59.4 $\pm$ 0.7	15.7 $\pm$ 2.4	36.0 $\pm$ 0.9	25.0 $\pm$ 3.1	23.0 $\pm$ 2.7	31.8
	ANLL	✗	59.2 $\pm$ 0.5	15.9 $\pm$ 2.2	36.0 $\pm$ 1.3	24.9 $\pm$ 3.3	22.8 $\pm$ 3.0	31.8
	SC	✓	64.4 $\pm$ 0.6	21.4 $\pm$ 0.6	78.7 $\pm$ 3.3	52.8 $\pm$ 1.3	40.5 $\pm$ 2.1	51.6
	SCW	✓	64.5 $\pm$ 0.8	23.0 $\pm$ 0.3	51.8 $\pm$ 3.0	37.4 $\pm$ 2.8	25.7 $\pm$ 2.4	40.5

Table 2: Best-of-

N

selection accuracy. BBF : black-box friendly. For non-RCS methods, underline marks the best baseline per setting. Among RCS variants, bold indicates the best RCS method per column. Item-level McNemar tests (Appendix Table 9 ) show that RCS

{}_{\text{uni}}

significantly outperforms self-consistency in 15 of 25 model

\times

benchmark cells (

p<0.05

). Additional results in Appendix Table 8 .

Figure 2: Average performance over five benchmarks for different numbers of sampling responses

N

. Each subplot corresponds to an LLM backbone, showing how accuracy changes with increasing

N

Model	Method	N	Arithmetics				Form.Log.
Model	Method	N	R=0	R=1	R=2	Avg.	R=0	R=1	R=2	Avg.
Qwen2.5-3B	SC	10	94.5 $\pm$ 4.0	60.4 $\pm$ 1.6	34.6 $\pm$ 5.8	63.1	44.8 $\pm$ 2.5	35.3 $\pm$ 2.9	27.6 $\pm$ 5.4	35.9
	RCS ${}_{\text{uni}}$	10	94.3 $\pm$ 3.7	59.5 $\pm$ 4.1	34.0 $\pm$ 11.0	62.6	46.0 $\pm$ 2.3	35.8 $\pm$ 5.3	28.8 $\pm$ 6.0	36.9
	RCS ${}_{\text{freq}}$	10	95.2 $\pm$ 3.6	61.4 $\pm$ 1.2	34.7 $\pm$ 5.4	63.8	44.8 $\pm$ 2.8	36.1 $\pm$ 3.3	28.7 $\pm$ 5.5	36.5
	SC	20	96.5 $\pm$ 0.5	24.8 $\pm$ 3.0	34.0 $\pm$ 3.3	51.8	42.3 $\pm$ 0.5	40.5 $\pm$ 1.6	35.7 $\pm$ 1.6	39.5
	RCS ${}_{\text{uni}}$	20	97.7 $\pm$ 1.0	27.3 $\pm$ 3.2	34.5 $\pm$ 5.1	53.2	46.6 $\pm$ 0.9	45.2 $\pm$ 1.4	37.8 $\pm$ 0.9	43.2

Table 3: Performance on Arithmetics and Form.Log. with Multi-agent Debate. Best values are bolded. R=0,1,2 indicate debate rounds, while Avg. denotes the average over all rounds.

(a) Method GPQA MMLU-Pro BBH-Date BBH-Navigate Avg. SC 13.1 47.6 76.8 39.2 44.2 SCW 12.6 47.6 76.2 39.7 44.0 RCS

{}_{\text{medoid}}

14.6 48.6 76.4 39.2 44.7 RCS

{}_{\text{uni}}

14.1 48.6 78.0 40.8 45.4 RCS

{}_{\text{freq}}

13.6 47.6 77.6 41.2 45.0 RCS

{}_{\text{cosine}}

13.6 47.6 77.2 40.5 44.7

(b) Method Complexity NLL, ANLL

\mathcal{O}(NL)

\mathcal{O}(N)

SCW

\mathcal{O}(N^{2}d)

\mathcal{O}(NL|V|)

ModeX

\mathcal{O}(N^{2}L+N^{3})

RCS

{}_{\text{medoid/cosine}}

\mathcal{O}(N^{2}d)

RCS

{}_{\text{uni/freq}}

\mathcal{O}(Nd)

RCS

{}_{\text{prob}}

\mathcal{O}(N(d+L))

Table 4: Performance under (a) black-box setting (

N{=}

10) and (b) complexity comparison.

Figure 4: Effect of encoders on Llama3.2-3B. Results for Qwen2.5-3B are in Appendix Figure 7 .

Figure 5: Effect of varying

\tau

using Qwen2.5-3B. Results for Llama3.2-3B are in Appendix Figure 8 .

Figure 6: Effect of reasoning paths on Arithmetics and Formal Logic.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
Temperature	1
Max new tokens (short-form QA)	32
Max new tokens (long-form tasks)	512

Appendix

Table 5: Sampling hyperparameters used for generation.

Tasks	Prompt
SciQ, GPQA	This is a bot that correctly answers questions.
Arithmetics, GSM8K	Make sure to state your final answer in curly brackets at the very end of your response, just like: {final answer: 12.34} . Let’s think step by step.
Form.Log., MMLU-Pro, BBH-Date	Make sure to state your final answer choice in curly brackets at the very end of your response, just like: {final answer: (A)} . Let’s think step by step.
BBH-Navigate	Make sure to state your final answer choice in curly brackets at the very end of your response, just like: {final answer: Yes} . Let’s think step by step.

Appendix

Table 6: System prompts used for generation across tasks.

Dataset	#Samples	Question	Answer
SciQ	1000	Compounds that are capable of accepting electrons, such as o 2 or f2, are called what?	oxidants
GPQA	198	Which of the following physical theories never requires UV regularization?	Superstring Theory
Arithmetics	200	What is $23+47$ ?	70
GSM8K	200	A robe takes 2 bolts of blue fiber and half that much white fiber. How many bolts in total does it take?	3
Formal Logic	126	Select the best translation into predicate logic: Sheena is a punk rocker. Choices: ("Sx", "xS", "sP", "Ps")	"Ps"
MMLU-Pro	105	Determine the number of men needed to build a boat in 77 days if it takes 36 men 132 days to build one. Choices: ("84 men", "36 men", "99 men", "132 men", "45 men", "70 men", "62 men", "50 men", "77 men", "120 men" )	"62 men"

Appendix

Table 7: Overview of evaluation benchmarks, including the number of samples and representative examples.

Model	Method	BBF	SciQ	GPQA	Arithmetics	GSM8K	Form.Log.	Average
Qwen2.5-7B	Greedy	✓	64.1	20.5	97.0	88.3	43.6	63.1
	Oracle	✓	82.0 $\pm$ 1.1	49.9 $\pm$ 1.6	100.0 $\pm$ 0.0	94.6 $\pm$ 0.8	67.7 $\pm$ 2.5	78.8
	NLL	✗	66.4 $\pm$ 0.1	18.3 $\pm$ 1.3	47.3 $\pm$ 5.7	32.7 $\pm$ 4.3	25.7 $\pm$ 0.5	38.1
	ANLL	✗	66.2 $\pm$ 0.4	18.1 $\pm$ 0.9	47.0 $\pm$ 5.6	32.7 $\pm$ 4.4	25.9 $\pm$ 0.5	38.0
	SC	✓	70.3 $\pm$ 0.9	22.1 $\pm$ 4.2	74.3 $\pm$ 2.0	52.3 $\pm$ 2.0	48.4 $\pm$ 0.0	53.5
	SCW	✓	70.2 $\pm$ 0.8	26.3 $\pm$ 3.2	52.8 $\pm$ 5.0	36.4 $\pm$ 5.2	32.8 $\pm$ 3.3	43.7

Appendix

Table 8: Extended Results: Best-of-

N

selection accuracy across settings. Best-of-

N

selection accuracy. BBF : black-box friendly. For non-RCS methods, underline marks the best baseline per setting. Among RCS variants, bold indicates the best RCS method per column. No baseline significantly outperforms RCS at the column level (

p<0.05

, paired t-test); a full per-cell (model

\times

benchmark) McNemar breakdown against all six baselines is in Appendix Table 9 .

Model	Baseline	SciQ	GPQA	Arithmetics	GSM8K	Form.Log.
Qwen2.5-3B	NLL	$<10^{-20}$	$<10^{-4}$	$<10^{-74}$	$<10^{-54}$	$<10^{-16}$
	ANLL	$<10^{-21}$	$<10^{-4}$	$<10^{-74}$	$<10^{-54}$	$<10^{-17}$
	SC	0.001	0.326	$<10^{-7}$	$<10^{-18}$	0.003
	SCW	0.001	1.000	0.108	$<10^{-6}$	0.388
	CE	$<10^{-4}$	$<10^{-4}$	$<10^{-4}$	$<10^{-19}$	0.036
	ModeX	$<10^{-9}$	0.007	$<10^{-39}$	$<10^{-28}$	$<10^{-6}$

Appendix

Table 9: Full per-cell (model

\times

benchmark) item-level McNemar

p

-value of RCS

{}_{\text{uni}}

against each of the six baselines. Bold = significant at

\alpha{=}0.05

in RCS

{}_{\text{uni}}

’s favor; red bold = significant in the baseline’s favor.

Figure 7: Effect of the sentence embedding model on Arithmetics and Form.Log. using Qwen2.5-3B.

Figure 8: Performance when varying correctness threshold (

\tau

) using Llama3.2-3B.

Model	Method	SciQ	GPQA	Arithmetics	GSM8K	Form.Log.
Qwen2.5-3B	NLL	61.7 $\pm$ 1.2	17.6 $\pm$ 3.1	49.2 $\pm$ 12.4	45.2 $\pm$ 4.7	28.0 $\pm$ 3.2
	ANLL	61.6 $\pm$ 1.1	17.1 $\pm$ 4.1	48.7 $\pm$ 11.4	46.0 $\pm$ 4.8	27.5 $\pm$ 3.3
	SC	63.8 $\pm$ 0.5	18.1 $\pm$ 2.7	78.7 $\pm$ 7.2	65.0 $\pm$ 0.5	39.4 $\pm$ 1.8
	SCW	64.0 $\pm$ 0.2	24.0 $\pm$ 2.6	46.8 $\pm$ 1.1	38.1 $\pm$ 6.2	27.0 $\pm$ 2.1
	CE	63.6 $\pm$ 0.3	15.7 $\pm$ 1.7	78.5 $\pm$ 7.4	66.8 $\pm$ 5.0	42.4 $\pm$ 0.2
	ModeX	61.2 $\pm$ 1.2	18.0 $\pm$ 2.4	46.0 $\pm$ 3.0	32.0 $\pm$ 0.5	24.9 $\pm$ 0.9

Appendix

Table 10: Best-of-

N

selection accuracy across settings (

N{=}5

Model	Method	SciQ	GPQA	Arithmetics	GSM8K	Form.Log.
Qwen2.5-3B	NLL	59.0 $\pm$ 0.4	16.4 $\pm$ 2.3	48.2 $\pm$ 11.9	40.6 $\pm$ 0.5	26.5 $\pm$ 2.6
	ANLL	58.4 $\pm$ 0.4	16.2 $\pm$ 2.6	47.8 $\pm$ 12.5	40.8 $\pm$ 1.1	26.5 $\pm$ 2.6
	SC	64.9 $\pm$ 0.6	20.0 $\pm$ 0.3	95.3 $\pm$ 5.1	88.3 $\pm$ 0.9	48.1 $\pm$ 3.6
	SCW	64.9 $\pm$ 0.6	24.4 $\pm$ 2.7	56.2 $\pm$ 4.6	33.8 $\pm$ 3.9	28.3 $\pm$ 1.8
	CE	64.2 $\pm$ 0.9	18.3 $\pm$ 1.3	95.3 $\pm$ 5.9	88.8 $\pm$ 0.9	47.1 $\pm$ 2.4
	ModeX	62.2 $\pm$ 0.8	21.9 $\pm$ 1.7	51.7 $\pm$ 3.4	37.9 $\pm$ 2.1	23.3 $\pm$ 1.8

Appendix

Table 11: Best-of-

N

selection accuracy across settings (

N{=}20

Model	Method	SciQ	GPQA	Arithmetics	GSM8K	Form.Log.
Qwen2.5-3B	NLL	56.3 $\pm$ 0.8	16.8 $\pm$ 1.8	45.3 $\pm$ 13.0	42.8 $\pm$ 1.5	25.9 $\pm$ 3.8
	ANLL	56.1 $\pm$ 0.9	16.8 $\pm$ 2.0	44.8 $\pm$ 13.3	43.0 $\pm$ 2.8	25.1 $\pm$ 3.8
	SC	65.2 $\pm$ 0.4	21.8 $\pm$ 1.4	97.3 $\pm$ 2.9	90.2 $\pm$ 1.1	49.5 $\pm$ 1.2
	SCW	65.3 $\pm$ 0.3	25.6 $\pm$ 2.3	54.5 $\pm$ 4.2	38.1 $\pm$ 1.8	30.7 $\pm$ 3.0
	CE	65.2 $\pm$ 0.3	20.4 $\pm$ 2.7	97.5 $\pm$ 2.6	90.4 $\pm$ 0.5	48.9 $\pm$ 3.2
	ModeX	63.0 $\pm$ 1.0	20.0 $\pm$ 0.5	49.8 $\pm$ 3.2	35.7 $\pm$ 3.1	25.7 $\pm$ 6.7

Appendix

Table 12: Best-of-

N

selection accuracy across settings (

N{=}40

Best-of-N Sampling

Best-of-N sampling generates multiple candidate outputs and selects one using reward, confidence, agreement, or task-specific evidence.

See all 33 papers

Boosting Self-Consistency with Ranking
Jun 3, 2026Maria Marina, Daniil Moskovskiy, Sergey Pletenev +3Self-Consistency Decoding LLM Reranking
AIRI · Skoltech · EPFL
When LLM Judge Scores Look Good but Best-of-N Decisions Fail
Mar 12, 2026Eddie LandesbergLLM Evaluation LLM-as-a-Judge
LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning
Jul 11, 2026Ning LiuLLM Answer Verification Test-Time Scaling
Independent Researcher

Jun 3, 2026J/K move · Enter open · S save

Boosting Self-Consistency with Ranking

Maria Marina, Daniil Moskovskiy, Sergey Pletenev, Mikhail Salnikov+2

AIRI · Skoltech · EPFL

Self-Consistency Decoding LLM Reranking Reasoning Consistency in Language Models Self-Consistency in Language Models

Open paper AskPDF arXiv

Self-consistency improves large language models by sampling multiple reasoning paths and selecting the most frequent answer, but majority voting often fails to recover correct answers that are already present among the samples. We address this limitation with Ranking-Improved Self-Consistency (RISC), which reformulates answer selection in self-consistency as a ranking problem. Instead of relying on a single uncertainty or confidence signal, RISC uses a lightweight LambdaRank model to score candidate answers with five carefully designed features that capture answer frequency, semantic centrality, and reasoning-trace consistency. We evaluate RISC on three datasets under a range of test-time budgets. Across datasets, RISC consistently achieves a better accuracy-efficiency trade-off than standard self-consistency and strong baselines, with particularly large gains on question answering benchmarks. Further analysis shows that the proposed features are individually useful and, more importantly, complementary, highlighting the value of learning to combine multiple informative signals for test-time answer selection.

Abstract

Best-of-N Sampling

Explore similar work

Boosting Self-Consistency with Ranking

When LLM Judge Scores Look Good but Best-of-N Decisions Fail

LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning