Self-Consistency Decoding

Follow this topic Ask about this topic

Self-consistency decoding studies whether a model can give more reliable answers by generating several responses and combining them, often by majority vote over chain-of-thought reasoning. Methods also cluster semantically similar answers, rank candidates using trace or confidence signals, or use verification, adaptive sampling, and early stopping to spend compute more selectively. Evaluations span math and science reasoning, question answering, coding, information extraction, and multimodal tasks, with gains that vary by model, task, and decoding budget. A central tension is that more samples can help find correct answers but correlated errors and weak selection can limit or reverse those gains, so fair comparisons must account for compute and stronger alternatives such as input variation or verification.

Latest papers

33 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Enhancing reasoning accuracy in large language models during inference time
Mar 22, 2026Vinay Sharma, Manish JainLLM Inference LLM Reasoning
Do not be greedy, Think Twice: Sampling and Selection for Document-level Information Extraction
Jan 26, 2026Mikel Zubillaga, Oscar Sainz, Oier Lopez de Lacalle +1Self-Consistency Decoding Document Information Extraction
HiTZ Center - Ixa, University of the Basque Country UPV/EHU
Optimal Self-Consistency for Efficient Reasoning with Large Language Models
Nov 15, 2025Austin Feng, Marius Alonso, Ambroise Odonnat +2Self-Consistency Decoding Inference-Time Scaling
1Yale University, New Haven, CT, USA · 2Noah’s Ark Lab, Paris, France · 3Inria, Rennes, France +1

Mar 22, 2026J/K move · Enter open · S save

Enhancing reasoning accuracy in large language models during inference time

Vinay Sharma, Manish Jain

LLM Inference LLM Reasoning Inference-Time Guidance Self-Consistency Decoding

Open paper AskPDF arXiv

Large Language Models (LLMs) often exhibit strong linguistic abilities while remaining unreliable on multi-step reasoning tasks, particularly when deployed without additional training or fine-tuning. In this work, we study inference-time techniques to improve the reasoning accuracy of LLMs. We systematically evaluate three classes of inference-time strategies: (i) self-consistency via stochastic decoding, where the model is sampled multiple times using controlled temperature and nucleus sampling and the most frequent final answer is selected; (ii) dual-model reasoning agreement, where outputs from two independent models are compared and only consistent reasoning traces are trusted; and (iii) self-reflection, where the model critiques and revises its own reasoning. Across all evaluated methods, we employ Chain-of-Thought (CoT) [1] prompting to elicit explicit intermediate reasoning steps before generating final answers. In this work, we provide a controlled comparative evaluation across three inference-time strategies under identical prompting and verification settings. Our experiments on LLM [2] show that self-consistency with nucleus sampling and controlled temperature value yields the substantial gains, achieving a 9% to 15% absolute improvement in accuracy over greedy single-pass decoding, well-suited for low-risk domains, offering meaningful gains with minimal overhead. The dual-model approach provides additional confirmation for model reasoning steps thus more appropriate for moderate-risk domains, where higher reliability justifies additional compute. Self-reflection offers only marginal improvements, suggesting limited effectiveness for smaller non-reasoning models at inference time.

Latest papers

Enhancing reasoning accuracy in large language models during inference time

Do not be greedy, Think Twice: Sampling and Selection for Document-level Information Extraction

Optimal Self-Consistency for Efficient Reasoning with Large Language Models