cs.CLOct 10, 2024

Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting

Authors: Tim Knappe, Ryan Li, Ayush Chauhan, Kaylee Chhua, Kevin Zhu, Sean O'Brien

Organizations: Algoverse AI Research

Abstract

While large language models (LLMs) have rapidly improved their performance on a broad number of tasks, they still often fall short on reasoning tasks. As LLMs become more integrated in diverse real-world tasks, advancing their reasoning capabilities is crucial to their effectiveness in nuanced, complex problems. Wang et al.'s self-consistency framework reveals that sampling multiple rationales before taking a majority vote reliably improves model performance across various closed-answer reasoning tasks. Standard methods based on this framework aggregate the final decisions of these rationales but fail to utilize the semantic information detailed in the step-by-step reasoning paths. Our work introduces semantic self-consistency, enhancing this approach by incorporating and analyzing both the reasoning paths of these rationales in addition to their final decisions before taking a majority vote. These methods not only improve the reliability of reasoning paths but also cause more robust performance on complex reasoning tasks.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Learning When to Sample: Confidence-Aware Selective Sampling for Efficient Chain-of-Thought Reasoning

    Mar 9, 2026Juming Xiong, Kevin Guo, Congning Ni +7CoT ReasoningEfficient Language Model Reasoning

  2. Reasoning Consensus: Structural Ensembling of LLM Reasoning via Weighted DAG Aggregation

    Jul 30, 2026Amruta Parulekar, Jinu Lee, Dilek Hakkani-Tür +1Structured ReasoningLLM Reasoning with Graphs

  3. Multidimensional Consistency Improves Reasoning in Language Models

    Mar 4, 2025Huiyuan Lai, Xiao Zhang, Malvina NissimReasoning Consistency in Language ModelsLLM Mathematical Reasoning