Self-Consistency in Language Models

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 18

All topics

Self-Repulsive Sampling for Diffusion Language Models
Sep 30, 2026Michael Helcig, Martin JaggiDiffusion Language Model Decoding Diverse Text Generation
ETH Zürich · EPFL
Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback
Sep 30, 2026Jingkai Huang, Yunfan Zhang, Will Ma +2LLM Inference Efficiency Adaptive Inference
Stern School of Business New York University · School of Management Zhejiang University
VAA-CSEC: Vote-guided Advantage Allocation for Chinese Semantic Error Correction
Sep 29, 2026Yitong Han, Nankai Lin, Juan Luo +3RL for Language Models Self-Consistency in Language Models
School of Information Science and Technology, Guangdong University of Foreign Studies · Guangdong Engineering Research Center of Data Security Governance and Privacy Computing · College of Computer Science and Technology, National University of Defense Technology
An Analysis of Training-Free Self-Reported Confidence in Language Models
Sep 17, 2026Lukas Meyer, Sofia Rossi, Wei Chen +2Confidence Estimation in Language Models Language Model Self-Assessment
DreamAI
Decomposing Wrong-Consensus Agreement in LLM Self-Consistency
Aug 19, 2026Lizhuo Zhang, Mengmeng Tang, Chenfeng Long +2LLM Evaluation LLM Reliability
College of Information and Intelligence, Hunan Agricultural University, Changsha 410128, China · Yuelushan Laboratory, Changsha 410128, China · School of Computer Science and Technology, Changsha University of Science and Technology, Changsha 410114, China +1
On-Policy Self-Distillation without Any Supervision
Aug 6, 2026Yijiang Li, Bingyang Wang, Yijun Liang +3On-Policy Self-Distillation Language Model Self-Improvement
UC San Diego · Georgia Institute of Technology · University of Maryland, College Park +1
Position: It's Time to Optimize LLMs for Self-Consistency
Jul 31, 2026Itamar Pres, Belinda Z. Li, Laura Ruis +6Reasoning Consistency in Language Models Language Model Robustness
MIT CSAIL · Goodfire AI
Controlled Reformulation Testing for Logical Consistency in Large Language Models
Jul 16, 2026Alexander Gu, Alan ChenLLM Evaluation Reasoning Consistency in Language Models
University of Texas at Austin
When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals
Jul 9, 2026Kaihua DingLLM-as-a-Judge LLM Reliability
University of Pennsylvania
Self-CTRL: Self-Consistency Training with Reinforcement Learning
Jun 16, 2026Itamar Pres, Laura Ruis, Melat Ghebreselassie +2RL for Language Models Faithfulness of Language Model Explanations
MIT CSAIL
The Consistency Dilemma in LLMs: Generator-Evaluator Agreement and Vulnerability to Mistakes
Jun 16, 2026Marina Mancoridis, Zoë HitzigLLM Evaluation Reasoning Consistency in Language Models
Massachusetts Institute of Technology · Harvard Society of Fellows
Agreement in Representation Space for Open-Ended Self-Consistency
Jun 10, 2026Paula Ontalvilla, Gorka Azkune, Aitor OrmazabalOpen-Ended Generation Self-Consistency Decoding
HiTZ Center - Ixa, University of the Basque Country (UPV/EHU)
Boosting Self-Consistency with Ranking
Jun 3, 2026Maria Marina, Daniil Moskovskiy, Sergey Pletenev +3Self-Consistency Decoding LLM Reranking
AIRI · Skoltech · EPFL
Self-Consistency via Marginal Sharpening
May 27, 2026Aleksei Arzhantsev, Otmane Sakhi, Nicolas ChopinEfficient Language Model Reasoning Reasoning Consistency in Language Models
Criteo AI Lab, CREST IP Paris · Criteo AI Lab · ENSAE, Institut Polytechnique de Paris
Reliable Chain-of-Thought via Prefix Consistency
May 8, 2026Naoto Iwase, Yuki Ichihara, Mohammad Atif Quamar +1Self-Consistency Decoding Reasoning Consistency in Language Models
Nagoya University · Nara Institute of Science and Technology · Mohamed bin Zayed University of Artificial Intelligence +1
Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generation
Apr 21, 2026Thomas Zollo, Jimmy Wang, Richard ZemelConfidence Estimation in Language Models Language Model Calibration
Columbia University
Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning
Apr 19, 2026Raman Saparkhan, Majd Hawasly, Md Rizwan Parvez +1LLM Inference Efficiency Self-Consistency Decoding
Carnegie Mellon University Qatar · Qatar Computing Research Institute
Optimal Self-Consistency for Efficient Reasoning with Large Language Models
Nov 15, 2025Austin Feng, Marius Alonso, Ambroise Odonnat +2Self-Consistency Decoding Inference-Time Scaling
1Yale University, New Haven, CT, USA · 2Noah’s Ark Lab, Paris, France · 3Inria, Rennes, France +1

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Self-Repulsive Sampling for Diffusion Language Models

Michael Helcig, Martin Jaggi

ETH Zürich · EPFL

Diffusion Language Model Decoding Diverse Text Generation Diffusion Language Models Self-Consistency in Language Models LLM Reasoning

Open paper AskPDF arXiv

Sampling several responses and voting over their answers can improve a language model's accuracy, but repeated answers limit the benefit of additional samples. Raising temperature increases diversity at a potential cost to per-sample accuracy. We introduce Self-Repulsion (SR), a sampler for masked diffusion language models that uses peer commitments to diversify the pool. At each penalized denoising step, each path lowers a token's logit according to how many peers have committed that token at the same position. Paths share a batched forward pass and then commit in sequence, so later paths observe choices made earlier in the same step. This coupling requires no training or additional forward or backward pass and can produce distinct paths even at temperature zero. When all paths commit a position together from identical logits, the update exactly maximizes total logit minus a convex duplication cost. On LLaDA-8B-Instruct with ten paths and 128 denoising steps, deterministic SR reaches 80.38% plurality accuracy on GSM8K, compared with 70.17% for the unpenalized greedy decoder. At temperature 0.6 and matched model-evaluation budgets, the count penalty improves over self-consistency by 2.06 percentage points in blocks of 32 and 14.50 under pure diffusion. Experiments on GSM8K, MATH and TruthfulQA show that voting gains arise mainly from higher coverage of correct answers, with gains that vary by benchmark and decoding regime.

Momentum

Related topics

Latest papers 18