Reasoning Consistency in Language Models

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 43

All topics

Training Language Models To Be Coherent Decision-Makers
Oct 6, 2026Khurram Yamin, Xavier Fernandes, Paul Koch +2LLM Decision-Making Decision-Making under Uncertainty
Carnegie Mellon University · Microsoft
Silent Dissent: LLM Agents That Yield to the Majority Still Represent Their Original Premise
Oct 2, 2026Ziang Ni, Peng ZouNormative Conformity in Language Models Multi-Agent Debate
Sun Yat-sen University Guangzhou, China
Understanding as No-Arbitrage: Bounded Dutch Books as a Definition and Training Objective for Language Models
Sep 30, 2026Daniel DragonevskiyAdversarial Training Reasoning Consistency in Language Models
When Scientific Contradictions Are Lost in Translation
Sep 29, 2026Tal Zeevi, Trey W. Jensen, Maxwell StromeReasoning Consistency in Language Models Scientific Reasoning in Language Models
ACTR: Aligning Thoughts and Responses for Multilingual Safety in Reasoning LLMs
Sep 29, 2026Xianhui Zhang, Jian Yu, Chengyu Xie +6Adversarial Attacks on LLMs LLM Safety Alignment
Nanjing University of Science and Technology, Nanjing, China · National University of Singapore, Singapore · Beihang University, Beijing, China
Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoning
Sep 28, 2026Yingjin Song, Denis Paperno, Albert GattVisual Spatial Reasoning Relational Reasoning
Utrecht University Utrecht, The Netherlands
Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfaces
Sep 27, 2026Han Chen, Yingrui LiReasoning Consistency in Language Models LLM Reliability
Independent Researcher
Can LLMs Follow Medical Expert Logic? A Benchmark for Hierarchical Logical Consistency in Risk-of-Bias Assessment
Sep 12, 2026Jiayu Huang, Zichen Tang, Qianhui Ling +2Reasoning Consistency in Language Models Clinical Language Model Evaluation
Beijing University of Posts and Telecommunications · Hypertension Center, Beijing Anzhen Hospital, Capital Medical University
Is This Your Final Answer? Cross-Contextual Consistency as a Measure of LLM Credibility
Aug 10, 2026Siyang Wu, Yibo Jiang, Bryon AragamPrompt Sensitivity LLM Evaluation
Data Science Institute University of Chicago Chicago, USA · Department of Computer Science University of Chicago Chicago, USA · Booth School of Business University of Chicago Chicago, USA
Reproducing and Stress-Testing Two Approaches to LLM Reasoning Reliability: Test-Time Probability Aggregation and Logic-Representation Editing
Aug 9, 2026Minhan Cho, Jimin KweonLLM Evaluation Reasoning Consistency in Language Models
Remember & Company AI Lab · DSAIL, Dept. of Applied AI · Sungkyunkwan University +2
TrAC: Trace-Conditioned Answer Consistency for Efficient Uncertainty Quantification in LLMs
Aug 1, 2026Dahai Yu, Lin Jiang, Rongchao Xu +1LLM Evaluation Reasoning Consistency in Language Models
Florida State University Tallahassee, Florida, USA
Position: It's Time to Optimize LLMs for Self-Consistency
Jul 31, 2026Itamar Pres, Belinda Z. Li, Laura Ruis +6Reasoning Consistency in Language Models Language Model Robustness
MIT CSAIL · Goodfire AI
Reason Popper-ly: Patching In-Context Reasoning with Inductive Logic Programming
Jul 25, 2026Zirong Chen, Meiyi MaLogic Programming Reasoning Consistency in Language Models
Department of Computer Science, Vanderbilt University, USA
Beyond Semantic Equivalence: Logical Graphs for LLM Uncertainty Quantification
Jul 18, 2026Yanni Dong, Minghua Liu, Meilin Zhu +2Reasoning Consistency in Language Models LLM Reliability
Institute of Intelligent Software, Guangzhou, China · Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Liverpool, Liverpool, UK
Controlled Reformulation Testing for Logical Consistency in Large Language Models
Jul 16, 2026Alexander Gu, Alan ChenLLM Evaluation Reasoning Consistency in Language Models
University of Texas at Austin
Implicit Reasoning Steering via Concept Chaining
Jul 15, 2026Xiao Ye, Sanika Chavan, Yuxi Huang +4Adversarial Attacks on LLMs Reasoning Consistency in Language Models
School of Computing and Augmented Intelligence, Arizona State University · Bellini College of AI, Cybersecurity, and Computing, University of South Florida, Tampa, Florida, USA · Department of Computer Science, University of California, Davis
Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework
Jul 9, 2026Riccardo Revalor, Jalees Rehman, Debjit PalCoT Faithfulness LLM Reasoning with Graphs
¶Department of Electrical and Computer Engineering · ‡Department of Biochemistry and Molecular Genetics
Reasoning Consistency Scanning: A Framework for Auditing Chain-of-Thought Validity in AI Safety Evaluations
Jul 8, 2026Silvia SantanoLLM Auditing Reasoning Consistency in Language Models
Clinical Reasoning Graphs: Structured Evaluation of LLM Diagnostic Reasoning Reveals Competence Without Consistency
Jun 29, 2026Nisarg A. PatelLLM Evaluation Reasoning Consistency in Language Models
University of California, San Francisco · Department of Oral and Maxillofacial Surgery, University of California, San Francisco, CA, USA · Department of Medicine, Division of Clinical Informatics and Digital Transformation, University of California, San Francisco, CA, USA
Invariant Reasoning Directions in Latent Trajectories of Language Models
Jun 28, 2026Arun Vignesh Malarkkan, Manan Roy Choudhury, Utkarsh Byahut +3Reasoning Consistency in Language Models Implicit Reasoning in Language Models
Arizona State University, Tempe, Arizona.
Decodable but Not Faithful: Coupling Natural-Language Rationales to Programmatic Verifiers
Jun 19, 2026Vatsal Ananthula, Adarsh KumarappanFaithfulness of Language Model Explanations LLM Auditing
University of California, Berkeley · 2California Institute of Technology
Coherence Under Commitment: Probing Generalization and Vacuous Memorization in LLM Logical Reasoning
Jun 19, 2026Noor Islam S. Mohammad, Mahmudul HasanLLM Evaluation Reasoning Consistency in Language Models
Department of Computer Science, Informatics Institute, Istanbul Technical University, ˙Istanbul, Türkiye · School of Information Technology, Deakin University, Geelong, VIC 3220, Australia
The Consistency Dilemma in LLMs: Generator-Evaluator Agreement and Vulnerability to Mistakes
Jun 16, 2026Marina Mancoridis, Zoë HitzigLLM Evaluation Reasoning Consistency in Language Models
Massachusetts Institute of Technology · Harvard Society of Fellows
Quantifying Consistency in LLM Logical Reasoning via Structural Uncertainty
Jun 15, 2026Baishali Chaudhury, Mengdie Flora Wang, Hyunji Hayley Park +3Reasoning Consistency in Language Models LLM Reliability
AWS Generative AI Innovation Center
Operadic consistency: a label-free signal for compositional reasoning failures in LLMs
Jun 11, 2026Nathaniel Bottman, Yinhong Liu, Kyle RichardsonCompositional Reasoning Reasoning Consistency in Language Models
Incubilate · University of Cambridge · Allen Institute for Artificial Intelligence
Agreement in Representation Space for Open-Ended Self-Consistency
Jun 10, 2026Paula Ontalvilla, Gorka Azkune, Aitor OrmazabalOpen-Ended Generation Self-Consistency Decoding
HiTZ Center - Ixa, University of the Basque Country (UPV/EHU)
Boosting Self-Consistency with Ranking
Jun 3, 2026Maria Marina, Daniil Moskovskiy, Sergey Pletenev +3Self-Consistency Decoding LLM Reranking
AIRI · Skoltech · EPFL
ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails
May 29, 2026Yan Wang, Zhixuan Chu, Zihao Xue +9LLM Guardrails LLM Safety Alignment
Alibaba Group · Zhejiang University · Huzhou Normal University +1
Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models
May 28, 2026Zizhuo Lin, Quanling Liu, Jinsheng Quan +6LLM Grounding Reasoning Consistency in Language Models
Zhejiang University · University of Science and Technology of China
Self-Consistency via Marginal Sharpening
May 27, 2026Aleksei Arzhantsev, Otmane Sakhi, Nicolas ChopinEfficient Language Model Reasoning Reasoning Consistency in Language Models
Criteo AI Lab, CREST IP Paris · Criteo AI Lab · ENSAE, Institut Polytechnique de Paris

Oct 6, 2026, cs.AIJ/K move · Enter open · S save

Training Language Models To Be Coherent Decision-Makers

Khurram Yamin, Xavier Fernandes, Paul Koch, Bryan Wilder+1

Carnegie Mellon University · Microsoft

LLM Decision-Making Decision-Making under Uncertainty Reasoning Consistency in Language Models Supervised Fine-Tuning

Open paper AskPDF arXiv

Reliable decision-making requires more than accurate prediction: a model must preserve its beliefs, apply the relevant utilities, and recognize when the information needed to justify an action is missing. We study whether language models can learn this decision procedure from supervised fine-tuning and generalize it across domains and differing natural-language expressions of the decision challenge. Across 20 datasets, we explore challenges of belief instability and decision-making errors by first eliciting probabilities of outcomes and then varying only the utilities and the framing of the decision problems, while holding the evidence fixed. We train models to preserve elicited beliefs while selecting the action that maximizes expected utility, and evaluate transfer to unseen application domains, held-out framings, and different classes of payoff structures. We further introduce incomplete-information settings in which required utilities are withheld and replaced with irrelevant text, testing whether models can distinguish missing decision-relevant information from merely additional context. We find that targeted fine-tuning substantially improves coherent decision-making and that in many situations, learning transfers across domains and framings to situations unobserved during training. Further, models trained for decidability learn to identify when action cannot be justified based on missing information. Finally, we show the value of a routed system that considers separately the recognition of decision completeness and utility-sensitive decision execution.

Momentum

Related topics

Latest papers 43