Reasoning Consistency in Language Models

Ask about this topic

Overview

The field studies whether language models preserve conclusions and sound reasoning when prompts are paraphrased, evidence changes, answers are resampled, or conversations continue. Evaluations use controlled reformulations and multi-turn tests, repeated-sample agreement, and checks of reasoning traces with formal solvers, causal interventions, or evidence-grounded verifiers across math, logic, coding, and medical tasks. Methods include self-consistency and uncertainty-based answer selection, consistency-aware training and rewards, and test-time verification, repair, or early-exit systems. A central tension is that consistency can signal stability without correctness or faithful reasoning: models may repeat the same error, abstain, or agree while relying on invalid reasoning, so reliability must be assessed alongside accuracy, faithfulness, and cost.

Latest papers

CardsList

123 papers24 papers in the last four weeks, up 60% on the four weeks before. 0.1% of all new papers.

  1. CogRAG: Tackling Heterogeneous Cognitive Demands in RAG via Stratified Retrieval and Reasoning

    Apr 1, 2026Xudong Wang, Zilong Wang, Kui Su +1Structured Reasoning in Language ModelsAdaptive RAG

    School of Computer and Computing Science, Hangzhou City University, Hangzhou, Zhejiang 310015, China · Institute of Digital Twin, Eastern Institute of Technology, Ningbo, Zhejiang 315200, China · Innovation Center of Yangtze River Delta, Zhejiang University, Jiaxing, Zhejiang 314100, China

  2. Grading the Unspoken: Evaluating Tacit Reasoning in Quantum Field Theory and String Theory with LLMs

    Apr 1, 2026Xingyang Yu, Yinghuan Zhang, Yufei Zhang +1Scientific Reasoning in Language ModelsLLM Evaluation

    Department of physics, Virginia Tech · Independent Researcher · Department of Computer Science and Engineering, Michigan State University

  3. The Necessity of Setting Temperature in LLM-as-a-Judge

    Mar 30, 2026Lujun Li, Lama Sleem, Yangjie Xu +4LLM-as-a-JudgeLLM Evaluation

    University of Luxembourg · ETH Zürich

  4. Distributionally Robust Token Optimization in RLHF

    Mar 27, 2026Yeping Jin, Jiaming Hu, Ioannis Ch. PaschalidisLanguage Model RobustnessReasoning Consistency in Language Models

    Department of System Engineering Boston University, Boston, MA 02215, USA · Department of Math & Statistics Boston University, Boston, MA 02215, USA

  5. PICon: A Multi-Turn Interrogation Framework for Evaluating Persona Agent Consistency

    Mar 26, 2026Minseo Kim, Sujeong Im, Junseong Choi +4Persona ConsistencyLLM Agent Evaluation

    KAIST

  6. Probing the Lack of Stable Internal Beliefs in LLMs

    Mar 26, 2026Yifan Luo, Kangping Xu, Yanzhen Lu +2Personality Modeling in Language ModelsReasoning Consistency in Language Models

    IIIS, Tsinghua University · Shanghai Qizhi Institute

  7. Multi-Agent Reasoning with Consistency Verification Improves Uncertainty Calibration in Medical MCQA

    Mar 25, 2026John Ray B. MartinezUncertainty CalibrationMedical QA

    Department of Data Science and Analytics, Harrisburg University of Science and Technology

  8. ThermoQA: A Three-Tier Benchmark for Evaluating Thermodynamic Reasoning in Large Language Models

    Mar 25, 2026Kemal DüzkarLLM EvaluationScientific Reasoning in Language Models

    Olivenet Kyrenia, Cyprus

  9. Explanation Generation for Contradiction Reconciliation with LLMs

    Mar 24, 2026Jason Chan, Zhixue Zhao, Robert GaizauskasLLM ReasoningReasoning Consistency in Language Models

    University of Sheffield, UK

  10. Reasoning or Rhetoric? An Empirical Analysis of Moral Reasoning Explanations in Large Language Models

    Mar 23, 2026Aryan Kasat, Smriti Singh, Aman Chadha +1Moral Reasoning in Language ModelsReasoning Consistency in Language Models

    UT Austin · Google GenAI

  11. Deep reflective reasoning in interdependence constrained structured data extraction from clinical notes for digital health

    Mar 20, 2026Jingwei Huang, Kuroush Nezafati, Zhikai Chi +9Clinical Information ExtractionReasoning Consistency in Language Models

    Quantitative Biomedical Research Center, Department of Health Data Science and Biostatistics, Peter O'Donnell School of Public Health, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390 · Department of Pathology, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390 · Department of Pediatrics, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390

  12. Correct Chains, Wrong Answers: Dissociating Reasoning from Output in LLM Logic

    Mar 19, 2026Abinav Rao, Sujan Rachuri, Nikhil VemuriLLM ReasoningLLM Evaluation

  13. Box Maze: A Process-Control Architecture for Reliable LLM Reasoning

    Mar 19, 2026Zou QiangLLM SafetyReasoning Consistency in Language Models

    1Independent Researcher

  14. Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structures

    Mar 17, 2026Oleg Somov, Mikhail Chaichuk, Gleb Ershov +4LLM ReasoningCausal Interventions in Language Models

    AIRI · MIPT · HSE University +2

  15. Amplification Effects in Test-Time Reinforcement Learning: Safety and Reasoning Vulnerabilities

    Mar 16, 2026Vanshaj Khattar, Md Rafi ur Rashid, Moumita Choudhury +4Test-Time RLLLM Safety

    Virginia Tech · Penn State University · University of Massachusetts Amherst +1

  16. BrainBench: Exposing the Commonsense Reasoning Gap in Large Language Models

    Mar 16, 2026Yuzhe TangCommonsense ReasoningLanguage Model Reasoning Evaluation

    Georgia Institute of Technology

  17. Inclusion-of-Thoughts: Mitigating Preference Instability via Purifying the Decision Space

    Mar 15, 2026Mohammad Reza Ghasemi Madani, Soyeon Caren Han, Shuo Yang +1Multiple-Choice Question AnsweringReasoning Consistency in Language Models

    School of Computing and Information Systems, The University of Melbourne

  18. Compositional Consistency-Guided Decoding for Three-Way Logical Question Answering

    Mar 12, 2026Tianyi Huang, Ming Hou, Jiaheng Su +2Question AnsweringLogical Reasoning

    Ryquo · App-In Club

  19. How Do Language Models Process Ethical Instructions? Deliberation, Consistency, and Other-Recognition Across Four Models

    Mar 11, 2026Hiroki FukuiMoral Reasoning in Language ModelsLLM Safety Alignment

    Research Institute of Criminal Psychiatry / Sex Offender Medical Center · Department of Neuropsychiatry, Kyoto University

  20. CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVR

    Mar 10, 2026Sijia Cui, Pengyu Cheng, Jiajun Song +6RL for Language Model ReasoningReinforcement Learning with Verifiable Rewards

    Qwen Large Model Application Team, Alibaba · Institute of Automation, Chinese Academy of Sciences

  21. Lost in Stories: Consistency Bugs in Long Story Generation by LLMs

    Mar 6, 2026Junjie Li, Xinrui Guo, Yuhao Wu +3Narrative GenerationReasoning Consistency in Language Models

    Microsoft, Beijing, China · Singapore University of Technology and Design

  22. The Fragility Of Moral Judgment In Large Language Models

    Mar 5, 2026Tom van Nuenen, Pratik S. SachdevaMoral Reasoning in Language ModelsPrompt Sensitivity

    D-Lab, University of California, Berkeley, USA

  23. When Shallow Wins: Silent Failures and the Depth-Accuracy Paradox in Latent Reasoning

    Mar 3, 2026Subramanyam Sahoo, Aman Chadha, Vinija Jain +1LLM Mathematical ReasoningReasoning Consistency in Language Models

    Independent · AWS Generative AI Innovation Center, Amazon Web Services · Stanford University +2

  24. Prompt Sensitivity and Answer Consistency of Small Open-Source Language Models for Clinical Question Answering in Low-Resource Healthcare

    Mar 1, 2026Shravani HariprasadClinical Language Model EvaluationPrompt Sensitivity

    Independent Researcher

  25. When the Pure Reasoner Meets the Impossible Object: Analytic vs. Synthetic Fine-Tuning and the Suppression of Genesis in Language Models

    Feb 26, 2026Amin AmouhadiLLM Fine-TuningKnowledge Conflicts in Language Models

    Institute for Artificial Intelligence, University of Georgia

  26. TRUE: A Trustworthy Unified Explanation Framework for Large Language Model Reasoning

    Feb 21, 2026Yujiao YangLLM InterpretabilityFaithfulness of Language Model Explanations

    Dalian University of Technology, Dalian, China

  27. Better Think Thrice: Learning to Reason Causally with Double Counterfactual Consistency

    Feb 18, 2026Victoria Lin, Xinnuo Xu, Rachel Lawrence +4Counterfactual Reasoning in Language ModelsCausal Reasoning in Language Models

    Carnegie Mellon University · Microsoft Research Cambridge · Microsoft Research India

  28. When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents

    Feb 12, 2026Aman MehtaAI Agent ReliabilitySelective Prediction

    Snowflake AI Research

  29. VERGE: Formal Refinement and Guidance Engine for Verifiable LLM Reasoning

    Jan 27, 2026Vikash Singh, Darion Cassel, Nathaniel Weir +2LLM Answer VerificationReasoning Consistency in Language Models

    Case Western Reserve University · Amazon Web Services

  30. Sandwich Reasoning: An Answer-Reasoning-Answer Approach for Low-Latency Query Correction

    Jan 7, 2026Chen Zhang, Kepu Zhang, Jiatong Zhang +2Efficient Language Model ReasoningQuery Reformulation

    Gaoling School of Artificial Intelligence, Renmin University of China