Reasoning Consistency in Language Models
Overview
The field studies whether language models preserve conclusions and sound reasoning when prompts are paraphrased, evidence changes, answers are resampled, or conversations continue. Evaluations use controlled reformulations and multi-turn tests, repeated-sample agreement, and checks of reasoning traces with formal solvers, causal interventions, or evidence-grounded verifiers across math, logic, coding, and medical tasks. Methods include self-consistency and uncertainty-based answer selection, consistency-aware training and rewards, and test-time verification, repair, or early-exit systems. A central tension is that consistency can signal stability without correctness or faithful reasoning: models may repeat the same error, abstain, or agree while relying on invalid reasoning, so reliability must be assessed alongside accuracy, faithfulness, and cost.
Latest papers
123 papers24 papers in the last four weeks, up 60% on the four weeks before. 0.1% of all new papers.