Moral Reasoning in Language Models

Follow this topic Ask about this topic

The field examines how language models recognize moral stakes, choose between competing values, and apply judgments in settings such as trolley dilemmas, clinical resource allocation, and interpersonal advice. Researchers compare model decisions with human judgments and use multilingual, counterfactual, and multi-turn adversarial tests to probe cultural variation and robustness; methods range from prompting and fine-tuning to multi-agent debate and mechanistic analysis of internal representations. Models can produce persuasive ethical explanations, but their choices may shift with framing, language, social pressure, or conversation history, and explanations do not always predict behavior. A central challenge is improving consistency and resilience without imposing one narrow moral standard across culturally diverse people and situations.

Latest papers

114 papers9 papers in the last four weeks, down 25% on the four weeks before. 0.1% of all new papers.

MedConstBench: Benchmarking How Constitutions Steer LLMs in Medical Ethics
Oct 27, 2026Yibo Yang, Zerui Huang, Haomiao Yang +1Medical LLMs Moral Reasoning in Language Models
Independent Researcher, China · Xinjin District People’s Hospital, China · University of Exeter, United Kingdom
Consistency Is Not Validity: Self-Rewarded Moral Argumentation Makes Language Models Decisive but Pliable
Oct 21, 2026Marius De Kuthy Meurers, Aaron Parisi, Bernd BohnetMoral Reasoning in Language Models Reasoning Consistency in Language Models
Google DeepMind
Formal Raising: Embedding Verifiable Ethical Properties into the Development of AI
Oct 18, 2026Rolf Drechsler, Christina PlumpAI Ethics AI Alignment
Moral Combat: Exploiting Moral Foundations Theory to Jailbreak LLMs
Oct 14, 2026Adel Khorramrouz, Yi-Chia Wang, Sharon LevyAdversarial Attacks on LLMs LLM Safety Evaluation
Rutgers University · Carnegie Mellon University
Rule vs.Consequence: Dissociable Internal Representations of Moral Reasoning in LLMs
Oct 12, 2026Eugenie ShiMoral Reasoning in Language Models LLM Interpretability
Department of Computer Science Stanford University Stanford, CA 94305, USA
Justice After Identity: Large Language Models and the View from Everywhere
Oct 6, 2026W. Russell NeumanMoral Reasoning in Language Models Moral Reasoning
Principled Under Pressure: Post-Training Decides Whether LLMs Act on Their Own Moral Judgment
Oct 6, 2026Orion Reblitz-RichardsonMoral Reasoning in Language Models Language Model Post-Training
Distiller Labs
OMIT the Action: Measuring Framing-Invariant Omission Bias under Philosophical Disagreement
Oct 6, 2026Sihyeon Lee, Jihun Song, Chanwoo Kim +2Moral Reasoning in Language Models Framing Effects in Language Models
Soongsil University
IREA: Intermediate Representation-based Embedding Alignment for Normative RAG
Oct 4, 2026Mirae Han, Sihyeong Yeom, Harksoo KimRetrieval-Augmented Generation Moral Reasoning in Language Models
Konkuk University · NAVER Cloud
Echoes of Deeds: Moral History Can Shape and Steer LLM Behavioral Choices
Sep 28, 2026Lucio La Cava, Andrea TagarelliMoral Reasoning in Language Models LLM Auditing
DIMES Dept., University of Calabria, Italy
Steering LLMs Responses Towards Moral Foundations on the Norwegian MFQ-30
Sep 18, 2026Hans Andersen, David DichasLanguage Model Steering Moral Reasoning in Language Models
Talking Past the Machine: Morality, Politeness, and Alignment in Human-AI Dialogue
Sep 18, 2026Marina Mitiaeva, Lu XiaoHuman-AI Interaction Moral Reasoning in Language Models
Arizona State University
Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Models
Sep 17, 2026Utkarsh Agarwal, Monojit ChoudhuryLLM Alignment Moral Reasoning in Language Models
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE.
Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Families
Sep 13, 2026Orion Reblitz-RichardsonLLM Alignment Moral Reasoning in Language Models
Distiller Labs.
FaithfulBench: Does AI Counsel Uphold or Undermine the User's Professed Faith?
Sep 12, 2026M Waleed Kadous, Benjamin Olsen, Walter Scheirer +3Moral Reasoning in Language Models Generative AI Evaluation
Islamic Alliance for Safe Ethical Responsible AI · Faith Family Technology Network · University of Notre Dame +2
Human-like moral judgments conceal divergent motive attributions in large language models
Sep 7, 2026Xiaoyan Wu, Jean-Claude DreherLLM Evaluation Moral Reasoning in Language Models
Department of Adult Psychiatry and Psychotherapy, University of Zurich, Zurich, Switzerland · Institut des Sciences Cognitives Marc Jeannerod, CNRS UMR 5229, Lyon, France
Measuring AI Accountability Through Argumentation Analysis: Can Model Reasoning Withstand Scrutiny?
Sep 4, 2026Daan R. Henselmans, Derck W. E. Prinzhorn, Arno LibertAI Accountability Language Model Auditing
Aithos Research Foundation, Keizersgracht 62, Amsterdam, 1015 CS, the Netherlands.
How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions
Sep 4, 2026Fernanda Mansilla, Aloysius Tok, Bahia Guellaï +2Moral Reasoning in Language Models Human-AI Decision Making
IRIT, Université de Toulouse, France and IPAL, Singapore · Institute for Infocomm Research (I2R), A*STAR, Singapore
Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment
Sep 4, 2026Arno Libert, Derck W. E. Prinzhorn, Daan R. HenselmansLLM Safety Alignment Behavioral Consistency of LLM Agents
Aithos Research Foundation, Keizersgracht 62, Amsterdam, 1015 CS, the Netherlands.
HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals
Sep 3, 2026Jasmine Brazilek, Miles Tidmarsh, Matthias Endres +2Moral Reasoning in Language Models LLM Safety Alignment
Compassion Aligned Machine Learning (CaML) · Department of Economics, University of Warwick
Representational alignment yields generalizable safety in language models
Sep 3, 2026Lingyu Li, Yan Teng, Yingchun Wang +1Moral Reasoning in Language Models LLM Safety Alignment
Shanghai Artiﬁcial Intelligence Laboratory, Shanghai, China
Caught in the Story: Narrative Captivity in Multi-turn LLMs Conversation
Sep 3, 2026Yuhe Wu, Guangyu Wang, Yujie Chen +7Moral Reasoning in Language Models LLM Reliability
1The Hong Kong University of Science and Technology (Guangzhou) · 2The Chinese University of Hong Kong, Shenzhen · 3Dongbei University of Finance and Economics
Rare Diseases, Common Dilemmas: LLMs Prioritize Equal Resource Distribution over Patient Benefit in Decision-Making
Aug 25, 2026Minda Zhao, Xu Han, Rishabh Goel +6Moral Reasoning in Language Models Clinical Decision-Making
Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA · Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA · Clalit Health Services, Tel Aviv, Israel +2
Incoherent by Design? On the Moral Self-Consistency of LLMs
Aug 15, 2026Pegah Nokhiz, Aravinda Kanchana Ruwanpathirana, Helen NissenbaumMoral Reasoning in Language Models Reasoning Consistency in Language Models
Cornell University, Cornell Tech · Cornell Tech · Nanyang Technological University
Synthetic Persona Pretraining: Alignment from Token Zero
Aug 13, 2026Julian Minder, Viktor Moskvoretskii, Raghav Singhal +12Synthetic Data Pretraining LLM Alignment
EPFL · SJTU · Northeastern University +8
Follow the Norm: Accounting for Fine-Tuning and Prompt Effects on Model Rationales
Aug 13, 2026Long Hoang Nguyen, Brice Valentin Kok-Shun, Guangyu Du +1Moral Reasoning in Language Models Normative Conformity in Language Models
1Technical University of Munich · University of Auckland
Toward Human Rights Benchmarking for LLMs: A Pilot Methodology
Aug 10, 2026Savannah Thais, Wm. Matthew Kennedy, Abhigyan Acherjee +3LLM Evaluation Moral Reasoning in Language Models
Hunter College, New York, USA · Oxford Internet Institute, University of Oxford, UK · King’s College London, UK +2
CORDA: A Benchmark for Hierarchical Harm-Centric Moral Reasoning in Large Language Models
Aug 8, 2026Siddarth Singh, Victoria Williams, Simon Rosen +6LLM Evaluation Moral Reasoning in Language Models
University of the Witwatersrand, Johannesburg, South Africa
Contextual Value Alignment via Multilayer Combinatorial Fusion
Aug 7, 2026Yuanhong Wu, Djallel Bouneffouf, D. Frank HsuLLM Alignment Moral Reasoning in Language Models
Dept. of Computer and Information Science, Fordham University, New York, NY, USA · T. J. Watson Research Center, IBM Research, Yorktown Heights, NY, USA
The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisions
Aug 6, 2026Hadi Hosseini, Samarth Khanna, Leona PierceMoral Reasoning in Language Models Clinical Decision-Making
Pennsylvania State University

Oct 27, 2026J/K move · Enter open · S save

MedConstBench: Benchmarking How Constitutions Steer LLMs in Medical Ethics

Yibo Yang, Zerui Huang, Haomiao Yang, Zixia Wang

Independent Researcher, China · Xinjin District People’s Hospital, China · University of Exeter, United Kingdom

Medical LLMs Moral Reasoning in Language Models LLM Safety Alignment

Open paper AskPDF arXiv

As large language models advance rapidly, they are increasingly used to answer medical questions, and many such models are now deployed locally. Once a model is deployed, its behavior is governed mainly by the ethical policy written into its constitution. However, most existing work focuses on factual correctness rather than the trade-off between competing ethical values. Even benchmarks that do evaluate ethics treat it as a fixed property of the model rather than as a variable, so how a constitution reshapes medical-ethics behavior remains largely unexplored. We introduce MedConstBench, a medical-ethics benchmark that treats the deployment constitution as a controlled variable, is organized around Beauchamp's four principles, and is scored by human-calibrated multi-model peer review. It includes MedDilemma, our own dataset of clinical dilemmas, and supports a systematic analysis of how constitutions steer LLMs. Evaluating ten open-source LLMs under four constitutions, we find that the effect of a constitution is dimension-selective and model-dependent, and that no constitution is universally optimal. Motivated by these findings, we propose a do-no-harm adaptive selector that chooses a constitution per model, matching the per-model oracle while guaranteeing a worst-case floor and generalizing to unseen models at low cost.