Moral Judgment

Moral judgment research explores how humans and artificial intelligence (AI) systems make ethical decisions, aiming to understand the underlying cognitive processes and develop more ethical AI. Current research focuses on analyzing moral judgments across cultures and languages, investigating the stability of moral preferences over time, and evaluating the alignment of AI models' moral judgments with human values using various methods, including reinforcement learning and contrastive learning. This work is crucial for building more responsible and human-aligned AI systems and for understanding the complex interplay between culture, cognition, and morality.

Papers

August 29, 2023

Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?
Jingyan Zhou, Minda Hu, Junan Li, Xiaoying Zhang, Xixin Wu, Irwin King, Helen Meng
Large Language Model Camera Lens Moral Judgment Machine Ethic Moral Theory Human Moral

June 20, 2023

Towards Theory-based Moral AI: Moral AI with Aggregating Models Based on Normative Ethical Theory
Masashi Takeshita, Rzepka Rafal, Kenji Araki
Model Aggregation Moral Judgment Moral Dilemma Ethical Principle Cultural Commonsense

January 13, 2023

The moral authority of ChatGPT
Sebastian Krügel, Andreas Ostermaier, Matthias Uhl
ChatGPT Generated Conversation Moral Judgment Moral Foundation Moral Value Chat Bot

December 20, 2022

ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations
Valentina Pyatkin, Jena D. Hwang, Vivek Srikumar, Ximing Lu, Liwei Jiang, Yejin Choi, Chandra Bhagavatula
Moral Judgment Moral Dilemma Defeasible Reasoning Clarification Question Ethical Principle Moral Development Auditing Delphi

May 17, 2022

Moral reinforcement learning using actual causation
Tue Herlau
Online Reinforcement Learning Causal Direction Actual Cause Moral Judgment Reinforcement Learning System

April 6, 2022

The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems
Caleb Ziems, Jane A. Yu, Yi-Chia Wang, Alon Halevy, Diyi Yang
New Benchmark Conversational Agent Open Domain Moral Judgment Moral Bias

January 26, 2022

Explainable Patterns for Distinction and Prediction of Moral Judgement on Reddit
Ion Stagkos Efstathiadis, Guilherme Paulino-Passos, Francesca Toni
Human Prediction High Quality Social Medium Site Reddit Moral Judgment Language Pattern POST Exploration Moral Value Interpretable Pattern

January 19, 2022

When Is It Acceptable to Break the Rules? Knowledge Representation of Moral Judgement Based on Empirical Data
Edmond Awad, Sydney Levine, Andrea Loreggia, Nicholas Mattei, Iyad Rahwan, Francesca Rossi, Kartik Talamadupula, Joshua Tenenbaum, Max Kleiman-Weiner
Reasoning Capability Knowledge Representation Simple RULE Empirical Evidence Moral Reasoning Moral Judgment Human Moral

November 7, 2021

A Word on Machine Ethics: A Response to Jiang et al. (2021)
Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, Adina Williams
Artificial Intelligence Non Negative Textual Response Real Text Word Moral Judgment Machine Ethic Et Al

Moral Judgment

Papers

Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?

Towards Theory-based Moral AI: Moral AI with Aggregating Models Based on Normative Ethical Theory

The moral authority of ChatGPT

ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations

Moral reinforcement learning using actual causation

The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems

Explainable Patterns for Distinction and Prediction of Moral Judgement on Reddit

When Is It Acceptable to Break the Rules? Knowledge Representation of Moral Judgement Based on Empirical Data

A Word on Machine Ethics: A Response to Jiang et al. (2021)