Moral Decision

Research on moral decision-making in artificial intelligence focuses on developing algorithms that enable AI systems to make ethically sound choices, mirroring or exceeding human capabilities. Current efforts explore reinforcement learning architectures augmented with "moral shields" based on normative reasons, and leverage large language models trained on diverse datasets to analyze and potentially mitigate biases in moral judgments. A key challenge lies in the lack of a universally accepted mathematical framework for moral reasoning, leading to investigations into the interpretability of AI's moral decision-making processes and the use of cognitive models to simulate human moral behavior for training purposes. This research is crucial for ensuring the responsible development and deployment of AI in high-stakes domains where ethical considerations are paramount.

Papers

November 11, 2024

Large-scale moral machine experiment on large language models
Muhammad Shahrul Zaim bin Ahmad, Kazuhiro Takemoto
Large Language Model Human Moral Moral Machine Moral Decision

November 6, 2024

Evaluating Moral Beliefs across LLMs through a Pluralistic Framework
Xuelin Liu, Yanfei Zhu, Shucheng Zhu, Pengyuan Liu, Ying Liu, Dong Yu
Medical LLM Moral Reasoning Moral Value Pluralistic Alignment Moral Decision

October 9, 2024

The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making
Basile Garcia, Crystal Qian, Stefano Palminteri
LLM Alignment LLM Evaluation Moral Dilemma Artificial Intelligence Bias Machine Perception Moral Machine Moral Decision

September 23, 2024

Acting for the Right Reasons: Creating Reason-Sensitive Artificial Moral Agents
Kevin Baum, Lisa Dargasz, Felix Jahn, Timo P. Gros, Verena Wolf
Moral Judgment Moral Agent Reinforcement Learning Architecture Shield Synthesis Moral Decision

July 10, 2024

Why should we ever automate moral decision making?
Vincent Conitzer
Moral Decision

July 2, 2024

Language Model Alignment in Multilingual Trolley Problems
Zhijing Jin, Max Kleiman-Weiner, Giorgio Piatti, Sydney Levine, Jiarui Liu, Fernando Gonzalez, Francesco Ortu, András Strausz, Mrinmaya Sachan, Rada Mihalcea, Yejin Choi, Bernhard Schölkopf
Language Model Human Moral Moral Machine Multilingual Reasoning Moral Decision

October 23, 2023

Moral Foundations of Large Language Models
Marwa Abdulhai, Gregory Serapio-Garcia, Clément Crepy, Daria Valter, John Canny, Natasha Jaques
Moral Foundation Human Moral Moral Decision

July 2, 2023

Minimum Levels of Interpretability for Artificial Moral Agents
Avish Vijayaraghavan, Cosmin Badea
Artificial Intelligence Inherent Interpretability Interpretable AI Moral Agent Minimum Number Moral Decision

January 20, 2023

Modeling Moral Choices in Social Dilemmas with Multi-Agent Reinforcement Learning
Elizaveta Tennant, Stephen Hailes, Mirco Musolesi
Multi Agent Reinforcement Learning Reward Function Social Dilemma Moral Agent Moral Decision

October 8, 2022

Cognitive Models as Simulators: The Case of Moral Decision-Making
Ardavan S. Nobandegani, Thomas R. Shultz, Irina Rish
Reinforcement Learning Case Relevance Interactive Simulation Cognitive Model Dilemma Game Moral Decision