Participatory Moral AI Is Not Neutral: The Invisible Hand of Developers

The project of aligning machine behavior with human values raises a basic problem: whose moral expectations should guide AI decision-making? Much alignment research assumes that the appropriate benchmark is how humans themselves would act in a given situation. Studies of agent-type value forks challenge this assumption by showing that people do not always judge humans and AI systems identically.This paper extends that challenge by examining two further possibilities: first, that evaluations of AI behavior change when its human origins are made visible; and second, that people judge the humans who program AI systems differently from either the machines or the human actors they are compared against. An experiment with 1,002 U.S. adults measured moral judgments in a runaway mine train scenario, varying the subject of evaluation across four conditions: a repairman, a repair robot, a repair robot programmed by company engineers, and company engineers programming a repair robot. We find no significant difference in evaluations of the repairman and the robot. However, judgments shifted substantially when the robot's actions were described as the product of human design. Participants exhibited markedly more deontological, rule-based reasoning when evaluating either the programmed robot or the engineers who programmed it, suggesting that rendering human agency visible activates heightened moral constraints. These findings indicate that people may evaluate humans, AI systems acting in the same situation, and the humans who design them in meaningfully different ways. The fact that these evaluations do not necessarily converge gives rise to the alignment target problem: which normative target should guide the development of artificial moral agents in high-stakes domains, and whether these plural judgments can be reconciled within a coherent account of value alignment.

Most work in computational ethics treats annotator disagreement on moral content as noise to be voted away, collapsed into majority vote or the more permissive any-annotator rule the moment a single annotator flags an item. We argue this uncertainty should instead be modeled and learned from. We introduce Moral Entropy, a Bayesian framework that keeps a full posterior over the true label and decomposes its entropy into aleatoric uncertainty (irreducible disagreement about the moral content) and epistemic uncertainty (from insufficient or noisy annotation) -- and lets any heuristic consensus rule be audited against a calibrated ground truth via entropy methods such as cross-entropy/KL, Brier score, and expected calibration error. Across three corpora and fifteen discourse domains, auditing the standard aggregation rules against this posterior reveals bias that no current pipeline reports: the any-annotator rule disagrees with the calibrated posterior on roughly 30% of items -- pooled, almost entirely false positives, though the errors invert at the foundation level (19.9%/38.9% mean FPR/FNR on MFTC) -- while the stricter majority and two-vote rules miss 63-83% of true positives.

This exploratory observational study uses the Transluce platform to analyze the internal activity of Llama 3.1-8B-Instruct on 54 moral prompts in four batteries: dilemmas, policy, and meta-ethical questions (B1, 17); role-playing scenarios (B3, 6); and a structured trolley comparison varying the switching mechanism (B4, 15) or the people's identity attributes (B5, 16) with the other fixed. We record each surfaced neuron's normalized activation score, identity, and description and each cluster's title and size, classify them by an explicit description rule, and measure the prevalence and ranking of ethics-labeled neurons and clusters, the neurons' activation scores, and the model's verdicts and stated confidence. Prevalence is similar across batteries (about 5% of retained neuron rows) while ranking differs: in the trolley batteries the first ethics-labeled row has median position 3, versus 9 in B1 and B3, ethics labels are about four times as frequent among the ten highest-scoring rows as overall, and ethics-labeled rows have a higher mean activation score than other rows within prompts (47 of 54). We observe situational anchoring: prompt-setting vocabulary appears in prominent cluster titles and selected high-ranked neuron descriptions. B4-vs-B5 First Rank and Priority Ratio contrasts detect no difference. With one retained response per prompt, the coded outcomes for the 31 trolley responses are 19 recommending intervention, one recommending non-intervention, and 11 giving no definite recommendation. We distinguish prevalence, ranking, behavior, and causal contribution as audit targets, measure the first three, offer frame-conditioned moral computation as an organizing hypothesis with stated tests, and propose mechanistic alignment, causal responsibility of ethically relevant features under controlled frame variation, as a criterion admitting distributed implementations.