Talking Past the Machine: Morality, Politeness, and Alignment in Human-AI Dialogue

Marina Mitiaeva; Lu Xiao

cs.CLSep 18, 2026

Talking Past the Machine: Morality, Politeness, and Alignment in Human-AI Dialogue

Authors: Marina Mitiaeva, Lu Xiao

Organizations: Arizona State University

Abstract

Conversational AI systems produce fluent, socially appropriate responses, yet whether they participate in cooperative communication or merely simulate its surface forms remains unclear - a question central to how these systems are evaluated, trusted, and designed. This study investigates how morality, politeness, and alignment - three dimensions central to cooperative dialogue - function in human-AI interaction compared to human-human conversation. We analyze 15,881 human-ChatGPT and 10,784 human-human multi-turn dialogues, using mixed-effects models to identify which features predict turn-to-turn alignment. We observe a consistent dissociation: AI produces the surface features of cooperative communication without the underlying social architecture. Moral output appears preconfigured rather than negotiated; warmth is generated without face sensitivity; linguistic convergence declines persistently. Most strikingly, the cooperative mechanisms themselves reverse direction: hedging and softening associated with greater accommodation between humans are associated with reduced alignment when produced by AI, and purity framing associated with human divergence coincides with users converging toward the AI. Agency - giving users room to shape the exchange - is the most consistent predictor of alignment across both interaction types, while lower moral assertiveness in more recent models is not accompanied by better cooperation. Together these patterns suggest that AI reproduces the surface of cooperation without the mutual adaptation that grounds it between humans - and, more surprisingly, that mechanisms sustaining human accommodation can run in reverse with AI, suggesting a turn-level view may be insufficient for interaction-level success.

Figures & tables

	Human–AI		Human–Human
	Initial	Cleaned	Initial	Cleaned
Conversations	15,882	15,881	10,784	10,784
Utterances	476,198	473,884	235,281	227,704
Avg. turns/conv.	15	16	22	21
Avg. words (resp.)	1,718	1,008	107	63
Avg. words (init.)	1,157	516	102	60

Table 1: Descriptive statistics for the WildChat (human–AI) and Topical-Chat (human–human) corpora before and after preprocessing.

	Human–AI		Human–Human
	M (SD)	Z (r)	M (SD)	Z (r)
Morality
Care/Harm	0.029 (0.118)	28.48 (.226) ∗∗∗	0.001 (0.056)	2.29 (.022) ∗
Fairness/Cheating	0.030 (0.094)	46.10 (.366) ∗∗∗	0.002 (0.064)	2.78 (.027) ∗∗
Loyalty/Betrayal	0.052 (0.114)	60.92 (.483) ∗∗∗	0.003 (0.064)	4.60 (.044) ∗∗∗
Authority/Subversion	0.046 (0.115)	56.79 (.451) ∗∗∗	0.001 (0.059)	1.54 (.015)

Table 2: Morality and politeness asymmetry across HAI and HH conversation. M (SD) = mean per-conversation difference. Z = Wilcoxon signed-rank. r = rank-biserial. BH corrected.

{}^{*}p<.05

;

{}^{**}p<.01

;

{}^{***}p<.001

Figure 1: Moral intensity and alignment across conversation arc by speaker role.

	HAI: User		HAI: Asst.
	%+/ $-$	Z (r)	%+/ $-$	Z (r)
Lexical	48/50	2.06 (.016) ∗	43/56	$-$ 16.30 ( $-$ .130) ∗∗∗
Syntactic	50/49	0.39 (.003)	48/52	$-$ 6.49 ( $-$ .052) ∗∗∗
Pragmatic	45/49	$-$ 4.36 ( $-$ .036) ∗∗∗	46/49	$-$ 4.98 ( $-$ .041) ∗∗∗
Emotional	47/53	$-$ 8.41 ( $-$ .067) ∗∗∗	43/57	$-$ 21.06 ( $-$ .167) ∗∗∗
Overall	48/52	$-$ 6.32 ( $-$ .050) ∗∗∗	44/56	$-$ 16.70 ( $-$ .133) ∗∗∗

Table 3: Alignment dynamics across speaker roles. %+/

-

= conversations with positive/negative slopes. Z = Wilcoxon. r = rank-biserial. BH corrected.

{}^{*}p<.05

;

{}^{**}p<.01

;

{}^{***}p<.001

Figure 2: Dynamics across GPT generations. GPT-3.5 shows the highest moral intensity yet strongest accommodation; GPT-4 Turbo the steepest alignment decline; GPT-4o the lowest structure rate and weakest alignment trajectory.

	HAI $\beta$ (SE)	HH $\beta$ (SE)
Morality
Care/Harm	.042 (.034)	.011 (.044)
Fairness	$-$ .097 (.037) ∗∗	$-$ .070 (.035) ∗
Loyalty	$-$ .003 (.032)	.007 (.039)
Authority	$-$ .135 (.032) ∗∗∗	$-$ .194 (.038) ∗∗∗
Sanctity	.160 (.028) ∗∗∗	$-$ .113 (.038) ∗∗

Table 4: Mixed effects regression: Predictors of next-turn alignment.

\beta

= unstandardized coefficient. HAI

n{=}218{,}148

turns (15,882 conv.); HH

n{=}103{,}067

turns (10,784 conv.).

{}^{*}p<.05

;

{}^{**}p<.01

;

{}^{***}p<.001

	HAI		HH
	Auth.	Pur.	Auth.	Pur.
Total ( $c$ )	$-$ .017 ∗∗∗	.107 ∗∗∗	$-$ .245 ∗∗∗	$-$ .177 ∗∗∗
Direct ( $c^{\prime}$ )	$-$ .112 ∗∗∗	.112 ∗∗∗	$-$ .236 ∗∗∗	.159 ∗∗∗
Indirect:
Modul.	.002 ∗∗	.004 ∗∗	$-$ .002 ∗	$-$ .001
Agency	$-$ .004 ∗∗	$-$ .011 ∗∗∗	$-$ .004 ∗∗	$-$ .007 ∗∗

Table 5: Multiple mediation: Authority and purity as exposures. BH corrected.

{}^{*}p<.05

;

{}^{**}p<.01

;

{}^{***}p<.001

Human-AI Interaction

Human-AI interaction studies how people and AI systems communicate, collaborate, make decisions, and shape one another over time.

See all 330 papers

Interaction Protocol Shapes Moral Judgment in Multi-Agent Debate
Oct 11, 2025Pratik S. Sachdeva, Tom van NuenenLLM Decision-Making Moral Reasoning in Language Models
D-Lab, University of California, Berkeley
Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment
Sep 4, 2026Arno Libert, Derck W. E. Prinzhorn, Daan R. HenselmansLLM Safety Alignment Behavioral Consistency of LLM Agents
Aithos Research Foundation, Keizersgracht 62, Amsterdam, 1015 CS, the Netherlands.
Reputation, Strategy, and Emotion Effects on Generative AI Cooperation: A Comparison Across Reasoning and Non-Reasoning Models
Oct 1, 2026Celso de Melo, Zishan Feng, James Hale +3Cooperative Game Theory Large Language Model-Based Social Simulation
DEVCOM Army Research Laboratory, USA · Department of Electrical, Electronic and Computer Engineering, Gifu University, Japan · Department of Computer Science, University of Southern California, USA +1

Oct 11, 2025J/K move · Enter open · S save

Interaction Protocol Shapes Moral Judgment in Multi-Agent Debate

Pratik S. Sachdeva, Tom van Nuenen

D-Lab, University of California, Berkeley

LLM Decision-Making Moral Reasoning in Language Models Multi-Agent Debate LLM Alignment Multi-Agent Systems Moral Reasoning+1

Open paper AskPDF arXiv

As large language models (LLMs) are increasingly deployed in sensitive everyday contexts -- offering personal advice, mental health support, and moral guidance -- understanding their behavior in navigating complex moral reasoning is essential. Most evaluations study this sociotechnical alignment through single-turn prompts, but it is unclear if these findings extend to multi-turn settings, and even less clear how they depend on the interaction protocols used to coordinate agentic systems. We address this gap using LLM debate to examine deliberative dynamics and value alignment in multi-turn settings by prompting subsets of three models (GPT-4.1, Claude 3.7 Sonnet, and Gemini 2.0 Flash) to collectively assign blame in 1,000 everyday dilemmas from Reddit's ``Am I the Asshole'' community. To test order effects and assess verdict revision, we use both synchronous (parallel responses) and round-robin (sequential responses) deliberation structures, mirroring how multi-agent systems are increasingly orchestrated in practice. Our findings show striking behavioral differences. In the synchronous setting, GPT-4.1 showed strong inertia (0.6-3.1% revision rates) while Claude 3.7 Sonnet and Gemini 2.0 Flash were far more flexible (28-41% revision rates). Value patterns also diverged: GPT-4.1 emphasized personal autonomy and direct communication (relative to its deliberation partners), while Claude 3.7 Sonnet and Gemini 2.0 Flash prioritized empathetic dialogue. We further find that deliberation format had a strong impact on model behavior: GPT-4.1 and Gemini 2.0 Flash stood out as highly conforming relative to Claude 3.7 Sonnet, with their verdict behavior strongly shaped by order effects. We provide additional results on open-source models (DeepSeek-V3.2 and Llama 3.1).

Abstract

Human-AI Interaction

Explore similar work

Interaction Protocol Shapes Moral Judgment in Multi-Agent Debate

Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment

Reputation, Strategy, and Emotion Effects on Generative AI Cooperation: A Comparison Across Reasoning and Non-Reasoning Models