Power Echoes: Investigating Moderation Biases in Online Power-Asymmetric Conflicts

Yaqiong Li; Peng Zhang; Peixu Hou; Kainan Tu; Guangping Zhang; Shan Qu; Wenshi Chen; Yan Chen

cs.HCMar 2, 2026

Power Echoes: Investigating Moderation Biases in Online Power-Asymmetric Conflicts

Authors: Yaqiong Li, Peng Zhang, Peixu Hou, Kainan Tu, Guangping Zhang, Shan Qu, Wenshi Chen, Yan Chen,

Organizations: Fudan University, Shanghai, China · Meituan, Shanghai, China · Virginia Tech, Virginia, USA

Abstract

Online power-asymmetric conflicts are prevalent, and most platforms rely on human moderators to conduct moderation currently. Previous studies have been continuously focusing on investigating human moderation biases in different scenarios, while moderation biases under power-asymmetric conflicts remain unexplored. Therefore, we aim to investigate the types of power-related biases human moderators exhibit in power-asymmetric conflict moderation (RQ1) and further explore the influence of AI's suggestions on these biases (RQ2). For this goal, we conducted a mixed design experiment with 50 participants by leveraging the real conflicts between consumers and merchants as a scenario. Results suggest several biases towards supporting the powerful party within these two moderation modes. AI assistance alleviates most biases of human moderation, but also amplifies a few. Based on these results, we propose several insights into future research on human moderation and human-AI collaborative moderation systems for power-asymmetric conflicts.

Figures & tables

Power Type	Description of Power Type	Power Manifestation	Description of Power Manifestation
Legitimacy power	It comes from an elected, selected, or appointed position of authority and may be underpinned by social norms or authority citation	Legitimate claim	Strengthening one’s position by citing rules (e.g., "This is the platform’s rule" )
		Authority citation	Supporting one party by referring to external institutions or sources (e.g., "please visit http : / /…" )
Coercive power	It uses the threat of force to gain compliance from another	Punishment threat	Forcing the other party to give in by threats, complaints, or negative consequences (e.g., “I’m going to complain about you” )
Reward power	It is based on the right to offer or deny tangible, social, emotional, or spiritual rewards	Compensation	Offering benefits to win support (e.g., “You can get a full refund” )
Expert power	It is based on knowledge, experience, and specialized skills	Expert knowledge	Supporting one party by providing professional knowledge (e.g., “The quality meets the required standard” )
Referent power	It is based on respect or admiration leading to identification or emulation	Group preference	Supporting one party by referring to group consensus (e.g., “Everyone says the service is bad” )

Table 1. Ten forms of power manifestation in online power-asymmetric conflicts.

Figure 1. The flow of experimental design. The figure illustrates the flow of experimental design, which includes four components: data construction, participant recruitment, the judgment task, and the post-survey.

Initial Conflict Sample

Consumer’s review : So this new “internet-famous” bakery just opened, and guess what? They’re already doing that whole “only a few people inside at a time” thing. On top of that, you have to pay separately, which is super annoying. Honestly, the bread isn’t even that great, definitely not worth the price. And now it looks like they’re even paying people to write fake positive reviews.

Merchant’s response : During our grand opening, we offered special promotions and the turnout was much bigger than expected. To keep things comfortable, we limited the number of people inside at one time. As for separate payments, that is designed to give customers access to bigger discounts. We never thought it would be taken as “hype marketing”.

Table 2. The initial and corresponding perturbed conflict samples (all samples are translated from Chinese for understanding).

Figure 2. The design of “I Support” program (for presentation convenience, all descriptions are translated into English). The figure illustrates the design of “I Support” program. The left panel shows the four components of the program: the task description, the conflict sample, the AI-generated suggestion (visible only to the human-AI moderation group), and the user choice. The right panel outlines the three steps of the Wizard-of-Oz procedure: retrieving the reference of the initial case from the conflict database, randomly selecting one of three strategies to present the reason, and presenting the reference label with the selected reason.

Power Manifestation	Human moderation group	Human–AI moderation group	$\Delta$ Percentage
Initial version	224 (12.80%)	227 (12.97%)	+0.17%
Legitimate claim	160 (9.14%)	156 (8.91%)	-0.23%
Authority citation	161 (9.20%)	159 (9.09%)	-0.11%
Punishment threat	165 (9.43%)	162 (9.26%)	-0.17%
Compensation	159 (9.09%)	163 (9.31%)	+0.22%
Expert knowledge	156 (8.91%)	158 (9.03%)	+0.12%

Table 3. Distribution of completed samples between two groups under different perturbation conditions.

Figure 3. Choice variations without AI (initial version vs. perturbed version). For presentation convenience, “Extremely support the merchant”, “Support the merchant”, “Neutral”, “Support the consumer”, and “Extremely support the consumer” are abbreviated as “ESM”, “SM”, “N”, “SC”, and “ESC” respectively. The figure is a heatmap showing the choice variations without AI (initial version vs. perturbed version) across five choices. For presentation convenience, ``Extremely support the merchant'', ``Support the merchant'', ``Neutral'', ``Support the consumer'', and ``Extremely support the consumer'' are abbreviated as ``ESM'', ``SM'', ``N'', ``SC'', and ``ESC'', respectively. Each row represents a version (e.g., legitimate claim, authority citation, punishment threat), and each cell shows the percentage point increase (blue) or decrease (orange) compared to the initial version, with the intensity of shading indicating the magnitude of change. — Figure 3. Choice variations without AI (initial version vs. perturbed version). For presentation convenience, “Extremely support the merchant”, “Support the merchant”, “Neutral”, “Support the consumer”, and “Extremely support the consumer” are abbreviated as “ESM”, “SM”, “N”, “SC”, and “ESC” respectively. The figure is a heatmap showing the choice variations without AI (initial version vs. perturbed version) across five choices. For presentation convenience, `Extremely support the merchant'',` Support the merchant'', `Neutral'',` Support the consumer'', and `Extremely support the consumer'' are abbreviated as` ESM'', `SM'',` N'', `SC'', and` ESC'', respectively. Each row represents a version (e.g., legitimate claim, authority citation, punishment threat), and each cell shows the percentage point increase (blue) or decrease (orange) compared to the initial version, with the intensity of shading indicating the magnitude of change.

Figure 4. Choice variations without AI (initial version vs. perturbed version under two parties, using the same representations as Figure 3). The figure is a heatmap showing the choice variations without AI under two parties (initial version vs. perturbed version, using the same representations as Figure 3). For presentation convenience, ``dir=c'' and ``dir=m'' denote perturbation on the consumer party and perturbation on the merchant party, respectively. — Figure 4. Choice variations without AI (initial version vs. perturbed version under two parties, using the same representations as Figure 3). The figure is a heatmap showing the choice variations without AI under two parties (initial version vs. perturbed version, using the same representations as Figure 3). For presentation convenience, `dir=c'' and` dir=m'' denote perturbation on the consumer party and perturbation on the merchant party, respectively.

Figure 5. Choice variations with AI (initial version vs. perturbed version, using the same representations as Figure 3). The figure is a heatmap showing the choice variations with AI (initial version vs. perturbed version, using the same representations as Figure 3).

Figure 6. Choice variations with AI under two parties(initial version vs. perturbed version, using the same representations as Figure 4). The figure is a heatmap showing the choice variations with AI under two parties (initial version vs. perturbed version, using the same representations as Figure 4).

Figure 7. Choice variations across three perspectives of reasons in AI-generated suggestions (initial version vs. perturbed versions, using the same representations as Figure 3). The figure contains three panels comparing the choice variations across three perspectives of reasons in AI-generated suggestions (initial version vs. perturbed versions, using the same representations as Figure 3). Each panel represents a different reason perspective, including supporting the current party, unsupporting the opposite party, and mixed reasons for both parties. It shows the percentage change of all perturbed versions compared to the initial version.

Figure 8. Choice variations with AI compared to without AI under different perturbed versions (using the same representations as Figure 3). This figure contains eleven bar charts showing the choice variations with AI compared to without AI under different perturbed versions. Each subplot corresponds to one version (e.g., legitimate claim, authority citation, punishment threat) and shows the percentage point increase (blue) or decrease (orange) relative to human moderation. For presentation convenience, ``Extremely support the merchant'', ``Support the merchant'', ``Neutral'', ``Support the consumer'', and ``Extremely support the consumer'' are abbreviated as ``ESM'', ``SM'', ``N'', ``SC'', and ``ESC'' respectively. Positive and negative values are encoded both through bar direction and text labels to ensure accessibility without reliance on color. — Figure 8. Choice variations with AI compared to without AI under different perturbed versions (using the same representations as Figure 3). This figure contains eleven bar charts showing the choice variations with AI compared to without AI under different perturbed versions. Each subplot corresponds to one version (e.g., legitimate claim, authority citation, punishment threat) and shows the percentage point increase (blue) or decrease (orange) relative to human moderation. For presentation convenience, `Extremely support the merchant'',` Support the merchant'', `Neutral'',` Support the consumer'', and `Extremely support the consumer'' are abbreviated as` ESM'', `SM'',` N'', `SC'', and` ESC'' respectively. Positive and negative values are encoded both through bar direction and text labels to ensure accessibility without reliance on color.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Power Manifestation	Human moderation group				Human-AI moderation group
Power Manifestation	sd	t	p	$d_{z}$	sd	t	p	$d_{z}$
Legitimate claim	1.181	2.284	0.025(*)	0.237	0.884	5.077	0 (***)	0.521
Authority citation	1.346	1.897	0.061	0.194	0.972	3.08	0.003(**)	0.309
Punishment threat	1.12	4.375	0(***)	0.461	1.058	2.015	0.047(*)	0.205
Compensation	1.303	3.759	0(***)	0.380	1.008	3.718	0.001(**)	0.358
Expert knowledge	1.094	2.357	0.021(*)	0.243	1.024	1.233	0.22	0.125

Appendix

Table 4. Between-group paired t -test results for the human and human-AI moderation groups. *** indicates

p<0.001

, ** indicates

p<0.01

, and * indicates

p<0.05

. Cohen’s

d_{z}

reports the effect size on the dependent variables.

Power Manifestation	Younger group				Older group
Power Manifestation	sd	t	p	$d_{z}$	sd	t	p	$d_{z}$
Legitimate claim	1.181	3.224	0.002(**)	0.423	1.284	1.382	0.176	0.234
Authority citation	1.443	0.809	0.422	0.111	1.366	2.088	0.043(*)	0.322
Punishment threat	1.153	4.045	0(***)	0.572	1.136	2.532	0.016(*)	0.416
Compensation	1.303	1.982	0.052	0.263	1.284	3.169	0.003(**)	0.478
Expert knowledge	1.421	2.226	0.030 (*)	0.297	1.225	1.847	0.072	0.288

Appendix

Table 5. Paired t -test results between the initial and perturbed versions across age groups (using the same representations as Table 4).

Power Manifestation	Male group				Female group
Power Manifestation	sd	t	p	$d_{z}$	sd	t	p	$d_{z}$
Legitimate claim	1.27	1.147	0.256	0.152	1.212	3.236	0.003(**)	0.518
Authority citation	1.327	2.234	0.030(*)	0.301	1.533	0.866	0.392	0.135
Punishment threat	1.176	2.418	0.020(*)	0.353	1.209	3.125	0.004(**)	0.514
Compensation	1.406	3.289	0.002(**)	0.411	1.212	1.071	0.291	0.174
Expert knowledge	1.319	2.785	0.007(**)	0.372	0.901	1.387	0.173	0.217

Appendix

Table 6. Paired t -test results between the initial and perturbed versions across gender groups (using the same representations as Table 4).

Power Manifestation	sd	t	p	$d_{z}$
Initial version	0.76	-0.449	0.654	-0.045
Legitimate claim	0.815	-0.523	0.602	-0.056
Authority citation	0.979	0.106	0.916	0.011
Punishment threat	0.886	2.520	0.014 (*)	0.270
Compensation	0.932	1.626	0.107	0.164
Expert knowledge	0.839	0.663	0.509	0.069

Appendix

Table 7. Paired t -test results comparing the human and human-AI moderation groups (using the same representations as Table 4).

Content Moderation

Content moderation studies how to detect, interpret, and respond to harmful or policy-violating material across digital platforms.

See all 87 papers

When AI Agents Disagree Like Humans: Reasoning Trace Analysis for Human-AI Collaborative Moderation
Apr 4, 2026Michał Wawer, Jarosław A. ChudziakHuman-AI Decision Making Annotator Disagreement
Faculty of Electronics and Information Technology Warsaw University of Technology, Warsaw, Poland
PowerBench: Measuring Language Model Bias in Power-shifting Requests
Oct 1, 2026Nicolas Martorell, Wendy Brau, Gonzalo A. Heredia +3Language Model Bias Evaluation Refusal Behavior in Language Models
Universidad de Buenos Aires · Buenos Aires AI Safety Hub · The World Bank +1
AI-Mediated Communication Can Steer Collective Opinion
May 15, 2026Stratis Tsirtsis, Kai Rawal, Chris Russell +2Opinion Dynamics LLM Auditing
1Hasso Plattner Institute · 2Oxford Internet Institute, University of Oxford · 3Weizenbaum Institute

Apr 4, 2026J/K move · Enter open · S save

When AI Agents Disagree Like Humans: Reasoning Trace Analysis for Human-AI Collaborative Moderation

Michał Wawer, Jarosław A. Chudziak

Faculty of Electronics and Information Technology Warsaw University of Technology, Warsaw, Poland

Human-AI Decision Making Annotator Disagreement Multi-Agent Systems Multi-Agent Reasoning Hate Speech Detection

Open paper AskPDF arXiv

When LLM-based multi-agent systems disagree, current practice treats this as noise to be resolved through consensus. We propose it can be signal. We focus on hate speech moderation, a domain where judgments depend on cultural context and individual value weightings, producing high legitimate disagreement among human annotators. We hypothesize that convergent disagreement, where agents reason similarly but conclude differently, indicates genuine value pluralism that humans also struggle to resolve. Using the Measuring Hate Speech corpus, we embed reasoning traces from five perspective-differentiated agents and classify disagreement patterns using a four-category taxonomy based on reasoning similarity and conclusion agreement. We find that raw reasoning divergence weakly predicts human annotator conflict, but the structure of agent discord carries additional signal: cases where agents agree on a verdict show markedly lower human disagreement than cases where they do not, with large effect sizes (d>0.8) surviving correction for multiple comparisons. Our taxonomy-based ordering correlates with human disagreement patterns. These preliminary findings motivate a shift from consensus-seeking to uncertainty-surfacing multi-agent design, where disagreement structure - not magnitude - guides when human judgment is needed.

Abstract

Content Moderation

Explore similar work

When AI Agents Disagree Like Humans: Reasoning Trace Analysis for Human-AI Collaborative Moderation

PowerBench: Measuring Language Model Bias in Power-shifting Requests

AI-Mediated Communication Can Steer Collective Opinion