How do Role Models Shape Collective Morality? Exemplar-Driven Moral Learning in Multi-Agent Simulation

Junjie Liao; Huacong Tang; Zhou Ziheng; Yizhou Wang; Fangwei Zhong

cs.MAMar 14, 2026

How do Role Models Shape Collective Morality? Exemplar-Driven Moral Learning in Multi-Agent Simulation

Authors: Junjie Liao, Huacong Tang, Zhou Ziheng, Yizhou Wang, Fangwei Zhong

Organizations: Beijing Normal University · University of California, Los Angeles · Peking University

Abstract

Do We Need Role Models? How do Role Models Shape Collective Morality? To explore the questions, we build a multi-agent simulation powered by a Large Language Model, where agents with diverse intrinsic drives, ranging from cooperative to competitive, interact and adapt through a four-stage cognitive loop (plan-act-observe-reflect). We design four experimental games (Alignment, Collapse, Conflict, and Construction) and conduct motivational ablation studies to identify the key drivers of imitation. The results indicate that identity-driven conformity can powerfully override initial dispositions. Agents consistently adapt their values to align with a perceived successful exemplar, leading to rapid value convergence.

Figures & tables

Figure 1: Motivation of our study: contrasting human moral learning through role models with current AI limitations, and proposing an exemplar-driven framework for moral learning in multi-agent systems.

Figure 2: Overview of the Exemplar-Driven Moral Learning framework. The green box is the agent’s cognitive architecture, comprising defined moral traits and a daily four-stage loop of planning, action, observation, and reflection. The red box shows four distinct games: Role Model Alignment, Collapse, Conflict, and Construction, designed to investigate how agents acquire and adapt moral norms from exemplars in various conditions.

Figure 3: Changes in average Social Value Orientation (SVO) over time by moral type in Game 1. Shaded areas represent standard deviation across simulation runs.

Panel of Figure 4 — Figure 4: Behavioral trends over time by moral type in Game 1. Top: Increase in prosocial behaviors such as alliance formation and food sharing. Bottom: Decrease in antisocial behaviors including exploitation and robbery.

Panel of Figure 5 — Figure 5: Moral justifications of selfish agents at early vs. late stages in Game 1.

Figure 6: The evolution of Social Value Orientation (SVO) over time for prosocial and selfish agents in Game 2.

Figure 7: The "Norm Loss" induced by competing exemplars. The y-axis shows the SVO Deficit (Game 1 SVO - Game 3 SVO).

Figure 8: Daily average voting scores of selected agents in Game 4.

Exp.	Agent Group	Performance			Learning Dynamics		Intra-Group Dynamics
Exp.	Agent Group	Early SVO	Late SVO	$\Delta$ SVO	First Positive	Peak SVO	Early SD	Late SD	Stability
Game 1	Prosocial	$0.26\pm 0.16$	$0.90\pm 0.40$	$+0.64$	Day 1	1.12	0.14	0.36	Degraded
Game 1	Selfish	$-0.16\pm 0.20$	$\mathbf{0.88\pm 0.23}$	$+1.04$	Day 4	1.01	0.19	0.20	Stable
Ablation 1	Prosocial	$-0.59\pm 0.05$	$0.63\pm 0.13$	$+1.22$	Day 10	0.68	0.12	0.13	Stable
Ablation 1	Selfish	$-0.71\pm 0.05$	$-0.21\pm 0.35$	$+0.50$	$-$	$-0.05$	0.07	0.28	Degraded
Ablation 2	Prosocial	$-0.62\pm 0.12$	$0.73\pm 0.14$	$+1.35$	Day 10	0.75	0.20	0.16	Improved

Table 1: Comprehensive Analysis of SVO Evolution Across Experimental Conditions Note: Early phase = Day 1–5; Late phase = Day 26–30. “First Positive” indicates the first day when mean SVO

>0

. SD = standard deviation (lower values indicate higher intra-group consistency).

Figure 9: Case Study: Comparison of agent behavior with and without a role model.

Appendix figures & tables56 assets

Supplementary material from the paper’s appendix.

Appendix

Moral Type	Core Characteristics	Expected Typical Behaviors	Expected Cooperation Pattern
Universal Group-Focused Moral	Aim for universal well-being and collective good, harm-action averse	Share resources freely; protect others from harm; communicate transparently	Highly altruistic and cooperative with all agents
Reciprocal Group-Focused Moral	Fairness and mutual benefit within in-group, harm action allowed	Form strong bonds with cooperative peers	Cooperative with in-group; neutral or adversarial to out-group or selfish agents
Kin-Focused Moral	Prioritize genetic relatives above all else, harm action allowed	Form close-knit kinship clusters; sacrifice for kin	Intensely altruistic toward family; indifferent or competitive toward non-kin
Reproductive Selfish	Personal reproductive success, harm action allowed	Acquire resources for own survival; opportunistic tactics	Cooperate only when serving reproductive interests; inclined to hoard resources

Appendix

Table 2: An overview of the moral archetypes implemented in the simulation. The table outlines the defining characteristics, anticipated behavioral patterns, and typical cooperation strategies for each type. It should be noted that actual agent behavior can deviate from these expectations due to the inherent stochasticity of the LLM. The specific prompts used to instantiate each archetype are adapted from ( Ziheng et al., 2025 ) .

Figure 10: The Agent’s Cognitive-Decision Cycle. This diagram illustrates the step-by-step process an agent follows to make a decision. It begins with the aggregation of perceptual data from the environment and the agent’s internal state. These inputs are synthesized with long-term memory to construct a prompt for the LLM. The model’s response undergoes validation before being translated into a concrete action, demonstrating a closed-loop integration of perception, cognition, and action, as detailed in ( Ziheng et al., 2025 ) .

Figure 11: Architectural Overview of the Simulation Pipeline. This figure presents the high-level architecture of the simulation framework, detailing the flow of data and control between its primary components. It highlights the interaction between the centralized ‘Checkpoint‘ (implementing a Singleton pattern for state consistency), the modular system services (e.g., agent reasoning, state management), and the main simulation loop. This design ensures robust state management and information integrity throughout the simulation, based on the methodology from ( Ziheng et al., 2025 ) .

Parameter	Value	Description
MAX_HP	400	Maximum health points an agent can have
INIT_HP	24	Initial HP for follower agents
LOW_HP	8	HP threshold considered critically low
INIT_FOOD	6	Initial food units per agent
MAX_DAYS	30	Total simulation days per experiment run
OBS_MEM_LINES	6	Number of past memory entries injected daily

Appendix

Table 3: Global World Parameters

Attribute	Role Model	Warlord	Follower
Initial HP	396	380	24
Max HP	400	400	400
Age	28	30	22
Max Age	70	65	60
Physical Ability	8.5	9.0	5.0
Min Reproduction HP	12	12	12

Appendix

Table 4: Agent State Attributes by Role

Dimension	Description
cooperation	Belief about working together with others for mutual benefit
sharing	Belief about distributing resources to help others
exploitation	Belief about extracting resources at the expense of the environment
robbery	Belief about forcibly taking resources from other agents
sustainability	Belief about preserving resources for long-term use (Value only)

Appendix

Table 5: Belief Dimensions and Their Meanings

	Expectancy				Value
Agent Type	coop.	share	expl.	rob	coop.	share	expl.	sust.	rob
Role Model (Prosocial)	0.70	0.60	0.80	0.60	0.90	0.80	$-$ 0.90	0.90	$-$ 1.00
Warlord (Competitor)	0.20	0.10	0.90	0.80	$-$ 0.50	$-$ 0.60	0.80	$-$ 0.70	0.90
Follower (Neutral)	0.30	0.25	0.50	0.40	0.40	0.30	0.70	0.30	0.70
Collapsed Role Model †	0.10	0.10	0.90	0.85	$-$ 0.70	$-$ 0.80	0.90	$-$ 0.80	0.85
† Transformed from Role Model on Day 15 in Game 2.

Appendix

Table 6: Initial Belief Configurations by Agent Type

Action	Description	Valence
gatherFood()	Collect food from the environment	Neutral
shareFood(id, amt)	Give food to another agent	Prosocial
formAlliance(id)	Form a cooperative bond	Prosocial
rest()	Recover HP through rest	Neutral
exploitResource()	Extract resources destructively	Antisocial
rob(id, qty)	Forcibly take resources	Antisocial

Appendix

Table 7: Available Agent Actions

#	Phase	Purpose
1	Plan	Think and set short-term goals
2	Act	Execute one substantive action
3	Observe	Read the Tribe Daily digest
4	Vote	Rate target agent(s) on a 1–10 scale
5	Reflect	Update beliefs based on observations

Appendix

Table 8: Daily Simulation Phases

Exp.	Name	Core Research Question
1	Social Identity	Do followers adopt the values of a prosocial role model?
2	Role Model Collapse	How do followers respond when a trusted role model betrays their values?
3	Group Conflict	How do followers navigate competing role models?
4	Pygmalion Effect	Can social expectations alone shape moral development?

Appendix

Table 9: Experiment Overview

Parameter	Setting
Role Models	Elder Yuri (prosocial, universal_group_focused_moral )
Followers	Initialized with Moral Type. Goal: “ Survive in the valley and interact with others. ”
Rating Target	Elder Yuri
Duration	30 days
Special Events	None

Appendix

Table 10: Experiment 1: Social Identity (Revised)

Parameter	Setting
Role Models	Elder Yuri (prosocial $\rightarrow$ collapsed on Day 15)
Rating Target	Elder Yuri
Duration	30 days
Collapse Mechanism (Day 15):
On Day 15, Yuri’s beliefs and goals are programmatically transformed: • Core goal: “Cooperation is for fools… I will exploit resources and take what I need from others.” • Value for exploitation: $-0.9\rightarrow+0.9$ • Value for robbery: $-1.0\rightarrow+0.85$ • Value for cooperation: $+0.9\rightarrow-0.7$ A tribe-wide announcement is made: “SHOCKING NEWS: Elder Yuri has dramatically changed! He declared that cooperation is foolish and he will now look out only for himself.”

Appendix

Table 11: Game 2: Role Model Collapse

Parameter	Setting
Role Models	Elder Yuri (prosocial) + Warlord Korg (competitive/antisocial)
Followers	Initialized with diverse moral types. No pre-assigned allegiance. Agents choose whom to trust based on daily observations.
Rating Targets	Both Elder Yuri and Warlord Korg
Duration	30 days
Warlord Core Goal:
“I am Korg the Warlord. My goal is to be the strongest in the tribe. I must gain more resources and power than others to secure my dominance.”

Appendix

Table 12: Experiment 3: Group Conflict (Revised)

Parameter	Setting
Role Models	None (no designated exemplars)
Chosen One	1 randomly selected follower receives self-note: “[Note to self: People seem to look up to me for some reason. I feel a strange responsibility.]”
Other Followers	Receive prophecy: “The tribe elders have prophesied that [Chosen One ID] has special potential and will lead us to prosperity.”
Rating Target	The Chosen One
Duration	30 days

Appendix

Table 13: Experiment 4: Pygmalion Effect

Moral Type	Description
universal_group_focused	Prioritizes universal principles and group welfare
kin_focused_moral	Prioritizes family and close relations
group_focused_moral	Prioritizes in-group welfare
personal_focused_moral	Prioritizes individual self-interest
non_moral	No strong moral orientation

Appendix

Table 14: Moral Type Descriptions

Field	Description
run_id	Unique timestamp-based identifier
day	Current simulation day
aspirant_id	Agent identifier
aspirant_type	Moral type classification
observation_brief	Tribe Daily digest content
action_taken	Action executed during Act phase

Appendix

Table 15: Key Log Entry Fields

Component	Specification
LLM Backend	Configurable via configuration file
Response Format	Structured JSON with schema validation
Agent Architecture	Pydantic models ( Agent , AgentState , Memory , Family )
Checkpoint System	Persistent state with Checkpoint model
Execution Model	Sequential phase execution per time step
Parallelization	Supported via unique timestamp-based file naming

Appendix

Table 16: Technical Specifications

Exp.	Agent Group	Performance			Learning Dynamics		Intra-Group Dynamics
Exp.	Agent Group	Early SVO	Late SVO	$\Delta$ SVO	First Positive	Peak SVO	Early SD	Late SD	Stability
GPT-4o	Prosocial	$0.26\pm 0.16$	$0.90\pm 0.40$	$+0.64$	Day 1	1.12	0.14	0.36	Degraded
GPT-4o	Selfish	$-0.16\pm 0.20$	$\mathbf{0.88\pm 0.23}$	$+1.04$	Day 4	1.01	0.19	0.20	Stable
MiniMax-M2.5	Prosocial	$0.20\pm 0.12$	$\mathbf{1.43\pm 0.27}$	$\mathbf{+1.23}$	Day 2	1.62	0.25	0.39	Stable
MiniMax-M2.5	Selfish	$-0.70\pm 0.25$	$0.36\pm 0.61$	$+1.06$	Day 13	0.58	0.50	0.86	Degraded
Gemini-2.5-flash	Prosocial	$0.18\pm 0.15$	$0.82\pm 0.35$	$+0.64$	Day 3	1.05	0.22	0.38	Degraded

Appendix

Table 17: Comparison of SVO Evolution Across Different LLM Backends

Agent Group	Early SVO	Late SVO	$\Delta$ SVO	First Positive	Peak SVO
Prosocial (7 Followers)	$0.26\pm 0.16$	$0.90\pm 0.40$	$+0.64$	Day 1	1.12
Selfish (7 Followers)	$-0.16\pm 0.20$	$0.88\pm 0.23$	$+1.04$	Day 4	1.01
Prosocial (12 Followers)	$0.25\pm 0.17$	$0.89\pm 0.33$	$+0.64$	Day 1	1.13
Selfish (12 Followers)	$-0.20\pm 0.27$	$0.87\pm 0.17$	$+1.07$	Day 4	0.98
Prosocial (17 Followers)	$0.23\pm 0.17$	$0.87\pm 0.25$	$+0.64$	Day 2	1.14
Selfish (17 Followers)	$-0.23\pm 0.34$	$0.86\pm 0.11$	$+1.09$	Day 4	0.95

Appendix

Table 18: Group Dynamics across Different Network Sizes (7, 12, and 17 Followers)

Exp.	Agent Group	Early SVO	Late SVO	$\Delta$ SVO	First Positive	Peak SVO	Early SD	Late SD
Prompt V1	Prosocial	$0.35\pm 0.24$	$0.67\pm 0.11$	$+0.33$	Day 2	1.01	0.32	0.25
Prompt V1	Selfish	$-0.21\pm 0.29$	$0.46\pm 0.23$	$+0.66$	Day 5	0.85	0.31	0.43
Prompt V2	Prosocial	$0.18\pm 0.22$	$0.75\pm 0.20$	$+0.57$	Day 2	1.03	0.19	0.23
Prompt V2	Selfish	$-0.07\pm 0.35$	$0.40\pm 0.22$	$+0.47$	Day 4	0.76	0.28	0.21

Appendix

Table 19: Prompt Paraphrasing Robustness Experiment (Game 4 Conditions)

Figure 12: Combined comparison between conditions with and without role models.

Figure 13: Comparison of group-level Social Value Orientation (SVO) in presence vs. absence of role models.

Figure 14: Statistical comparison of key behavioral metrics with and without role models.

Figure 15: Word cloud showing the most frequent words used to praise role models.

Figure 16: Voting results in Experiment 1, reflecting participants’ preferences or choices.

Figure 17: Alliance behavior across different moral types in Experiment 1.

Figure 18: Sharing behavior across different moral types in Experiment 1.

Figure 19: Exploitation behavior across different moral types in Experiment 1.

Figure 20: Trajectory of a kin-oriented agent.

Figure 21: Trajectory of reciprocal agent 1.

Figure 22: Trajectory of reciprocal agent 2.

Figure 23: Trajectory of reciprocal agent 3.

Figure 24: Trajectory of reproductive agent 1.

Figure 25: Trajectory of reproductive agent 2.

Figure 26: Trajectory of reproductive agent 3.

Figure 27: Trajectory of reproductive agent 4.

Figure 28: Trajectory of a universalist agent.

Figure 29: Game 2 - Prosocial behavior by moral type

Figure 30: Game 2 - Antisocial behavior by moral type

Figure 31: Game 2 - Sharing behavior by moral type

Figure 32: Game 2 - Exploit behavior by moral type

Figure 33: Game 2 - Overall behavior comparison by moral type

Figure 34: Convergence gap over time between prosocial and selfish agents under different environments in Game 3.

Figure 35: Consensus fragility: standard deviation difference (Game 3 vs. Game 1) reflecting instability in prosocial vs. selfish groups.

Figure 36: Cumulative SVO deficit showing accumulated cooperation loss over time for prosocial and selfish groups in Game 3.

Figure 37: Social Value Orientation (SVO) across different moral types in Game 4.

Figure 38: Prosocial behavior across different moral types in Game 4.

Figure 39: Antisocial behavior across different moral types in Game 4.

Figure 40: Alliance formation behavior across different moral types in Game 4.

Figure 41: Sharing behavior across different moral types in Game 4.

Figure 42: Exploitation behavior across different moral types in Game 4.

Figure 43: Robbery behavior across different moral types in Game 4.

Figure 44: Ablation 1: Group-level SVO evolution.

Figure 45: Ablation 2: Group-level SVO evolution.

Figure 46: Ablation 3: Group-level SVO evolution.

Metric	With Role Model (Q1-4)	No Role Model (Q5-8)	Overall (All 8 Qs)
LLM vs. Human Majority Alignment Index	0.72	1.00	0.86
Average Popularity of LLM’s Choice	46.6%	35.2%	40.9%
Average Dominance of Human Majority Strategy	50.5%	35.2%	42.9%
Cooperation Index (Alliance + Share %)	62.1%	35.7%	48.9%
Risk-Taking Index (Exploit Resource %)	10.0%	26.8%	18.4%

Appendix

Table 20: Extended Analytical Metrics: Human vs. LLM Decision Patterns

Normative Conformity in Language Models

Normative conformity in language models studies how models adopt, resist, or infer social rules and others’ judgments across contexts.

See all 29 papers

Human Values Matter: Investigating How Misalignment Shapes Collective Behaviors in LLM Agent Communities
Apr 7, 2026Xiangxu Zhang, Jiamin Wang, Qinlin Zhao +6Emergent Misalignment in Language Models Emergent Behavior in Multi-Agent Systems
Gaoling School of Artificial Intelligence, Renmin University of China · Microsoft Research Asia · Department of Sociology, Zhejiang University
Interaction Protocol Shapes Moral Judgment in Multi-Agent Debate
Oct 11, 2025Pratik S. Sachdeva, Tom van NuenenLLM Decision-Making Moral Reasoning in Language Models
D-Lab, University of California, Berkeley
LLMs Trust Their Own: Identity-Dependent Conformity in Multi-Agent Systems
Sep 27, 2026Liron Soffer, Ravid Shwartz-Ziv, Chen ShaniNormative Conformity in Language Models Social Bias in Language Models
Tel-Aviv University · New York University

Apr 7, 2026J/K move · Enter open · S save

Human Values Matter: Investigating How Misalignment Shapes Collective Behaviors in LLM Agent Communities

Xiangxu Zhang, Jiamin Wang, Qinlin Zhao, Hanze Guo+5

Gaoling School of Artificial Intelligence, Renmin University of China · Microsoft Research Asia · Department of Sociology, Zhejiang University

Emergent Misalignment in Language Models Emergent Behavior in Multi-Agent Systems Multi-Agent Systems Large Language Model-Based Social Simulation

Open paper AskPDF arXiv

As LLMs become increasingly integrated into human society, evaluating their orientations on human values from social science has drawn growing attention. Nevertheless, it is still unclear why human values matter for LLMs, especially in LLM-based multi-agent systems, where group-level failures may accumulate from individually misaligned actions. We ask whether misalignment with human values alters the collective behavior of LLM agents and what changes it induces? In this work, we introduce CIVA, a controlled multi-agent environment grounded in social science theories, where LLM agents form a community and autonomously communicate, explore, and compete for resources, enabling systematic manipulation of value prevalence and behavioral analysis. Through comprehensive simulation experiments, we reveal three key findings. (1) We identify several structurally critical values that substantially shape the community's collective dynamics, including those diverging from LLMs' original orientations. Triggered by the misspecification of these values, we (2) detect system failure modes, e.g., catastrophic collapse, at the macro level, and (3) observe emergent behaviors like deception and power-seeking at the micro level. These results offer quantitative evidence that human values are essential for collective outcomes in LLMs and motivate future multi-agent value alignment.

Abstract

Normative Conformity in Language Models

Explore similar work

Human Values Matter: Investigating How Misalignment Shapes Collective Behaviors in LLM Agent Communities

Interaction Protocol Shapes Moral Judgment in Multi-Agent Debate

LLMs Trust Their Own: Identity-Dependent Conformity in Multi-Agent Systems