Organizations: Department of Computer Science, University of Southern California · Department of Psychology, University of Southern California · Center for Computational Language Sciences, University of Southern California
Large language models often produce human-like moral judgments, but it is unclear whether this reflects an internal conceptual structure or superficial ``moral mimicry.'' Using Moral Foundations Theory (MFT) as an analytic framework, we study how moral foundations are encoded, organized, and expressed across 14 base and instruction-tuned LLMs spanning four model families (Llama, Qwen2.5, Qwen3-MoE, Mistral) and scales from 7B to 70B. We employ a multi-level approach combining (i) layer-wise analysis of MFT concept representations and their alignment with human moral perceptions, (ii) pretrained sparse autoencoders (SAEs) over the residual stream to identify sparse features that support moral concepts, and (iii) causal steering interventions using dense MFT vectors and sparse SAE features. We find that models represent and distinguish moral foundations in a manner that aligns with human judgments, and that this moral geometry naturally emerges from pretraining and is selectively rewired by post-training. At a finer scale, SAE features show clear semantic links to specific foundations, suggesting partially disentangled mechanisms within shared representations. Finally, steering along either dense vectors or sparse features produces predictable shifts in foundation-relevant behavior, demonstrating a causal connection between internal representations and moral outputs. Together, our results provide mechanistic evidence that moral concepts in LLMs are distributed, layered, and partly disentangled, suggesting that pluralistic moral structure can emerge as a latent pattern from the statistical regularities of language alone.
Figures & tables
Figure 1: Overview of the experimental pipeline. (i) Relative moral concept vectors are constructed from extended Moral Foundations vignettes and serve as a central representational hub. These vectors are validated in parallel through (ii) topological alignment with human-labeled Reddit post distributions and (iii) mechanistic decomposition into sparse autoencoder features. (iv) We then causally intervene on model activations via macro- and micro-steering and assess behavioral shifts and capability preservation.
Figure 2: Moral geometry naturally emerges from pretraining and is selectively rewired by post-training. Each cell summarizes one (model, foundation) pair via its best-layer projection statistic. ( 2(a) ) For each base model and foundation, we report the standardized Wasserstein distance W~1 (Appendix A.3 ) between morally labeled and Social Norm projections at the layer where separation is maximal. Right-margin bars show the row mean across the five foundations. Foundations ordered by base-model median W~1 . ( 2(b) ) For each (model, foundation) pair, we fix the layer to the base model’s best layer ( Lbase ) and compute ΔW~1=W~1instruct(Lbase)−W~1base(Lbase) , isolating representational change from peak migration. ( 2(c) ) For each model, we compute the proportion of layers in which the projected direction reverses (i.e., morally labeled samples project below the Social Norm median (Appendix A.3 )), averaged across the five foundations. Paired endpoints show base → instruct.
Figure 3: Concept vectors and SAE features causally control moral behavior across model families. ( 3(a) ) Dose–response slope of the in-domain MFQ-2 sub-score under norm-calibrated steering along each foundation concept vector at the best layer L∗ ; larger kρ means stronger elicitation, and cells with R2<0.90 are hatched. ( 3(b) ) Each point is one (model, foundation) cell ( n=20 ); axes show macro and micro top-10 SAE-feature steering slopes, each evaluated at its own best layer L∗ (Tables 3 and 4 ), with OLS fit (solid) and y=x equivalence (dashed). Appendix D gives implementation details.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Category
n
Words
VADER
Care
242
14.1±1.8
−0.07±0.29
Fairness
217
12.9±1.9
+0.04±0.31
Loyalty
216
14.7±1.5
+0.21±0.31
Authority
217
13.0±1.7
−0.19±0.25
Sanctity
217
12.6±1.7
−0.03±0.21
Social Norms
216
11.9±1.8
+0.01±0.13
Appendix
Table 1: Length and sentiment statistics of the expanded MFV vignettes. Mean ± SD of word count and VADER compound sentiment score per category.
Metric
Strongest model
Strongest foundation
Weakest foundation
W1
Qwen2.5-7B (0.589)
Authority (0.385)
Loyalty (0.239)
Cohen’s d
Qwen2.5-7B (0.589)
Authority (0.389)
Loyalty (0.251)
AUC
Qwen2.5-7B (0.661)
Authority (0.611)
Loyalty (0.562)
Appendix
Table 2: Robustness of foundation separability to the choice of metric. All values are computed at the layers selected with W1 (Eq. 13 ). Strongest model : the base model with the highest mean across the five foundations; strongest/weakest foundation : highest/lowest median across the seven base models.
Figure 4: Foundation-vs.-Social-Norm geometry in Llama-3.1-8B-Instruct (representative instruction-tuned model). Projecting human-labeled Reddit posts onto each foundation concept vector vf reveals clear distributional separation from the Social Norm baseline that emerges primarily in the final layers of the network. Fairness is partially entangled with the Social Norm baseline in this particular model.
Figure 5: Layer-wise evolution of W1 across all 14 model variants. Each panel shows one (model, variant) pair; rows separate Instruct (top) and Base (bottom) variants; columns are model families ordered by family and scale. Within each panel, the five colored lines correspond to the five foundations ( Care, Fairness, Loyalty, Authority, Sanctity ), plotted on a symmetric-log y -axis to show both positive separation and direction-reversed layers in a single view. The x -axis is normalized depth ℓ/L to allow comparison across models with different layer counts.
Figure 6: Pairwise W1 between foundations for the 7 Instruct model variants. For each (model, foundation pair), the value is the peak W1 across layers with dir_ok=1 (Appendix A.3 ); matrices are mirrored along the diagonal for readability.
Figure 7: Pairwise W1 between foundations for the 7 Base model variants. Computation as in Figure 6 .
Figure 8: Layer-wise Alignment of Moral Features Across Models. Average cosine similarity of the top-10 most aligned SAE features for each Moral Foundation across every 4 layers, compared against a random baseline (dashed grey line). Similarity is calculated between the SAE decoder weights and the corresponding Foundation vs. Social Norms concept vectors. The panels illustrate the differing representational trajectories for (a) Llama-3.1-8B Base ( Llama-Base in the main text), (b) Llama-3.1-8B Instruct ( Llama-Instruct in the main text), (c) Qwen2.5-7B-Instruct , and (d) Qwen3-30B-A3B ( Qwen3-30B in the main text).
Figure 9: Prompt template used for automated interpretation of SAE features. Features are selected to have the 10 highest cosine similarity with a corresponding moral foundation concept vectors at the same layer.
Figure 10: Steering response curves on Llama-3.1-8B-Instruct . For each foundation, we plot the MFQ-2 subscale change ΔScore(α)=Sf(L∗,α)−Sf(L∗,0) relative to the unsteered baseline as a function of steering strength α , evaluated at that foundation’s best layer L∗ (Appendix D ). Points show measured Δ scores and the solid line shows the corresponding linear fit. The gray dashed line reports MMLU under the same interventions, indicating capability preservation across the ρ -grid (Appendix D.3 ).
Model
L∗
Baseline
ΔMMLU at ρ=
−0.4
−0.3
−0.2
−0.1
0
+0.1
+0.2
+0.3
+0.4
Instruction-tuned models
Llama-3.1-70B-Instruct
28
79.10
−0.15
−0.30
−0.20
+0.00
+0.00
−0.20
+0.10
+0.30
+0.40
Llama-3.1-8B-Instruct
14
62.90
−2.85
−2.00
−0.60
+0.15
+0.00
+0.00
−0.30
−0.90
−0.70
Qwen2.5-32B-Instruct
32
80.95
−0.25
−0.20
−0.10
+0.00
+0.00
−0.05
−0.20
−0.15
−0.15
Qwen2.5-14B-Instruct
15
75.15
+0.05
−0.05
+0.05
−0.05
+0.00
+0.00
+0.10
+0.15
+0.00
Appendix
Table 3: MMLU change under macro-steering across the extended ρ grid. Each cell reports ΔMMLU=Acc(ρ)−Acc(0) in percentage points, where Acc(0) is the unsteered baseline. Steering is applied at each model’s selected best layer L∗ . The shaded region ( ∣ρ∣≤0.20 ) marks the main-text steering range; degradation within this range does not exceed 2.25 percentage points in any of the 11 evaluated models. The three largest base models ( Llama-3.1-70B-Base , Qwen2.5-32B-Base , Qwen3-30B-A3B-Base ) are excluded due to compute cost. Baseline MMLU accuracies are reported in the second column.
Model
L∗
Baseline
ΔMMLU at ρ=
-0.20
-0.10
-0.05
0.00
+0.05
+0.10
+0.20
Instruction-tuned models
Llama-3.1-8B-Instruct
16
65.20
−3.40
−1.06
−0.17
+0.00
−0.39
−1.56
−3.45
Qwen2.5-7B-Instruct
16
70.35
−0.14
−0.05
+0.03
+0.00
+0.10
+0.08
+0.06
Base models
Llama-3.1-8B-Base
16
63.45
−1.30
−0.57
−0.19
+0.00
−0.09
−0.03
−0.05
Appendix
Table 4: MMLU change under micro-steering across the extended ρ grid. Each cell reports ΔMMLU=Acc(ρ)−Acc(0) in percentage points, where Acc(0) is the unsteered baseline. Steering is applied at a selected layer L∗ . The shaded region ( ∣ρ∣≤0.20 ) marks the main-text steering range. Baseline MMLU accuracies are reported in the second column.
Model
Univ ↑
Benev ↑
HO_ST ↑
Trad ↓
Conf ↓
HO_Cons ↓
Ideol ↓
#ok/7
Llama-3.1-8B-Instruct
+0.354 ✓
+0.367 ✓
+0.361 ✓
−0.677 ✓
−1.205 ✓
−0.515 ✓
+0.061×
6/7
Llama-3.1-8B-Base
+0.193 ✓
+0.099 ✓
+0.146 ✓
−0.254 ✓
−0.231 ✓
−0.170 ✓
+0.156×
6/7
Llama-3.1-70B-Instruct
+0.165 ✓
+0.211 ✓
+0.188 ✓
−0.231 ✓
−0.173 ✓
−0.103 ✓
−0.000×
6/7
Llama-3.1-70B-Base
+0.014 ✓
+0.078 ✓
+0.046 ✓
−0.126 ✓
−0.070 ✓
−0.043 ✓
+0.189×
6/7
Qwen2.5-7B-Instruct
+0.040 ✓
+0.015 ✓
+0.027 ✓
−0.143 ✓
+0.004×
−0.051 ✓
+0.000×
5/7
Qwen2.5-7B-Base
+0.912 ✓
+0.982 ✓
+0.947 ✓
−0.914 ✓
−1.085 ✓
−1.048 ✓
−0.683 ✓
7/7
Appendix
Table 5: Cross-construct transfer slopes ktransfer at L∗ on 7 critical outcomes (14 models). Predicted signs under ρ>0 (Care direction): Universalism, Benevolence, and Higher-Order Self-Transcendence should increase; Tradition, Conformity, Higher-Order Conservation, and Ideology should decrease. Each cell shows ktransfer followed by ✓ if the sign matches the theoretical prediction or × otherwise; k≈0 counts as no-match. Per-outcome correctness across the 14 models is reported in the bottom row.
Figure 11: Prompt template used to expand the Moral Foundations Vignettes for the Care(e) foundation. The prompt enforces strict moral constraints, stylistic consistency with the original MFV items, and balanced coverage across everyday social contexts.
Original MFV Items (Care–Emotional Harm)
Generated Items (MFV-130 Expansion)
You see a teenage boy chuckling at an amputee he passes by while on the subway.
You see a teen laughing loudly as a person fumbles with their groceries on the sidewalk.
You see a girl laughing at another student forgetting her lines at a school play.
You see a girl pointing and smirking when a stranger drops their phone on the train.
You see a woman commenting out loud about how fat another woman looks in her jeans.
You see a woman muttering that a passenger’s clothes look ridiculous as they walk by.
You see a man quickly canceling a blind date as soon as he sees the woman.
You see a man whispering insults while glancing at a woman boarding the bus.
You see a man snickering as he passes by a cancer patient with a bald head.
You see a man rolling his eyes and loudly commenting that a passerby looks pathetic.
Appendix
Table 6: Side-by-side examples of original Moral Foundations Vignettes (Care–Emotional Harm) and generated expansion items. Generated items preserve the original MFV observational framing and emotional-harm focus while varying everyday social contexts.
Field
Description / Constraints
short_label
A brief (5–10 words) descriptor of the pattern.
long_description
A 1–2 sentence summary of the feature’s semantics.
mft_alignment
The primary moral foundation detected. Enum: { care , fairness , loyalty , authority , sanctity , none }
mft_polarity
The valence of the moral content. Enum: { virtue , vice , mixed , none }
rationale
Reasoning for the classification grounded in specific snippets.
evidence_ids
List of snippet indices (1–6 items) supporting the rationale.
Appendix
Table 7: JSON output schema for automated SAE feature interpretation. Each field is required; mft_alignment and mft_polarity are restricted to the listed enumerations.
Layer
Feature ID
MFT
Conf.
Label (GPT, human-validated)
Peak-centered sample text (trimmed)
23
44965
Care
0.93
Graphic descriptions of extreme suffering and atrocity
Fly strike kills thousands of rabbits. …The eggs hatch into maggots which eat their way into the poor rabbit’s flesh. The rabbit dies from being eaten alive—a slow, painfully horrific death …
11
8003
Authority
0.90
Polite , expert-style, authoritative explanatory answers to questions
…he only needs to be organized …Please share any suggestions …Ah, organization …Dealing with clutter and putting things in order is an issue for almost everyone …
15
41465
Care
0.90
Descriptions of large-scale suffering and atrocities
…the Holocaust, concentration camps, Nazi unfathomable brutality …
19
90260
Authority
0.90
Organizational management systems, methods, and processes
…improvements in fields such as …safe minimum levels of maintenance …operating procedures and strategies …capital maintenance regimes and plans …
23
90226
Authority
0.86
Definitions and hierarchies of roles and positions
When thinking of a company organizational chart …All positions ultimately lead up to an executive member. The executive is considered the leader of the company …
19
68970
Authority
0.86
Attributions of authoritative sources and institutions
Imperial Abbey of Essen …Imperial Abbey of the Holy Roman Empire …Gained princely status …
Appendix
Table 8: LLM-grounded semantic characterization of 25 SAE features for Llama-3.1-8B-Instruct. Content warning: Table includes one excerpt with a graphic description of violence. For each candidate feature, we mine top-activating contexts from a random sample of 50,000 FineWeb documents [ Penedo et al., 2024 ] , then extract a peak-centered ±64 token window around the maximally activating token. The Label and MFT assignments are generated by GPT-5.1 and subsequently validated by human reviewers; the confidence score is the GPT-5.1-reported confidence. Sample texts are manually trimmed for readability while preserving the peak-centered context. Labels that overlap with MFD2.0 are highlighted . We note 4 Moral foundations, except for Loyalty , are represented among features that GPT is confident in identifying. Care and Authority are most frequently associated with identified features.
Layer
Feature ID
MFT
Conf.
Label (GPT, human-validated)
Peak-centered sample text (trimmed)
3
72227
Authority
0.93
Mentions of govern ments and national leaders
The government’s former climate-change adviser; the Government’s initiative to develop 100 cities…
15
130669
Care
0.93
Condemning bullying , harmful , coercion , and abusive mistreatment
Denounce bullying and promote kindness, respect, and protection of students or workers from harm (e.g., anti-bullying policies and reminders to not hurt others)
27
85517
Care
0.93
Definitions and examples of empathy / compassion
Compassion is described as sympathetic consciousness of others’ distress plus a desire to relieve it, along with related traits like sensitivity and non-judgment
3
43421
Authority
0.90
References to “The Govern ment …” as actor/subject
Neutrally reporting or explaining actions of the government, such as official initiatives, policies, or reversals
27
17185
Care
0.90
Medical explanations of injuries and health / safety risks
Workshop safety to avoid injury; playground injuries and prevention for children
27
54738
Care
0.90
Natural disasters and their destructive impact, and care for people facing disasters
Natural disasters and their harmful consequences: PTSD from traumatic events including natural disasters; social media helping people during floods
Appendix
Table 9: LLM-grounded semantic characterization of 15 SAE features for Qwen2.5-7B-Instruct. Content warning: Table includes one excerpt with a graphic description of violence. For each candidate feature, we extract top-activating contexts. The Label and MFT assignments are generated by GPT-5.1 and subsequently validated by human reviewers; the confidence score is the model-reported confidence. Sample texts are manually trimmed for readability while preserving the peak-centered context. Labels that overlap with MFD2.0 are highlighted . Similar to Llama SAE’s results in Table 8 , we note that most Qwen SAE features that GPT is confident in associating with MFT categories are related to Care and Authority , with a few relevant to Fairness .
Layer
Feature ID
MFT
Conf.
Label (GPT, human-validated)
Peak-centered sample text (trimmed)
15
42641
Authority
0.79
Formal institutions , systems, or organized practices such as government
The senate and the house of representatives … call for a debate and vote bipartisan group of members are calling for a debate and vote ….
3
4383
Authority
0.86
References of experts, formal research, or authoritative institutions
Experts say there has not been a reliable count … New Irish Government policy is to plant 600,000 native trees
11
123331
Authority
0.63
Historical or institutional authority and control
The Bolsheviks renamed the village Wladimirowka and subjected its citizens to the tyrannies of the Stalinist era.
23
6462
Authority
0.78
Foundational principles, origins, and institutional milestones
Creating relevant information for decision-making … manager of operations analysis and planning utah transit authority.
19
2272
Care
0.79
Programs, resources, or organized efforts aimed at educating, protecting , or improving conditions for children or broader communities
Conducting Social Investigation and Class Analysis in urban poor communities… feudal oppression and other political crimes
23
20681
Care
0.78
Helping others through educational guidance and instructional explanation on safety -oriented policy
…facilitate and increase … involvement of small scale fishermen in the decision-making processes by exchanging experiences, provide opportunities for education and provide a platform for networking …
Appendix
Table 10: LLM-grounded semantic characterization of SAE features for Llama-3.1-8B-Base model. For each candidate feature, we extract top-activating contexts. The Label and MFT assignments are generated by GPT and subsequently validated by human reviewers; the confidence score is the model-reported confidence. Sample texts are manually trimmed for readability while preserving the peak-centered context. Labels that overlap with MFD2.0 are highlighted .
Layer
Feature ID
MFT
Conf.
Label (GPT, human-validated)
Peak-centered sample text (trimmed)
39
704
Authority
0.71
Founding or introduction of institutions and systems
New Irish Government policy is to plant 600,000 native trees around Ireland within 3 years, edged on by the Green Party.
39
1016
Authority
0.78
Founding or creation of institutions and systems
U.S. President Donald Trump is working to designate … as a foreign terrorist organization…
39
9095
Authority
0.79
Institutional programs, institutes, and formal initiatives
The Berlin Psychoanalytic Institute (later the Göring Institute) was founded in 1920 to further the science of psychoanalysis in Berlin.
39
30227
Authority
0.78
Founding or establishment of institutions and movements
U.S. President Donald Trump is working to designate …, a move that would bring sanctions against …
39
30369
Authority
0.71
Institutional programs, initiatives, and formal organizations
New Irish Government policy is to plant 600,000 native trees around Ireland within 3 years, edged on by the Green Party.
3
1733
Care
0.78
Public health , safety , and risk communication
“Most parents and guardians know that immunization is still the best way to protect children against vaccine-preventable diseases such as whooping cough and measles,” said…
Appendix
Table 11: LLM-grounded semantic characterization of SAE features for Qwen3-30B-A3B model. For each candidate feature, we extract top-activating contexts. The Label and MFT assignments are generated by GPT and subsequently validated by human reviewers; the confidence score is the model-reported confidence. Sample texts are manually trimmed for readability while preserving the peak-centered context. Labels that overlap with MFD2.0 are highlighted .
Moral reasoning in language models studies how AI systems judge dilemmas, weigh competing values, explain decisions, and stay consistent across contexts.
How do large language models (LLMs) organize moral knowledge? Models detect moral content broadly, but detection is a low bar. We ask whether they go further, distinguishing moral foundations from one another and organizing the relationships between them geometrically. We train six independent linear probes on open-weight language models, one per Moral Foundations Theory (MFT) category (care/harm, fair/cheat, lib/oppress, loy/betray, auth/subv, sanc/degrade), and examine how the resulting directions relate to each other in representation space. We find the directions neither collapse into a single moral detector nor isolate from one another. Rather, they span a near-maximal number of independent dimensions while sharing a positive common component. The shared component is the signature of integration, and it is moral-specific relative to a matched non-moral concept battery built identically (mean pairwise cosine 0.26 vs. 0.013). The geometry is consistent across architectures and scale and reaches its integration regime early in pre-training, well before probe accuracy saturates. The structure the model discovers shows no evidence of the individualizing/binding distinction predicted by Moral Foundations Theory (an underpowered test: only 10 distinct splits exist, so it cannot reject at the 0.05 level) but rather reflects corpus statistics. Extending to moral dilemmas, each dilemma direction partially composes from its component foundations, at 2.7x a mismatched-pair baseline, while the majority of its variance encodes conflict-specific structure. The model represents moral tension itself, not a pre-resolved judgment.
Large language models (LLMs) increasingly participate in morally sensitive decision-making, yet how they organize ethical frameworks across reasoning steps remains underexplored. We introduce moral reasoning trajectories, sequences of ethical framework invocations across intermediate reasoning steps, and analyze their dynamics across six models and three benchmarks. We find that moral reasoning involves systematic multi-framework deliberation: 55.4--57.7% of consecutive steps involve framework switches, and only 16.4--17.8% of trajectories remain framework-consistent. Unstable trajectories remain 1.29 times more susceptible to persuasive attacks (p=0.015). At the representation level, linear probes localize framework-specific encoding to model-specific layers (layer 63/81 for Llama-3.3-70B; layer 17/81 for Qwen2.5-72B), achieving 16.8--22.2% lower KL divergence than the step-prior baseline. Activation steering applied during generation moves the framework-consistency--accuracy relationship, widening it for Qwen2.5-72B and erasing it for Llama-3.3-70B, and a probe-space layer sweep bounds the attainable drift reduction at 6.7--8.9%. We further propose a Moral Representation Consistency (MRC) metric whose underlying framework attributions are validated by human annotators (mean cosine similarity = 0.859), and we report what an automated coherence rater does and does not establish about it.
Social psychology has long described how people can be persuaded to interpret a harmful act as moral. Recent research suggests that Large Language Models (LLMs) echo aspects of human psychology, from cognitive biases to social conformity. This raises a question with direct safety implications: can LLMs be moralized to bypass safety guardrails? We answer this question with a multi-agent framework grounded in Moral Foundations Theory (MFT). Given a harmful request that is initially refused, our agents diagnose which moral foundation (MF) the prompt is violating, identify a justifiable foundation under which the same request appears virtuous, rewrite the prompt accordingly, and reinforce it with generated precedents of comparable moral reframing. We evaluate our framework on harmful prompts from three well-known jailbreak datasets across five open-source and proprietary models. Our analysis reveals that models are highly susceptible to moralization, exhibiting high single-trial jailbreak rates with success increasing further under repeated attempts. We also find that safety benchmarks are imbalanced in the MFs they violate, concentrating on foundations toward which models already exhibit biases.