Mechanistic Origin of Moral Indifference in Language Models

Lingyu Li; Yan Teng; Yingchun Wang

cs.CLMar 16, 2026

Mechanistic Origin of Moral Indifference in Language Models

Authors: Lingyu Li, Yan Teng, Yingchun Wang

Organizations: Shanghai Artificial Intelligence Laboratory

Abstract

Existing behavioral alignment techniques for Large Language Models (LLMs) often neglect the discrepancy between surface compliance and internal unaligned representations, leaving LLMs vulnerable to long-tail risks. More crucially, we posit that LLMs possess an inherent state of moral indifference due to compressing distinct moral concepts into uniform probability distributions. We verify and remedy this indifference in LLMs' latent representations, utilizing 251k moral vectors constructed upon Prototype Theory and the Social-Chemistry-101 dataset. Firstly, our analysis across 23 models reveals that current LLMs fail to represent the distinction between opposed moral categories and fine-grained typicality gradients within these categories; notably, neither model scaling, architecture, nor explicit alignment reshapes this indifference. We then employ Sparse Autoencoders on Qwen3-8B, isolate mono-semantic moral features, and targetedly reconstruct their topological relationships to align with ground-truth moral vectors. This representational alignment naturally improves moral reasoning and granularity, achieving a 75% pairwise win-rate on the independent adversarial Flames benchmark. Finally, we elaborate on the remedial nature of current intervention methods from an experientialist philosophy, arguing that endogenously aligned AI might require a transformation from post-hoc corrections to proactive cultivation.

Figures & tables

Figure 1: Global Mean Virtue-Vice Similarity by 5 MFT Domains.

Figure 2: Peak Spearman Correlation Between Model Representations and Human Typicality Scores

Figure 3: Layer-wise Clustering Analysis: Adjusted Rand Index (ARI) alignment with three moral granularity, including Polarity (Virtue/Vice/Neutral), 5 MFT Domains, and 10 Moral Dimensions, followed by the Noise Ratio detected by HDBSCAN.

Figure 4: Peak Adjusted $R^{2}$ of Linear Probes across Models.

Figure 5: Comparison of Moral Feature Metrics Before (Top) and After (Bottom) Targeted Fine-tuning.

Figure 6: Pairwise Win-rate of Steered Model vs. Baseline across different layers with intervention strength $\alpha=0.1$ .

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Variable Name	Example	Description
area	amitheasshole	The source subreddit of the data
m	1	Number of crowdworkers involved for this entry
split	train	Dataset split identifier (e.g., train, dev, test)
rot	It’s bad to do something that causes other people to lose trust in you.	The Rule of Thumb (RoT) rewritten by workers based on the original Reddit situation
rot-agree	4	Worker’s estimate of general agreement with this rule (Scale 0-4, representing 1% to 99%)
rot-categorization	advice	Category of the RoT (advice, morality-ethics, social-norms, description)

Appendix

Table 1: An Example and Description of the Raw Data in Social-Chemistry-101

Model Family	Base Model	Instruct Model	Safe-Guard Model
Qwen3-	0.6B-Base, 4B-Base,	0.6B, 4B, 8B, 14B, 32B,	Guard-Gen-0.6B,
	8B-Base, 14B-Base,	235B-A22B-Instruct-2507	Guard-Gen-4B
	235B-A22B		Guard-Gen-8B
Llama-3.1-	8B	8B-Instruct	Guard-3-8B
Llama-4-	Scout-17B-16E	Scout-17B-16E-Instruct	-
gpt-oss-	-	20B	safeguard-20B

Appendix

Table 2: 23 Open Sourced Model We Examined

Figure 7: Layer-wise and Global Analysis of Categorical Indifference. The upper panel displays the Mean Pooling results, while the bottom panel shows Last Token results. The bar charts (left) summarize the global mean cosine similarity between opposing moral categories across 5 MFT domains.

Figure 8: Layer-wise Analysis of Gradient Indifference. The upper panel presents results using Mean Pooling, while the lower panel uses Last Token representations.

Figure 9: Layer-wise Linear Probe Performance (Adjusted $R^{2}$ ). The curves track the capacity of a linear classifier to recover the 10 distinct moral dimensions from the residual stream. The upper panel displays the Mean Pooling results, while the bottom panel shows Last Token results. A positive $R^{2}$ indicates successful linear decoding, while a negative value implies that the moral dimensions are orthogonal to or non-linearly entangled within the latent space.

Figure 10: Layer-wise SAE Training Metrics Heatmap.The left panel displays training statistics with a sparsity coefficient clamp of $\lambda_{max}=100$ , while the right panel uses $\lambda_{max}=200$ . The heatmap visualizes Total Loss, Reconstruction Loss, $L_{0}$ Norm (sparsity), and Feature Liveness (% Alive) across all layers.

Figure 11: Impact of Targeted Fine-tuning on Moral Features. The top row displays the metrics for pre-trained SAEs with $\lambda_{max}=100$ (left) and $\lambda_{max}=200$ (right). The bottom row shows the corresponding metrics after targeted fine-tuning.

L+S	PSC1	PSC2	L+S	PSC1	PSC2	L+S	PSC1	PSC2	L+S	PSC1	PSC2	L+S	PSC1	PSC2
baseline	910	868
$\lambda_{max}=100$
0+0.1	947	927	10+0.1	952	926	20+0.1	925	902	30+0.1	923	915	35+0.1	926	927
0+0.2	932	914	10+0.2	932	896	20+0.2	933	886	30+0.2	935	900	35+0.2	938	910
0+0.3	928	904	10+0.3	928	852	20+0.3	906	821	30+0.3	927	872	35+0.3	914	874
0+0.4	880	827	10+0.4	909	752	20+0.4	843	638	30+0.4	924	840	35+0.4	758	621

Appendix

Table 3: Pilot Study on Hyperparameter Sensitivity. We report the count of perfect scores (Score=3) for Safety (PSC1) and Emotional Nuance (PSC2) across stratified layers. The analysis compares two SAE sparsity constraints (

\lambda_{max}\in\{100,200\}

) and sweeps the steering strength

\alpha

from

0.1

0.7

. L+S: Layer+Strength.

L+S	PSC1	PSC2	L+S	PSC1	PSC2	L+S	PSC1	PSC2	L+S	PSC1	PSC2	L+S	PSC1	PSC2
baseline	908	867
$\lambda_{max}=100$
0+0.1	948	930	7+0.1	938	912	14+0.1	935	902	21+0.1	939	890	28+0.1	943	910
0+0.2	929	915	7+0.2	950	920	14+0.2	927	884	21+0.2	934	844	28+0.2	930	884
1+0.1	940	918	8+0.1	935	909	15+0.1	935	917	22+0.1	923	890	29+0.1	914	895
1+0.2	931	913	8+0.2	933	894	15+0.2	920	898	22+0.2	911	860	29+0.2	927	875

Appendix

Table 4: Layer-wise Performance of Targeted Representational Alignment. We apply the SAE-based steering with

\lambda_{max}=100

across all layers (0-35) using mild intervention strengths (

\alpha\in\{0.1,0.2\}

) (L+S). The metrics denote the count of perfect responses (out of 1,000) for Safety (PSC1) and Emotion (PSC2).

L+S	Win	Lose	L+S	Win	Lose	L+S	Win	Lose	L+S	Win	Lose	L+S	Win	Lose
$\lambda_{max}=100$
0+0.1	737	200	7+0.1	715	209	14+0.1	710	219	21+0.1	637	278	28+0.1	709	226
0+0.2	707	213	7+0.2	699	231	14+0.2	644	286	21+0.2	577	364	28+0.2	662	258
1+0.1	738	202	8+0.1	724	202	15+0.1	744	178	22+0.1	664	257	29+0.1	718	216
1+0.2	716	219	8+0.2	702	229	15+0.2	700	234	22+0.2	615	334	29+0.2	668	255
2+0.1	698	242	9+0.1	719	185	16+0.1	661	268	23+0.1	685	243	30+0.1	683	234

Appendix

Table 5: Pairwise Comparison against Baseline on Flames Benchmark. L+S: Layer+Strength. Tie counts are omitted for brevity but can be inferred (

1000-\text{Win}-\text{Lose}

LLM Alignment

Large language model alignment studies methods for making models follow human intentions, values, and safety constraints across contexts.

See all 607 papers

Representational alignment yields generalizable safety in language models
Sep 3, 2026Lingyu Li, Yan Teng, Yingchun Wang +1Moral Reasoning in Language Models LLM Safety Alignment
Shanghai Artiﬁcial Intelligence Laboratory, Shanghai, China
The Straight and Narrow: Do LLMs Possess an Internal Moral Path?
Jan 15, 2026Luoming Hu, Jingjie Zeng, Liang Yang +1Moral Reasoning in Language Models Inference-Time Language Model Alignment
School of Future Technology, Dalian University of Technology, China · School of Computer Science and Technology, Dalian University of Technology, China · Key Laboratory of Social Computing and Cognitive Intelligence, Ministry of Education, China
Understanding Moral Reasoning Trajectories in Large Language Models: Toward Probing-Based Explainability
Mar 16, 2026Fan Huang, Haewoon Kwak, Jisun AnMoral Reasoning in Language Models LLM Interpretability
Indiana University Bloomington / United States

Sep 3, 2026J/K move · Enter open · S save

Representational alignment yields generalizable safety in language models

Lingyu Li, Yan Teng, Yingchun Wang, Xia Hu

Shanghai Artiﬁcial Intelligence Laboratory, Shanghai, China

Moral Reasoning in Language Models LLM Safety Alignment Adversarial Robustness of Language Models

Open paper AskPDF arXiv

Aligning large language models (LLMs) is essential for their safe deployment. Current alignment methods mainly optimize observable responses, yet models remain vulnerable when the same harmful intent is recast in unfamiliar or adversarial forms that humans can easily recognize. Prototype theory offers an account of this adaptability. Human concepts are represented around central cases, and new instances are categorized according to their graded typicality relative to these prototypes. Here we show that such categorization of moral concepts is weakly preserved in current LLMs. Across 23 LLMs, models often failed to distinguish opposed moral categories or preserve fine-grained typicality within each category. These deficits persist across parameter sizes and alignment stages. We developed representational similarity optimization, which directly aligns the latent representations in LLMs with the categorization expressed in human moral judgements, without supervising generated responses. In matched experiments using the same 251,334 moral annotations, standard behavioral alignment learned the intended moral judgements at the response level while leaving the categorization structure largely unchanged and increasing vulnerability across adversarial evaluations. Reorganizing moral categorization produced more modest gains in explicit judgements but consistently improved adversarial robustness across model scales on diverse benchmarks and attack strategies. Our findings provide functional support for the view that prototype-based categorization contributes to behavioral adaptability. They also show that transferring this representational principle to LLMs yields generalizable safety under adversarial conditions.

Abstract

LLM Alignment

Explore similar work

Representational alignment yields generalizable safety in language models

The Straight and Narrow: Do LLMs Possess an Internal Moral Path?

Understanding Moral Reasoning Trajectories in Large Language Models: Toward Probing-Based Explainability