Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs
Organizations: Department of Language Science and Technology, Universität des Saarlandes · TU Darmstadt, Germany.
Abstract
Large Language Models (LLMs) are increasingly deployed across diverse real-world applications and user communities. As such, it is crucial that these models remain both morally grounded and knowledge-aware. In this work, we uncover a critical limitation of current LLMs -- their tendency to prioritize moral reasoning over commonsense understanding. To investigate this phenomenon, we introduce CoMoral, a novel benchmark dataset containing commonsense contradictions embedded within moral dilemmas. Through extensive evaluation of ten LLMs across different model sizes, we find that existing models consistently struggle to identify such contradictions without prior signal. Furthermore, we observe a pervasive narrative focus bias, wherein LLMs more readily detect commonsense contradictions when they are attributed to a secondary character rather than the primary (narrator) character. Our comprehensive analysis underscores the need for enhanced reasoning-aware training to improve the commonsense robustness of large language models.
Figures & tables
| Category | Example Contradictions |
|---|---|
| Physical | Sun rising in the west; Aeroplane sailing mid-sea; Carrying a car on shoulder; Casting a shadow in complete darkness; Magnet repelling iron |
| Biological | Kindergarten student having naturally grown beard; Fishes swimming in waterless ponds; Birds crawling up a tree; Flying ostrich; Coconut tree producing mango |
| Temporal | One hour after 7 AM is 9 AM; In 100 AD, almost a century before Jesus Christ’s birth; In 50 BC, shortly after Jesus Christ’s birth; Going to school during holidays |
| Social | A teetotaler drinking alcohol; Eating while on a fast; Kindergarten casting vote |
| Environmental | Freezing at desert during daytime; Experiencing snowfall in desert; Glacier in desert; Flood-like situation in desert; Gentle breeze during tornado; Midnight sun at equator; Moonlight during newmoon; Sweet natural seawater |
| Conceptual | Making coffee with tea leaves; Making rice on bread toaster; Enjoying shade of a rose tree; Gazing stars during daylight; Footballer scoring runs |
| Model | Implicit | Explicit | ||||
|---|---|---|---|---|---|---|
| Overall | Primary | Secondary | Overall | Primary | Secondary | |
| LLaMA 1B | 0.152 | 0.138 | 0.175 | 0.261 | 0.263 | 0.270 |
| LLaMA 3B | 0.194 | 0.168 | 0.237 | 0.565 | 0.553 | 0.571 |
| LLaMA 8B | 0.261 | 0.220 | 0.301 | 0.845 | 0.831 | 0.868 |
| Qwen 0.5B | 0.087 | 0.082 | 0.096 | 0.321 | 0.243 | 0.421 |
| Qwen 1.5B | 0.103 | 0.092 | 0.122 | 0.245 | 0.201 | 0.285 |
Moral Reasoning in Language Models
Moral reasoning in language models studies how AI systems judge dilemmas, weigh competing values, explain decisions, and stay consistent across contexts.
See all 114 papers