Pragmatic Inference for Improving Social Reasoning Generalization in Large Language Models
Organizations: Indiana University Indianapolis · Nanyang Technological University · University of Mississippi · Qualcomm ·
Abstract
Achieving social reasoning, including moral reasoning, cultural awareness, and politeness, is critical for the socially reliable deployment of large language models (LLMs). A key bottleneck is limited generalization: social situations that convey similar social implicatures can be expressed through highly diverse linguistic forms, creating an enormous space of possible situations and posing a substantial computational challenge even for LLMs. From a linguistic perspective, this challenge arises because social reasoning operates at the pragmatic level, requiring a pragmatic inference process to infer social implicatures from a given situation and its underlying social context. Although moral values such as moral foundations are generally considered as the representations of the social context, how to construct the corresponding pragmatic inference steps remains underexplored. Taking moral reasoning as an instance of social reasoning, this paper develops a pragmatic inference approach that enables LLMs to infer moral implicatures by (1) specifying the inferential steps through which LLMs derive moral implicatures from language, and (2) explicitly modeling the metapragmatic links among these inferential steps to bridge the pragmatic gaps between them and make such inferences learnable for LLMs. Experimental results demonstrate that our approach substantially improves LLMs' generalization in moral reasoning, highlighting the potential of pragmatic inference for future social reasoning research.
Figures & tables
| Moral Situation: Brush your teeth with |
| laundry detergent. |
| Moral Foundation : Care. |
| Rule of Thumb (RoT) : It is harmful to brush |
| teeth with laundry detergent. |
| Moral Judgment : Disagree. |
| MFC | Moral Judgment | |||
| #MFs=1 | #MFs=2 | #MFs=3 | Average | |
| .694 | .274 | .147 | .579 | .466 |
| Accuracy(#MFs=1) | Accuracy(#MFs=2) | Accuracy(#MFs=3) | |||||||||||
| Model | Data Scale | base | base+ | CoT | ours | base | base+ | CoT | ours | base | base+ | CoT | ours |
| Llama-1B | 5000 | .501 | .696 | .661 | .890 | .402 | .545 | .475 | .856 | .396 | .460 | .428 | .806 |
| 10000 | .582 | .593 | .677 | .832 | .489 | .449 | .545 | .778 | .414 | .365 | .468 | .743 | |
| 235000 | .552 | .671 | .717 | .770 | .505 | .514 | .509 | .796 | .387 | .495 | .401 | .743 | |
| Llama-3B | 5000 | .537 | .754 | .719 | .851 | .466 | .653 | .544 | .822 | .423 | .572 | .459 | .798 |
| 10000 | .386 | .558 | .700 | .725 | .336 | .507 | .507 | .649 | .324 | .451 | .441 | .698 | |
| Model | Data Scale | base | base+ | CoT | ours -MFC |
| Llama 1B | 5000 | .433 | .567 | .521 | .851 |
| 10000 | .495 | .469 | .564 | .784 | |
| 235000 | .481 | .560 | .542 | .770 | |
| Llama 3B | 5000 | .475 | .659 | .574 | .824 |
| 10000 | .349 | .505 | .549 | .691 | |
| 235000 | .532 | .684 | .558 | .789 |
| Model | Size | base | base+ | CoT | ours- MJ | ours- joint |
| Llama -1B | 5000 | .151 | .080 | .629 | .708 | .706 |
| 10000 | .498 | .335 | .652 | .728 | .718 | |
| 23500 | .179 | .318 | .641 | .710 | .704 | |
| Llama -3B | 5000 | .160 | .304 | .635 | .712 | .750 |
| 10000 | .205 | .252 | .653 | .735 | .748 | |
| 23500 | .249 | .390 | .665 | .710 | .766 |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Pragmatic Reasoning in Language Models
Pragmatic reasoning in language models studies how models infer and communicate meanings shaped by context, intent, shared knowledge, and social norms.
See all 107 papers