Pragmatic Reasoning in Language Models

Follow this topic Ask about this topic

This field studies how language models interpret what people mean beyond literal wording, including implicatures, indirect requests, ambiguity, and culturally shaped conventions. Researchers use theory-based tests such as Rational Speech Act models, controlled benchmarks, and interactive dialogue tasks to examine how models track speaker beliefs, common ground, and conversational context. Results show that models can often recognize or judge pragmatic meanings more reliably than they can use them when generating responses, and explicit prompts or targeted training sometimes help. A central challenge is making these abilities robust across speakers, languages, and real interactions, where fluent or accurate answers do not necessarily show that a model understood the user’s intent.

Latest papers

103 papers16 papers in the last four weeks, down 16% on the four weeks before. 0.1% of all new papers.

Hey, guess what: CUES lead to inferences about typicality
Oct 23, 2026Alison Chi, Sanghyeok Choi, Hannah Rohde +1Pragmatic Reasoning in Language Models Commonsense Reasoning
University of Edinburgh
CEMIR vs. FISER: a comparative study of reasoning-based and enrichment-based approaches to pragmatic intent recognition
Oct 23, 2026Ziya Hosseynzadeh, Farjad Fallah, Farnoosh Fallah +3Multimodal Intent Recognition Pragmatic Reasoning in Language Models
Speech and Language Processing Laboratory (SLPL), Sharif University of Technology, Tehran, Iran · School of Electrical and Computer Engineering, University of Tehran, Iran · Department of Computer Engineering, Sharif University of Technology, Tehran, Iran
Reading Too Much Into It? Interpretive Preferences and Commitment in LLM Mentalisation under Ambiguity
Oct 20, 2026Stephanie Fong, Yiwen Jiang, Qingyang Xu +7Language Model Reasoning Evaluation Theory of Mind
Monash University · University of Liverpool · Orygen, The University of Melbourne
Why Do LLMs Accept Generics? Explaining Acceptability using Psycholinguistic and Model-Derived Factors
Oct 20, 2026Mervat Abassy, Zhaoxin Feng, Tatsuki Kuribayashi +2Pragmatic Reasoning in Language Models
MBZUAI · The Hong Kong Polytechnic University · Tohoku University +1
Honeyed Words, Hidden Daggers: Benchmarking and Training LLMs to See Through Mock Politeness
Oct 19, 2026Caixin Kang, Sitong Gong, Tianyu Yan +3Pragmatic Reasoning in Language Models LLM Evaluation
The University of Tokyo · Alaya Lab
Assertion and Belief: Moore's Paradox in Language Models
Oct 18, 2026Anshun Asher Zheng, Weili Cao, Fangcong Yin +2Pragmatic Reasoning in Language Models Language Model Reasoning Evaluation
The University of Texas at Austin · Duke University · New York University +1
A Thousand Faces: A Persuasion Benchmark with Experience-Grounded Persuadees Informed by Psychology and Pragmatics
Oct 18, 2026Ziqing Zhuang, Dingyi Zhang, Deyu ZhouPersuasion in Language Models Pragmatic Reasoning in Language Models
Southeast University
Are Large Reasoning Models Good Everyday Reasoners? A Comparison with Non-Reasoning Models on Logical Reasoning Tasks
Oct 15, 2026Kentaro Ozeki, Risako Ando, Takanobu Morishita +3Language Model Reasoning Evaluation LLM Reasoning
Keio University
Not X, but When: Diagnosing a Mandarin LLM Writing Tic through Contextual Appropriateness
Oct 15, 2026Yihang Peng, Yi Jing, Dewen Liu +2Pragmatic Reasoning in Language Models Language Model Generation Evaluation
Tsinghua University · Fudan University
CLUE: Uncovering Social Context for Pragmatic Understanding
Oct 15, 2026Chengfei Wu, Zhaoqing Wu, Dan GoldwasserPragmatic Reasoning in Language Models Social Reasoning
Purdue University
Entertain, then Cancel: How Thinking Makes Reasoning Models Read Some and Or Literally
Oct 15, 2026Zhizhuo Kou, Jian Yang, Pengfei Hao +4Pragmatic Reasoning in Language Models Large Reasoning Models
1The Hong Kong University of Science and Technology · 2The Hong Kong University of Science and Technology (Guangzhou) · Peking University
Lexical Redefinition Reveals Context-Sensitive Instruction Following
Oct 12, 2026Roksana Goworek, Haim DubossarskyLLM Prompting Pragmatic Reasoning in Language Models
Queen Mary University of London · University of Cambridge
Evaluating Bias in Chinese Large Language Models across Tasks and Registers: Task Operationalization Effects and Irony Classification Risks
Oct 10, 2026Mo ZhouLLM Evaluation Language Model Safety Evaluation
School of Chinese Language and Literature, Nanjing Normal University, China
Not Just an Evaluation: A Case Study of Discourse Particle Sensitivity in Language Models
Oct 9, 2026William Sheffield, Jiahui Chen, Amy Zhang +3Pragmatic Reasoning in Language Models Language Modeling
The University of Texas at Austin · University of California, San Diego
The AI's Poker Face: Contrasting Persuasion and Sycophancy in Voice-Based Multi-Turn Conversations
Oct 9, 2026Ilia Frantzi, Bernie Hogan, Max M. Lang +1Persuasion in Language Models LLM Sycophancy
University of Munich, Germany · University of Oxford, UK · GESIS - Leibniz-Institute for the Social Sciences, Germany
Evidence and Intervention: A Coupled Active-Inference Extension of Rational Speech Act Models
Oct 3, 2026Yonghyeon Gwon, Elliot Murphy, Chun Kee ChungPragmatic Reasoning in Language Models Active Inference
Neuroscience Research Institute, Medical Research Center, Seoul National University, Seoul, Republic of Korea. · Vivian L. Smith Department of Neurosurgery, McGovern Medical School, UTHealth, Houston, TX, USA · Texas Institute for Restorative Neurotechnologies, UTHealth, Houston, TX, USA
Prospective Interpretation Risk: Principled Communication Control Between LLMs
Sep 27, 2026Wanrong Yang, Rehan Deen, Julian Ma +7Multi-Agent LLM Systems Pragmatic Reasoning in Language Models
University of Liverpool · Independent Researcher · University College London +1
Larry Caused the Car to Stop, But the Model Didn't Notice: Transformer Blindness to the M-Heuristic
Sep 27, 2026Stefania Butnaru, Claudiu CreangaPragmatic Reasoning in Language Models
Faculty of Foreign Languages, University of Bucharest, Romania · Faculty of Mathematics and Computer Science, University of Bucharest, Romania · Interdisciplinary School of Doctoral Studies, University of Bucharest, Romania +1
Agentic Detection of Online Conspiracies
Sep 24, 2026Lior Biton, Oren TsurSocial Media Analysis Pragmatic Reasoning in Language Models
Department of Computer and Information Science Ben-Gurion University of the Negev, Israel
Polite but Misaligned: Evaluating LLM Politeness Judgments Against Human Pragmatic Norms
Sep 24, 2026Rong Wang, Kun Sun, Yadong GuoLLM Evaluation Pragmatic Reasoning in Language Models
University of Tübingen · Tongji University
Playing log(N)-Questions over Wikipedia Abstracts: How Per-Round Errors Compound Under Information Asymmetry
Sep 16, 2026Peter PotashLLM Evaluation Pragmatic Reasoning in Language Models
Structured Claim-Level Discourse Representations for Dense Health Narratives
Sep 16, 2026Farnoushsadat Nilizadeh, Elham Pourabbas Vafa, Shirin Nilizadeh +1Argument Mining Stance Detection
Temple University · University of Texas at Arlington
Domain-specific Pretraining Profile and Transformer Performance: Evidence from Modeling Digital Pragmatics in Arabic-English Code-switching
Sep 13, 2026Fahad Al Hussen, King Saud University, Riyadh +5Arabic NLP Pragmatic Reasoning in Language Models
King Saud University, Riyadh, Saudi Arabia · Ibb University, Ibb, Yemen
Learning to Refer from Estimated Listener Gaze
Sep 13, 2026Téa Wright, Alane SuhrText Generation Pragmatic Reasoning in Language Models
Department of Computer Science University of California, Berkeley Berkeley, CA 94704, USA
Calibrated Ambiguity in Multimodal Language Models: Humans reach for cultural references, while models describe the picture
Sep 11, 2026Cody Kommers, Mingrui Ye, Evelyn Gius +4Pragmatic Reasoning in Language Models Multimodal Large Language Models
King’s College London London, United Kingdom · University of Edinburgh Edinburgh, United Kingdom
Do LLMs Trust the Accuser or the Accusation? Measuring Belief Shifts in Werewolf
Sep 11, 2026Yu-Yu Yang, Ti-Rong Wu, Hung Guei +2Social Deduction Games Pragmatic Reasoning in Language Models
Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan · College of Artificial Intelligence, National Yang Ming Chiao Tung University, Taiwan +1
xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems
Sep 7, 2026Yongchang Peng, Qingshui Gu, Liya Zhu +31Pragmatic Reasoning in Language Models LLM Agent Evaluation
ByteDance Seed · M-A-P
XYBench: Can LLMs Respond Pragmatically to Queries with Misconceptions?
Sep 6, 2026Akhila Yerukola, Jena D. Hwang, Mingqian Zheng +5Pragmatic Reasoning in Language Models
Carnegie Mellon University · Allen Institute for AI · NVIDIA +1
You Really Didn't Get That? Benchmarking Social Pragmatic Inference for Indirect and Playful Chinese Online Comments
Sep 3, 2026Shiwei Hong, Junjie Ma, Emma Jiren Wang +5Pragmatic Reasoning in Language Models LLM Evaluation
George Mason University · Virginia Tech · University of Toronto +2
To What Extent Do Large Language Models Understand Bangla Idioms?
Sep 3, 2026Mousumi Akter, Md. Faiyaz Abdullah Sayeedi, Nurul Labib Sayeedi +1LLM Evaluation Pragmatic Reasoning in Language Models
Technical University Dortmund · Research Center Trustworthy Data Science and Security · BRAC University +1

Oct 23, 2026J/K move · Enter open · S save

Hey, guess what: CUES lead to inferences about typicality

Alison Chi, Sanghyeok Choi, Hannah Rohde, Emily Allaway

University of Edinburgh

Pragmatic Reasoning in Language Models Commonsense Reasoning

Open paper AskPDF arXiv

Throughout life, we accumulate knowledge of how typical certain events are. We use this knowledge in daily conversation, often interpreting meaning beyond what has been said explicitly. In this work, we take steps to better understand this meaning recovery, investigating both atypicality and typicality inferences. First, we construct two corpora, EventTyp and the human-validated EventTypGold. We conduct large-scale human and LLM experiments and find that for humans, three CUES predict inferences about typicality: Commonsense Knowledge, Utterance Emphasis, and Snark. However, in most experimental settings, we find that LLMs lack the prerequisite Commonsense Knowledge necessary to accurately assess an event's general typicality, and therefore struggle to follow these CUES.