Pragmatic Reasoning in Language Models

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, down 20% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 54

All topics

Prospective Interpretation Risk: Principled Communication Control Between LLMs
Sep 27, 2026Wanrong Yang, Rehan Deen, Julian Ma +7Multi-Agent LLM Systems Pragmatic Reasoning in Language Models
University of Liverpool · Independent Researcher · University College London +1
Agentic Detection of Online Conspiracies
Sep 24, 2026Lior Biton, Oren TsurSocial Media Analysis Pragmatic Reasoning in Language Models
Department of Computer and Information Science Ben-Gurion University of the Negev, Israel
Polite but Misaligned: Evaluating LLM Politeness Judgments Against Human Pragmatic Norms
Sep 24, 2026Rong Wang, Kun Sun, Yadong GuoLLM Evaluation Pragmatic Reasoning in Language Models
University of Tübingen · Tongji University
Playing log(N)-Questions over Wikipedia Abstracts: How Per-Round Errors Compound Under Information Asymmetry
Sep 16, 2026Peter PotashLLM Evaluation Pragmatic Reasoning in Language Models
Structured Claim-Level Discourse Representations for Dense Health Narratives
Sep 16, 2026Farnoushsadat Nilizadeh, Elham Pourabbas Vafa, Shirin Nilizadeh +1Argument Mining Stance Detection
Temple University · University of Texas at Arlington
Do LLMs Trust the Accuser or the Accusation? Measuring Belief Shifts in Werewolf
Sep 14, 2026Yu-Yu Yang, Ti-Rong Wu, Hung Guei +2Social Deduction Games Pragmatic Reasoning in Language Models
Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan · College of Artificial Intelligence, National Yang Ming Chiao Tung University, Taiwan +1
Calibrated Ambiguity in Multimodal Language Models: Humans reach for cultural references, while models describe the picture
Sep 14, 2026Cody Kommers, Mingrui Ye, Evelyn Gius +4Pragmatic Reasoning in Language Models Multimodal Large Language Models
King’s College London London, United Kingdom · University of Edinburgh Edinburgh, United Kingdom
xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems
Sep 7, 2026Yongchang Peng, Qingshui Gu, Liya Zhu +31Pragmatic Reasoning in Language Models LLM Agent Evaluation
ByteDance Seed · M-A-P
To What Extent Do Large Language Models Understand Bangla Idioms?
Sep 3, 2026Mousumi Akter, Md. Faiyaz Abdullah Sayeedi, Nurul Labib Sayeedi +1LLM Evaluation Pragmatic Reasoning in Language Models
Technical University Dortmund · Research Center Trustworthy Data Science and Security · BRAC University +1
VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages
Sep 1, 2026Usneek Singh, Poorvaja Veera Balaji Kumar, Parth Nanda +4Multilingual Language Model Evaluation LLM Evaluation
Georgia Institute of Technology · University of Texas at Austin
Consistency Without Alignment: Item-Sensitive Language Models Indistinguishable From Random
Sep 1, 2026Cris HuynhLLM Evaluation Pragmatic Reasoning in Language Models
Independent researcher
You Shouldn't Have Asked: A Pragmatics-Inspired Taxonomy for Evaluating LLM Refusals
Aug 31, 2026Ruoxuan Li, Pinqiao Wang, Sheng Li +1Language Model Safety Evaluation Pragmatic Reasoning in Language Models
Stony Brook University · University of Virginia
Mind the Gap: Theory-of-Mind-Grounded Friction for Epistemic Alignment
Aug 31, 2026Yifan Zhu, Kyeongmin Rim, James PustejovskyLLM Alignment Pragmatic Reasoning in Language Models
Brandeis University, Waltham, MA, USA
When Models Hear What They Expect: Diagnosing Prosodic Heuristics in Multimodal Sarcasm Detection
Aug 31, 2026Yongjian Chen, Pengfei Wei, Yiqun Sun +2Audio-Language Model Evaluation Pragmatic Reasoning in Language Models
Magellan Technology Research Institute (MTRI) · Center for Language and Cognition, University of Groningen, the Netherlands
Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It
Aug 18, 2026Quang Minh Nguyen, Luis Frentzen SalimKnowledge Conflicts in Language Models Pragmatic Reasoning in Language Models
1KAIST · 2National Taiwan University of Science and Technology
Toward a Gricean Retreat: Probing LLMs for Knowledge Boundaries and Referent Specificity
Aug 13, 2026Dananjay Srinivas, Saksham Khatwani, Maria PachecoPragmatic Reasoning in Language Models Factual Knowledge in Language Models
University of Colorado, Boulder
PragMatch: Separating Pragmatic Incongruity from Cross-Modal Mismatch in Large Vision-Language Models
Aug 10, 2026Zhanna Mukhametsharip, Vera Demberg, Varsha SureshVLM Evaluation Pragmatic Reasoning in Language Models
Saarland University, Germany · Max Planck Institute for Informatics, Germany
Pragmatic Attack Surface: Vulnerabilities of Implicit Context in Large Language Models
Aug 10, 2026Bocheng Chen, Han Zi, Roucheng Ou +5Pragmatic Reasoning in Language Models Adversarial Attacks on LLMs
University of Mississippi · Indiana University Indianapolis · Michigan State University +1
Focus particles and scalar inferences across humans and language models
Aug 8, 2026Catherine M. Brousse, Nelu D. RadpourLLM Evaluation Pragmatic Reasoning in Language Models
Department of Psychology, Florida State University
DS@GT ARC at Touché: Large Language Models for Retrieval-Augmented Debate
Aug 8, 2026Anthony Miyaguchi, Conor JohnstonLLM Evaluation LLM-as-a-Judge
Georgia Institute of Technology, North Ave NW, Atlanta, GA 30332, USA
Does Machine "know" interpersonal pragmatics? Evidence from MARBERT's learning of emoji pragmatics in Arabic digital discourse
Aug 2, 2026Mohammed Q. ShormaniArabic NLP Pragmatic Reasoning in Language Models
Ibb University · Ibb University, Ibb, Yemen
Evaluating Communicative Belief Updates in Large Language Models via Implicature Recognition and Cancellation
Jul 27, 2026Cesare Spinoso-Di Piano, Verna Dankers, Marius Mosbach +1LLM Evaluation Pragmatic Reasoning in Language Models
Mila - Quebec AI Institute & McGill University · Canada CIFAR AI Chair
Towards High-Level Semantic Intelligence
Jul 27, 2026Xiujie Song, Gefei Yang, Yining You +6Pragmatic Reasoning in Language Models
X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Nanjing University, Nanjing, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China
Computational models of pragmatic reasoning with flexible generation of meaning and expression alternatives
Jul 20, 2026Polina Tsvilodub, Fausto Carcassi, Michael FrankeComputational Cognitive Modeling Pragmatic Reasoning in Language Models
University of Tübingen · University of Amsterdam
It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief
Jul 20, 2026Kevin Du, Clara Kümpel, Michelle Wastl +1LLM Evaluation Knowledge Conflicts in Language Models
ETH Zürich · University of Zurich · UC San Diego
Interactive Task Alignment as a POMDP
Jul 17, 2026Andy Dai, Zexue He, Zhenyu Zhang +2Pragmatic Reasoning in Language Models AI Agent Benchmarks
1Stanford University
Epistemic Stance Flexibility Probing: Measuring Prompt-Conditioned Register Shift in Large Language Models
Jul 14, 2026Binwen Liu, Yilin RenLLM Evaluation Pragmatic Reasoning in Language Models
Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University Xi’an, China
Beyond Sally-Anne: Evaluating Theory of Mind in LLMs using Epistemic Schelling Points
Jul 13, 2026Roberta Rocca, Sami Boukortt, Geoff Keeling +1LLM Evaluation Theory of Mind
1Google
Flout at Your Own Risk: LLMs Struggle with Pragmatic Cooperativity Under Epistemic Asymmetry
Jul 13, 2026Hannah VanderHoeven, Abhijnan Nath, Nikhil KrishnaswamyPragmatic Reasoning in Language Models Multi-Agent Collaboration
Situated Grounding and Natural Language (SIGNAL) Lab Department of Computer Science Colorado State University Fort Collins, CO 80523, USA
Understanding Interpretation Difficulty in Harmful Online Communication: Insights from Cybercrime Communities
Jul 8, 2026Tomohiro Okatsu, Naoki Takada, Yin Min Pa Pa +2Pragmatic Reasoning in Language Models
Yokohama National University, Japan

Sep 27, 2026, cs.MAJ/K move · Enter open · S save

Prospective Interpretation Risk: Principled Communication Control Between LLMs

Wanrong Yang, Rehan Deen, Julian Ma, Yuheng Fan+6

University of Liverpool · Independent Researcher · University College London+1

Multi-Agent LLM Systems Pragmatic Reasoning in Language Models Multi-Agent Communication LLM Agent Reliability

Open paper AskPDF arXiv

Large language model (LLM) agentic systems increasingly rely on models communicating with one another, yet existing uncertainty and multi-agent methods rarely estimate how a particular receiver will interpret a message before it is sent. This matters in heterogeneous systems, where capable receivers can reconstruct different tasks from the same message. We model this as a sender-receiver problem with a latent receiver type and define prospective interpretation risk (PIR): the probability that a receiver reconstructs a task other than intended. Rather than model an LLM's full input-output behaviour, we use black-box probes relating messages, intended tasks, and receiver-specific reconstructions, yielding scalable supervision while separating interpretation from downstream capability failure. Offline, heterogeneous frozen receivers provide supervision for receiver-conditioned risk and the effects of predefined mutable message features. At deployment, history induces a posterior over receiver types, guiding message revision and selection. We introduce value of interpretation information (VoII), querying for receiver information only when its expected communication benefit exceeds its cost. Our theory characterises when receiver information has decision value and bounds such queries. Empirically, interpretation-failure rates vary by 4-13x across receivers. Receiver information reduces PIR calibration error by 68% relative to a receiver-agnostic predictor, largely by correcting receiver-specific risk levels. PIR-guided revision reduces interpretation failure by 44% relative to the original message and 40% relative to a generic rewrite, mostly through a repair that helps every receiver. VoII outperforms information-gain and random querying at matched cost on the interpretation objective it optimises, lowering interpretation failure from 3.84% to 3.79% while querying 18.2% of episodes.

Momentum

Related topics

Latest papers 54