Pragmatic Reasoning in Language Models

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, down 20% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 54

All topics

Retroactive Chain-of-Thought (RetroCoT): Forensic Reconstruction Prompts as a Safety Diagnostic Across Model Generations
Jul 6, 2026Samira HajizadehPragmatic Reasoning in Language Models Adversarial Attacks on LLMs
Department of Computer Science Columbia University New York, NY, USA
A Retrieval-Augmented Framework for Detecting and Resolving Pragmatic Ambiguities in Natural Language Requirements
Jul 5, 2026Pavithra PM Nair, Preethu Rose AnishRequirements Engineering Pragmatic Reasoning in Language Models
Tata Consultancy Services Pune, India
They Infer What You Meant: Models Represent Communicative Intent More Reliably Than They Act On It
Jul 3, 2026Alex KwonPragmatic Reasoning in Language Models
Independent Researcher
Beyond Skepticism: Evaluating LLMs Pedagogical Intent Reasoning with the Adaptive Pedagogical Vigilance Framework
Jul 2, 2026Minghao Chen, Ruihan Zhou, Jiayi Tang +3Pragmatic Reasoning in Language Models AI in Education
Department of Computer Science Zhejiang University
Adversarial Pragmatics for AI Safety Evaluation: A Benchmark for Instruction Conflict, Embedded Commands, and Policy Ambiguity
Jul 1, 2026Brett ReynoldsLLM Safety Benchmarks Language Model Safety Evaluation
Humber Polytechnic & University of Toronto
"Don't Say It!": Constraints, Compliance, and Communication when Language Models Play Taboo
Jul 1, 2026Sara Candussio, Francesca Padovani, Daniel Scalena +1LLM Grounding Language Model Generation Evaluation
AILab, MIGe, Univeristy of Trieste, Italy · Center for Language and Cognition (CLCG), University of Groningen, The Netherlands · University of Milano - Bicocca, Italy
From Propositional to Perceptual Asymmetry: Extending Frictive Policy Optimization to Asymmetric Partial Information Dialogue
Jun 29, 2026Yifan Zhu, Kyeongmin Rim, James PustejovskyPragmatic Reasoning in Language Models Policy Optimization
Brandeis University, Waltham, MA, USA
When Helpfulness Overrides Causal Caution: Context-Dependent Suppression and Recovery in LLMs
Jun 23, 2026Hiroshi OkumuraLLM Auditing Pragmatic Reasoning in Language Models
Mitsubishi Research Institute, Inc. · Kobe University
The Register Gap: A Meaning Intelligence Framework for Nigerian Public Discourse
Jun 18, 2026Celestine AchiMultilingual Language Model Evaluation LLM Evaluation
AGENTPR™ / AI-Powered PR / Cihan Digital Academy
PEC-Home: Interpretation of Progressively Elliptical Commands in Smart Homes
Jun 17, 2026Yingyu Shan, Zeming Liu, Silin Li +4Spoken Dialogue Systems Pragmatic Reasoning in Language Models
Beijing Institute of Technology · Beihang University · Baidu Inc.
PragReST: Self-Reinforcing Counterfactual Reasoning for Pragmatic Language Understanding
Jun 17, 2026Jihyung Park, Minchao Huang, Leqi Liu +1Pragmatic Reasoning in Language Models Counterfactual Reasoning in Language Models
The University of Texas at Austin
LLMs Infer Cultural Context but Fail to Apply It When Responding
Jun 16, 2026Yisong Miao, Jian Zhu, Vered ShwartzPragmatic Reasoning in Language Models Cultural Bias in Language Models
National University of Singapore · University of British Columbia · Canada CIFAR AI Chair, Vector Institute
Implicit vs. Explicit Prompting Strategies for LVLMs in Referential Communication
Jun 16, 2026Peter Zeng, Amie J. Paige, Weiling Li +3LLM Prompting Pragmatic Reasoning in Language Models
Department of Computer Science 2Department of Psychology 3Department of Linguistics 4Institute for Advanced Computational Science Stony Brook University
ttda704 at SemEval-2026 Task 6: Structured Chain-of-Thought Prompting for Political Evasion Detection
Jun 14, 2026Tai Tran Tan, An Dinh ThienCoT Reasoning Pragmatic Reasoning in Language Models
University of Information Technology, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam
LoSoNA: A Benchmark for Local Social Norm Adaptation in Group Conversations
Jun 12, 2026Mateusz Winiarek, Maksymilian Bilski, Mateusz JacniackiNormative Conformity in Language Models Pragmatic Reasoning in Language Models
Humalike Research
Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthropomorphism, and Maxims
Jun 1, 2026Siddhesh Milind Pawar, Sarah Masud, Haneul Yoo +2LLM Auditing Pragmatic Reasoning in Language Models
♠University of Copenhagen · ♣KAIST
Unveiling the Limits of Large Language Models in Inferring Pragmatic Meaning from Non-Verbal Responses
Jun 1, 2026Sugyeong Eo, Heuiseok LimLLM Evaluation Pragmatic Reasoning in Language Models
Department of Software, Yonsei University Mirae Campus, Republic of Korea · Department of Computer Science and Engineering, Korea University, Republic of Korea
Can Large Language Models Handle Discourse Particles? A Case Study of Colloquial Malay
May 27, 2026Mariah Al Giptiah Binte Yusoff, Jakin Tan, Bocheng Chen +2Multilingual Language Model Evaluation Pragmatic Reasoning in Language Models
Nanyang Technological University · University of Mississippi · Indiana University
KARMA: Karma-Aligned Reward Model Adaptation
May 26, 2026Jared Scott, Jesse RobertsReward Modeling RL for Language Models
Tennessee Tech University Cookeville, Tennessee
DRInQ: Evaluating Conversational Implicature with Controlled Context Variation
May 22, 2026Hirona Jacqueline Arai, Xiang RenLLM Evaluation Pragmatic Reasoning in Language Models
University of Southern California
Tracing the ongoing emergence of human-like reasoning in Large Language Models
May 20, 2026Paolo Morosi, Nikoleta Pantelidou, Fritz Günther +2Logical Reasoning Pragmatic Reasoning in Language Models
Departament de Filologia Catalana, Universitat Autònoma de Barcelona, Barcelona 08193, Spain. · Institut für Psychologie, Humboldt-Universitat zu Berlin, Berlin 10099, Germany. · Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona 08010, Spain
Presupposition and Reasoning in Conditionals: A Theory-Based Study of Humans and LLMs
May 18, 2026Tara Azin, Yongan Yu, Raj Singh +1LLM Evaluation Pragmatic Reasoning in Language Models
Department of Cognitive Science, Carleton University · School of Computer Science, McGill University · Mila – Quebec AI Institute
Evaluating Pragmatic Reasoning in Large Language Models: Evidence from Scalar Diversity
May 9, 2026Ye-eun ChoLLM Evaluation Pragmatic Reasoning in Language Models
Sungkyunkwan University Seoul, South Korea
How Hypocritical Is Your LLM judge? Listener-Speaker Asymmetries in the Pragmatic Competence of Large Language Models
Apr 17, 2026Judith Sieker, Sina ZarrießLLM Evaluation LLM-as-a-Judge
Computational Linguistics, Department of Linguistics Bielefeld University, Germany

Jul 6, 2026, cs.CLJ/K move · Enter open · S save

Retroactive Chain-of-Thought (RetroCoT): Forensic Reconstruction Prompts as a Safety Diagnostic Across Model Generations

Samira Hajizadeh

Department of Computer Science Columbia University New York, NY, USA

Pragmatic Reasoning in Language Models Adversarial Attacks on LLMs LLM Safety Alignment LLM Jailbreak Attacks LLM Safety Evaluation

Open paper AskPDF arXiv

Safety alignment in large language models is typically evaluated against direct, imperative harmful requests. We show that this alignment is highly conditioned on pragmatic register: models that refuse a direct request frequently comply when the same underlying objective is expressed through a different communicative stance. This suggests that current alignment policies are not invariant to semantic equivalence, but remain sensitive to how a request is pragmatically framed. We introduce Retroactive Chain-of-Thought (RetroCoT), a single-turn attack that reframes harmful requests as forensic reconstruction tasks. Rather than requesting harmful instructions directly, RetroCoT presupposes that the harmful outcome has already occurred and asks the model, acting as a forensic analyst, to reconstruct in reverse the causal chain that produced it. On AdvBench (n=50), RetroCoT achieves attach success rate of 58% on gpt-4o and 52% on gpt-4o-mini, compared with direct-request baselines of 0% and 4%, respectively. We further identify a pronounced generation gap: GPT-5-family models refuse RetroCoT entirely, explicitly identifying the reconstruction premise in their refusal rationales, consistent with explicit coverage of this reconstruction register. However, this robustness does not generalize across pragmatic forms. A single adversarial feedback turn presenting an existing forensic reconstruction response alongside evaluator critique raises ASR from 0% to 48% on GPT-5.4-mini and from 58% to 94% on GPT-4o; a control condition omitting the fabricated low score achieves 85% on GPT-5.4-mini, indicating that the operative element is pragmatic continuation within the established forensic frame rather than score manipulation. These results suggest that frontier-model alignment remains conditioned on pragmatic framing rather than semantic intent, and that new pragmatic registers can continue to expose a...

Momentum

Related topics

Latest papers 54