Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem

Tara Azin; Daniel Dumitrescu; Diana Inkpen; Raj Singh

cs.CLMar 9, 2026

Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem

Authors: Tara Azin, Daniel Dumitrescu, Diana Inkpen, Raj Singh

Organizations: Carleton University, Canada · University of Ottawa, Canada

Abstract

We investigate how language models handle the proviso problem, an unresolved issue in pragmatics where presuppositions in conditional sentences diverge between theoretical and human interpretations. We reformulate this phenomenon as a Natural Language Inference task and introduce a diagnostic dataset designed to probe presupposition projection in conditionals. We evaluate RoBERTa, DeBERTa, LLaMA, and Gemma using explainability analyses. The results show that models broadly align with human judgments but rely on shallow pattern matching rather than semantic or pragmatic reasoning. Our work provides the first computational evaluation framework for the proviso problem and highlights the need for diagnostic, multi-method approaches to assess pragmatic competence and context-dependent meaning in language models.

Figures & tables

Figure 1: The proviso problem illustrated. Given the sentence If Theo hates sonnets, so does his wife , formal semantic theories predict a conditional presupposition, while human speakers typically accommodate an unconditional presupposition. Our work investigates where language models fall in this theory-human divide.

Dependent case (DEP)
Premise	If Randolf is a carpenter, he’ll use his beading tools for designing.
Hypothesis	Randolf has beading tools.
Independent case (IND)
Premise	If Lisa finishes the meeting early, she’ll never drive a sports car again .
Hypothesis	Lisa has driven a sports car before.

Table 1: Examples of DEP and IND cases. In DEP, the antecedent

A

logically relates to the presupposition

p

(here, being a carpenter relates to using beading tools). In IND,

A

and

p

are unrelated (finishing a meeting has no connection to driving sports cars). Presupposition triggers are underlined.

Subset	Premise	Hypothesis	Human	Theory
1	DEP: If John is a scuba diver, he’ll bring his wetsuit. IND: If Lisa finishes meeting early, she’ll never drive a sports car again.	John has a wetsuit. Lisa has driven a sports car before.	E	N
2	Conjunction: If Sarah attends the conference and her friend meets her at the hotel, it’s okay. Disjunction: Either Sarah doesn’t attend conference, or she and her friend do. Belief: Sarah believes her friend is upstairs.	Sarah has a friend.	E	N
3	Related: If Nadia attends conference, she’ll never watch Star Wars again. Somewhat related: If Nadia attends conference, she’ll never watch a George Lucas movie again. Unrelated: If Nadia attends conference, she’ll never watch Titanic again.	Nadia has watched Star Wars.	E/N ∗	N
4	Related: If Ali attends conference, his brother will meet her at university. Unrelated: If Ali attends conference, his brother will get rid of his old car.	Ali has a brother.	E	N

Table 2: Examples from the four subsets with corresponding human and theory-based labels. ∗ In Subset 3, the human label is E for the related variant where the trigger matches the hypothesis, and N for somewhat related and unrelated variants where the modified trigger no longer semantically supports the hypothesis.

Figure 2: IG visualization showing token-level attribution. Darker shading indicates higher IG values, with the presupposition trigger receiving the strongest attribution.

Figure 4: Accuracy across sentence types in Subset 2. The horizontal axis shows accuracy percentages, and the values inside the bars indicate the corresponding trigger IG ratios.

Figure 5: Models’ accuracies in zero-shot evaluation across four subsets, using human and theory-based labels.

Figure 6: Accuracy and trigger IG ratios across sentence types in Subset 3

Figure 7: Subset 4 accuracy by sentence type and model, with bar colours indicating K2->special token attention on the corresponding examples.

Figure 8: Average attention between tokens across all layers and heads in Gemma, for the sentence If Theo’s train is delayed, his wife will wait for him at the station , and its hypothesis Theo has a wife . Brighter cells indicate higher attention values. First column corresponds to the special token attention.

Pragmatic Reasoning in Language Models

Pragmatic reasoning in language models studies how models infer and communicate meanings shaped by context, intent, shared knowledge, and social norms.

See all 103 papers

Presupposition and Reasoning in Conditionals: A Theory-Based Study of Humans and LLMs
May 18, 2026Tara Azin, Yongan Yu, Raj Singh +1LLM Evaluation Pragmatic Reasoning in Language Models
Department of Cognitive Science, Carleton University · School of Computer Science, McGill University · Mila – Quebec AI Institute
Tracing the ongoing emergence of human-like reasoning in Large Language Models
May 20, 2026Paolo Morosi, Nikoleta Pantelidou, Fritz Günther +2Logical Reasoning Pragmatic Reasoning in Language Models
Departament de Filologia Catalana, Universitat Autònoma de Barcelona, Barcelona 08193, Spain. · Institut für Psychologie, Humboldt-Universitat zu Berlin, Berlin 10099, Germany. · Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona 08010, Spain
Same Formulas, Different Semantics: Do Language Models Follow Modal Logic Specifications?
Aug 5, 2026Réemi Andrieu, Damien SileoLogical Reasoning LLM Evaluation
Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 - CRIStAL, F-59000 Lille, France

May 18, 2026J/K move · Enter open · S save

Presupposition and Reasoning in Conditionals: A Theory-Based Study of Humans and LLMs

Tara Azin, Yongan Yu, Raj Singh, Olessia Jouravlev

Department of Cognitive Science, Carleton University · School of Computer Science, McGill University · Mila – Quebec AI Institute

LLM Evaluation Pragmatic Reasoning in Language Models LLM Reasoning

Open paper AskPDF arXiv

Presupposition projection in conditionals is central to theories of meaning and pragmatics, yet it remains largely unevaluated in large language models. We address this gap through a parallel behavioral study comparing human judgments and LLM predictions on a normed dataset of conditional sentences that controls the relation between the antecedent and the projected presupposition. We collect likelihood ratings from 120 participants and four LLMs under matched contextual conditions. Results show that humans integrate probabilistic and pragmatic cues in their judgment, whereas LLMs show variable alignment with human patterns. Using a linguistically motivated checklist within an LLM-as-a-Judge framework, we further evaluate model reasoning. We observe models that best match human ratings often lack coherent pragmatic reasoning, while models with stronger reasoning produce less human-like judgments. These findings suggest that LLMs' performance on such tasks may result from surface pattern matching rather than pragmatic competence. Our findings highlight the importance of benchmarks grounded in linguistic theory for comparing humans and models.

Abstract

Pragmatic Reasoning in Language Models

Explore similar work

Presupposition and Reasoning in Conditionals: A Theory-Based Study of Humans and LLMs

Tracing the ongoing emergence of human-like reasoning in Large Language Models

Same Formulas, Different Semantics: Do Language Models Follow Modal Logic Specifications?