Larry Caused the Car to Stop, But the Model Didn't Notice: Transformer Blindness to the M-Heuristic
Authors: Stefania Butnaru, Claudiu Creanga
Organizations: Faculty of Foreign Languages, University of Bucharest, Romania · Faculty of Mathematics and Computer Science, University of Bucharest, Romania · Interdisciplinary School of Doctoral Studies, University of Bucharest, Romania · HLT Research Center, University of Bucharest, Romania
Modern transformer models excel at capturing semantic relationships through sentence embeddings, yet their ability to perform pragmatic reasoning remains understudied. This paper investigates whether encoder-based transformers such as DeBERTa employ the M-Heuristic (the neo-Gricean principle that marked linguistic forms implicate marked meanings). We test this hypothesis by contrasting lexical causatives (e.g., Larry stopped the car'') with periphrastic causatives (e.g., Larry caused the car to stop'') using a Natural Language Inference framework. Our experiments across 188 conditions with 15 ambitransitive verbs reveal that DeBERTa, RoBERTa, and BART show no evidence of capturing the pragmatic distinction between these forms, with DeBERTa predicting ``Neutral'' for 100% of cases. Probing analysis initially suggested a representation-use dissociation, but control experiments reveal the probe was tracking syntactic complexity, not causative pragmatics. Semantic similarity over 30 triplets places periphrastic causatives closer to unmediated manner descriptions in 29/30 cases, opposite to M-Heuristic predictions in the embedding space. Under explicit metalinguistic framing, Gemini Flash-Lite reaches 100% with item-specific traces, so the principle is available under instruction yet unused in default NLI.
Figures & tables
Figure 1: One fixed NLI instance from our tests (M-Heuristic gold: Entailment ). DeBERTa, RoBERTa, BART, and Gemini under direct NLI all predict Neutral on this item. Across all 188 items: chain-of-thought recovers 21.8%; metalinguistic prompting reaches 100%.
Figure 2
Figure 3: Human description preferences from Wolff (2003) . For unmediated events, participants chose lexical causatives ( ∼ 58%) over periphrastic ( ∼ 38%); for mediated events, the pattern reversed sharply, with periphrastic descriptions selected ∼ 85% of the time and lexical only ∼ 5%. This double dissociation is the human baseline our NLI models fail to replicate.
Cond.
Premise Type
Hypothesis Type
Expected Label
1
Lexical causative
Normal interpretation
Entailment
2
Lexical causative
Unusual interpretation
Contradiction
3
Periphrastic causative
Normal interpretation
Contradiction
4
Periphrastic causative
Unusual interpretation
Entailment
Table 1: Experimental Conditions and Expected Human Judgments
Premise Type
Premise
Hypothesis
Expected
Lexical
The chef melted the butter.
The chef did this in a normal, direct way.
Entailment
Periphrastic
The chef caused the butter to melt.
The chef did this in an unusual or indirect way.
Entailment
Table 2: Sample Experimental Items
Model
Correct / Total
Accuracy (%)
DeBERTa-v3-large
0 / 188
0.0
RoBERTa-large
5 / 188
2.7
BART-large
0 / 188
0.0
Table 3: Overall Accuracy on M-Heuristic Inference Task
Expected ↓ / Predicted →
Entailment
Neutral
Contradiction
Entailment
0
94
0
Contradiction
0
94
0
Table 4: Confusion Matrix: DeBERTa-v3-large
Prompt condition
Correct / Total
Accuracy (%)
Direct NLI
0 / 188
0.0
Chain-of-thought
41 / 188
21.8
Metalinguistic (M-Heuristic)
188 / 188
100.0
Table 5: Gemini Flash-Lite accuracy on pragmatic gold labels by prompt condition ( n=188 )
Layer Range
Accuracy
Initial Interpretation
Layer 0 (embedding)
50.0%
Chance level
Layers 1–3
80–99%
Distinction emerges
Layers 4–11
100.0%
Perfect separation
Layers 12–24
94–99%
Slight degradation
Table 6: Linear Probe Accuracy: Lexical vs. Periphrastic Causatives
Layer
Causative Probe
Control Probe
Layer 4
100.0%
87.5%
Layer 8
100.0%
87.5%
Layer 12
97.8%
87.5%
Layer 16
96.7%
97.5%
Layer 20
98.9%
97.5%
Table 7: Probe Accuracy on Syntactic Controls by Layer
Figure 4: Probe accuracy on syntactic controls by layer. Probes achieve 87.5–97.5% accuracy on non-causative periphrastic constructions, matching causative probe accuracy and indicating that the original 100% result tracks syntactic complexity rather than causative pragmatics.
Figure 5: Semantic similarity results. Periphrastic causative anchors cluster with unmediated (direct) descriptions in 29/30 cases, showing that DeBERTa’s representations encode the pragmatically wrong grouping relative to M-Heuristic predictions.
Pragmatic reasoning in language models studies how models infer and communicate meanings shaped by context, intent, shared knowledge, and social norms.