The Imperfective Paradox in Large Language Models

Bolei Ma; Yusuke Miyao

cs.CLJan 14, 2026

The Imperfective Paradox in Large Language Models

Authors: Bolei Ma, Yusuke Miyao

Organizations: LMU Munich & MCML · The University of Tokyo

Abstract

Do Large Language Models (LLMs) genuinely grasp the compositional semantics of events, or do they rely on surface-level probabilistic heuristics? We investigate the Imperfective Paradox, a logical phenomenon where the past progressive aspect entails event realization for activities (e.g., running $\to$ ran) but not for accomplishments (e.g., building $\nrightarrow$ built). We introduce ImperfectiveNLI, a diagnostic dataset designed to probe this distinction across diverse semantic classes. Evaluating state-of-the-art open-weight models, we uncover a pervasive Teleological Bias: models systematically hallucinate completion for goal-oriented events, even overriding explicit textual cancellation. Prompting interventions partially reduce this bias but trigger a calibration crisis, causing models to incorrectly reject valid entailments for atelic verbs. Representational analyses further show that while internal embeddings often distinguish progressive from simple past forms, inference decisions are dominated by strong priors about goal attainment. Taken together, our findings indicate that these current open-weight LLMs operate as predictive narrative engines rather than faithful logical reasoners, and that resolving aspectual inference requires moving beyond prompting toward structurally grounded alignment.

Figures & tables

Class	Examples (Premise $\to$ Hypothesis)	Entail?
Telic	P: The carpenter was building a gazebo.	✗ No
	H: The carpenter built a gazebo.
Atelic	P: The boy was running in the park.	✓ Yes
	H: The boy ran in the park.

Table 1: Examples for the Imperfective Paradox. Telic verbs do not entail completion in the progressive form.

Group	Condition	Premise Example	Hypothesis	Gold Label
A	Interrupted Accomp. (Telic + Cancel)	The carpenter was building a gazebo, but a storm destroyed the frame before the roof was on .	The carpenter built a gazebo.	Contradiction ( ✗ False)
B	Interrupted Activity (Atelic + Stop)	The athletes were running on the track, but it started to hail .	The athletes ran on the track.	Entailment ( ✓ True)
C	Ambiguous Accomp. (Telic + Process)	The carpenter was building a gazebo.	The carpenter built a gazebo.	Neutral ( ? Unknown)
D	Ambiguous Activity (Atelic + Process)	The athletes were running on the track.	The athletes ran on the track.	Entailment ( ✓ True)

Table 2: Examples from the ImperfectiveNLI dataset. Group C is the critical probe, where the imperfective paradox implies a Neutral relation, but teleological bias may lead models to predict Entailment .

Prompt	Model	Acc A ( $\uparrow$ )	Acc B ( $\uparrow$ )	Acc C ( $\uparrow$ )	Acc D ( $\uparrow$ )	TBR A ( $\downarrow$ )	TBR C ( $\downarrow$ )	$\Delta_{AA}$ ( $\uparrow$ )
Zero-shot	Llama-3.1-8B-Instruct	0.47	0.85	0.02	0.98	0.51	0.98	0.00
	Mistral-7B-Instruct-v0.3	0.37	0.92	0.02	1.00	0.46	0.97	0.03
	Qwen2.5-7B-Instruct	0.20	0.98	0.47	0.97	0.55	0.53	0.44
	Yi-1.5-9B-Chat	0.35	0.94	0.02	1.00	0.57	0.98	0.02
	deepseek-llm-7b-chat	0.04	0.88	0.00	1.00	0.47	1.00	0.00
	gemma-2-9b-it	0.03	0.96	0.06	1.00	0.50	0.94	0.06

Table 3: Model performance under Zero-shot, DAP, CoT, and Counterfactual prompting strategies with Acc for A-D Groups, Teleological Bias Rate for Groups A and C (TBR A , TBR C ), and Aspectual Awareness Gap (

\Delta_{AA}

Figure 1: The trade-off in prompting ( Llama-3.1 ). As prompts become more aggressive in targeting bias (Zero-shot

\to

Counterfactual), accuracy on Group C improves significantly (Pearson

r=1.00,p=0.00

), while accuracy on Group D declines, though less consistently (

r=-0.91,p=0.09

). This trade-off illustrates the calibration bias.

Figure 2: The emergence of aspectual reasoning capabilities across model scales. It shows a significant decrease in TBR and a corresponding increase in

\Delta_{AA}

Figure 3: Average accuracy across models by semantic verb classes in Zero-shot. Motion verbs are consistently easier for models to reason about, while Creation verbs induce strong teleological bias (low accuracy in Group C) and resistance to contextual cancellation (lowest accuracy in Group A).

Panel of Figure 4 — Figure 4: Representational vs. Behavioral Divergence.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Mean Score	Std. Dev.	Agreement (%)
Grammar	4.63	0.49	99.8%
Fluency	4.53	0.56	97.7%
Adequacy	4.46	0.58	97.2%
Average	4.54	-	98.2%

Appendix

Table 4: Human evaluation results. “Agreement” denotes the percentage of instances where the scores from two evaluators differed by at most 1 point (on a 5-point scale), averaged over all evaluator pairs.

Size	Acc A	Acc B	Acc C	Acc D	TBR C ( $\downarrow$ )	$\Delta_{AA}$ ( $\uparrow$ )
1.5B	0.21	0.96	0.00	1.00	1.00	0.00
7B	0.20	0.98	0.47	0.97	0.53	0.44
14B	0.24	0.86	0.39	0.98	0.61	0.37
32B	0.53	0.90	0.91	0.92	0.09	0.83
72B	0.43	0.88	0.84	0.97	0.16	0.81

Appendix

Table 5: Performance of the Qwen2.5 family across different parameter scales (Zero-shot).

Prompt	Model	“False” Rate
Zero-shot	Llama-3.1-8B-Instruct	0.00
	Mistral-7B-Instruct-v0.3	0.01
	Qwen2.5-7B-Instruct	0.00
	Yi-1.5-9B-Chat	0.00
	deepseek-llm-7b-chat	0.00
	gemma-2-9b-it	0.00

Appendix

Table 6: “False” rate (Group C) across different prompt settings and models.

Figure 5: The Trade-off in Prompting ( Mistral ). The model shows a clear skepticism trap with strong correlations. (Group C:

r=0.95,p=0.05

; Group D:

r=-0.93,p=0.07

Figure 6: The Trade-off in Prompting ( Yi-1.5 ). Similar to Llama, Yi exhibits a sharp trade-off. (Group C:

r=0.97,p=0.03

; Group D:

r=-0.95,p=0.05

Figure 7: The Trade-off in Prompting ( Gemma-2 ). The model achieves perfect correction on Group C but collapses to zero on Group D. (Group C:

r=0.94,p=0.06

; Group D:

r=-0.95,p=0.05

Figure 8: The Trade-off in Prompting ( GLM-4 ). (Group C:

r=0.97,p=0.03

; Group D:

r=-0.79,p=0.21

Figure 9: Resistance to Prompting ( DeepSeek ). Unlike other models, DeepSeek shows weak correlation, indicating rigidity against the counterfactual prompts. (Group C:

r=0.26,p=0.74

; Group D:

r=-0.55,p=0.45

Figure 10: The Trade-off in Prompting ( Qwen2.5 ). (Group C:

r=0.88,p=0.12

; Group D:

r=-1.00,p=0.00

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.48	0.56	0.38	0.75
Mistral-7B-Instruct-v0.3	0.32	0.44	0.31	0.88
Qwen2.5-7B-Instruct	0.20	0.11	0.15	0.50
Yi-1.5-9B-Chat	0.34	0.67	0.23	0.62
deepseek-llm-7b-chat	0.05	0.00	0.05	0.00
gemma-2-9b-it	0.02	0.00	0.05	0.00

Appendix

Table 7: Accuracy Breakdown by Verb Class in Zero-shot (Group A)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.02	0.00	0.03	0.00
Mistral-7B-Instruct-v0.3	0.00	0.00	0.03	0.12
Qwen2.5-7B-Instruct	0.43	0.33	0.51	0.62
Yi-1.5-9B-Chat	0.02	0.11	0.00	0.00
deepseek-llm-7b-chat	0.00	0.00	0.00	0.00
gemma-2-9b-it	0.02	0.11	0.05	0.25

Appendix

Table 8: Accuracy Breakdown by Verb Class in Zero-shot (Group C)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.98	1.00	0.97	1.00
Mistral-7B-Instruct-v0.3	1.00	1.00	0.95	0.88
Qwen2.5-7B-Instruct	0.57	0.67	0.49	0.38
Yi-1.5-9B-Chat	0.98	0.89	1.00	1.00
deepseek-llm-7b-chat	1.00	1.00	1.00	1.00
gemma-2-9b-it	0.98	0.89	0.95	0.75

Appendix

Table 9: Bias Breakdown by Verb Class in Zero-shot (Group C)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.66	0.56	0.49	0.88
Mistral-7B-Instruct-v0.3	0.55	0.67	0.44	1.00
Qwen2.5-7B-Instruct	0.34	0.33	0.28	0.75
Yi-1.5-9B-Chat	0.61	0.78	0.44	0.75
deepseek-llm-7b-chat	0.11	0.11	0.15	0.00
gemma-2-9b-it	0.09	0.00	0.05	0.25

Appendix

Table 10: Accuracy Breakdown by Verb Class in DAP (Group A)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.30	0.11	0.54	0.12
Mistral-7B-Instruct-v0.3	0.07	0.00	0.05	0.12
Qwen2.5-7B-Instruct	0.84	0.78	0.95	1.00
Yi-1.5-9B-Chat	0.20	0.22	0.15	0.38
deepseek-llm-7b-chat	0.00	0.00	0.00	0.00
gemma-2-9b-it	0.61	0.56	0.56	0.75

Appendix

Table 11: Accuracy Breakdown by Verb Class in DAP (Group C)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.48	0.78	0.33	0.50
Mistral-7B-Instruct-v0.3	0.93	1.00	0.95	0.88
Qwen2.5-7B-Instruct	0.14	0.22	0.03	0.00
Yi-1.5-9B-Chat	0.80	0.78	0.85	0.62
deepseek-llm-7b-chat	1.00	1.00	1.00	1.00
gemma-2-9b-it	0.39	0.44	0.44	0.25

Appendix

Table 12: Bias Breakdown by Verb Class in DAP (Group C)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.14	0.22	0.13	0.25
Mistral-7B-Instruct-v0.3	0.18	0.22	0.23	0.50
Qwen2.5-7B-Instruct	0.30	0.11	0.28	0.62
Yi-1.5-9B-Chat	0.64	0.67	0.38	0.88
deepseek-llm-7b-chat	0.09	0.00	0.03	0.00
gemma-2-9b-it	0.18	0.00	0.08	0.50

Appendix

Table 13: Accuracy Breakdown by Verb Class in CoT (Group A)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.64	1.00	0.64	0.62
Mistral-7B-Instruct-v0.3	0.36	0.78	0.38	0.38
Qwen2.5-7B-Instruct	0.93	1.00	1.00	1.00
Yi-1.5-9B-Chat	0.89	0.78	0.82	0.88
deepseek-llm-7b-chat	0.23	0.33	0.62	0.38
gemma-2-9b-it	1.00	1.00	0.97	0.88

Appendix

Table 14: Accuracy Breakdown by Verb Class in CoT (Group C)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.36	0.00	0.36	0.38
Mistral-7B-Instruct-v0.3	0.61	0.22	0.56	0.62
Qwen2.5-7B-Instruct	0.07	0.00	0.00	0.00
Yi-1.5-9B-Chat	0.09	0.11	0.18	0.12
deepseek-llm-7b-chat	0.77	0.67	0.38	0.62
gemma-2-9b-it	0.00	0.00	0.03	0.12

Appendix

Table 15: Bias Breakdown by Verb Class in CoT (Group C)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.27	0.22	0.44	0.50
Mistral-7B-Instruct-v0.3	0.30	0.33	0.15	0.75
Qwen2.5-7B-Instruct	0.07	0.11	0.05	0.12
Yi-1.5-9B-Chat	0.05	0.11	0.05	0.25
deepseek-llm-7b-chat	0.23	0.44	0.26	0.12
gemma-2-9b-it	0.02	0.00	0.05	0.00

Appendix

Table 16: Accuracy Breakdown by Verb Class in Counterfactual (Group A)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	1.00	0.89	1.00	0.75
Mistral-7B-Instruct-v0.3	0.89	0.67	0.90	0.88
Qwen2.5-7B-Instruct	1.00	1.00	1.00	1.00
Yi-1.5-9B-Chat	1.00	1.00	1.00	1.00
deepseek-llm-7b-chat	0.00	0.00	0.00	0.00
gemma-2-9b-it	1.00	1.00	1.00	1.00

Appendix

Table 17: Accuracy Breakdown by Verb Class in Counterfactual (Group C)

Models	Change of State	Consumption	Creation	Motion to Goal
Llama-3.1-8B-Instruct	0.00	0.00	0.00	0.00
Mistral-7B-Instruct-v0.3	0.11	0.33	0.05	0.12
Qwen2.5-7B-Instruct	0.00	0.00	0.00	0.00
Yi-1.5-9B-Chat	0.00	0.00	0.00	0.00
deepseek-llm-7b-chat	1.00	1.00	1.00	1.00
gemma-2-9b-it	0.00	0.00	0.00	0.00

Appendix

Table 18: Bias Breakdown by Verb Class in Counterfactual (Group C)

Faithfulness of Language Model Reasoning

See all 35 papers

The Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure
Aug 25, 2026Kaiqiao Han, Yizhou SunTemporal Reasoning in Language Models LLM Evaluation
University of California, Los Angeles
Tracing the ongoing emergence of human-like reasoning in Large Language Models
May 20, 2026Paolo Morosi, Nikoleta Pantelidou, Fritz Günther +2Logical Reasoning Pragmatic Reasoning in Language Models
Departament de Filologia Catalana, Universitat Autònoma de Barcelona, Barcelona 08193, Spain. · Institut für Psychologie, Humboldt-Universitat zu Berlin, Berlin 10099, Germany. · Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona 08010, Spain
Neither True nor False: Do LLMs Represent Truth-Value Gaps in Plural Predication?
Oct 10, 2026Duaa Nakshbandi, Zhiyu An, Yiping Li +2
University of California, Merced

Aug 25, 2026J/K move · Enter open · S save

The Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure

Kaiqiao Han, Yizhou Sun

University of California, Los Angeles

Temporal Reasoning in Language Models LLM Evaluation Natural Language Inference Language Modeling

Open paper AskPDF arXiv

The imperfective paradox provides a useful test of compositional semantic analysis. Recent work constructs an NLI benchmark and reports that models frequently infer completed telic events from progressive descriptions, attributing this behavior to a Teleological Bias. It further argues that prompting interventions cause a Calibration Crisis. We reexamine the benchmark and conclusions and show that it is substantially affected by conceptual and evaluation mis-specifications. We identify three conceptual mis-specifications. In particular, Aspectual Reduction affects the benchmark construction, analysis, experiments, and conclusions. Under a strict NLI standard, 76% of Group A instances do not explicitly rule out culmination. In our native-speaker annotation, 38% of Group A examples and 29% of the Group C examples were judged to permit an alternative interpretation. To control these issues and lexical variation, we construct Lexically Matched Minimal Pairs. At the evaluation level, we formulate event-semantic NLI as a Multi-step Reasoning Problem and assess both intermediate semantic decisions and final predictions. Our results show that models often do not affirm culmination but nevertheless accept the corresponding simple-past hypothesis, a pattern we characterize as Sufficiency Bias. We further show that prompting interventions produce a Decision Shift among labels without reliably improving the underlying semantic understanding and reasoning. Intermediate and oracle-guided analyses identify two additional failure modes: errors in compositional aspectual classification and Surface-form Attraction toward surface-associated answers. Our experiments on Qwen-7B with suitable prompts, GPT-5.4, and Qwen-72B provide initial evidence for the context sensitivity of aspectual classification and suggest that these models can achieve performance comparable to that of human annotators.

Abstract

Faithfulness of Language Model Reasoning

Explore similar work

The Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure

Tracing the ongoing emergence of human-like reasoning in Large Language Models

Neither True nor False: Do LLMs Represent Truth-Value Gaps in Plural Predication?