Natural Language Inference

Also known as NLI

Natural language inference (NLI) asks whether one text entails, contradicts, or leaves undetermined a claim in another, supporting tasks such as fact checking, argument analysis, and document review. Common approaches fine-tune transformer models or prompt large language models, while linguistic features, semantic graphs, and neuro-symbolic methods aim to make the reasoning more structured and interpretable. Benchmarks range from general-purpose datasets to adversarial, multilingual, clinical, and legal settings, with diagnostics probing phenomena such as negation, lexical changes, and implicit premises. A central challenge is that high benchmark scores can coexist with reliance on dataset artifacts, sensitivity to wording, and gaps between model judgments and human interpretations; annotation ambiguity and benchmark quality also complicate evaluation.

Latest papers

40 papers4 papers in the last four weeks, down 60% on the four weeks before. 0.0% of all new papers.

An Experimental Study of Hybrid Retrieval, NLI and Open-source LLMs for Scientific Claim Verification
Nov 2, 2026Lakshmi Narayanan P, Narenkarthik K, Naren Kumar CScientific Claim Verification LLM Evaluation
Shiv Nadar University Chennai, Chennai, India
FOLLIE: Automated Reasoning over the First Dataset of Sentences with their First-Order Logical Representation for French
Nov 1, 2026Maximos SkandalisFirst-Order Logic Natural Language Inference
LIRMM, CNRS & University of Montpellier, Montpellier, France
Dual-Axis Policy Entailment: A Benchmark for Policy-Change Reasoning
Oct 31, 2026Charles Chang, Yuen Yuen Ang, Bolun SunPolicy Evaluation Language Model Reasoning Evaluation
Duke Kunshan University · Johns Hopkins University · Northwestern University
Entertain, then Cancel: How Thinking Makes Reasoning Models Read Some and Or Literally
Oct 15, 2026Zhizhuo Kou, Jian Yang, Pengfei Hao +4Pragmatic Reasoning in Language Models Large Reasoning Models
1The Hong Kong University of Science and Technology · 2The Hong Kong University of Science and Technology (Guangzhou) · Peking University
Learning Proposition-Level Links from Sentence-Level Supervision for Neuro-Symbolic NLI
Oct 10, 2026Xuyao Feng, Antonis BikakisNatural Language Inference Neuro-Symbolic Reasoning
University College London, United Kingdom
Verdicts Without Annotated Evidence: Rejection Sampling or Label-Only Post-Training for Evidence Recovery?
Oct 3, 2026Nishanth Nayakanti, Prasang Gupta, Ashutosh Bilthare +1Language Model Post-Training Rejection Sampling
PricewaterhouseCoopers, U.S.
UniBuc at SemEval-2024 Task 2: Tailored Prompting with Solar for Clinical NLI
Sep 30, 2026Marius Micluta-Campeanu, Claudiu Creanga, Ana-Maria Bucur +2Clinical Language Model Evaluation Clinical NLP
Interdisciplinary School of Doctoral Studies, ♡HLT Research Center University of Bucharest, Romania · Faculty of Mathematics and Computer Science
E-CONAN (Entailment, CONtradition And Neutral) Diagnostics Dataset Investigating Linguistic Phenomena in Arabic Natural Language Understanding
Sep 27, 2026Khloud AL Jallad, Nada Ghneim, Ghaida RebdawiLLM Evaluation Arabic NLP
Informatics Department, Higher Institute for Applied Sciences and Technology, Damascus, Syria. · Faculty of Informatics and Communication, Arab International University, Daraa, Syria.
Explaining Textual Entailment with Lexical Entailments: Using LLMs to Supply Lexical Relations for Formal Proofs
Sep 26, 2026Jorryt de Jong, Stefan Moraca, Ettore Cesari +1LLM Reasoning Natural Language Inference
Utrecht University, the Netherlands · Institute for Language Sciences, Utrecht University, the Netherlands
UO-FIE: Combining Exact-Label Supervision with Graded Utility for Factivity Inference
Sep 23, 2026Xinchen XiaoOrdinal Regression Classification
Xinjiang University
Same Scores, Different Decisions: Evaluating JEV and Language Models for Legal Document Understanding
Sep 23, 2026Fan Zhang, Yankai Chen, Zhuohan Xie +11LLM Evaluation Efficient Language Model Inference
The University of Tokyo · MBZUAI · McGill University +8
Semantic Abstraction for Natural Language Inference: a Methodological Framework for Discovering and Compensating Semantic Knowledge and Reasoning Gaps in Large Language Models
Sep 22, 2026David Torres-Moreno, Jorge Hermosillo-ValadezKnowledge Augmentation for Language Models Natural Language Inference
Linguistic Features for Interpretable Textual Entailment
Sep 21, 2026David Torres-Moreno, Jorge Hermosillo-Valadez, Asela Reig-AlamilloInterpretable ML Natural Language Inference
Centro de Investigación en Ciencias, Universidad Autónoma del Estado de Morelos · Centro Interdisciplinario de Investigación en Humanidades, Universidad Autónoma del Estado de Morelos
Can We Do Interpretable NLI with Graphs Based on Atomic Propositions?
Sep 15, 2026Younes Boufouss, Luc Pommeret, Thomas Gerald +2Interpretable ML Natural Language Inference
STL, LISN · STL, CNRS, LISN · LISN, STL
E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets
Sep 10, 2026Khloud AL Jallad, Nada Ghneim, Ghaida RebdawiMultilingual Language Model Evaluation Arabic NLP
Higher Institute for Applied Sciences and Technology, Damascus, Syria. · Arab International University, Daraa, Syria.
LexFlip: A Dissociation Diagnostic for Legal Meaning Preservation Metrics
Sep 4, 2026Gaurab BaralLegal NLP Semantic Textual Similarity
University of Cincinnati
The Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure
Aug 25, 2026Kaiqiao Han, Yizhou SunTemporal Reasoning in Language Models LLM Evaluation
University of California, Los Angeles
Structure, Association, and Decision Value: Representation-Based Difficulty Estimation for Adaptive Inference in African-Language NLI
Aug 19, 2026Toheeb OgunadeAdaptive Inference African Language Processing
University of Lagos
Identifying Implicit Premises for Logical Reconstruction of Argument Graphs
Aug 19, 2026Xuyao Feng, Anthony HunterComputational Argumentation Neuro-Symbolic Reasoning
Department of Computer Science, University College London, United Kingdom
Selection Shapes the Boundary: A Replication of Monotonicity and Label Agreement in Unselected NLI Populations
Jul 21, 2026Haram ChoiAnnotator Disagreement Selection Bias
University of Bremen
How Much Human Label Variation Does Formal Semantic Structure Explain? Group-Level Effects and Item-Level Ceilings in NLI
Jul 17, 2026Haram ChoiAnnotator Disagreement Natural Language Inference
University of Bremen
SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs
Jun 3, 2026Jingyao Wu, Ashley Wang, Keane Ong +2RL for Language Models LLM Alignment
MIT Media Lab, Massachusetts Institute of Technology · National University of Singapore
From Script to Semantics: Prompting Strategies for African NLI
Jun 2, 2026Anuj Tiwari, Terry Oko-odion, Hannah NwokochaMultilingual Language Model Evaluation LLM Prompting
Noida Institute of Engineering and Technology1, India1 · ML Collective1,2,3, Nigeria2,3
SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding
Jun 2, 2026Peerawat Chomphooyod, Jian Gang Ngui, Yosephine Susanto +5Multilingual Language Model Evaluation Cross-Cultural Language Model Evaluation
1Chulalongkorn University · 2AI Singapore · 3MBZUAI +1
Sample-Size Scaling of the African Languages NLI Evaluation
Jun 2, 2026Anuj Tiwari, Oluwapelumi Ogunremu, Terry Oko-odion +2Multilingual Language Model Evaluation Low-Resource Language Processing
Noida Institute of Engineering and Technology1 · ML Collective1,2,3,4,5
Fixing FOLIO and MALLS: Verified Annotations and an LLM-assisted Framework to Focus Human Relabeling
Jun 1, 2026Andrea Brunello, Cristian Curaba, Luca Geatti +3LLM-Assisted Annotation Human-in-the-Loop Annotation
University of Udine, Italy
Metric-Dependent Annotation Saturation for Learning from Label Distributions
May 28, 2026Guneet KohliSoft-Label Learning Annotator Disagreement
Apple
Is Inference Mediated by Distinct Semantic Structures in LLMs? A Mechanistic Interpretation
May 25, 2026Nura Aljaafari, Marco Valentino, André FreitasMechanistic Interpretability Language Modeling
University of Manchester, United Kingdom · University of Sheffield, United Kingdom · Idiap Research Institute, Switzerland +1
Tracing the ongoing emergence of human-like reasoning in Large Language Models
May 20, 2026Paolo Morosi, Nikoleta Pantelidou, Fritz Günther +2Logical Reasoning Pragmatic Reasoning in Language Models
Departament de Filologia Catalana, Universitat Autònoma de Barcelona, Barcelona 08193, Spain. · Institut für Psychologie, Humboldt-Universitat zu Berlin, Berlin 10099, Germany. · Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona 08010, Spain
ViLegalNLI: Natural Language Inference for Vietnamese Legal Texts
Apr 30, 2026Nhung Thi-Hong Duong, Mai Ngoc Ho, Tin Van Huynh +1Legal Reasoning Natural Language Inference
University of Information Technology, Ho Chi Minh City, Vietnam. · Vietnam National University, Ho Chi Minh City, Vietnam.

Nov 2, 2026J/K move · Enter open · S save

An Experimental Study of Hybrid Retrieval, NLI and Open-source LLMs for Scientific Claim Verification

Lakshmi Narayanan P, Narenkarthik K, Naren Kumar C

Shiv Nadar University Chennai, Chennai, India

Scientific Claim Verification LLM Evaluation Natural Language Inference

Open paper AskPDF arXiv

Claim verification is the task of deciding whether a source text supports a claim, contradicts it, or lacks sufficient evidence. Natural Language Inference (NLI) models can judge whether a single sentence supports or contradicts a claim, but they struggle when the evidence is spread across several sentences. Conversely, Large Language Models (LLMs) can reason across sentences, but may draw conclusions the text does not support. In this experimental study, we test whether a conservative fusion rule, labeling a claim Support or Contradict only when both models agree and defaulting to Not Enough Evidence otherwise, combines their strengths. We construct a benchmark of 2,035 claims from SciFact and CliniFact (858 Support, 152 Contradict, 1,025 Not Enough Evidence). We retrieve the top five most relevant sentences per claim using a hybrid of semantic and lexical (BM25) retrieval, which slightly outperforms either method alone. These retrieved sentences are then provided to every verification model. We then compare a DeBERTa-v3-base-mnli-fever-anli NLI baseline against Llama 3.1 8B, Qwen3 8B, and Mistral 7B. Qwen3 achieves the highest Macro-F1 (0.7937), followed by Llama 3.1 (0.7204) and Mistral (0.6388), while the NLI baseline reaches only 0.3157. Contrary to our hypothesis, fusion degrades performance, dropping Macro-F1 to 0.7392, 0.6730, and 0.5991, respectively. To understand this decline, we analyze agreements, disagreements, and common error patterns between the NLI and LLM verifiers. Our error analysis shows that NLI and LLM disagreements often result in correct LLM predictions being overturned.