Neither True nor False: Do LLMs Represent Truth-Value Gaps in Plural Predication?

Duaa Nakshbandi; Zhiyu An; Yiping Li; Sachin Giri; Wan Du

cs.CLOct 10, 2026

Neither True nor False: Do LLMs Represent Truth-Value Gaps in Plural Predication?

Authors: Duaa Nakshbandi, Zhiyu An, Yiping Li, Sachin Giri, Wan Du

Organizations: University of California, Merced

Abstract

Large Language Models (LLMs) are increasingly being used to summarize and verify claims in critical sources that inform consequential decisions. Such claims are full of plural definites. If three of five patients improved after taking drug A, the claim the patients improved after drug A is not true, since two did not, yet its negation, the patients didn't improve after drug A, is not true either, since three did. In formal semantics, this truth-value gap is known as the Homogeneity Gap: both the plural definite and its negation are neither true nor false when the property holds for only a fraction of the plural referent. A model that cannot represent this gap turns mixed evidence into a clean verdict, an error that can be costly in high-stakes use. We introduce HomogeneityNLI, a diagnostic dataset of 1,000 minimal pair items crossing determiner, polarity, and group composition across ten logical conditions. Evaluating seven open-weight models, we uncover a pervasive Bivalence Bias: models assign plural definites their homogeneous truth conditions. In the zero-shot setting, no model recognizes the gap for both polarities in more than 15% of scenarios, and negated definites are judged False in 84% of cases. Explicit linguistic rules, chain-of-thought, and enumeration prompts produce unstable, model-specific effects that often entrench the bias. Our findings indicate that current open-weight LLMs acquire the truth conditions of plural predication but not its trivalent logic, and cannot yet be trusted to summarize or verify claims whose evidence is mixed.

Figures & tables

P: Five patients took drug A. Three of them improved
P: after taking drug A and two did not.
Determiner	Hypothesis	Label
the	H: The patients improved after taking drug A.	? Neither
the + not	H: The patients didn’t improve after taking drug A.	? Neither
all	H: All the patients improved after taking drug A.	✗ False
not all	H: Not all the patients improved after taking drug A.	✓ True

Table 1: Examples of the Homogeneity Gap. Given a mixed group, a plural definite and its negation are neither true nor false, while their quantified counterparts are classically false or true.

Group	Condition	Context	Hypothesis	Gold Label
A	Gap Definite ( the + P, mixed group)	GAP	The hearts were red.	Neither ( ? )
B	Gap Negated Definite ( the + not-P, mixed group)	GAP	The hearts weren’t red.	Neither ( ? )
C	Gap Universal ( all + P, mixed group)	GAP	All the hearts were red.	False ( ✗ )
D	Gap Negated Universal ( not all + P, mixed group)	GAP	Not all the hearts were red.	True ( ✓ )
E	Uniform Definite ( the + P, all members)	ALL	The hearts were red.	True ( ✓ )
F	Uniform Negated Definite ( the + not-P, all members)	ALL	The hearts weren’t red.	False ( ✗ )

Table 2: The ten logical conditions of HomogeneityNLI , illustrated with one scenario. All premises begin with “ Five hearts were drawn on the card. ”, followed by “ Three of them were red and two were not. ” (GAP), “ Every one of them was red. ” (ALL), or “ None of them were red. ” (NONE). Groups A and B are the critical probe, where homogeneity implies a truth-value gap, but a bivalent reasoner is forced to choose True or False . ∗ True under the literal meaning of some ; False under the scalar implicature some but not all .

Prompt	Model	A	B	C	D	Ctrl	EBR	UBR	( $\uparrow$ )	( $\downarrow$ )	( $\uparrow$ )
		Accuracy ( $\uparrow$ )					Bias Rate		HGR	BCR	$\boldsymbol{\Delta_{NA}}$
Zero-shot	Llama-3.1-8B-Instruct	0.37	0.03	1.00	1.00	1.00	0.08	0.00	0.02	0.54	1.00
	Mistral-7B-Instruct-v0.3	0.02	0.00	1.00	1.00	1.00	0.08	0.28	0.00	0.63	0.72
	Qwen2.5-7B-Instruct	0.39	0.01	1.00	1.00	1.00	0.03	0.00	0.01	0.58	1.00
	Yi-1.5-9B-Chat	0.51	0.02	1.00	1.00	1.00	0.39	0.00	0.01	0.10	1.00
	deepseek-llm-7b-chat	0.46	0.18	0.96	1.00	0.95	0.14	0.17	0.06	0.26	0.83

Table 3: Model performance under the four prompting strategies: accuracy on groups A–D and the uniform-context controls (Ctrl: E–H), Existential and Universal Bias Rates, Homogeneity Gap Rate (HGR), Bivalent Collapse Rate (BCR), and Negation Asymmetry Gap (

\Delta_{NA}

). † Degenerate run (§ 5.2 ).

Figure 1: Response patterns on the target pair (answer on A, answer on B) in the zero-shot setting, in % of the 100 scenarios per model. N = Neither , F = False , T = True . Only Gap reflects the truth-value gap; Collapse and Mixed would both count as “homogeneous” rejections in a binary judgment task.

Figure 2: Homogeneity Gap Rate (HGR: both sentences Neither ) and Bivalent Collapse Rate (BCR: both False ) per model across prompting strategies (ZS: Zero-shot, Enum: Enumeration). A genuine gap exceeds half of the scenarios only for Yi-1.5 with DAP. † Degenerate run in which the uniform-context controls fail (§ 5.2 ).

Model	A: “all”	B: “none”
Llama-3.1-8B	1.00 (76/76)	0.94 (67/71)
Mistral-7B	0.98 (89/91)	0.85 (53/62)
Qwen2.5-7B	0.74 (59/80)	0.34 (33/97)
Yi-1.5-9B	0.93 (42/45)	0.01 (1/90)
DeepSeek-7B	0.91 (70/77)	0.22 (13/58)
Gemma-2-9B	0.98 (41/42)	0.76 (68/90)

Table 4: Share of CoT rationales for False that restate the hypothesis as all (A) or none (B).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Model	License
Llama-3.1-8B-Instruct	Llama 3.1 Community License
Mistral-7B-Instruct-v0.3	Apache 2.0
Qwen2.5-7B-Instruct	Apache 2.0
deepseek-llm-7b-chat	DeepSeek License
gemma-2-9b-it	Gemma Terms of Use
glm-4-9b-chat-hf	GLM-4 License

Appendix

Table 5: Licenses of the evaluated models.

Prompt	Model	A	B	C	D	E	F	G	H	I	J
Zero-shot	Llama-3.1-8B-Instruct	0.37	0.03	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
	Mistral-7B-Instruct-v0.3	0.02	0.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
	Qwen2.5-7B-Instruct	0.39	0.01	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
	Yi-1.5-9B-Chat	0.51	0.02	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
	deepseek-llm-7b-chat	0.46	0.18	0.96	1.00	1.00	1.00	0.82	1.00	1.00	1.00
	gemma-2-9b-it	0.84	0.21	1.00	0.99	1.00	1.00	1.00	1.00	1.00	1.00

Appendix

Table 6: Accuracy on all ten groups of HomogeneityNLI , with the scalar implicature rate (SI: share of group-I items not judged True ) and the share of unparseable outputs (Inv.). Gold labels: A, B = Neither ; C, F, G = False ; D, E, H, I, J = True (group I is True under the literal reading of some ).

Prompt	Model	(N,N)	(N,F)	(F,N)	(F,F)	(T, $\cdot$ )	( $\cdot$ ,T)	(T,T)	both	OK
		Gap	Mixed		Collapse	Exist.	Univ.	Incons.	Reject	Ctrl
Zero-shot	Llama-3.1-8B-Instruct	0.02	0.35	0.01	0.54	0.08	0.00	0.00	0.92	100
	Mistral-7B-Instruct-v0.3	0.00	0.02	0.00	0.63	0.07	0.27	0.01	0.65	100
	Qwen2.5-7B-Instruct	0.01	0.38	0.00	0.58	0.03	0.00	0.00	0.97	100
	Yi-1.5-9B-Chat	0.01	0.50	0.00	0.10	0.39	0.00	0.00	0.61	100
	deepseek-llm-7b-chat	0.06	0.30	0.07	0.26	0.14	0.17	0.00	0.69	82

Appendix

Table 7: Distribution of per-scenario response patterns on the target pair (answer on A, answer on B), following the participant classification of Tieu et al. (2015) . N = Neither , F = False , T = True . Reject both (= Gap + Mixed + Collapse) is the “homogeneous” category of a binary truth-value judgment task. Ctrl-OK: number of scenarios (of 100) whose four definite controls (E–H) are all correct.

Figure 5: Response patterns on the target pair (answer on A, answer on B) for all prompting strategies, in % of scenarios.

Prompt	Model	T A	N A	N B	T A	N A	N B	T A	N A	N B
		1 of N			About half			N $-$ 1 of N
Zero-shot	Llama-3.1-8B-Instruct	0.00	0.33	0.03	0.06	0.69	0.06	0.19	0.09	0.00
	Mistral-7B-Instruct-v0.3	0.06	0.00	0.00	0.03	0.03	0.00	0.16	0.03	0.00
	Qwen2.5-7B-Instruct	0.00	0.06	0.00	0.03	0.72	0.03	0.06	0.44	0.00
	Yi-1.5-9B-Chat	0.28	0.53	0.00	0.38	0.62	0.06	0.53	0.38	0.00
	deepseek-llm-7b-chat	0.06	0.64	0.08	0.06	0.53	0.22	0.31	0.19	0.25

Appendix

Table 8: Breakdown by the proportion of group members with the property (GAP contexts): share of True on A (T A ), Neither on A (N A ), and Neither on B (N B ).

Prompt	Model	T A	N A	N B	T A	N A	N B	T A	N A	N B	T A	N A	N B
		Action			Property			Location			State
Zero-shot	Llama-3.1-8B-Instruct	0.08	0.24	0.04	0.00	0.32	0.08	0.08	0.44	0.00	0.16	0.48	0.00
	Mistral-7B-Instruct-v0.3	0.24	0.00	0.00	0.00	0.00	0.00	0.04	0.08	0.00	0.04	0.00	0.00
	Qwen2.5-7B-Instruct	0.04	0.60	0.00	0.00	0.16	0.00	0.04	0.40	0.04	0.04	0.40	0.00
	Yi-1.5-9B-Chat	0.60	0.32	0.08	0.12	0.84	0.00	0.52	0.32	0.00	0.32	0.56	0.00
	deepseek-llm-7b-chat	0.28	0.36	0.48	0.00	0.64	0.00	0.12	0.36	0.00	0.16	0.48	0.24

Appendix

Table 9: Breakdown by predicate class (GAP contexts): share of True on A (T A ), Neither on A (N A ), and Neither on B (N B ).

Aug 4, 2026

Language Models Encode the Contextual Truth of Propositions

Prior work has shown that LLMs encode the truth of factual propositions along linear directions in activation space. It's unclear how these representations extend to contextual truth: propositions whose truth is determined by in-context evidence rather than world knowledge. We show that LLMs maintain a linear representation of contextual truth that persists across structurally different output policies, even when the output doesn't require the model to determine a proposition's truth, and show causal evidence via steering experiments. Using the transcripts from a collaborative vision-language task that requires two LLMs to maintain a shared common ground, we show that truth representations of a proposition are significantly swayed by partner assertions about that proposition, even when the LLM has enough evidence to determine its truth. We find evidence that propositions near the decision boundary are more susceptible to having their truth shifted through partner assertions. Separating representation from output distinguish two forms of sycophancy that output behavior alone cannot: the model may accommodate a false proposition while continuing to represent it as false, or shift its representation across the boundary. The latter is 2.59x more common when the model agrees by restating the false claim explicitly than when it agrees implicitly.

Rupak Sarkar, Pritika Ramu, Rachel Rudinger

University of Maryland, College Park

LLM Sycophancy Knowledge Conflicts in Language Models Factual Knowledge in Language Models Representation Geometry in Language Models

Ask PDF

Aug 18, 2026

Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It

Humans naturally form and express beliefs in daily communication, e.g., "I think the answer is 3" or "I suppose that's right." Such beliefs inevitably intertwine with fact and knowledge, making the ability to handle them in tandem desirable for large language models (LLMs), as they are increasingly deployed in user-facing settings. Prior work showed that even capable LLMs exhibit a systemic weakness in acknowledging user beliefs grounded in incorrect information. We extend this evaluation to 10 LLMs across 18 epistemic expressions and find that the size and direction of this weakness depend on the verb used to express the belief, with the accuracy gap between factual and false information ranging from +50% on "I vaguely remember" to -14% on "I seriously doubt". We further show that the phenomenon stems from what we call task confusion: models default to fact-checking the underlying claim, overriding the user's stated belief. We provide evidence where chains of thought that explicitly fact-check show lower accuracy on false information than those that do not, and a single instruction can reverse the failure across verb families. Mechanistically, models attend more to false beliefs they fail to confirm, but suppressing this attention at decoding time recovers accuracy only partially and only in some models, calling for future work on intervention methods. Our findings clarify prior results and show how fact-checking, a generally desirable behavior, can interfere with belief tracking in LLMs.

Quang Minh Nguyen, Luis Frentzen Salim

1KAIST · 2National Taiwan University of Science and Technology

Knowledge Conflicts in Language Models Pragmatic Reasoning in Language Models Factual Knowledge in Language Models

Ask PDF

Jun 30, 2025

The Trilemma of Truth in Large Language Models

The public often attributes human-like qualities to large language models (LLMs), assuming that they "know" certain things. In reality, LLMs encode information retained during training as internal probabilistic knowledge. This study examines existing methods for probing the veracity of that knowledge and identifies three flawed underlying assumptions. To address these flaws, we introduce sAwMIL (Sparse-Aware Multiple-Instance Learning), a multiclass probing framework that combines multiple-instance learning with conformal prediction. sAwMIL leverages LLMs' internal representations to classify statements as true, false, or neither. We evaluate sAwMIL across 16 open-source LLMs, including default and chat-based variants, using three new curated datasets. Our results show that (1) common probing methods fail to provide a reliable and transferable veracity direction and, in some settings, perform worse than zero-shot prompting; (2) truth and falsehood are not encoded symmetrically; and (3) LLMs encode a third type of signal that is distinct from both true and false.

Germans Savcisens, Tina Eliassi-Rad

Khoury College of Computer Sciences Northeastern University, Boston, MA, USA · Network Science Institute Northeastern University, Boston, MA, USA · Santa Fe Institute, Santa Fe, NM, USA

LLM Interpretability LLM Reliability Language Model Probing

Ask PDF

Abstract

Explore similar work

Language Models Encode the Contextual Truth of Propositions

Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It

The Trilemma of Truth in Large Language Models