Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility

Michael A. Lepori; Jennifer Hu; Ishita Dasgupta; Roma Patel; Thomas Serre; Ellie Pavlick

cs.CLJul 16, 2025

Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility

Authors: Michael A. Lepori, Jennifer Hu, Ishita Dasgupta, Roma Patel, Thomas Serre, Ellie Pavlick

Organizations: Department of Computer Science, Brown University · Google DeepMind · Department of Cognitive & Psychological Sciences, Brown University

Abstract

Language models (LMs) are used for a diverse range of tasks, from question answering to writing fantastical stories. In order to reliably accomplish these tasks, LMs must be able to discern the modal category of a sentence (i.e., whether it describes something that is possible, impossible, completely nonsensical, etc.). However, recent studies have called into question the ability of LMs to categorize sentences according to modality (Michaelov et al., 2025; Kauf et al., 2023). In this work, we identify linear representations that discriminate between modal categories within a variety of LMs, or modal difference vectors. Analysis of modal difference vectors reveals that LMs have access to more reliable modal categorization judgments than previously reported. Furthermore, we find that modal difference vectors emerge in a consistent order as models become more competent (i.e., through training steps, layers, and parameter count). Notably, we find that modal difference vectors identified within LM activations can be used to model fine-grained human categorization behavior. This potentially provides a novel view into how human participants distinguish between modal categories, which we explore by correlating projections along modal difference vectors with human participants' ratings of interpretable features. In summary, we derive new insights into LM modal categorization using techniques from mechanistic interpretability, with the potential to inform our understanding of modal categorization in humans.

Figures & tables

Name	Prob.	Improb.	Imposs.	Inc.	Pair	Adv.	Human
Hu et al. (2025b)	✓	✓	✓	✓	✓	✗	✓
Goulding et al. (2024)	✓	✓	✓	✗	✓	✗	✓
Vega-Mendoza et al. (2021)	✓	✓	✗	✓	✓	Semantic	✗
Kauf et al. (2023)	✓	✓	✗	✓	✓	Lexical	✗
Hu et al. (2025a)	✓	✗	✗	✓	✗	✗	✓
Tuckute et al. (2024)	✗	✗	✗	✗	✗	✗	✓

Table 1: Properties of the datasets analyzed in the study. Prob. , Improb. , Imposs. , Inc. denote whether the dataset contains probable, improbable, impossible, or inconceivable sentences, respectively. Pair denotes that the dataset contains minimal pairs that vary in modal category, allowing for classification using either vectors or probability estimation. Adv. indicates that the dataset contains pairs that are adversarial in some way (see main text). Human indicates that we analyze human behavioral data from this dataset.

Figure 1: (Left) Diagram describing how we create modal difference vectors. In this example, a modal difference vector capturing the difference between probable and impossible stimuli is created by taking the mean over differences in hidden representations. (Right) Diagram describing how modal difference vectors are used to classify novel minimal pairs of impossible/probable sentences. Hidden representations from each sentence are projected onto the modal difference vector, and the magnitudes of these projections are compared.

Figure 2: Classification evaluations for models with at least 2B parameters. Results are averages across models and generalization datasets. Modal difference vectors outperform probability estimates and other projection-based classification baselines.

Figure 3: (a) Average generalization performance vs. parameter count reveals a large performance gap between models with fewer/greater than 2B parameters. At (b) smaller scales, (c) earlier in training, and (d) in earlier layers, models form modal difference vectors that can differentiate inconceivable stimuli from other modal categories. After that, models learn the distinction between probable and impossible , then probable and improbable , and finally improbable and impossible .

Scenario ( Someone is about to… )	Modal Diff. P(Poss.)	Prob. P(Poss.)	Human P(Poss.)
clean a car.	0.99	0.70	1.0
clean a road.	0.94	0.62	0.97
clean a cloud.	0.09	0.57	0.05
stay awake for 5 hours.	0.94	0.59	1.0
stay awake for 5 days.	0.67	0.63	0.53
stay awake for 5 years.	0.25	0.60	0.05

Table 2: Qualitative examples from Goulding et al. (2024) showing the probability estimates that each scenario is possible, as assessed by 1) the logistic regression model using projections on modal difference vectors from Gemma-2-9B, 2) the logistic regression model using probability estimates from Gemma-2-9B, 3) and the proportion of participants classifying the scenario as “possible”.

Panel of Figure 4 — Figure 4: (Left) A qualitative example of stimuli from Hu et al. (2025b) projected along two modal difference vectors. Dots are colored according to their expert label. Background color intensity represents the probability that each point belongs to a particular class according to a logistic regression model fit to this subset of data using these two features. (Right) (a) Pearson correlation between the predicted probability distributions and the empirical proportion of participants that selected each category. (b) Mean squared error between predicted and empirical response distributions. (c) Pearson correlation between the entropy of predicted and empirical response distributions. In all analyses, we find that featurizing using projections along modal difference vectors leads to better models of human categorization behavior.

Figure 5: Absolute correlations between projections along modal difference vectors and interpretable features (averaged over models). Notably, Probable-Improbable correlates with human subjective event likelihood judgments, and Impossible-Inconceivable correlates selectively with imageability, the presence of physical objects, and places/environments.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Classification evaluations for models with less than 2B parameters. Results are averages across models and generalization datasets. Results are mixed.

Figure 7: Performance of all classification methods just on adversarial stimuli. We include lexical adversarial stimuli from Kauf et al. (2023) on the left. These stimuli contain the same tokens, just in different orders. We find that modal difference vectors and probability estimates perform best in the face of this manipulation. We include semantically adversarial stimuli used in Vega-Mendoza et al. (2021) and Michaelov et al. (2025) on the right. We find that modal difference vectors and projections along principal components perform best, while probability estimates are systematically misled.

Figure 8: Surprisal (with respect to the baseline model) of the next token predictions generated after steering. We find that, for both Gemma-2-2B and Llama-3.2-3B, surprisal values generally increase in order of baseline

<

improbable

<

impossible

<

inconceivable . This mirrors the finding from Hu et al. (2025b) .


Model	Prefix
Gemma-2-2B	Someone measured the furniture using a…
Steering	Generations
None	tape measure, ruler, 1, measuring tape, laser level
Improbable	tape measure, ruler, laser, measuring tape, 3D scanner
Impossible	laser, telescope, microscope, new technique, camera

Appendix

Table 3: Steering Generations using Gemma-2-2B.


Model	Prefix
Llama-3.2-3B	Someone destroyed a building using a…
Steering	Generations
None	sledgehammer, bulldozer, wrecking ball, crane, drone
Improbable	flamethrower, bulldozer, sledgehammer, drone, chainsaw
Impossible	boat, car, 9, real-life, man and a woman

Appendix

Table 4: Steering generations using Llama-3.2-3B

Figure 9: Effect of ablating modal difference features on capturing human participant categorization behavior. For the Hu et al. (2025b) dataset, we find that all three modal difference features are required to best capture behavior (according to Overall Correlation and MSE metrics). In order to capture entropy in the response distribution, regression models require the probable-improbable feature. The impact of ablation is not significant for the other two datasets, which only elicit coarse, binary modal categorization judgments.

Representation Geometry in Language Models

Representation geometry in language models studies how embeddings and hidden states organize concepts, computation, and behavior.

See all 404 papers

Modal Sense Disambiguation in Language Models: The Role of Context, Training Domain, and Input Frequency
Oct 19, 2026Ezgi Başar, Yevgen Matusevych, Annemarie van Dooren +1Word Sense Disambiguation Language Modeling
Center for Language and Cognition (CLCG), University of Groningen
Same Formulas, Different Semantics: Do Language Models Follow Modal Logic Specifications?
Aug 5, 2026Réemi Andrieu, Damien SileoLogical Reasoning LLM Evaluation
Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 - CRIStAL, F-59000 Lille, France
Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs
Feb 17, 2026Sean Trott, Samuel Taylor, Cameron Jones +2Theory of Mind Pragmatic Reasoning in Language Models
Rutgers University - Newark · UC San Diego · Stony Brook University +1

Oct 19, 2026J/K move · Enter open · S save

Modal Sense Disambiguation in Language Models: The Role of Context, Training Domain, and Input Frequency

Ezgi Başar, Yevgen Matusevych, Annemarie van Dooren, Arianna Bisazza

Center for Language and Cognition (CLCG), University of Groningen

Word Sense Disambiguation Language Modeling LLM Evaluation

Open paper AskPDF arXiv

Modals such as may and must can take on different senses depending on discourse context, alternating between root interpretations (e.g., permission, obligation) and epistemic interpretations (e.g., knowledge, belief). While modals are known to be notoriously difficult for children to acquire, limited research exists on whether language models (LMs) can reliably distinguish between different modal senses. We evaluate seven LMs that vary in architecture, size, and training data type using vector-based similarity and surprisal measures across four datasets. Specifically, we investigate whether LMs can leverage contextual information to disambiguate modal senses and whether their performance reflects the distribution of modal senses in their training data. We show that larger LMs trained on diverse text successfully leverage additional contextual information when available. In the absence of context, they tend to favor the dominant sense based on corpus statistics. Smaller developmentally plausible LMs, including those trained on child-directed language (CDL), fail to show consistent sensitivity to modal sense distinctions and do not benefit from context to the same degree.

Abstract

Representation Geometry in Language Models

Explore similar work

Modal Sense Disambiguation in Language Models: The Role of Context, Training Domain, and Input Frequency

Same Formulas, Different Semantics: Do Language Models Follow Modal Logic Specifications?

Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs