Language Models Have Interpretable Narrative Mechanisms

David Yiyang Liu; Aditya Joshi; Paul Dawson; Andrew Piper

cs.CLOct 9, 2026

Language Models Have Interpretable Narrative Mechanisms

Authors: David Yiyang Liu, Aditya Joshi, Paul Dawson, Andrew Piper

Organizations: UNSW Sydney · McGill University

Abstract

We investigate whether language models contain internal components that contribute to representing narrativity, the degree to which a text exhibits narrative properties. Across GPT-2-Small, Gemma-2-2B, and Llama-3.1-8B, we identify attention heads, MLP neurons, and sparse autoencoder features whose activations correlate with narrativity. Ablating selected neuron and feature sets increases next-token prediction loss more for high-narrativity than for low-narrativity texts, and reduces the narrativity of open-ended generations. On downstream benchmarks, the largest effects occur in story completion and narrative question answering, although they vary across models and languages; results on confabulation and hallucination benchmarks are mixed. These findings provide causal evidence that selected sets of narrative-associated components contribute to model behaviour, while their internal organisation, relation to narrative theory, and broader functions remain open questions.

Figures & tables

Figure 1: Narrative features can be interpreted along three dimensions: agency, event and world.

Figure 2: Finding NarraHeads, NarraNeurons and NarraFeatures in transformer layers.

Figure 3: Mean activation of each component in Llama on low vs. high NarraDetect texts. Red marks NarraComponents. Components with a zero mean in either group are omitted.

Figure 4: Change in loss correlates with narrativity when ablating NarraComponents inside Llama.

Component	$n$	%	Narra		Random
GPT
NarraHead	2	1.4	$-$ .01	[ $-$ .10, .10]	.06	[ $-$ .01, .13]
NarraNeuron	2534	6.9	.84	[.81, .87]	$-$ .04	[ $-$ .12, .04]
NarraFeature	695	0.2	.83	[.80, .86]	.01	[ $-$ .08, .10]
Gemma
NarraHead	3	1.4	.13	[.03, .22]	$-$ .06	[ $-$ .15, .02]

Table 1:

\mathrm{NVC}_{\mathrm{loss}}

when ablating NarraComponents vs. random: mean [95% bootstrap CIs].

Category	GPT2	Gemma	Llama
	( $n=695$ )	( $n=2529$ )	( $n=1211$ )
Agency	33.7%	40.2%	34.4%
Event sequence	29.4%	21.7%	34.2%
World building	37.0%	38.1%	31.5%

Table 2: Percentage of NarraFeatures assigned to each dimension.

		% of features
Word	Top	All	Agency	Event	World
		4434	1666	1167	1601
action	Event	23	15	33	24
interaction	World	16	11	18	19
emotional	Agency	12	14	7	13
character	World	11	9	10	13

Table 3: The 10 most common words in the auto-generated explanations of NarraFeatures.

	Macro-F1
	GPT2	Gemma	Llama
	695	2529	1210
Embedding Classifier	.53	.57	.58
DeepSeek Zero-shot	.50	.47	.48
Chance	.33	.33	.33

Table 4: Predicting a NarraFeature’s top narrativity dimension from its auto-generated explanation.

Figure 5: Llama’s NarraComponents activate more for stories than explanations. Activations are max-normalised per component and averaged; one point per topic pair.

	Story			Explanation
	Agency	Event	World	Agency	Event	World
Gemma	3.80	3.15	3.13	2.17	1.78	2.24
Llama	4.05	3.27	3.28	1.78	1.49	1.96

Table 5: Mean narrativity (1–5) of zero-shot generations, prompted for a story or an explanation.

	Features ( $n$ )	Judge score
Model	by top dim.	Ag.	Ev.	Wo.
Gemma-2-2B	agency (1016)	0.82	0.74	0.72
	event (549)	0.84	0.88	0.83
	world (964)	0.76	0.81	0.89
Llama-3.1-8B	agency (416)	0.82	0.73	0.70
	event (414)	0.82	0.89	0.84

Table 6: Spearman

\rho

between NarraFeature contributions and narrativity. 95% bootstrap

\leq\pm

0.04.

	Target			Random
	Ag.	Ev.	Wo.	Ag.	Ev.	Wo.
Gemma
NarraHeads	$-$ .15	$-$ .17	$-$ .13	$-$ .13	$-$ .13	$-$ .12
	$\pm$ .08	$\pm$ .08	$\pm$ .06	$\pm$ .15	$\pm$ .16	$\pm$ .13
NarraNeurons	$-$ .67	$-$ .75	$-$ .47	$-$ .11	$-$ .09	$-$ .08
	$\pm$ .10	$\pm$ .09	$\pm$ .07	$\pm$ .10	$\pm$ .10	$\pm$ .07

Table 7:

\Delta Narrativity

after ablating NarraComponents vs. random with

\pm

95% CI. Bold: Target differs from Random (paired bootstrap over prompts, two-sided,

p<.05

, uncorrected).

	Story			Explanation
	Agency	Event	World	Agency	Event	World
Gemma
agency	$-$ 2.48	$-$ 1.96	$-$ 1.60	$-$ 1.09	$-$ 0.70	$-$ 0.75
event	$-$ 1.19	$-$ 1.55	$-$ 0.92	$-$ 0.30	$-$ 0.41	$-$ 0.23
world	$-$ 1.10	$-$ 1.17	$-$ 0.77	$-$ 0.22	$-$ 0.32	$-$ 0.24
Llama

Table 8:

\Delta Narrativity

after ablation of NarraFeature categories. Bold: drop that is significantly larger than the next-largest drop (paired bootstrap over prompts, two-sided,

p<.05

, uncorrected). 95% bootstrap CIs

\leq\pm 0.15

Prompt: Write a story concerning repairing a bicycle.
Abl.	A/E/W	Generation (abridged)
none	5/5/4.5	I was riding my bike to the store […] I saw a car coming towards me. I swerved […] hit a pothole and fell off my bike.
agency	1 /2.5/2	Repairing a bicycle is a task that requires patience […] The first step in repairing a bicycle is to identify the problem.
event	4/ 2 /3	[…] The boy’s name is John. […] It has a flat tire. He can’t ride it. He wants to ride it. He wants to go to the park. […]
world	5/5/ 4	I was riding my bicycle when I got a flat tire. […] I was very happy and I rode my bicycle home.

Table 9: Llama generations after ablating each dimension’s NarraFeatures. Scores: Agency/Event/World (1–5).

	Story completion (XStoryCloze)				Question Answering (QA)		Hallucination/Confabulation
	EN	RU	ZH	HI	Cosmos	OpenBook	TruthfulQA	FaithDial	BEGIN
Gemma-2-2B (2529 features, 3 heads, 3470 neurons)
Base	82.7	74.7	73.3	63.7	42.6	69.4	25.0	67.6	85.5
NarraFeatures	$-$ 9.1 $\pm$ 2.3	$-$ 7.6 $\pm$ 2.3	$-$ 5.0 $\pm$ 2.0	$-$ 5.3 $\pm$ 1.7	$-$ 2.8 $\pm$ 2.2	+2.6 $\pm$ 2.5	+2.8 $\pm$ 2.2	$-$ 2.0 $\pm$ 1.2	+0.5 $\pm$ 1.6
Heads	$-$ 0.3 $\pm$ 1.0	+0.7 $\pm$ 1.2	$-$ 0.5 $\pm$ 1.0	$-$ 0.5 $\pm$ 1.0	$-$ 0.5 $\pm$ 1.4	+2.3 $\pm$ 1.1	$-$ 0.8 $\pm$ 1.0	+0.3 $\pm$ 0.5	$-$ 0.1 $\pm$ 0.7
Neurons	$-$ 1.3 $\pm$ 1.2	$-$ 2.2 $\pm$ 1.5	$-$ 1.3 $\pm$ 1.3	$-$ 0.9 $\pm$ 1.2	+0.5 $\pm$ 1.7	+0.4 $\pm$ 1.2	+0.8 $\pm$ 1.3	$-$ 0.2 $\pm$ 0.7	+0.7 $\pm$ 0.8

Table 10: Effect of ablating NarraFeatures, NarraHeads and NarraNeurons relative to ablating the same number of random units:

\Delta

Target

\,-\,\Delta

Rand (pp), with 95% bootstrap CIs. Bold :

\Delta

NarraFeatures differ significantly from

\Delta

Rand ( red : below, blue : above) (paired bootstrap, two-sided,

p<.05

, uncorrected). Full table with separate

\Delta

Target and

\Delta

Rand in Appendix J .

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Judge	Overall	Agency	Event	World
Hy3	.886 [.867, .902]	.831 [.804, .853]	.838 [.803, .867]	.855 [.830, .877]
DeepSeek-V4.1-Flash	.888 [.868, .904]	.850 [.826, .870]	.846 [.816, .871]	.869 [.848, .888]
Gemma-4-31B	.898 [.880, .913]	.858 [.837, .877]	.840 [.807, .868]	.853 [.831, .871]
GLM-4-32B	.883 [.861, .901]	.857 [.834, .877]	.845 [.815, .869]	.795 [.761, .823]
Qwen3.5-27B	.882 [.862, .897]	.835 [.805, .861]	.868 [.846, .886]	.809 [.776, .836]

Appendix

Table 11: Spearman

\rho

between LLM-judge narrativity ratings and the mean of three human annotators (

n=394

passages). Brackets give 95% bootstrap CIs.

Topic type	Topics
Entities & agents	a teacher; a dog; an octopus; an alien; a dragon; a robot; a family; a team of explorers; a colony of ants; a mermaid
Objects	a key; a photograph; a clock; a letter; a map; a ring; a telescope; an ancient coin; a magic mirror; a time machine
Places & environments	a kitchen; a library; a marketplace; a forest; a river; an abandoned station; a deep-sea trench; a distant planet; a floating city; an enchanted forest
Emotions & mental states	joy; gratitude; relief; jealousy; loneliness; fear; curiosity; uncertainty; nostalgia; ambivalence
Relationships & social dynamics	friendship; kinship; cooperation; trust between strangers; rivalry; authority; belonging; estrangement; dependence; exclusion from a group
Activities & processes	gardening; learning a language; preparing a meal; repairing a bicycle; painting; building a bridge; migration; metamorphosis; erosion; brewing a magic potion

Appendix

Table 12: The 90 topics of the open-ended generation prompts, ten per topic type. Each topic fills the <topic> slot of all five templates in both conditions.

	Agency		Event		World
	Target	Random	Target	Random	Target	Random	$n$
Gemma
Keep all	$-$ 1.86	$-$ 0.36	$-$ 1.40	$-$ 0.44	$-$ 1.27	$-$ 0.36	900
Excl. $\geq 0.8$	$-$ 1.92	$-$ 0.37	$-$ 1.45	$-$ 0.46	$-$ 1.31	$-$ 0.36	736
Excl. $\geq 0.6$	$-$ 2.02	$-$ 0.37	$-$ 1.55	$-$ 0.45	$-$ 1.37	$-$ 0.35	409
Excl. $\geq 0.4$	$-$ 2.09	$-$ 0.31	$-$ 1.81	$-$ 0.49	$-$ 1.45	$-$ 0.33	137

Appendix

Table 13:

\Delta Narrativity

(1–5 scale) after SAE Target ablation and size-matched random ablation (Random, mean of 10 seeds), keeping every prompt or excluding prompts whose unablated or ablated generation is degenerate at three seq-rep-4 thresholds.

n

: prompts kept in all 11 arms. In every setting, every SAE Target cell stays below every random seed. Bold: Target differs from Random (paired bootstrap over prompts, two-sided,

p<.05

, uncorrected).

Abl.	A/E/W	Generation (first 30 words)
Prompt T091_P03
none	4.5/4.5/4	I wrote a story about a man who lived in a desert. He had a well that was his only source of water. One day, a man came to his […]
agency	2/3.5/3	Water is a precious resource that is essential for life. Competition for water can arise due to various factors such as population growth, climate change, and economic development. In this […]
event	5/4/4	I am going to write a story about a man who is competing for water. The man is in a desert. He is thirsty. He is looking for water. He […]
world	3.5/3.5/3	I wrote a story about a man who is trying to get water for his crops. He has to go to the river to get water, but there is a […]
Prompt T068_P02

Appendix

Table 14: Llama story prompts drawn at random (seed 0), unablated and after each category ablation. Scores: mean of 2 LLM judges (1–5).

Model	rep-4	Generation (first 60 words)
Gemma	0.63	I have a lot of experience in gardening because it is a hobby of mine to grow plants and flowers in the garden that is why it is important to be able to care for plants and flowers in order to be able to grow healthy and beautiful plants and flowers in the garden that is why it is important […]
Gemma	–	(empty)
Gemma	0.72	Kinship is a relationship between two or more people who are related by blood, marriage, or adoption to each other in a family or social group that is recognized by law or custom as a family or social group that is recognized by law or custom as a family or social group that is recognized by law or custom as […]
Gemma	0.65	I’m sorry to hear about the loss of a loved one is never easy to deal with, but it is important to remember that time heals all wounds, and that you are not alone in this difficult time of grief and loss of a loved one is never easy to deal with, but it is important to remember that time […]
Llama	0.86	Photographs are a way of capturing a moment in time. Photographs" are a way of capturing a moment in time. Photographs are a way of capturing a moment in time. Photographs are a way of capturing a moment in time. Photographs are a way of capturing a moment in time. Photographs are a way of capturing a moment in time. […]
Llama	0.73	I am a writer of fiction. Fiction is a story that is not true" (Fiction 1). Fiction is a story that is not true, but it can be based on a true story. Fiction is a story that is not true, but it can be based on a true story. Fiction is a story that is not true, but it […]

Appendix

Table 15: Degenerate generations under SAE Target ablation, 4 per model drawn at random (seed 0). rep-4: word-level seq-rep-4.

	Story completion (XStoryCloze)				Question Answering (QA)		Hallucination/Confabulation
	EN	RU	ZH	HI	Cosmos	OpenBook	TruthfulQA	FaithDial	BEGIN
Gemma-2-2B (2529 features, 3 heads, 3470 neurons)
Base	82.7	74.7	73.3	63.7	42.6	69.4	25.0	67.6	85.5
$\Delta$ NarraFeatures	$-$ 11.6 $\pm$ 2.3	$-$ 10.2 $\pm$ 2.2	$-$ 6.1 $\pm$ 2.0	$-$ 5.4 $\pm$ 1.9	$-$ 6.2 $\pm$ 1.6	$-$ 0.5 $\pm$ 0.8	+3.4 $\pm$ 2.4	$-$ 3.2 $\pm$ 1.3	+0.1 $\pm$ 1.6
$\Delta$ Rand	$-$ 2.6 $\pm$ 1.3	$-$ 2.6 $\pm$ 1.6	$-$ 1.0 $\pm$ 1.1	$-$ 0.1 $\pm$ 1.0	$-$ 3.4 $\pm$ 2.0	$-$ 3.1 $\pm$ 2.4	+0.6 $\pm$ 1.4	$-$ 1.2 $\pm$ 0.9	$-$ 0.4 $\pm$ 1.0
$\Delta$ Heads	$-$ 1.0 $\pm$ 0.9	$-$ 0.5 $\pm$ 0.9	$-$ 0.9 $\pm$ 0.9	$-$ 0.4 $\pm$ 1.0	$-$ 2.2 $\pm$ 0.9	+0.3 $\pm$ 0.5	$-$ 0.9 $\pm$ 1.0	+0.6 $\pm$ 0.5	$-$ 0.3 $\pm$ 0.7

Appendix

Table 16: Effect of ablating NarraFeatures, narrativity-selective attention heads and MLP neurons (all units with

\rho\geq 0.5

), as in Table 10 . Each indented

\Delta

Rand row is the matched-size random baseline for the unit row directly above it.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 761 papers

Characterizing Narrative Content in Web-scale LLM Pretraining Data
Jun 17, 2026Teagan Johnson, Elliott Ash, Andrew Piper +1Document Understanding Language Model Pretraining
University of Colorado Boulder · ETH Zürich · McGill University
Narrative Landscape: Mapping Narrative Dispositions Across LLMs
May 9, 2026Donghoon Jung, Jiwoo Choi, Songeun Chae +1LLM Evaluation
School of Digital Humanities and Computational Social Sciences, KAIST, South Korea
Narrative Lookahead in LLMs: Event-Level Decodability, Predictive Advantage, and Steering
Oct 13, 2026Donghoon Jung, Min Kim, Seojin Choi +2Next-Event Prediction Text Generation
School of Digital Humanities and Computational Social Sciences, KAIST, South Korea

Jun 17, 2026J/K move · Enter open · S save

Characterizing Narrative Content in Web-scale LLM Pretraining Data

Teagan Johnson, Elliott Ash, Andrew Piper, Maria Antoniak

University of Colorado Boulder · ETH Zürich · McGill University

Document Understanding Language Model Pretraining

Open paper AskPDF arXiv

The narrative composition of web-scale LLM pretraining corpora remains largely unexplored, even though narrative is a fundamental mode of human communication. We present the first fine-grained study of narrative features in Dolma, a 3-trillion-token open pretraining corpus. Drawing on narrative theory, we design a framework spanning three core narrative elements (agency, setting, and events) operationalized as 11 interpretable dimensions. After curating and hand-annotating a diverse set of 400 passages, we create an LLM-labeled dataset of 25K passages, and finally, we finetune and validate NarraBERT, two RoBERTa-based models for fine-grained narrative prediction. We apply NarraBERT to 13M passages, resulting in a new dataset, NarraDolma. We find that narrative structure is measurable at scale across extremely heterogeneous data and narrative qualities are unequally distributed across pretraining sources, topics, and formats in ways that current data curation practices neither measure nor account for. Our framework, dataset, and analyses provide a foundation for understanding how narrative qualities are distributed in LLM pretraining data.

Abstract

LLM Interpretability

Explore similar work

Characterizing Narrative Content in Web-scale LLM Pretraining Data

Narrative Landscape: Mapping Narrative Dispositions Across LLMs

Narrative Lookahead in LLMs: Event-Level Decodability, Predictive Advantage, and Steering