Scaling Long-Form Story Generation via Narrative State Tracking

Zhennan Wan; Jianfei Chen

cs.CLSep 28, 2026

Scaling Long-Form Story Generation via Narrative State Tracking

Authors: Zhennan Wan, Jianfei Chen

Organizations: Dept. of Comp. Sci. and Tech., Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint ML Center, Tsinghua University

Abstract

LLMs have demonstrated strong capabilities in creative writing. However, scaling them to full-length novels remains challenging, as maintaining narrative consistency becomes increasingly difficult. Existing story-generation methods typically focus on stories of up to about ten thousand words, leaving their ability to scale to full-length novels underexplored. In this work, we introduce Narrative State Tracking Agent (NstAgent), a training-free agentic framework that allows LLMs to track a structured narrative state including characters, past events and future requirements. We extend an existing benchmark to compare narrative consistency across lengths, and use it together with a writing-quality benchmark to systematically evaluate stories ranging from 10K to 100K words. We show that NstAgent achieves better narrative consistency and writing quality as stories grow longer, and neither of them degrades noticeably as length increases, suggesting that it provides an effective approach to scaling story generation toward full-length novels.

Figures & tables

Panel of Figure 1 — Figure 1: Instance CED (left, lower is better) and writing quality (right, higher is better) of NstAgent and RollSum on DeepSeek-V4-Flash as the target length grows from 10K to 100K words.

Method	Memory Type	Update Type	Access	Future Obligations	Context per Chapter
Direct	–	–	–	–	Prompt only
DOME	Structured	Incremental	Passive	No	Outline; retrieved triples
StoryWriter	Free-text	Rewrite	Passive	No	Event plan; compressed history
RollSum	Free-text	Rewrite	Passive	Implicit	Outline; summary
NstAgent	Structured	Incremental	Active	Explicit	Outline; state

Table 1: Memory design of the compared methods. Rewrite regenerates the memory after each chapter, whereas incremental updates add or replace individual entries. Passive memory is supplied to the writer by the pipeline; active memory lets the writer decide what to read or search. Direct writes the whole story in a single call.

Figure 2: Overview of NstAgent . Our framework first plans a frozen outline from the story prompt, and then writes the story chapter by chapter: in state-guided generation, the model writes chapter $t{+}1$ from the outline and state $t$ , optionally searching or reading earlier chapters and correcting errors; in narrative state tracking, the update call turns state $t$ into state $t{+}1$ .

Model	Method	Subtype CED ( $\downarrow$ )	Instance CED ( $\downarrow$ )	Writing Quality ( $\uparrow$ )	Avg. Words
DeepSeek-V4-Flash	Direct	6.225	9.293	8.705	9,416
	DOME	9.607	16.765	4.870	9,990
	StoryWriter	7.395	11.803	7.353	9,830
	RollSum	5.981	9.171	8.946	10,186
	NstAgent	4.541	6.392	8.998	11,365
GPT-5.6 Luna	Direct	5.428	7.961	9.233	9,721

Table 2: Results on 10K. Best in bold , second best underlined ; average word counts are not ranked.

Model	Method	Target Length	Subtype CED ( $\downarrow$ )	Instance CED ( $\downarrow$ )	Writing Quality ( $\uparrow$ )	Avg. Words
DeepSeek-V4-Flash	RollSum	10K	5.981	9.171	8.946	10,186
		20K	6.761	10.917	8.986	20,187
		50K	7.162	11.458	9.027	51,332
		100K	7.239	11.820	8.964	100,470
		10K	4.541	6.392	8.998	11,365
		20K	5.382	8.350	9.140	22,673

Table 3: Results from 10K to 100K. Better of the two methods at each length in bold . 10K–50K use the 100 English prompts and 100K uses 50 of them.

Variant	Subtype CED ( $\downarrow$ )	Instance CED ( $\downarrow$ )	Writing Quality ( $\uparrow$ )	Reads and searches per story
NstAgent	5.382	8.350	9.140	31.4
$-$ State	5.902	9.400	9.046	52.9
$-$ Lookback	6.577	9.786	8.980	–
RollSum (reference)	6.761	10.917	8.986	–

Table 6: Ablations on DeepSeek-V4-Flash at 20K words.

-

State removes the narrative state, its update tool, and the state-maintenance guidance;

-

Lookback removes the read, search, and correct tools. RollSum is shown for reference.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Subtypes
Characterization	memory, knowledge, skill/power fluctuation, forgotten ability
Factual detail	appearance, nomenclature, quantitative
Narrative style	perspective, tone, style shift
Timeline and plot	absolute time, duration, simultaneity, causeless effect, causal logic, abandoned plot element
World building	core rules, social norms, geography

Appendix

Table 7: The 19 error subtypes of ConStory-Bench grouped by category.

Model	Length	Tool calls per story					Final state size
Model	Length	read	search	write	correct	update	Characters	Past events	Open req.
DeepSeek-V4-Flash	10K	15.9	1.7	18.6	0.3	13.3	5.2	34.3	0.8
	20K	25.1	6.3	33.0	0.4	20.6	5.7	50.0	1.7
	50K	47.0	27.2	56.3	1.3	31.1	9.6	87.6	6.7
	100K	61.5	17.3	81.3	2.3	54.3	11.8	158.0	9.2
GPT-5.6 Luna	10K	3.6	0.0	15.8	0.1	10.4	6.3	35.4	1.2

Appendix

Table 8: Average NstAgent tool calls per story (including rejected write and update attempts) and final state size. Stories have 10, 15, 25, and 40 chapters at 10K, 20K, 50K, and 100K words.

Target length	Prefix separating the two halves	Target subtype recall	Target category recall	Controls reported as target type
10K	0–9.3K	28/30	29/30	0/6
20K	4.4K–15.3K	30/30	30/30	2/6
50K	4.5K–46.8K	29/30	30/30	2/6
100K	5.4K–101.0K	30/30	30/30	2/6

Appendix

Table 9: Detection of injected contradictions by DeepSeek-V4-Pro under the terminal-window protocol. The second column gives the range of mean word distances between the two halves across the three injection depths. Controls contain no contradiction, so any report is a false positive.

Model	Method	10K	20K	50K	100K
DeepSeek-V4-Flash	NstAgent	6.06 / 0.33	7.82 / 0.53	8.14 / 0.77	6.39 / 0.85
DeepSeek-V4-Flash	RollSum	8.49 / 0.68	10.21 / 0.70	10.25 / 1.21	10.64 / 1.18
GPT-5.6 Luna	NstAgent	5.80 / 1.02	6.64 / 1.06	6.15 / 1.55	5.31 / 1.90
GPT-5.6 Luna	RollSum	5.79 / 0.86	7.78 / 1.19	7.43 / 1.67	7.43 / 1.94

Appendix

Table 10: Instance CED split into window-local subtypes and the globally scoped abandoned_plot_elements subtype (local / global). The two add up to the instance CED of Table 3 .

Category	Subtype	Direct	DOME	StoryWriter	RollSum	NstAgent
Characterization	memory	1.10	2.17	1.05	1.18	0.64
	knowledge	0.09	0.10	0.13	0.09	0.06
	skill/power	0.12	0.31	0.11	0.06	0.05
	forgotten ability	0.00	0.00	0.01	0.00	0.01
	all	1.30	2.58	1.30	1.33	0.75
Factual detail	appearance	0.73	1.12	1.00	0.89	0.62

Appendix

Table 11: Instance CED by error category and subtype at 10K words on DeepSeek-V4-Flash, in errors per 10K checked words. Rows marked all (shaded) give the category total. The lowest value of each row is in bold ; totals equal the instance CED of Table 2 . NstAgent is lowest in all five categories and in 15 of the 19 subtypes, counted on unrounded values.

Category	Subtype	Direct	DOME	StoryWriter	RollSum	NstAgent
Characterization	memory	0.82	1.65	0.66	0.53	0.68
	knowledge	0.01	0.05	0.01	0.03	0.00
	skill/power	0.09	0.10	0.12	0.08	0.07
	forgotten ability	0.00	0.00	0.00	0.00	0.00
	all	0.92	1.80	0.79	0.64	0.76
Factual detail	appearance	0.57	0.95	0.97	0.93	0.69

Appendix

Table 12: Instance CED by error category and subtype at 10K words on GPT-5.6 Luna, in errors per 10K checked words. Rows marked all (shaded) give the category total. The lowest value of each row is in bold ; totals equal the instance CED of Table 2 . NstAgent is lowest in factual detail and in 4 of the 19 subtypes, counted on unrounded values.

Category	Subtype	20K		50K		100K
Category	Subtype	RollSum	NstAgent	RollSum	NstAgent	RollSum	NstAgent
Characterization	memory	1.62	1.07	2.01	1.29	2.09	1.05
	knowledge	0.09	0.06	0.05	0.06	0.02	0.08
	skill/power	0.08	0.11	0.09	0.06	0.00	0.06
	forgotten ability	0.01	0.02	0.01	0.00	0.00	0.00
	all	1.79	1.25	2.16	1.41	2.12	1.19

Appendix

Table 13: Instance CED by error category and subtype from 20K to 100K words on DeepSeek-V4-Flash, in errors per 10K checked words. Rows marked all (shaded) give the category total. The better method of each pair is in bold ; totals equal the instance CED of Table 3 . NstAgent is lower in all five categories at every length and in 17, 15, and 15 of the 19 subtypes at 20K, 50K, and 100K, counted on unrounded values.

Category	Subtype	20K		50K		100K
Category	Subtype	RollSum	NstAgent	RollSum	NstAgent	RollSum	NstAgent
Characterization	memory	1.02	0.90	0.99	0.83	1.35	0.84
	knowledge	0.02	0.05	0.03	0.01	0.04	0.02
	skill/power	0.11	0.12	0.08	0.04	0.09	0.08
	forgotten ability	0.00	0.00	0.00	0.00	0.00	0.00
	all	1.14	1.07	1.10	0.88	1.48	0.94

Appendix

Table 14: Instance CED by error category and subtype from 20K to 100K words on GPT-5.6 Luna, in errors per 10K checked words. Rows marked all (shaded) give the category total. The better method of each pair is in bold ; totals equal the instance CED of Table 3 . NstAgent is lower in all five categories at every length and in 13, 14, and 14 of the 19 subtypes at 20K, 50K, and 100K, counted on unrounded values.

Category	Subtype	NstAgent	$-$ State	$-$ Lookback	RollSum
Characterization	memory	1.07	1.49	1.40	1.62
	knowledge	0.06	0.06	0.05	0.09
	skill/power	0.11	0.06	0.08	0.08
	forgotten ability	0.02	0.00	0.00	0.01
	all	1.25	1.61	1.52	1.79
Factual detail	appearance	0.72	0.64	0.72	1.08

Appendix

Table 15: Instance CED by error category and subtype for the ablations of Table 6 (DeepSeek-V4-Flash, 20K words), in errors per 10K checked words. Rows marked all (shaded) give the category total. The lowest value of each row is in bold ; totals equal the instance CED of Table 6 . NstAgent is lowest in four of the five categories and in 9 of the 19 subtypes, counted on unrounded values.

Model	Length	Subtype CED	Instance CED	Local-only Instance CED	Writing Quality
DeepSeek-V4-Flash	10K	$-1.44$ [ $-1.86$ , $-1.03$ ] ∗∗∗	$-2.78$ [ $-3.65$ , $-1.95$ ] ∗∗∗	$-2.42$ [ $-3.28$ , $-1.61$ ] ∗∗∗	$+0.05$ [ $-0.05$ , $+0.15$ ]
	20K	$-1.38$ [ $-1.83$ , $-0.94$ ] ∗∗∗	$-2.57$ [ $-3.52$ , $-1.63$ ] ∗∗∗	$-2.40$ [ $-3.31$ , $-1.48$ ] ∗∗∗	$+0.15$ [ $+0.05$ , $+0.26$ ] ∗
	50K	$-1.43$ [ $-1.91$ , $-0.96$ ] ∗∗∗	$-2.59$ [ $-3.58$ , $-1.61$ ] ∗∗∗	$-2.15$ [ $-3.10$ , $-1.22$ ] ∗∗∗	$+0.22$ [ $+0.14$ , $+0.29$ ] ∗∗∗
	100K	$-2.38$ [ $-3.20$ , $-1.59$ ] ∗∗∗	$-4.58$ [ $-6.37$ , $-2.85$ ] ∗∗∗	$-4.25$ [ $-6.01$ , $-2.54$ ] ∗∗∗	$+0.24$ [ $+0.09$ , $+0.40$ ] ∗
GPT-5.6 Luna	10K	$+0.21$ [ $-0.30$ , $+0.70$ ]	$+0.17$ [ $-0.69$ , $+1.01$ ]	$+0.01$ [ $-0.85$ , $+0.84$ ]	$+0.03$ [ $-0.05$ , $+0.11$ ]
	20K	$-0.86$ [ $-1.38$ , $-0.35$ ] ∗∗	$-1.27$ [ $-2.11$ , $-0.44$ ] ∗	$-1.14$ [ $-1.93$ , $-0.34$ ] ∗	$+0.05$ [ $-0.01$ , $+0.11$ ]

Appendix

Table 16: Paired differences between NstAgent and RollSum ( NstAgent minus RollSum) at each length, with 95% bootstrap confidence intervals. ∗ , ∗∗ , ∗∗∗ mark

p<0.05

p<0.01

p<0.001

after Holm correction within each metric, over the four lengths in that column of a backbone’s block. Local-only instance CED excludes the globally scoped abandoned_plot_elements subtype; see the discussion of Table 10 .

Variant $-$ NstAgent	Subtype CED	Instance CED	Writing Quality
$-$ State	$+0.52$ [ $+0.10$ , $+0.94$ ] ∗	$+1.05$ [ $+0.16$ , $+1.95$ ] ∗	$-0.09$ [ $-0.17$ , $-0.02$ ] ∗
$-$ Lookback	$+1.20$ [ $+0.74$ , $+1.65$ ] ∗∗∗	$+1.44$ [ $+0.61$ , $+2.23$ ] ∗∗	$-0.16$ [ $-0.23$ , $-0.09$ ] ∗∗∗
RollSum	$+1.38$ [ $+0.94$ , $+1.83$ ] ∗∗∗	$+2.57$ [ $+1.63$ , $+3.52$ ] ∗∗∗	$-0.15$ [ $-0.26$ , $-0.05$ ] ∗

Appendix

Table 17: Paired differences for Table 6 (variant minus the full agent, so positive CED means the variant is worse), with 95% bootstrap confidence intervals and Holm correction within each metric, over the three comparisons in that column. Removing either memory channel makes the agent measurably worse on all three metrics.

Model	10K	20K	50K	100K
DeepSeek-V4-Flash	$-2.60$ [ $-3.89$ , $-1.32$ ]	$-2.48$ [ $-3.74$ , $-1.24$ ]	$-2.76$ [ $-4.13$ , $-1.37$ ]	$-4.58$ [ $-6.37$ , $-2.85$ ]
GPT-5.6 Luna	$+0.48$ [ $-0.75$ , $+1.65$ ]	$-1.29$ [ $-2.48$ , $-0.12$ ]	$-2.03$ [ $-3.26$ , $-0.83$ ]	$-2.15$ [ $-3.47$ , $-0.85$ ]

Appendix

Table 18: Instance CED difference ( NstAgent minus RollSum, negative favors NstAgent ) computed on the same 50 prompts at every length, with 95% bootstrap confidence intervals.

n=50

except DeepSeek-V4-Flash at 50K, where RollSum completed 49 of the 50.

Checkpoint	Subtype CED ( $\downarrow$ )	Instance CED ( $\downarrow$ )	Writing Quality ( $\uparrow$ )
Initial (Qwen3.5-4B)	9.617	15.627	4.80
Step 16	9.471	16.756	4.60
Step 32	9.263	14.373	4.86
Step 48	7.672	12.135	5.20

Appendix

Table 19: RL checkpoints of NstAgent on Qwen3.5-4B at 10K words (20 prompts, shared outlines). Best in bold .

Source	Excerpt
RollSum, Chapter 2	“He drew it out. The runes shifted like sand in a current, and the stone sang—a hum that answered something deep in the water, in the coral, in her.”
RollSum, Chapter 3	“The stone sang—a thin, high note, a string that had been waiting to be plucked.”
RollSum summary before Chapter 8	“The stone has never sung for him; it only pulses warmly.” … “The prophecy-stone hums in the braided voices of the found souls …”
RollSum, Chapter 8	“The prophecy-stone at Kaelen’s hip pulsed, warm as a coal, and for the first time since the Ember-Scar it began to sing.”
NstAgent state, past event nerys_joins_kaelen	“On the drowned Tidal spire, Kaelen found Nerys … The prophecy-stone sang in her presence …”
NstAgent state, past event cyra_joins_kaelen	“At the Gyre Bastion … the prophecy-stone sang for Cyra, the disgraced wind-rider.”

Appendix

Table 20: Case 1. The rolling summary rewrites earlier singing as humming, and the next chapter declares a “first time”. NstAgent keeps each occurrence as a keyed past event.

Source	Excerpt
RollSum, Chapter 13	Dr. Vale: “I was born in 1961.”
RollSum summary before Chapter 36	“Vale, born approximately 1963, spent summers at a farmhouse with her younger sister Miriam …”
RollSum, Chapter 38	“Irena Vale. Born 6 February 1963.”
RollSum, Chapter 2	“Daniel Reyes, born March 14, 1994, Newark, New Jersey.”
RollSum summary before Chapter 36	“Daniel, born 17 October 1994, directly viewed the tape …”
RollSum, Chapter 36	“You’re Daniel Mateo Reyes. Born October seventeenth, nineteen ninety-four.”

Appendix

Table 21: Case 2. In RollSum, birth dates drift when the summary is regenerated and the drifted values reach the final chapters. In NstAgent , character snapshots restate the recorded year at every update.

Source	Excerpt
RollSum, Chapter 0	“Behind him, Bishop Creel had said, ‘Continue.’ ”
RollSum, Chapter 8	“For one breath it was Bishop Creel, broad and pale beneath his red skullcap.”
RollSum, Chapters 0 and 4	“Mara had been nine, narrow-shouldered and solemn …” … “… nine years old, wrists bound, candles guttering whenever she breathed.”
NstAgent , Chapter 0	“Bishop Armitage ordered Elias to conduct the exorcism, though he had been ordained only two years and had never performed one alone.”
NstAgent , Chapter 4	“Bishop Haldane said the demon would imitate her, and that hesitation would damn her.”
NstAgent state, past event	“Elias confessed that Mara had asked him to stop during her exorcism … because he feared disobeying Bishop Haldane more than losing her.”

Appendix

Table 22: Case 3. NstAgent renames the bishop in Chapter 4 and records the new name in the state, after which every later chapter uses it; RollSum keeps one name and one age throughout.

Explore similar work

From Personas to Plot: Character-Grounded Multi-Agent Story Generation for Long-Form Narratives
Jul 1, 2026Aayush Aluru, Chloe Ho, Muhammad Hammouri +5Narrative Generation Long-Form Generation
Pocket FM · Princeton University · University of Michigan +2
IS-CoT: Breaking the Long-form Generation Collapse via Interleaved Structural Thinking
Jun 8, 2026Zechen Sun, Yuyang Sun, Zecheng Tang +6Long-Form Generation Large Language Model Generation
Institute of Computer Science and Technology, Soochow University · 2Information Research Center of Military Science, PLA Academy of Military Science
NARRA-Gym for Evaluating Interactive Narrative Agents
May 8, 2026Yue Huang, Yuchen Ma, Jiayi Ye +14Skillgym Emotion
University of Notre Dame · 2LMU Munich · 11Munich Center for Machine Learning +8

Jul 1, 2026, cs.CLJ/K move · Enter open · S save

From Personas to Plot: Character-Grounded Multi-Agent Story Generation for Long-Form Narratives

Aayush Aluru, Chloe Ho, Muhammad Hammouri, Kerry Luo+4

Pocket FM · Princeton University · University of Michigan+2

Narrative Generation Long-Form Generation Narratives Atlas Chart

Open paper AskPDF arXiv

Although large language models (LLMs) have demonstrated impressive creative fiction generation, they struggle to maintain narrative consistency and coherent plot lines in long-form stories. In this work, we introduce a unified framework for long-form narrative generation and verification. MAGNET, a multi-agent goal-driven narrative engine for storytelling, generates stories with persona-grounded character agents that propose actions based on a shared world state and evolving story goals, while ATLAS is a graph-based pipeline that compares scene-level world representations across a generated story to detect hallucinations. By evaluating MAGNET using an LLM editor, pairwise rubric scoring, and ATLAS, we show that our framework produces coherent narratives compared to single-model prompting and IBSEN. At 100 pages, MAGNET reduced annotations and hallucinations by 41 and 50%, respectively, compared to the single model baseline and by 34 and 45%, respectively, compared to IBSEN, with pairwise rubric evaluation showing similar results. These results suggest that long-form narratives can emerge from explicit world-state tracking and goal-driven multi-agent generation, providing a foundation for controllable and structurally coherent long-form narrative generation.

Abstract

Explore similar work

From Personas to Plot: Character-Grounded Multi-Agent Story Generation for Long-Form Narratives

IS-CoT: Breaking the Long-form Generation Collapse via Interleaved Structural Thinking

NARRA-Gym for Evaluating Interactive Narrative Agents