Steering Readerly Dynamics: Trajectory-Guided Story Generation with Large Language Models

Danrui Li; Yiwei Zhao; Mubbasir Kapadia; Lingxiao Li

cs.CLOct 30, 2026

Steering Readerly Dynamics: Trajectory-Guided Story Generation with Large Language Models

Authors: Danrui Li, Yiwei Zhao, Mubbasir Kapadia, Lingxiao Li

Organizations: Department of Computer Science, Rutgers University, USA · Netflix, USA

Abstract

Large language models (LLMs) are increasingly used to generate stories, but these stories struggle to reproduce engaging readerly dynamics, the evolving reader responses that shape narrative experience. To address this challenge, we propose a framework that learns readerly dynamics from high-quality human-written stories and uses them to guide story generation. Instead of directly modeling high-dimensional story content, we train a small LLM to predict future reader responses as low-dimensional trajectories. We then leverage the strong instruction-following capability of a larger, off-the-shelf writer LLM to generate story segments that follow these predicted trajectories within a given story setting. Our framework improves trajectory-based fidelity to human-written stories by 12%-35% while largely preserving story coherence.

Figures & tables

Figure 1: Our work finetunes a small LLM to steer off-the-shelf larger LLM in writing stories segment by segment, enabling more natural readerly dynamics in the narration.

Panel of Figure 2 — Figure 2: Our framework jointly generate story text and readerly dynamics (left), where the steering model (green) is finetuned using the readerly dynamics extracted from human stories (right).

Panel of Figure 4 — Figure 4: Readerly-dynamics fidelity versus Story coherence for different methods, using two base writer models in suspense and humor. Lower trajectory error (x axis) and higher coherence (y axis) are better. Gray bands are 95% bootstrap intervals for LLM judge’s scores on human-authored stories.

Panel of Figure 5 — Figure 5: Top: LLM measurement scores versus human mean scores (rescaled from 1–9 to 0–100) at each story position. Each point represents a unique story prefix; $r$ and $\rho$ denote Pearson and Spearman correlations. Bottom: Probability that human and LLM scores agree on the direction of change between two story positions, given an absolute LLM score gap of $X$ . Curves show bootstrap medians with 95% confidence intervals.

	Suspense	Humor
All complete rows	94/114 (82.5%)	91/126 (72.2%)
Excluding 3:2 votes	75/82 (91.5%)	65/83 (78.3%)

Table 1: Agreement between the five-annotator human majority and the measure model scoring.

Panel of Figure 6 — Figure 6: Ablation studies on the conditioning settings of (a) measurement, (b) steering, (c) writer model, and (d) best-of-N. Use suspense data on GPT-5.5 writer. Bars show 95% bootstrap intervals; $*$ marks paired $p<.05$ .

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Humor
	GPT-5.5 writer				Qwen3.5-27B writer
	Thinking off		Thinking on		Thinking off		Thinking on
Prompt condition	Min- $K$ MAE	CRPS	Min- $K$ MAE	CRPS	Min- $K$ MAE	CRPS	Min- $K$ MAE	CRPS
Off-the-shelf	14.94 [14.08, 15.83]	17.53 [16.64, 18.46]	15.14 [14.27, 16.04]	17.85 [16.96, 18.78]	8.97 [8.30, 9.67]	11.50 [10.76, 12.26]	7.62 $\Downarrow$ [7.06, 8.20]	9.84 $\Downarrow$ [9.22, 10.48]
One-shot ICL	15.08 [14.21, 15.98]	17.82 [16.93, 18.72]	15.65 [14.78, 16.56]	18.40 [17.49, 19.34]	9.67 [8.99, 10.37]	12.26 [11.51, 13.02]	7.82 ‡ $\Downarrow$ [6.33, 9.40]	9.97 ‡ $\Downarrow$ [8.46, 11.63]
Explicit off-the-shelf	15.30 [14.46, 16.18]	18.18 [17.27, 19.12]	16.10 [15.20, 17.02]	18.54 [17.62, 19.48]	10.14 [9.43, 10.90]	13.04 [12.25, 13.86]	8.88 $\Downarrow$ [8.22, 9.56]	11.35 $\Downarrow$ [10.61, 12.09]

Appendix

Table 2: Effect of prompt configuration and thinking mode with GPT-5.5 and Qwen3.5-27B under autoregressive evaluation.

\Downarrow

denotes a decreasing point estimate of the metric compared to off-the-shelf counterparts. It does not represent a significant difference.

Dataset	Reasoning free	Reasoning on	Total multiplier
Suspense	54.3	4,493.0	82.7 $\times$
Humor	75.5	3,881.2	51.4 $\times$

Appendix

Table 3: Output token usage per segment for Qwen3.5-27B writer, with and without reasoning mode.

Method	Min- $K$ MAE $\downarrow$
Ours	2.49 [2.13, 2.89]
Numeric prediction	3.01 [2.61, 3.44]
Delta sampling	4.76 [4.11, 5.44]
GPT-5.5 direct prediction	7.06 [6.19, 7.98]

Appendix

Table 4: Position-wise prediction performance of our method and baselines.

Suspense
		Ours		Selection-only		$\Delta$ (ours $-$ Selection-only)
Writer	$N$	Min- $K$ MAE	CRPS	Min- $K$ MAE	CRPS	Min- $K$ MAE	CRPS
Qwen3.5-27B	1	9.25 [8.13, 10.40]	12.04 [10.82, 13.31]	8.29 [7.19, 9.42]	11.04 [9.79, 12.29]	+0.95 [-0.04, +1.94]	+1.01 [-0.19, +2.22]
Qwen3.5-27B	6	7.32 [6.44, 8.25]	10.11 [9.08, 11.19]	7.40 [6.41, 8.41]	10.33 [9.14, 11.55]	-0.08 [-0.85, +0.71]	-0.22 [-1.13, +0.71]
GPT-5.5	1	6.94 [6.12, 7.80]	9.56 [8.53, 10.59]	7.32 [6.40, 8.29]	10.00 [8.84, 11.20]	-0.38 [-1.10, +0.35]	-0.44 [-1.19, +0.35]
GPT-5.5	6	6.14 [5.43, 6.87]	8.70 [7.82, 9.61]	6.51 [5.61, 7.49]	9.25 [8.17, 10.39]	-0.37 [-1.07, +0.27]	-0.55 * [-1.09, -0.02]

Appendix

Table 5: Our method versus Selection-only ablation in autoregressive setting. Both methods use best-of-

N

selection, but only our method provides score guidance to the writer. At

N=1

, Selection-only reduces to off-the-shelf generation.

\Delta

denotes Ours minus Selection-only (negative favors ours). Lower is better; brackets show 95% bootstrap confidence intervals, with ∗ indicating intervals excluding zero.

Task	Suspense	Humor
Readerly dynamics rating	27.85	23.63
Head-to-head comparison	33.81	38.62
Story quality rating	12.45	11.13
Total participant time	74.11	73.38

Appendix

Table 6: User study annotation time (hours).

Table 7: Suspense conditioning ablations for the measurement, steering, and writer components.

\dagger

represents our default setting. Brackets give 95% bootstrap confidence intervals.

Table 8: Humor conditioning ablations. Measurement-model rows use 20 ten-rater human-rated stories; generation rows use 514 stories and a GPT-5.5 writer. Lower error and raw gap rates are better; higher direction accuracy is better. Brackets give 95% whole-story bootstrap confidence intervals. Bold and blue mark the best and second-best point estimates.

Table 9: Readerly dynamics fidelity of different methods, averaged across stories. Metrics are shown as means with 95% bootstrap percentile intervals; lower is better. Bold marks each block’s best metric, and blue marks its second-best metric.

Table 10: Story quality evaluations for autoregressive outputs and human references on the humor dataset. Scores are medians of expected values on a 1–5 scale with 95% bootstrap intervals; higher is better. The final column of the LLM-scoring table gives mean word count per story and is not a quality score.

Table 11: Story quality evaluations for autoregressive outputs and human references on the suspense dataset. Scores are medians of expected values on a 1–5 scale with 95% bootstrap intervals; higher is better. The final column of the LLM-scoring table gives mean word count per story and is not a quality score.

Humor
Method	MAE $\downarrow$	$\Delta$ to Off-the-shelf $\downarrow$	$p$
Off-the-shelf LLM	1.44 [0.90, 2.03]	–	–
Ours	0.95 [0.66, 1.19]	-0.49 [-1.22, +0.18]	0.25
One-shot ICL	1.48 [0.95, 2.08]	+0.04 [-0.62, +0.64]	0.91
SWAG w/o DPO	1.34 [0.88, 1.84]	-0.10 [-0.86, +0.64]	0.80
Suspense

Appendix

Table 12: Human-annotated Min-K MAE on the 1–9 rating scale, using 8 stories. Lower is better.

Setting	Value
Model
Base model	Qwen3.5-27B
LoRA
Rank / scaling / dropout	16 / 32 / 0.05
Training
Epoch	1

Appendix

Table 13: Training and inference configuration for the steering model

q_{\theta_{S}}

Output Format	Min-K MAE $\downarrow$	CRPS $\downarrow$
Structured YAML	2.42 [2.03, 2.86]	6.70 [6.09, 7.37]
Plain Integer	2.53 [2.17, 2.92]	7.09 [6.46, 7.76]

Appendix

Table 14: Effect of reader dynamics representation on steering model performance on the suspense dataset. Lower values indicate better performance.

	Dynamics fidelity $\downarrow$		Story quality $\uparrow$
$T$	Min- $K$ MAE	CRPS	Relevance	Coherence	Empathy	Surprise	Engagement	Complexity
Off-the-shelf Qwen
0.7	8.80 [7.64, 10.03]	11.67 [10.37, 13.04]	4.31 [4.28, 4.34]	4.20 [4.17, 4.21]	3.05 [3.03, 3.08]	2.65 [2.61, 2.67]	3.58 [3.54, 3.61]	3.16 [3.13, 3.20]
1.0	8.30 [7.19, 9.47]	11.49 [10.19, 12.87]	4.31 [4.27, 4.32]	4.19 [4.17, 4.20]	3.04 [3.01, 3.07]	2.65 [2.61, 2.70]	3.58 [3.55, 3.60]	3.17 [3.12, 3.21]
1.5	7.82 [6.73, 8.96]	10.45 [9.27, 11.70]	4.27 [4.24, 4.30]	4.17 [4.14, 4.18]	3.05 [3.01, 3.09]	2.68 [2.63, 2.71]	3.58 [3.56, 3.61]	3.19 [3.15, 3.24]
Ours (Qwen writer)

Appendix

Table 15: Autoregressive suspense fidelity and story quality by Qwen writer temperature. Fidelity values are means across stories (lower is better); quality values are medians on a 1–5 scale (higher is better). Brackets give 95% bootstrap confidence intervals.

Figure 9: Suspense measurement under seven segmentations of the 65-segment Sandman stimulus [ 37 ] . The black curve is the mean of 23 human ratings on the original 65 segments; shading shows one standard error. Each model curve is separately affine-mapped to the observed minimum and maximum of the human mean.

		Suspense				Humor
Model	Metric	0–100	1–9	1–5 Likert	Token logits	0–100	1–9	1–5 Likert	Token logits
GPT-5.4-mini	Pearson $r$	.87 [.80, .93]	.88 [.79, .93]	.88 [.81, .92]	–	.83 [.78, .88]	.79 [.73, .85]	.76 [.69, .84]	–
	Direction (%)	78.1 [69.4, 85.7]	68.4 [57.5, 77.4]	61.0 [51.8, 68.0]	–	66.3 [59.4, 73.4]	57.7 [47.5, 69.2]	48.5 [39.0, 60.0]	–
Qwen3.5-27B	Pearson $r$	.80 [.70, .88]	.85 [.78, .91]	.86 [.78, .92]	.86 [.78, .93]	.75 [.69, .83]	.72 [.63, .82]	.72 [.64, .81]	.72 [.64, .82]
	Direction (%)	69.0 [60.9, 79.4]	71.1 [63.9, 79.7]	55.1 [47.2, 63.3]	80.7 [74.7, 87.3]	66.3 [59.1, 75.0]	65.6 [59.0, 75.2]	52.1 [44.8, 60.7]	66.3 [58.0, 74.8]
GPT-5.5	Pearson $r$	.87 [.81, .92]	.89 [.84, .92]	.87 [.81, .91]	–	.81 [.75, .87]	.79 [.72, .86]	.81 [.75, .86]	–

Appendix

Table 16: Measurement-model agreement with ten-rater human means on the “Readerly Dynamics Rating” user survey data. Brackets give 95% source-story bootstrap intervals. Dashes indicate unavailable token-logit scores.

Position-wise
Learning rate	Epochs	Min- $K$ MAE	CRPS
$1\times 10^{-4}$	1	2.59 [2.22, 2.98]	7.20 [6.51, 7.92]
$1\times 10^{-4}$	2	2.69 [2.29, 3.13]	7.12 [6.40, 7.89]
$2\times 10^{-4}$	1	2.74 [2.34, 3.17]	7.46 [6.74, 8.22]
$2\times 10^{-4}$	2	2.88 [2.50, 3.30]	7.26 [6.59, 7.96]
Autoregressive

Appendix

Table 17: Suspense fidelity of SFT baseline by learning rate and training epochs. Lower is better; bold marks the best metric and blue marks the second-best metric within each evaluation protocol.

Stage	Removed	Remaining
Source records	—	24,438
Fewer than five candidate segments	14,994	9,444
More than ten segments or preprocessing failure	1,080	8,364
LLM-based genre and quality filtering	4,268	4,096
Remove duplicates	84	4,012
Post-split leakage audit	88	3,924

Appendix

Table 18: Humor dataset size after each filtering stage.

Stage	Removed	Remaining
Corpus
Raw records available	—	3,715
Source collection and initial cleaning	653	3,062
200–1,000 words	965	2,097
Fable-family restriction	158	1,939
Near-duplicate removal	12	1,927

Appendix

Table 19: Suspense dataset size after each filtering.

Narrative Generation

See all 38 papers

StoryAlign: Evaluating and Training Reward Models for Story Generation
May 6, 2026Haotian Xia, Hao Peng, Yunjia Qi +4Reward Modeling Language Model Generation Evaluation
Department of Computer Science and Technology, Tsinghua University
The Garden of Forking Prompts: How Users Explore Narrative Space in Story Generation
Sep 13, 2026Advait Deshmukh, Nora Benedict, Melanie Walsh +1LLM Prompting Narrative Generation
University of Colorado Boulder · University of Georgia · University of Washington
Do Large Language Models Always Tell The Same Stories?
Jun 15, 2026Thennal DK, Hans Ole HatzelLLM Evaluation Diverse Text Generation
Language Technology Group University of Hamburg

May 6, 2026J/K move · Enter open · S save

StoryAlign: Evaluating and Training Reward Models for Story Generation

Haotian Xia, Hao Peng, Yunjia Qi, Xiaozhi Wang+3

Department of Computer Science and Technology, Tsinghua University

Reward Modeling Language Model Generation Evaluation Human Preference Modeling Text Generation Reward Model Evaluation Narrative Generation

Open paper AskPDF arXiv

Story generation aims to automatically produce coherent, structured, and engaging narratives. Although large language models (LLMs) have significantly advanced text generation, stories generated by LLMs still diverge from human-authored works regarding complex narrative structure and human-aligned preferences. A key reason is the absence of effective modeling of human story preferences, which are inherently subjective and under-explored. In this work, we systematically evaluate the modeling of human story preferences and introduce StoryRMB, the first benchmark for assessing reward models on story preferences. StoryRMB contains

1,133

high-quality, human-verified instances, each consisting of a prompt, one chosen story, and three rejected stories. We find existing reward models struggle to select human-preferred stories, with the best model achieving only

66.3\%

accuracy. To address this limitation, we construct roughly

100,000

high-quality story preference pairs across diverse domains and develop StoryReward, an advanced reward model for story preference trained on this dataset. StoryReward achieves state-of-the-art (SoTA) performance on StoryRMB, outperforming much larger models. We also adopt StoryReward in downstream test-time scaling applications for best-of-n (BoN) story selection and find that it generally chooses stories better aligned with human preferences. We will release our dataset, model, and code to facilitate future research. Related code and data are available at https://github.com/THU-KEG/StoryReward.

Abstract

Narrative Generation

Explore similar work

StoryAlign: Evaluating and Training Reward Models for Story Generation

The Garden of Forking Prompts: How Users Explore Narrative Space in Story Generation

Do Large Language Models Always Tell The Same Stories?