Critic's Eye: Review-Grounded Graph Reasoning for Long-Form Motif Discovery

Xinyu Sun; Zhangyue Yin; Xinyi Ju; Xin Wang

cs.AIOct 26, 2026

Critic's Eye: Review-Grounded Graph Reasoning for Long-Form Motif Discovery

Authors: Xinyu Sun, Zhangyue Yin, Xinyi Ju, Xin Wang

Organizations: College of Computer Science and Artificial Intelligence, Fudan University · Shanghai Key Laboratory of Intelligent Information Processing

Abstract

Deep narrative understanding requires distinguishing what happens from how it is presented, moving beyond the chronological fabula to decode the sjuzet, the discursive organization through which themes are staged. We study this through the recurrence of literary motifs (blood in Macbeth, for instance), where symbolic significance emerges not from local cues, but through repetition-with-variation across the whole work. We operationalize this setting as Computational Motif Discovery, a transductive task that predicts missing line-motif links using the full narrative structure of a play. We propose Critic's Eye, which models the work as a heterogeneous Narrative Topology Graph and performs discriminative inference over global structural evidence. Critic's Eye achieves 84.8% Hit@5 on our benchmark, nearly quadrupling state-of-the-art proprietary foundation models (Gemini-3-pro-preview, ~22.6%). Our analysis reveals that despite massive scaling, sequence-based models hit a "semantic ceiling", struggling to resolve dispersed dependencies. These findings suggest that explicit structural priors are a far more effective inductive bias than parameter scaling for decoding the complex architecture of long-form literature.

Figures & tables

Figure 1: Why motif evidence is spread out. Bottom: the fabula , the play’s events in chronological order. Middle: recurring images, each attaching to events that lie far apart on that order. Top: the themes those images carry.

Figure 2: The Critic’s Eye Graph Construction Pipeline. We fuse the explicit Narrative Backbone (parsed from TEI XML) with a latent Thematic Layer derived from literary criticism. (1) An LLM extracts structured tuples (Reference, Theme, Device) from unstructured commentary. (2) These mentions are semantically clustered into canonical motif nodes. (3) The resulting heterogeneous graph encodes both the linear fabula and the critical sjužet , serving as the substrate for transductive inference. The extraction prompt and output schema are in Appendix B .

Figure 3: Stage 1: Multi-Task Structural Pre-training. Before accessing motif labels, the encoder is forced to internalize the narrative “anatomy” via three intrinsic objectives: (A) Adjacency Continuity ( $u_{t}\to u_{t+1}$ ) captures local discourse flow; (B) Hierarchical Localization ( $u\to s$ ) recovers macro-structural positioning; and (C) Persona Alignment ( $u\to a$ ) learns character-specific stylistic features. This structure-first strategy aligns the embedding space with the narrative topology.

Node Type	Count	Edge Type	Count
Act / Scene	5 / 28	Hierarchical ( $\mathcal{E}_{\textsc{hier}}$ )	3,397
Speech Section	865	Sequential ( $\mathcal{E}_{\textsc{seq}}$ )	2,602
Atomic Line	2,456	Metadata ( $\mathcal{E}_{\textsc{agent}}$ )	1,058
Character	44	Total Explicit	7,057
Latent Motif	281	Thematic Link	923

Table 1: Statistics of the Macbeth Narrative Graph. Thematic Link counts every criticism-derived edge in the graph, including relations between two textual units and relations anchored to a speech or scene rather than to a single line. The line

\rightarrow

motif membership links that supply supervision and evaluation gold are the subset plotted in Figure 4 (351 links over 283 lines and 193 motifs).

Stratum (Type)	Focus & Role in Narrative Graph
1. Humanist	Focus: Character agency, ethical conflicts, and tragedy.
(12 Monographs	Role : High-Density Supervision. Comprising foundational book-length studies and major critical volumes.
& Volumes)	Key Sources: Bradley (1904) , Bloom (1987) , Fletcher (1844) , Hazlitt (1903) , Weiss (1876) .
2. Socio-Cultural	Focus: Folklore, politics, history, and theatre practice.
(8 Mixed Sources)	Role : Contextual Anchoring. Captures world-building motifs (e.g., Witchcraft , Kingship ) missed by pure text analysis.
(Books/Essays)	Key Sources: Thiselton-Dyer (1884) (Folklore), Adams (1917) (Theatre), Tolstoy (1906) (Counter-criticism), Moschovakis (2008) .

Table 2: Stratification of the Critical Corpus. Sources are categorized by interpretive lens. Note that the Humanist stratum comprises of extensive monographs and volumes, providing a dense backbone of character analysis, while other strata introduce targeted, fine-grained perspectives. Full bibliographic details are in Appendix A .

Method	Hit				nDCG				MRR
	@1	@3	@5 ∗	@10	@1	@3	@5	@10
Embedding SOTA (Best of dense retrievers)
Sentence-BERT	0.042	0.074	0.106	0.106	0.041	0.053	0.064	0.074	0.081
Open-Source LLMs (Highlighting Inverse Scaling)
GPT-OSS-20B (+RAG)	0.042	0.088	0.117	0.120	0.042	0.070	0.081	0.082	0.070
GPT-OSS-120B (+RAG) †	0.032	0.064	0.085	0.085	0.032	0.049	0.058	0.058	0.050

Table 3: Performance Comparison with Frontier Models. Even the strongest proprietary model (Gemini-3-pro-preview) hits a ceiling (

\sim

23% Hit@5), lagging significantly behind Critic’s Eye ’s structural reasoning.

Figure 4: The Severe Long-Tail Distribution of Literary Motifs. MotifBench follows a sharp power law (Imbalance Ratio = 35.0), with over 96% of active motifs (187/193) in the sparse “tail” (freq $\leq$ 5). This extreme sparsity renders standard random splits ineffective, motivating our transductive formulation to capture these rare signals.

Model Configuration	Hit				nDCG				MRR
	@1	@3	@5	@10	@1	@3	@5	@10
w/o pre-training (frozen random)	0.1237	0.2544	0.3145	0.3746	0.1237	0.1710	0.1944	0.2134	0.2189
Critic’s Eye ( Hetero-only )	0.2792	0.4947	0.5901	0.7208	0.2792	0.3742	0.4165	0.4657	0.4245
Critic’s Eye ( Seq-only )	0.4876	0.6608	0.7562	0.8198	0.4876	0.5543	0.5960	0.6223	0.6019
Critic’s Eye ( Full Topology )	0.5654	0.7809	0.8481	0.9293	0.5654	0.6618	0.6979	0.7321	0.6942

Table 4: Structural pre-training and topology ablation (

N{=}283{}

query lines;

|\mathcal{M}|{=}281{}

motifs). We evaluate with a frozen-encoder probing protocol to measure representation quality. Pre-training is necessary (random vs. pre-trained), and integrating richer structure yields monotonic gains ( Hetero-only

<

Seq-only

<

Full Topology ).

Figure 5: Structural resilience under edge dropout. Mean Reciprocal Rank (MRR) as a function of inference-time edge dropout rate $p$ . The Full topology degrades more gracefully than Sequence-only across a wide range of dropout rates, indicating higher robustness to partial graph observations.

Figure 6: Structural reasoning beyond local cues. (a) Critic’s Eye anchors a metaphor to its scene context, avoiding a lexical-similarity trap. (b) Critic’s Eye propagates cross-speaker causality via interaction structure, resolving a specific motif where a sequential baseline predicts a generic one.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Author / Source	Title
Johnson (1765)	Preface to Shakespeare
Fletcher (1844)	Macbeth:—Shakespearian Criticism and Acting
Weiss (1876)	Wit, Humor, and Shakspeare: Twelve Essays
Thiselton-Dyer (1884)	Folk-lore of Shakespeare
Hazlitt (1903)	Characters of Shakespeare’s Plays
Bradley (1904)	Shakespearean Tragedy: Lectures on Hamlet, Othello, King Lear, Macbeth

Appendix

Table 5: The Critical Corpus for Macbeth Motif Extraction (Weak Supervision Sources).

Panel of Figure 7 — Figure 7: Prompt templates for relation extraction from criticism (Section 3.3 ), condensed. The full prompt also carries four worked examples, two of which show passages that must be rejected.

Panel of Figure 8 — Figure 8: Prompt templates for the zero-shot and retrieval-augmented LLM baselines (Section 5.1 ). This is a separate procedure from the extraction pipeline of Figure 7 .

Method	Hit				nDCG				MRR
	@1	@3	@5	@10	@1	@3	@5	@10
Embedding Baselines
Random Guess	0.0035	0.0106	0.0177	0.0353	0.0035	0.0075	0.0104	0.0161	0.0220
Sentence-BERT	0.0424	0.0742	0.1060	0.1060	0.0406	0.0531	0.0637	0.0736	0.0812
BGE-Large-v1.5	0.0389	0.0707	0.0813	0.1307	0.0371	0.0488	0.0521	0.0665	0.0748
Open-Source LLMs

Appendix

Table 6: Performance on motif identification (

N=283{}

). Comparison against Open and Closed SOTA LLMs.

Computational Literary Studies

Computational Literary Studies uses data analysis and language technologies to examine literary style, themes, narratives, influence, and change.

See all 49 papers

Beyond Linear Context: Graph-Guided Evidence Navigation for Long-Novel Reasoning with a Local 9B Language Model
Sep 19, 2026Wenji FuLong-Context QA LLM Reasoning with Graphs
Research Institute of Economics and Management Southwestern University of Finance and Economics
ClueWeaver: Reward-Guided Dual-Agent Evidence Reasoning for Compact LLMs on Literary Long Narratives
Aug 26, 2026Jihao Zhu, Zhiwei Yang, Wenxiao Zhang +7Long-Context QA Multi-Agent LLM Systems
University of Aberdeen, Aberdeen, United Kingdom · Institute of Information Engineering, CAS, Beijing, China · The University of Western Australia, Perth, Australia +1
Storyline Trees: Hierarchical Representations for Long-Form Narratives
Jun 18, 2026Litu Ou, Mirella LapataLong-Context Retrieval Long-Context QA
School of Informatics University of Edinburgh

Sep 19, 2026J/K move · Enter open · S save

Beyond Linear Context: Graph-Guided Evidence Navigation for Long-Novel Reasoning with a Local 9B Language Model

Wenji Fu

Research Institute of Economics and Management Southwestern University of Finance and Economics

Long-Context QA LLM Reasoning with Graphs Long-Context Retrieval Graph-Based Retrieval Knowledge Graph Question Answering

Open paper AskPDF arXiv

Long-context models read a novel the way a person reads a printout: one token after another, in narrative order, with the whole history competing for a fixed budget of attention. A detective does not work that way. They sort what happened when, and they keep a map of who relates to whom, so a clue from chapter one can meet a question asked at the end of the book. We test whether a frozen knowledge graph can give a small local model that same freedom. Thirty detective novels and 234 multiple-choice questions are answered by one fixed qwen3.5:9b reader under nine conditions: five graph routes, a recent-window baseline, whole-book compression, ordinary vector retrieval, and a question-only control. The strongest graph route reaches 53.85% (126/234) against 46.15% for the recent window, 51.28% for compression, 51.71% for vector retrieval and 40.17% for question-only. On the subset that no model can answer without the book, the graph route reaches 42.86%. None of the fifteen graph-baseline contrasts survives Holm correction, so we present the result as exploratory evidence about a design. Two structural findings survive scrutiny better than the headline number: annotated evidence concentrates in the topological core of these graphs (2.35x enrichment, pooled), and the two graph-building pipelines differ so much in annotation coverage (16% versus 73% of clue paragraphs) that pooled accuracy alone would hide which bottleneck is being measured.

Abstract

Computational Literary Studies

Explore similar work

Beyond Linear Context: Graph-Guided Evidence Navigation for Long-Novel Reasoning with a Local 9B Language Model

ClueWeaver: Reward-Guided Dual-Agent Evidence Reasoning for Compact LLMs on Literary Long Narratives

Storyline Trees: Hierarchical Representations for Long-Form Narratives