cs.CLApr 13, 2026
SaveThink Before you Write: QA-Guided Reasoning for Character Descriptions in Books
Organizations: Institute of Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB
Abstract
Character description generation is an important capability for narrative-focused applications such as summarization, story analysis, and character-driven simulations. However, generating accurate character descriptions from long-form narratives (e.g., novels) is challenging: models must track evolving attributes (e.g., relationships and events), integrate evidence scattered across the text, and infer implicit details. Despite the success of reasoning-enabled LLMs on many benchmarks, we find that for character description generation their performance improves when built-in reasoning is disabled (i.e., an empty reasoning trace). Motivated by this, we propose a training framework that decouples reasoning from generation. Our approach, which can be applied on top of long-context LLMs or chunk-based methods, consists of a reasoning model that produces a structured QA reasoning trace and a generation model that conditions on this trace to produce the final character description. Experiments on two datasets (BookWorm and CroSS) show that QA-guided reasoning improves faithfulness, informativeness, and grounding over strong long-context baselines.
Figures & tables
Figure 1: Overview of QA-based guided reasoning for character description generation. Given a book and character name, the reasoning model generates a structured QA trace capturing salient character information. This trace is injected into the thinking tokens of the generation model , which produces the final character description. The reasoning model can be trained with GRPO and the generation model with SFT.
| BookWorm | CroSS | |
| Books | 324 | 126 |
| Characters | 9.74 | 6.56 |
| Samples | 5,869 | 824 |
| Input Length | 97,685 | 129,113 |
| Output Length | 88.79 | 295.27 |
Table 1: Dataset statistics: unique books, average characters per book, total examples (book–description pairs), and average input/output length in words.
| Method | Trace | Desc | PRISMA | QA | NLI | EntMent | Rouge-L | |
| Qwen-3-8b | No Context | — | ZS | 11.97 | 10.31 | — | 25.97 | 17.55 |
| Lead-32k | — | ZS | 15.72 | 12.35 | 61.01 | 30.60 | 17.79 | |
| BM25-32k | — | ZS | 17.09 | 13.56 | 58.58 | 34.06 | 18.18 | |
| BM25-32k | — | SFT | 16.17 † | 13.26 | 60.00 † | 33.11 † | 18.99 † | |
| + reasoning | built-in | ZS | 17.06 | 13.50 | 58.44 | 35.13 | 16.51 † | |
| + guided-QA | ZS | ZS | 17.84 | 14.66 † | 59.96 † | 36.23 † | 17.43 † |
Table 2: Results on BookWorm. Trace: — (none), built-in (model’s default), ZS (zero-shot guided-QA), GRPO (GRPO-trained guided-QA). Desc: ZS (zero-shot) or SFT (supervised fine-tuning). † indicates statistically significant difference from the corresponding baseline without reasoning (i.e., first row for each method group); bold indicates best per metric.
| Model | Method | Trace | Desc | PRISMA | QA | NLI | EntMent | Rouge-L |
| Qwen-3-8b | No Context | — | ZS | 7.69 | 2.92 | — | 19.83 | 16.40 |
| Coref-32k | — | ZS | 23.13 | 13.03 | 50.01 | 35.90 | 17.44 | |
| + reasoning | built-in | ZS | 22.26 † | 12.52 | 46.84 † | 33.50 † | 16.02 † | |
| + guided-QA | ZS | ZS | 21.88 † | 13.11 | 49.63 | 35.46 | 17.24 † | |
| + guided-QA | GRPO | ZS | 22.63 † | 13.08 † | 49.70 | 35.38 † | 17.33 † | |
| Lead-128k (w/ YaRN) | — | ZS | 22.22 | 13.70 | 66.39 | 35.48 | 17.21 |
Table 3: Results on CroSS. Trace: — (none), built-in (model’s default), ZS (zero-shot guided-QA), GRPO (GRPO-trained guided-QA). Desc: ZS (zero-shot) or SFT (supervised fine-tuning). † indicates statistically significant difference from the corresponding baseline without reasoning (i.e., first row for each method group); bold indicates best per metric.
| QA reasoning | Description | |||||
| Method | # QA | Prec | Rec | F1 | PRISMA | QA |
| No reasoning | — | — | — | — | 15.20 | 14.42 |
| No chunking | 6.50 | 16.10 | 16.93 | 16.50 | 14.02 | 14.03 |
| guided-QA | 8.60 | 15.07 | 17.68 | 16.27 | 14.45 | 14.45 |
| guided-QA (SFT) | 6.60 | 14.84 | 15.98 | 15.38 | 14.11 | 14.08 |
| guided-QA (GRPO) | 11.07 | 15.25 | 19.10 | 16.96 | 15.25 | 14.79 |
Table 4: Comparison of question generation methods for the reasoning model on the BookWorm dataset (validation set) with coreference-based retrieval. The generation model is zero-shot in all experiments.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Hyperparameter | Value |
| actor learning rate | |
| kL coefficient | |
| train batch size | |
| samples per prompt | |
| prompt max length | |
| generation max length |
Table 5: Training hyperparameters for Group Relative Policy Optimization (GRPO) training.
| Hyperparameter | Value |
| learning rate | |
| max input length | |
| batch size | |
| gradient accumulation steps | |
| alpha | |
| rank |
Table 6: Training hyperparameters for supervised finetuning (SFT) training with LoRA.
| Context: {context} |
| Describe character {character} from book {book} based on the given context. |
| Return your output as a single paragraph (close to {length} words) including the important information. |
Table 7: Character description generation prompt template. We replace the variable length with the average output length of the corresponding dataset.
| Context: {context} |
| Your task is to generate questions answer pairs about character: {character} from book: {book} given the above chunk of the book. You should focus on understanding aspects of the character (e.g., role, relationships, personality, events) that are mentioned in the context. Each qa-pair should be labelled as role, relationship, personality, event or other. We provide the definitions for these below. |
| Definitions: |
| Role: defines what part the character plays in the story, narrator, major/minor character. |
| Relationship: connections the character has with others, such as friendships or family ties. |
Table 8: QA generation prompt template. We adopt the definitions for the different categories from Papoudakis et al. (2024)
| QA Reasoning | Description | ||||||
| Method | #QA | P | R | F1 | PRI | QA | Ent |
| guided-QA | 8.60 | 15.07 | 17.68 | 16.27 | 14.45 | 14.45 | 30.41 |
| w/o expl. | 12.05 | 13.89 | 16.93 | 15.26 | 14.34 | 15.01 | 29.94 |
| w/o types | 11.44 | 12.46 | 15.87 | 13.95 | 13.41 | 14.34 | 30.37 |
| w/o expl., types | 15.46 | 13.90 | 12.16 | 13.61 | 13.71 | 14.92 | 30.08 |
| w/o expl., ans., types | 13.44 | — | — | — | 14.02 | 14.22 | 29.62 |
Table 9: Ablation of QA reasoning components using Qwen-3-8b with coreference retrieval on BookWorm validation set (zero-shot). P/R/F1 = reasoning precision/recall/F1; PRI = PRISMA F1; Ent = EntMent F1.
| Reasoning | Description | ||||
| Method | #QA | Tok | Uni | Tok | Uni |
| None | — | — | — | 111 | 71 |
| built-in | — | 270 | 60 | 109 | 78 |
| guided-QA (ZS) | 8 | 391 | 36 | 118 | 73 |
| guided-QA (GRPO) | 11 | 535 | 31 | 111 | 71 |
Table 10: Statistics for generated traces and their corresponding descriptions for different reasoning methods using retrieved context on BookWorm validation set. #QA = QA pairs; Tok = avg. tokens; Uni = unique unigrams (%).
| Method | PRISMA | QA | EntMent | R-L |
| Coref-32k | 13.65 | 12.49 | 27.93 | 16.20 |
| + reasoning | 11.90 | 11.11 | 25.60 | 15.49 |
| + guided-QA | 13.10 | 13.60 | 28.82 | 16.61 |
| Lead-128k | 11.76 | 11.78 | 26.39 | 15.99 |
| + reasoning | 11.16 | 10.67 | 25.66 | 15.16 |
| + guided-QA | 13.12 | 13.32 | 29.34 | 16.62 |
Table 11: Results on BookWorm dataset using Gemma-12b-it model. Zero-shot experiments without any reasoning trace, explicitly prompting the model to reason and QA-guided reasoning for coref-based retrieval and Lead-128k methods.
| Method | # QA | Precision | Recall | F1 |
| guided-QA | 8.60 | 17.68 | 15.07 | 16.27 |
| Savaal | 12.01 | 15.23 | 13.92 | 14.54 |
| Plan-based QA | 11.90 | 17.34 | 11.71 | 13.98 |
Table 12: Experiments with different QA generation methods using coref-based retrieval on BookWorm dataset. We report the average number of QA-pairs along with precision, recall and F1 score for reasoning evaluation.
| Character: Demon Copperhead Book: Demon Copperhead |
| Lead-128k: Demon Copperhead is a complex and resilient character shaped by a life of hardship, neglect, and survival. Born to a drug-addicted mother, he was abandoned in a trailer park in Lee County, Virginia, and nearly died at birth due to his mother’s drug use, which led to his being born in the caul—a rare occurrence that his mother claimed was a sign of her own superstitions. He was taken in by the Peggots, who were not his real family but became his de facto parents, giving him a sense of belonging and a nickname, "Demon," that stuck with him as he grew. His early life was marked by instability, including being sent to live with the brutal Crickson family on a farm, where he endured harsh treatment and emotional abuse. He also experienced foster care with Miss Barks, who became a maternal figure but eventually left for a better-paying job as a teacher. Demon’s life took a tragic turn when his mother died in an overdose, leaving him in foster care again and further alienated. He later ran away to Nashville, where he encountered a series of misfortunes, including theft, abuse, and being caught in a pawnshop with a hooker. Eventually, he found refuge with his grandmother Betsy Woodall, who had long been estranged from him but now took him in after learning of his mother’s death. Demon is marked by his red hair and blue eyes, traits he inherited from his father—a Melungeon man he never knew but who left an indelible mark on his identity. He is also defined by his survival instincts, having learned to navigate the harsh realities of poverty, neglect, and abuse. |
| Lead-128k + guided-QA (GRPO): Demon Copperhead, the protagonist of the novel, is a resilient and determined young man who navigates a life marked by hardship, neglect, and a complex web of relationships. Born to an addicted mother, he faces a difficult start in life, including a traumatic birth and early experiences with foster care. His early years are shaped by the Peggots, who become a significant part of his life, though their relationship is strained and complicated. He develops a close friendship with Maggot, another foster child, and later forms a bond with Fast Forward, who becomes a mentor and friend. Demon’s nickname "Demon" is given to him by others, reflecting his troubled past and the stigma he carries. His relationship with Stoner, his stepfather, is fraught with tension and abuse, leading to a confrontation that results in Demon’s injury and eventual placement in foster care. He is placed with Mr. Crickson, a harsh farmer who runs a slave-boy farm, but eventually finds a more supportive home with Coach Winfield and his family in Jonesville. Demon’s personality is marked by his determination to prove his worth despite his circumstances, as well as his artistic talent for drawing superheroes and comic strips. He has a complex relationship with his biological mother, who dies from an overdose, and his biological father, whose death he never fully understands. His journey includes running away from home, stealing a tree for Christmas, and eventually finding love and support with Dori, who becomes his girlfriend and later his fiancée. Demon’s story is one of resilience, as he struggles with addiction but ultimately seeks redemption through his art and relationships. His character is defined by his search for belonging and self-worth in a world that often overlooks him. |
| Goldstandard Description: Demon Copperhead is the eponymous, first-person narrator and protagonist of the novel. Because the novel’s narrative voice deals with both the events of Demon’s life and his reflection on those events, the novel is structured like a memoir. It is also a bildungsroman in which Demon endures several life-altering conflicts and eventually finds peace with himself. The world has been cruel to Demon, and his early life demonstrates The Failure of Society to Protect Its Children. He was born without a father, and his mother struggled with addiction throughout her short life. He navigates several different forms of false families, such as life with his abusive stepfather, placements in two abusive foster homes, and rejection from his father’s mother. The lack of love in Demon’s childhood and early adolescence makes him radically self-reliant, but it also makes him distrustful of the people who do provide him with security and care. Demon is independent because he must be to survive, but he looks for people to believe in—often misguidedly, as with Fast Forward and Dori. Similarly, he has an attachment to Lee County because it is a space he at least understands, even if it keeps bringing him down. Demon eventually learns that he must focus on himself to receive real love from other people. [truncated] Fact Lead-128k Lead-128k guided-QA Demon Copperhead is the eponymous narrator of the novel. False True Demon Copperhead is the first-person narrator of the novel. False False Demon Copperhead is the protagonist of the novel. False True The narrative voice of the novel "Demon Copperhead" deals with the events of Demon’s life. False True The narrative voice of the novel "Demon Copperhead" includes Demon’s reflection on those events. False False The novel "Demon Copperhead" is structured like a memoir. False False "Demon Copperhead" is a bildungsroman. False True In "Demon Copperhead," Demon endures several life-altering conflicts. True True In "Demon Copperhead," Demon eventually finds peace with himself. False False The world has been cruel to Demon Copperhead. True True Demon’s early life demonstrates The Failure of Society to Protect Its Children. True True Demon was born without a father. True False Demon’s mother struggled with addiction throughout her short life. False True Demon navigates several different forms of false families. True True Demon lived with an abusive stepfather. False True Demon was placed in two abusive foster homes. True True Demon experienced rejection from his father’s mother. False False The lack of love in Demon’s childhood made him radically self-reliant. True True [Facts truncated] |
Table 13: Example of generated descriptions in CroSS dataset with QA-guided Lead-128k and the corresponding baseline. We demonstrate facts evaluated based on the PRISMA-recall metric.
| Character: Demon Copperhead Book: Demon Copperhead |
| Reasoning for 1st Chunk: Q1: What is Demon Copperhead’s role in the story? E1: Demon Copperhead is the protagonist and narrator of the story, experiencing various challenges and relationships throughout his life. A1: Protagonist T1: Role Q2: Who is Demon Copperhead’s mother? E2: Demon Copperhead’s mother is a central character in the story, struggling with addiction and neglect. A2: Addicted mother T2: Relationship Q3: What is Demon Copperhead’s personality like? E3: Demon Copperhead is portrayed as a resilient and determined character, despite facing numerous hardships. A3: Resilient and determined T3: Personality Q4: What event leads to Demon Copperhead’s birth? E4: Demon Copperhead is born to a mother who is struggling with addiction, leading to a difficult start in life. A4: Difficult birth T4: Event Q5: Who is Demon Copperhead’s foster family? E5: Demon Copperhead is raised by the Peggots, who become a significant part of his life. A5: The Peggots T5: Relationship Q6: What is Demon Copperhead’s relationship with Maggot? E6: Demon Copperhead and Maggot are close friends who grow up together in the same community. A6: Close friend T6: Relationship Q7: What is Demon Copperhead’s nickname? E7: Demon Copperhead is given the nickname "Demon" by others, which becomes a part of his identity. A7: Demon T7: Personality Q8: What event leads to Demon Copperhead’s time with the Peggots? E8: Demon Copperhead spends time with the Peggots after his mother’s struggles with addiction and his own difficult circumstances. A8: Time with Peggots T8: Event Q9: What is Demon Copperhead’s relationship with Stoner? E9: Demon Copperhead has a complex relationship with Stoner, who becomes his stepfather and has a significant impact on his life. A9: Stepfather T9: Relationship Q10: What event leads to Demon Copperhead’s conflict with Stoner? E10: Demon Copperhead conflicts with Stoner due to Stoner’s strict parenting style and his own struggles with identity and self-worth. A10: Conflict with Stoner T10: Event Reasoning for 2st Chunk: Q11: What event leads to Demon Copperhead being placed in foster care? E11: Demon Copperhead’s mother becomes addicted to drugs and is involved in a violent relationship, leading to his placement in foster care. A11: Mother’s addiction and abuse. T11: Event Q12: Who becomes Demon Copperhead’s foster father? E12: Demon Copperhead’s foster father is Mr. Crickson, a farmer who takes in foster children. A12: Mr. Crickson. T12: Relationship Q13: What event causes Demon Copperhead to run away from home? E13: Demon Copperhead runs away from home after a violent confrontation with his stepfather, Stoner. A13: Confrontation with Stoner. T13: Event Q14: What is Demon Copperhead’s attitude towards his foster family? E14: Demon Copperhead has a complex attitude towards his foster family, initially feeling out of place but gradually forming bonds with some members. A14: Complex attitude. T14: Personality Q15: What event leads to Demon Copperhead being placed in a temporary foster home? E15: Demon Copperhead is placed in a temporary foster home after his mother is hospitalized and unable to care for him. A15: Mother’s hospitalization. T15: Event [Trace truncated] |
Table 14: Example of generated reasoning trace in CroSS dataset. The trace is from the GRPO-trained Qwen-3-8b reasoning model using Lead-128k.
| Character: John Montraville Book: Charlotte Temple |
| Lead-128k: John Montraville is a young, impetuous soldier who becomes infatuated with Charlotte Temple, a pure and innocent girl, and secretly courts her despite his duty and honor. He is torn between his love for Charlotte and his sense of responsibility, but his passion leads him to elope with her, only to discover she was deceived by Mademoiselle La Rue. Montraville’s heart breaks upon learning of Charlotte’s betrayal and her subsequent suffering, ultimately leading to his tragic downfall and death, which he brings upon himself in a fit of vengeance against Belcour. Fact John Montraville is a young soldier. True John Montraville is an impetuous soldier. False John Montraville becomes infatuated with Charlotte Temple. True Charlotte Temple is a pure and innocent girl. False John Montraville secretly courts Charlotte Temple despite his duty. False John Montraville secretly courts Charlotte Temple despite his honor. False John Montraville is torn between his love for Charlotte Temple and his sense of responsibility. False John Montraville’s passion leads him to elope with Charlotte Temple. False John Montraville discovers that Charlotte Temple was deceived by Mademoiselle La Rue. False Montraville’s heart broke upon learning of Charlotte’s betrayal. False Montraville’s heart broke upon learning of Charlotte’s subsequent suffering. False Montraville’s heart breaking led to his tragic downfall. False Montraville’s tragic downfall led to his death. False Montraville brought his death upon himself in a fit of vengeance against Belcour. False |
| Lead-128k + guided-QA: John Montraville is a Lieutenant in the army, an ardent and impetuous soldier who becomes infatuated with Charlotte Temple, a young, innocent girl he meets at a boarding school. He secretly courts her, leading her to elope with him, but later discovers her relationship with Belcour, his friend, which causes his heartbreak. Despite his initial honor and compassion, he succumbs to passion, abandons Charlotte, and marries Julia Franklin. His guilt over betraying Charlotte and the death of her child lead to a tragic end, as he kills Belcour in a fit of rage and dies shortly after, haunted by remorse. Fact John Montraville is a Lieutenant in the army. False John Montraville is an ardent soldier. False John Montraville is an impetuous soldier. False John Montraville became infatuated with Charlotte Temple. True Charlotte Temple is a young and innocent girl. False John Montraville met Charlotte Temple at a boarding school. False John Montraville secretly courted Charlotte Temple. False John Montraville led Charlotte Temple to elope with him. False John Montraville later discovered Charlotte Temple’s relationship with Belcour. False Belcour is a friend of John Montraville. True John Montraville’s discovery of Charlotte Temple’s relationship with Belcour caused him heartbreak. False John Montraville initially displayed honor and compassion. False John Montraville succumbed to passion. True John Montraville abandoned Charlotte Temple. True John Montraville married Julia Franklin. True John Montraville felt guilt over betraying Charlotte Temple. True John Montraville felt guilt over the death of Charlotte Temple’s child. False John Montraville killed Belcour in a fit of rage. False John Montraville died shortly after killing Belcour. False John Montraville was haunted by remorse after his actions. True |
| Goldstandard Description: A charming, dashing young officer in the British Army, John Montraville falls in love with Charlotte Temple in Chichester and hopes to marry her. However, his father refuses to endorse a marriage to a poor woman, so Montraville makes Charlotte his mistress and brings her to America with him. He tries to treat her honorably but is easily deceived by his friend Belcour, who ruins his relationship with Charlotte so he can have her as his own mistress. Despite his inability to control his womanizing impulses, Montraville feels guilty about the misfortune he causes Charlotte and others. |
Table 15: Examples of generated descriptions in BookWorm dataset with QA-guided Lead-128k and the corresponding baseline. We demonstrate the facts evaluated based on PRISMA-precision metric.
| Character: John Montraville Book: Charlotte Temple |
| Reasoning for 1st Chunk: Q1: What role does John Montraville play in the story? E1: John Montraville is a major character in the story, a young soldier who becomes involved with Charlotte Temple. A1: Major character T1: Role Q2: Who is John Montraville related to in the story? E2: John Montraville is related to Charlotte Temple, as he becomes romantically involved with her. A2: Charlotte Temple T2: Relationship Q3: What is John Montraville’s personality like? E3: John Montraville is portrayed as a tender, eloquent, and ardent lover, but also impetuous and driven by his passions. A3: Ardent and impetuous T3: Personality Q4: What event leads John Montraville to meet Charlotte Temple? E4: John Montraville meets Charlotte Temple when he encounters her and her governess at a church in Chichester. A4: Meeting Charlotte Temple T4: Event Q5: What decision does John Montraville make regarding his relationship with Charlotte Temple? E5: John Montraville decides to elope with Charlotte Temple, despite knowing their families would not approve. A5: Decide to elope T5: Event Q6: What is John Montraville’s profession? E6: John Montraville is a soldier, as he is mentioned as a Lieutenant in the army. A6: Soldier T6: Role Q7: What is John Montraville’s relationship with Belcour? E7: John Montraville is friends with Belcour, who accompanies him on his visits to Charlotte Temple. A7: Friend T7: Relationship Q8: What is John Montraville’s attitude towards marriage? E8: John Montraville is initially cautious about marriage, as his father warns him against rushing into it. A8: Cautious T8: Personality Q9: What is one of the events involving John Montraville and Charlotte Temple? E9: One event involving John Montraville and Charlotte Temple is their clandestine meetings at a summer-house. A9: Clandestine meetings T9: Event Q10: What is John Montraville’s relationship with his father? E10: John Montraville has a respectful relationship with his father, who gives him advice about marriage and duty. A10: Respectful T10: Relationship Reasoning for 2st Chunk: Q11: Who is John Montraville romantically involved with? E11: John Montraville is romantically involved with Charlotte Temple and Julia Franklin. A11: Charlotte Temple T11: Relationship Q12: What event leads to John Montraville’s conflict with Charlotte Temple? E12: John Montraville’s conflict with Charlotte Temple arises from her elopement and his subsequent discovery of her relationship with Belcour. A12: Elopement T12: Event Q13: What event leads to John Montraville’s conflict with Julia Franklin? E13: John Montraville’s conflict with Julia Franklin arises from his indecision about marrying her while still being involved with Charlotte Temple. A13: Indecision T13: Event Q14: What event leads to John Montraville’s discovery of Charlotte’s infidelity? E14: John Montraville discovers Charlotte’s infidelity when he finds her in bed with Belcour. A14: Discovery T14: Event Q15: What is John Montraville’s role in the story’s resolution? E15: John Montraville plays a significant role in the story’s resolution by ultimately leaving Charlotte and pursuing a relationship with Julia Franklin. A15: Pursuing relationship T15: Event Q16: What is John Montraville’s personality trait regarding honor? E16: John Montraville is portrayed as having a strong sense of honor, which conflicts with his romantic entanglements. A16: Strong sense of honor T16: Personality [Trace truncated] |
Table 16: Example of generated reasoning trace in BookWorm dataset. The trace is from the GRPO-trained Qwen-3-8b reasoning model with Lead-128k as input context.
More in this topic
Text Generation
Text generation studies methods for producing coherent, useful language from prompts, data, and dialogue context.
See all 422 papers