cs.CLMar 28, 2026
SaveImproving Attributed Long-form Question Answering with Intent Awareness
Organizations: Allen Institute for AI · Carnegie Mellon University · University of Washington
Abstract
Large language models (LLMs) are increasingly being used to generate comprehensive, knowledge-intensive reports. However, while these models are trained on diverse academic papers and reports, they are not exposed to the reasoning processes and intents that guide authors in crafting these documents. We hypothesize that enhancing a model's intent awareness can significantly improve the quality of generated long-form reports. We develop and employ structured, tag-based schemes to better elicit underlying implicit intents to write or cite. We demonstrate that these extracted intents enhance both zero-shot generation capabilities in LLMs and enable the creation of high-quality synthetic data for fine-tuning smaller models. Our experiments reveal improved performance across various challenging scientific report generation tasks, with an average improvement of +2.9 and +12.3 absolute points for large and small models over baselines, respectively. Furthermore, our analysis illuminates how intent awareness enhances model citation usage and substantially improves report readability.
Figures & tables
Figure 1: Current long-form question answering systems don’t consider intents when generating responses. The figure above shows how having explicit citation intents and paragraph intents helps reason about the text and generate better responses.
| Intent Category & Type | Description | |
|---|---|---|
| Citation Intent | Background | The citation provides relevant information for this domain |
| Motivation | The citation illustrates the need for data, goals, methods, etc. | |
| Uses | The sentence uses data, methods, etc. from the citation | |
| Extension | The sentence extends the referenced work’s data, methods, etc. of the citation | |
| Comparison or Contrast | The sentence expresses similarity/differences to the referenced work | |
| Future | The citation identifies the reference as a potential avenue for future work | |
Table 1: The types and descriptions for our intent awareness schemes. We adopt the citation intent types from ACL-ARC ( Jurgens et al., 2018 ) and extend the paragraph intent types from the discourse modes studied in ( Song et al., 2017 ) .
| Method | SQA-CS-V2 | ||||
|---|---|---|---|---|---|
| Overall | Rubrics | Ans. P | Citation P | Citation R | |
| o3 | 85.1 | 91.4 | 96.5 | 89.4 | 63.4 |
| + intent | 86.0 | 90.7 | 96.6 | 89.9 | 66.9 |
| gemini-2.5-pro | 88.1 | 82.6 | 94.1 | 93.2 | 82.4 |
| + intent | 89.7 | 82.6 | 94.5 | 95.7 | 86.1 |
| Claude opus-4 | 85.4 | 84.3 | 87.9 | 89.6 | 79.6 |
Table 2: Performance comparison across various models on SQA-CS-V2. Overall denotes the macro-average of other sub-metics. Bold indicates the best-performing row for overall metrics. +intent denotes the use of our intent-aware-writing framework with both paragraph and citaiton intents.
| Method | DeepScholar Bench (DSB) | RQA | ||||
|---|---|---|---|---|---|---|
| Overall | Nug. Cov. | Org. | Cite-P | Claim Cov | Rubrics | |
| o3 | 46.8 | 47.0 | 61.1 | 39.1 | 40.2 | 76.3 |
| + intent | 43.2 | 49.1 | 64.1 | 27.2 | 34.3 | 79.3 |
| gemini-2.5-pro | 54.8 | 49.0 | 63.1 | 53.0 | 54.2 | 71.9 |
| + intent | 57.8 | 49.0 | 58.0 | 61.1 | 63.3 | 74.0 |
| Claude opus-4 | 58.1 | 54.0 | 64.1 | 56.6 | 57.6 | 74.3 |
Table 3: Performance comparison on DeepScholar Bench and ReseachQA. RQA denotes ResearchQA. Overall denotes the macro-average of other sub-metics. Bold indicates the best-performing row for overall metrics.
| Base Model | Variant | Overall | Rubrics | Answer P | Citation P | Citation R |
|---|---|---|---|---|---|---|
| gemini-2.5-pro(ref) | - | 88.1 | 82.6 | 94.1 | 93.2 | 82.4 |
| qwen3-8b | no training | 80.7 | 82.1 | 90.4 | 83.2 | 66.9 |
| baseline SFT | 83.2 | 78.7 | 94.3 | 85.8 | 73.9 | |
| intent-explicit SFT | 88.0 | 80.5 | 93.0 | 93.6 | 85.0 | |
| intent-implicit SFT | 87.1 | 78.9 | 94.0 | 92.5 | 82.9 | |
| intent-multiview SFT | 88.6 | 81.4 | 94.7 | 93.7 | 84.7 |
Table 4: SQA-CS-V2 Performance Across different base models and method variants. For each of the intent-aware method variants, the inference prompt explicitly asks the model to use intents.
Figure 2: (left) average portion of retrieved candidates used in the generated reports; (right) average citation coverage between small model variants and gemini-2.5-pro . All average scores are computed at a query level. default and verb. intent denotes the different instructions. verb. intent denotes the augmentation of intent awareness. The analysis is done on SQA-CS-V2.
| Method | Variant | Overall | Rubrics | Answer P | Citation P | Citation R |
| verbalized intent (gemini) | no | 88.1 | 82.6 | 94.1 | 93.2 | 82.4 |
| all | 89.7 | 82.6 | 94.5 | 95.7 | 86.1 | |
| citation-only | 88.6 | 81.5 | 91.7 | 95.3 | 86.2 | |
| paragraph-only | 89.1 | 82.7 | 92.9 | 95.2 | 85.6 | |
| other inference methods | CoT | 81.3 | 71.5 | 94.5 | 83.3 | 76.1 |
| ReAct | 77.6 | 67.4 | 94.6 | 76.5 | 72.0 |
Table 5: SQA-CS-V2-dev Performance results with verbalized intents and gemini-2.5-pro . We bold the best row for the Overall metric.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Base Model | Variant | Overall | Rubrics | Answer P | Citation P | Citation R |
|---|---|---|---|---|---|---|
| gemini-2.5-pro(ref) | - | 88.1 | 82.6 | 94.1 | 93.2 | 82.4 |
| qwen3-8b | no training | 80.7 | 82.1 | 90.4 | 83.2 | 66.9 |
| -verb. intent | 80.9 | 80.2 | 92.6 | 81.3 | 69.8 | |
| SFT | 83.2 | 78.7 | 94.3 | 85.8 | 73.9 | |
| -verb. intent | 84.6 | 79.0 | 94.6 | 87.6 | 76.9 | |
| intent-explicit SFT | 86.7 | 79.4 | 91.7 | 92.3 | 83.6 |
Table 7: SQA-CS-V2 Performance Results Across Base Models and Variants
| Base Model | Variant | Overall | Rubrics | Answer P | Citation P | Citation R |
|---|---|---|---|---|---|---|
| o3 | default | 85.1 | 91.4 | 96.5 | 89.4 | 63.4 |
| + pre-planning | 86.5 | 90.5 | 95.1 | 91.7 | 68.8 | |
| + intents | 86.2 | 89.6 | 95.1 | 90.5 | 69.8 | |
| gemini-2.5-pro | default | 88.1 | 82.6 | 94.1 | 93.2 | 82.4 |
| + pre-planning | 88.4 | 81.3 | 93.7 | 93.4 | 85.3 | |
| + intents | 90.7 | 80.6 | 94.0 | 97.2 | 90.9 |
Table 8: SQA-CS-V2 Performance Results with Pre-planning
| Method | DeepScholar Bench (DSB) | ||||
|---|---|---|---|---|---|
| Overall | Nug. Cov. | Org. | Cite-P | Claim Cov | |
| o3 | 46.8 | 47.0 | 61.1 | 39.1 | 40.2 |
| + intent | 43.2 | 49.1 | 64.1 | 27.2 | 34.3 |
| + intent (paragraph-only) | 49.3 | 48.0 | 66.0 | 39.1 | 44.3 |
| qwen3-8b | 56.0 | 46.0 | 59.0 | 57.0 | 62.0 |
| intent-explicit | 59.5 | 48.2 | 68.0 | 61.2 | 63.1 |
Table 9: Extended performance comparison on Deepscholar Bench. Bold indicates the best-performing row for overall metrics.
| Method | Variant | Overall | Rubrics | Answer P | Citation P | Citation R |
|---|---|---|---|---|---|---|
| verbalized intent (gemini) | free | 89.3 | 82.4 | 92.0 | 96.1 | 86.7 |
| current | 89.7 | 82.6 | 94.5 | 95.7 | 86.1 | |
| mix | 91.6 | 83.1 | 95.0 | 97.3 | 91.0 |
Table 10: SQA-CS-V2-dev Performance results with verbalized intents and gemini-2.5-pro . We compare variants of intent schema design. free denotes the use of model improvised types. current denotes the use of our schema. mix denotes the use of most frequent types in our schema and let the model has freedom on adding their own. We bold the best row for the Overall metric.
Figure 3: A screenshot of the instructions to the users. Besides the tasks shown in the figure, the users will also be provided with a step-by-step guide on the annotation tasks and the key points to remember. The instructions can be revisited during the annotation task by the users by clicking a “Click to Expand Instruction” button.
Figure 4: A screenshot of the PIT questions for the baseline system. Users are shown with the section titles and paragraph first sentences to answer the questions.
Figure 5: A screenshot of the CIT question for the baseline system. Users are shown with a specific paragraph with one highlighted citation to answer the question. The snippet will show as the users move their mouse over it.
Figure 6: A screenshot of the PIT questions for our intent-aware reading system. Users are shown with the section titles, the paragraph-level intents, and paragraph first sentences to answer the questions.
Figure 7: A screenshot of the CIT question for the intent-aware reading system. Users are shown with a specific paragraph with one highlighted citation to answer the question. The potential citation intent and the snippet will show as the users move their mouse over it.
More in this topic
Long-Form Document Generation
Long-form document generation studies methods for producing coherent, factual, well-structured books, reports, and other extended texts with AI.
See all 32 papers