GRAFT: Graph-Distilled Generative Retrieval for Facet-Aware Scientific Literature Exploration
Organizations: Department of Informatics King’s College London London, England, UK
Abstract
Scientific papers may relate by problem, method, result, or contribution, but document-level retrievers collapse these into a single similarity score without saying why they are related. Citation- and similarity-based retrieval alone also confines search to the neighbourhood of what is already known, whereas generative retrieval generates document identifiers directly, enabling the exploratory retrieval that scientific discovery depends on. We connect papers in a graph whose edges are typed by these four facets, derived from facet items and citation signals, and distil it into a generative retriever whose identifiers are the papers' own facet text. Two graph properties do not survive naive distillation. First, because every training pair is an edge, naive enumeration indexes just 84% of the corpus. Coverage-aware distillation makes every paper learnable through a reverse-neighbour fallback, a minimum-coverage threshold, and edge-importance weighting. Second, constrained decoding guarantees that every generated identifier is a valid paper, but not that the graph connects it to the query. Graph-weighted reciprocal rank fusion scales each candidate's rank term by its query-candidate edge weight, dropping unsupported ones. On LitWeave, our constructed corpus of 11,359 NLP papers, Graft recovers 91% of its graph teacher's Recall@20 with no nearest-neighbour index or encoder at inference, and outperforms the graph teacher on query papers outside the corpus. It reproduces the graph's own facet labels at 0.922 precision, so every returned paper arrives labelled with the facet that surfaced it rather than an opaque score.
Figures & tables
| Method | R@20 | R@50 |
| Dense-graph | 0.357 | 0.560 |
| MiniLM (title+abstract) | 0.295 | 0.440 |
| Dense-only (facet) | 0.274 | 0.440 |
| BM25 (title+facets) | 0.274 | 0.433 |
| SPECTER2 (title+abstract) | 0.220 | 0.341 |
| S2 Recommendations API | 0.124 | 0.129 |
| Method | R@5 | R@10 | R@20 | R@50 |
|---|---|---|---|---|
| Dense-graph | 0.218 | 0.360 | 0.501 | 0.657 |
| BM25 | 0.164 | 0.230 | 0.309 | 0.434 |
| MiniLM | 0.152 | 0.225 | 0.293 | 0.408 |
| SPECTER2 | 0.121 | 0.170 | 0.225 | 0.314 |
| Graft | 0.367 | 0.453 | 0.548 | 0.709 |
| Fusion | R@20 | R@50 |
|---|---|---|
| Best single facet (problems) | 0.115 | 0.115 |
| Uniform RRF (no graph) | 0.284 | 0.447 |
| MINDER combination | 0.310 | 0.464 |
| Graph-RRF (all four facets) | 0.322 | 0.447 |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Facet | Mean | |
|---|---|---|
| Problems | 4.68 | 47/50 (94%) |
| Methods | 4.83 | 49/50 (98%) |
| Results | 4.77 | 47/50 (94%) |
| Contributions | 4.85 | 50/50 (100%) |
| All | 4.78 | 193/200 (96.5%) |
| Year | Train | Dev | Test |
|---|---|---|---|
| 2019 | 1,479 | ||
| 2020 | 1,727 | ||
| 2021 | 1,839 | ||
| 2022 | 1,830 | ||
| 2023 | 1,543 | ||
| 2024 | 1,582 |
| Venue | Venue | ||
|---|---|---|---|
| 2,891 | ACL | 137 | TACL |
| 2,842 | EMNLP | 135 | AACL-IJCNLP |
| 1,273 | NAACL | 111 | CoNLL |
| 438 | EACL | 102 | WMT |
| 398 | LREC | 91 | INLG |
| 335 | Findings of ACL | 78 | SIGDIAL |
| R@20 | R@50 | ||
|---|---|---|---|
| 0.00 | 1.00 | 0.323 | 0.524 |
| 0.25 | 0.75 | 0.339 | 0.552 |
| 0.50 | 0.50 | 0.356 | 0.568 |
| 0.75 | 0.25 | 0.363 | 0.569 |
| 1.00 | 0.00 | 0.273 | 0.437 |
| Coverage | |
|---|---|
| 10 | 52.9% |
| 20 | 84.0% |
| 50 | 98.5% |
| Walk depth | R@20 | R@50 |
|---|---|---|
| 1-hop | 0.318 | 0.465 |
| 2-hop (ours) | 0.357 | 0.560 |
| 3-hop | 0.357 | 0.565 |
| Facet | Stored facet item | Identifier (first 12 tokens) |
|---|---|---|
| Problems | Lack of systematic, comparable evaluation of post-hoc explainability techniques: existing studies focus on narrow setups (single method, task, or architecture) making it hard to choose suitable explainers for a given model and task. | lack of systematic, comparable evaluation of post-hoc explainability |
| Methods | Empirically test these diagnostic measures across three text-classification datasets (e-SNLI, TSE, IMDB) and three model architectures (CNN, LSTM, Transformer), including multiple random seeds and randomly initialized models to assess consistency. | empirically test these diagnostic measures across three text-classification datasets |
| Results | Rationale and dataset consistency measures show low-to-moderate correlations overall; no single explainer is universally best for rationale consistency, indicating that faithfulness, human-alignment, confidence indication, and consistency are separable properties. | rationale and dataset consistency measures show low-to-moderate correlations |
| Contributions | A comprehensive, practical suite of diagnostic properties for explainability methods together with explicit, automated measures (MAP, AUC-TP, SD+LR MAE, Spearman correlations) that are applicable across tasks and architectures. | a comprehensive, practical suite of diagnostic properties for explainability methods |
| Facet | Nodes | Leaves |
|---|---|---|
| Problems | 396,133 | 41,997 |
| Methods | 481,796 | 47,490 |
| Results | 469,724 | 46,827 |
| Contributions | 447,281 | 47,974 |
| Total | 1,794,934 | 184,288 |
| Facet | Query retrieved paper | Rank | Shared, in different words |
|---|---|---|---|
| problems | Higher-order comparisons of sentence encoder representations Designing and interpreting probes with control tasks | 92 | Whether an interpretability method measures the representation or the instrument: via representational geometry, and via probe expressivity. |
| methods | Findings from the Bambara–French machine translation competition AmericasNLI | 84 | Building a parallel resource for a genuinely low-resource language and adapting a multilingual pretrained model to it. |
| results | Semantic representation for dialogue modeling Entity, relation and event extraction with contextualized span representations | 74 | An explicit graph over a contextual encoder recovers long-range structure the encoder alone misses: across utterances, and across sentences. |
| contributions | A challenge dataset for conversational stance detection Build it break it fix it for dialogue safety | 80 | A multi-turn dialogue dataset built to capture what single-turn collection misses, for stance and for safety. |
| DocID scheme | R@20 |
|---|---|
| Numeric codebook | 0.121 |
| C2T cluster keywords | 0.260 |
| Facet-item bullet | 0.296 |
| R@20 | |
|---|---|
| 1 | 0.299 |
| 2 | 0.304 |
| 3 | 0.322 |
| 4 | 0.294 |
| Fusion | R@20 | |
|---|---|---|
| Graph-RRF (all four facets) | 0.322 | |
| problems | 0.303 | -0.020 |
| methods | 0.321 | -0.002 |
| results | 0.317 | -0.005 |
| contributions | 0.316 | -0.006 |
| Query facet | Top retrieved related paper |
|---|---|
| Problems | Flexible Text Generation for Counterfactual Fairness Probing |
| Methods | YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering |
| Results | Optimal and Efficient Text Counterfactuals Using Graph Neural Networks |
| Contributions | DISCO: Distilling Counterfactuals with Large Language Models |
| Variant | R@20 | R@50 |
|---|---|---|
| Uniform sampling (base) | 0.296 | 0.430 |
| CE reweighting by edge score | 0.303 | 0.436 |
| Graph-weighted sampling (ours) | 0.307 | 0.442 |
| Exhaustive Cartesian pairing | 0.242 | 0.328 |
| Hard negatives, top-50 | 0.211 | 0.277 |
| Hard negatives, top-100 | 0.201 | 0.275 |
| Model | R@20 | R@50 |
|---|---|---|
| LoRA | 0.169 | 0.232 |
| LoRA | 0.205 | 0.282 |
| LoRA | 0.217 | 0.293 |
| LoRA | 0.230 | 0.314 |
| LoRA | 0.237 | 0.337 |
| Full fine-tuning | 0.322 | 0.447 |
| Base model | R@20 | R@50 |
|---|---|---|
| Llama-3.2 1B | 0.322 | 0.447 |
| Qwen3 0.6B | 0.248 | 0.332 |
| SmolLM2 360M | 0.089 | 0.130 |
| SmolLM2 135M | 0.084 | 0.112 |
| Package | Version | Licence |
|---|---|---|
| accelerate | 1.12.0 | Apache-2.0 |
| beartype | 0.22.9 | MIT |
| bitsandbytes | 0.49.2 | MIT |
| fastapi | 0.135.1 | MIT |
| leidenalg | 0.12.0 | GPL-3.0 |
| matplotlib | 3.10.7 | PSF |
Information Retrieval
Information Retrieval studies how to find, rank, and organize relevant information in large collections of data.
See all 459 papers