SurveyGraph: Generating Academic Surveys via Methods-Entity Knowledge Graphs
Organizations: School of Artificial Intelligence, Shenzhen University, Shenzhen, China
Abstract
Most automated survey generation systems retrieve papers as flat documents, limiting their ability to capture research structure. We present SurveyGraph, a framework built around a Methods-Entity Knowledge Graph (MEKG) encoding method-level follow-on, competition, and shared-resource relations. The graph supports core-paper discovery, retrieval, hybrid outlining, and evidence-gated narrative synthesis. We evaluate retrieval, semantic edges, citation grounding, collection coverage, and generation-stage components. In a controlled seven-method equal-budget benchmark, MEKG-anchored fusion balances recall and structural density, with an independent co-citation check. In a separate archived graph-native diagnostic, graph-aware retrieval nearly doubles recall over flat lexical baselines across eight runs (p = 0.004). A semantic evidence gate achieves AUC = 0.75. Audits of 160 generated sentences find 80.6% anchored-or-partial citation support. Human adjudication of 108 stratified semantic edges finds 68.5% supported-or-partial relations (binary-support Cohen's kappa = 0.64), with improves strongest (80.6%) and extends weakest (52.8%). A SurveyBench-facing collection probe reaches mean reference coverage of 0.521 across four topics with an eight-paper budget. Same-backend Qwen diagnostics on a 30-paper/74-relation graph show reduced organization signals when hybrid outlining is removed. Concurrent-system comparisons provide bounded context rather than matched-backend end-to-end evidence. The results position MEKG as an auditable structural layer for typed diagnostics, relation-specific filtering, and controlled survey generation. Code and artifacts: https://github.com/xzzvsxd/SurveyGraph.
Figures & tables
| Cat. | Relations | Layer | Conf. | # |
|---|---|---|---|---|
| Struct. | cites, cited_by, co_cite, bib_coup | 1 | 1.0 | 4 |
| Semantic | extends, improves, applies, … | 4 | 0.6 | 8 |
| Concept. | shares_*, is_variant/instance_of | 2–3 | 0.7–0.9 | 5 |
| Temporal | precedes, succeeds | 1 | 1.0 | 2 |
| Author. | same_author, same_group | 1 | 1.0 | 2 |
| Total | 21 |
| Method | R@5 | P@5 | Str. | Den. | CoCit |
|---|---|---|---|---|---|
| Flat lexical | .252 | .132 | .558 | .134 | .089 |
| Biblio. RRF | .309 | .194 | .530 | .132 | .142 |
| TF-IDF GraphRAG | .347 | .176 | .485 | .140 | .118 |
| MEKG typed | .153 | .089 | .392 | .228 | .156 |
| Safe fusion | .261 | .134 | .528 | .179 | .147 |
| Biblio+TF-IDF | .357 | .200 | .498 | .134 | .131 |
| Variant | Sec./pg. | Tab./vis. | Rubric | MEKG | Global | Boundary condition |
|---|---|---|---|---|---|---|
| Full Qwen | 7/17 | 4/4 | 4.135 | .562 | 3.80 | Complete core pipeline artifact; external evaluator was stopped after PDF generation and is not claimed. |
| No outline critic / no post-refine | 7/18 | 4/5 | 4.110 | .619 | 3.80 | Bounded by the globally disabled text post-refiner; isolates outline-critic removal under that setting. |
| No evolutionary analyzer | 7/17 | 4/– | 4.135 | .562 | 3.50 | Post-visual diagnostic with evolutionary analyzer disabled, not a full content regeneration. |
| No hybrid outline | 5/23 | 4/4 | 3.955 | .779 | 3.20 | Same 30-paper/74-relation substrate; bounded LLM outline timeout fell back to rule-based MEKG outline. |
| Check | SurveyGraph | Flat | |
|---|---|---|---|
| Rubric score | 4.135 | 4.030 | +0.105 |
| Citation grounding | 89.1% (49/55) | 86.3% (44/51) | +2.8 pp |
| Section alignment | 75.0% (3/4) | 60.0% (3/5) | directional |
| Global reflection | 3.80 | 3.20 | +0.60 |
Appendix figures & tables58 assets
Supplementary material from the paper’s appendix.
Appendix
| Archived run | Overall | Ref. | Struct. | Content | Ethics | Media |
|---|---|---|---|---|---|---|
| Large Language Model Function Calling | 76.77 | 77.50 | 77.40 | 64.83 | 82.17 | 89.24 |
| Document Visual Question Answering | 75.19 | 78.58 | 60.77 | 72.50 | 82.77 | 85.65 |
| Memory in the Age of AI Agents | 80.94 | 81.15 | 64.67 | 79.23 | 95.50 | 90.57 |
| Embodied Question Answering in City Airspace (run 1) | 77.17 | 87.44 | 57.50 | 74.94 | 80.37 | 86.81 |
| Text-to-SQL Agent | 80.65 | 94.00 | 61.96 | 75.07 | 80.75 | 92.55 |
| Embodied Question Answering in City Airspace (run 2) | 71.60 | 82.30 | 47.85 | 66.82 | 74.94 | 90.10 |
| Archived run | WCR | CC | SC | TC | Bib. Int. |
|---|---|---|---|---|---|
| Large Language Model Function Calling | 0.25 | 1.00 | 0.50 | 0.75 | 1.00 |
| Document Visual Question Answering | 0.42 | 0.83 | 0.17 | 1.00 | 1.00 |
| Memory in the Age of AI Agents | 0.37 | 1.00 | 0.17 | 0.83 | 1.00 |
| Embodied Question Answering in City Airspace (run 1) | 0.70 | 1.00 | 0.20 | 1.00 | 0.86 |
| Text-to-SQL Agent | 0.80 | 1.00 | 0.20 | 1.00 | 1.00 |
| Embodied Question Answering in City Airspace (run 2) | 0.41 | 1.00 | 0.00 | 1.00 | 1.00 |
| Fresh run | Pages | Sections | Relations | Fast overall |
|---|---|---|---|---|
| GLM single-PDF smoke | 40 | 7 | 0 | 79.28 |
| Qwen single-PDF smoke | 36 | 7 | 0 | 74.30 |
| Run | Papers/rel. | Sec./pages | Tables/vis. | Rubric | MEKG | Artifact summary | Scope boundary |
|---|---|---|---|---|---|---|---|
| Full Qwen | 30/74 | 7/17 | 4/4 | 4.135 good | .5617 | qwen_full_artifact_summary.json | Core pipeline completed; external evaluator was stopped after PDF generation, so no external-evaluator score is claimed. |
| No outline critic / no post-refine | 30/74 | 7/18 | 4/5 | 4.110 good | .6186 | qwen_no_critic_ablation_summary.json | Bounded by globally disabled text post-refinement; isolates outline-critic removal under that setting. |
| No evolutionary analyzer | 30/74 | 7/17 | 4/– | 4.135 good | .5617 | qwen_no_evolutionary_ablation_summary.json | Post-visual-stage diagnostic; disables evolutionary analysis rather than regenerating all content from scratch. |
| No hybrid outline | 30/74 | 5/23 | 4/4 | 3.955 acceptable | .7795 | qwen_no_hybrid_ablation_summary.json | Reuses the same MEKG; after bounded LLM outline timeout, falls back to a rule-based MEKG outline with five sections. |
| Topic | Papers | Relations | Sections |
|---|---|---|---|
| LLM-based Multi-Agent | 30 | 121 | 7 |
| Evaluation of LLMs | 30 | 31 | 7 |
| RAG for LLMs | 30 | 104 | 7 |
| Multimodal LLMs | 30 | 190 | 8 |
| Instruction Tuning | 30 | 202 | 7 |
| System | Count | Pages mean | Text chars mean | Citation mean | Section mean | Completeness signal |
|---|---|---|---|---|---|---|
| SurveyGraph-GLM-fresh | 1 | 40.00 | 169009 | 94.00 | 31.00 | TeX/Bib present |
| SurveyGraph-Qwen-fresh | 1 | 36.00 | 149616 | 19.00 | 31.00 | TeX/Bib present |
| AutoSurvey | 3 | 31.00 | 89238 | 221.33 | 0.33 | PDF-only released artifacts |
| SurveyForge | 10 | 27.80 | 186486 | 406.50 | 39.50 | PDF-only released artifacts |
| SurveyX | 8 | 23.00 | 86917 | 273.12 | 32.88 | PDF-only released artifacts |
| ARISE | 27 | 29.78 | 204664 | 453.30 | 60.19 | 100% TeX/Bib/review; 96.3% multi-round |
| System | Orig. | Dedup | Page | Cit. | Sec. |
|---|---|---|---|---|---|
| ARISE | 27 | 25 | -0.18 | -16.46 | -0.31 |
| SurveyForge | 10 | 9 | +0.64 | +18.39 | +0.50 |
| SurveyX | 8 | 7 | -0.43 | -8.55 | -0.73 |
| System | N | TOC | TOC=Sec. | Ref. |
|---|---|---|---|---|
| ARISE | 27 | 27 | 100.0% | 100.0% |
| AutoSurvey | 3 | 0 | 0.0% | 0.0% |
| SurveyForge | 10 | 10 | 100.0% | 100.0% |
| SurveyGraph-GLM-fresh | 1 | 1 | 100.0% | 100.0% |
| SurveyGraph-Qwen-fresh | 1 | 1 | 100.0% | 100.0% |
| SurveyX | 8 | 8 | 100.0% | 100.0% |
| System | N | Cit./1k | Head./10k | Fig+Tbl./10k | Editable |
|---|---|---|---|---|---|
| ARISE | 25 | 14.95 | 22.10 | 9.69 | 100.0% |
| AutoSurvey | 3 | 16.68 | 0.00 | 0.00 | 0.0% |
| SurveyForge | 9 | 14.75 | 14.28 | 0.00 | 0.0% |
| SurveyGraph-fresh | 2 | 2.56 | 14.81 | 13.12 | 100.0% |
| SurveyX | 7 | 22.28 | 28.14 | 13.45 | 0.0% |
| System | Setup | Ext-core | Outline | Runtime / note |
|---|---|---|---|---|
| SGraph | archived DocVQA | 0.083 @20 | 7 sec / 34 n. | 133.4 min; 24 refs |
| SciSage | live DocVQA outline-only | – | 3 sec / 9 n. | 117.2 s; gate passed |
| Topic | SGraph | SciSage | Role recall | Missing roles |
| DocVQA | 7 sec / 34 nodes | 3 sec / 9 nodes | 0.571 | motivation, evaluation, conclusion |
| Text-to-SQL | 7 sec / 40 nodes | 3 sec / 9 nodes | 0.429 | task background, motivation, other, conclusion |
| LLM function calling | 8 sec / 34 nodes | 3 sec / 9 nodes | 0.429 | task background, motivation, other, conclusion |
| Memory in AI agents | 7 sec / 38 nodes | 2 sec / 4 nodes | 0.286 | task background, motivation, other, challenges, conclusion |
| Mean | 7.25 sec / 36.5 nodes | 2.75 sec / 7.75 nodes | 0.429 | motivation and conclusion recur across all 4 |
| Topic / metric | Flat@5 | Simple@5 | Full@5 |
| LLM function calling recall | 0.000 | 0.077 | 0.077 |
| DocVQA ext-proxy recall | 0.125 | 0.125 | 0.167 |
| Memory in AI agents recall | 0.214 | 0.214 | 0.214 |
| Text-to-SQL ext-proxy recall | 0.200 | 0.200 | 0.200 |
| Mean ext-proxy recall | 0.135 | 0.154 | 0.164 |
| Mean ext-proxy precision | 0.087 | 0.090 | 0.105 |
| Method | R@5 | P@5 | MEKG dens. | Act. dens. |
|---|---|---|---|---|
| Flat lexical | 0.252 | 0.132 | 0.134 | 0.000 |
| TF-IDF kNN graph | 0.347 | 0.176 | 0.140 | 0.167 |
| Structural graph | 0.135 | 0.075 | 0.217 | 0.217 |
| Full typed MEKG | 0.153 | 0.089 | 0.228 | 0.228 |
| Safe fusion full | 0.261 | 0.134 | 0.179 | 0.179 |
| k | Min sim. | R@5 | P@5 | Active rels |
|---|---|---|---|---|
| 2 | 0.05 | 0.300 | 0.139 | 245.4 |
| 3 | 0.05 | 0.291 | 0.134 | 358.6 |
| 5 | 0.05 | 0.347 | 0.176 | 569.7 |
| 8 | 0.05 | 0.342 | 0.165 | 859.1 |
| Sweep min | – | 0.274 | 0.133 | 27.0 |
| Sweep max | – | 0.347 | 0.176 | 914.3 |
| Method | R@5 | P@5 | Quality | Cites |
|---|---|---|---|---|
| Flat lexical | 0.252 | 0.132 | 0.369 | 6.6 |
| OA quality only | 0.210 | 0.457 | 0.670 | 115.5 |
| Top-20 rerank | 0.280 | 0.156 | 0.393 | 11.7 |
| Flat + quality | 0.280 | 0.150 | 0.384 | 11.1 |
| Lexical–biblio RRF | 0.309 | 0.194 | 0.452 | 17.3 |
| Method | R@5 | P@5 | Align | Dens. |
|---|---|---|---|---|
| Flat lexical | 0.252 | 0.132 | 0.558 | 0.134 |
| Bibliometric RRF | 0.309 | 0.194 | 0.530 | 0.132 |
| TF-IDF GraphRAG-lite | 0.347 | 0.176 | 0.485 | 0.140 |
| SurveyGraph MEKG | 0.153 | 0.089 | 0.392 | 0.228 |
| Safe fusion MEKG | 0.261 | 0.134 | 0.528 | 0.179 |
| Biblio + TF-IDF fusion | 0.357 | 0.200 | 0.498 | 0.134 |
| System | Status | Decisive interpretation |
|---|---|---|
| SurveyForge | Partial | Released artifacts analyzable; live rerun still blocked by missing local database/ and embedding assets |
| SurveyX | Analyzable | Released artifacts available, but no same-environment live rerun claimed |
| AutoSurvey | Analyzable | Released artifacts available, but no same-environment live rerun claimed |
| SciSage | Partial | Same-topic outline-only live baseline succeeds; full-paper parity still unverified |
| ARISE | Partial | Released artifacts analyzable; code path still expects extra provider/runtime setup for live rerun |
| Source | N | Cov. | Match |
|---|---|---|---|
| SurveyForge released refs | 10 | 0.465 | 50.0 |
| Human survey refs | 10 | 0.629 | 148.8 |
| SurveyGraph archive overlaps | 15 | 0.078 | 3.9 |
| Topic | Budget | Enhanced | Exact-topic | |
|---|---|---|---|---|
| LLM-based Multi-Agent | 8 | 0.333 | 0.000 | +0.333 |
| Evaluation of LLMs | 8 | 0.250 | 0.120 | +0.130 |
| RAG for LLMs | 8 | 1.000 | 0.188 | +0.812 |
| Multimodal LLMs | 8 | 0.500 | 0.357 | +0.143 |
| Mean | 8 | 0.521 | 0.166 | +0.355 |
| Metric | Flat | Graph aware | Graph better | |
|---|---|---|---|---|
| Core recall | 0.216 | 0.373 | +0.157 | 8/8 |
| Comm. cov. | 0.758 | 0.863 | +0.104 | 5/8 |
| Avg PageRank | 0.481 | 0.630 | +0.149 | 8/8 |
| Relation dens. | 0.162 | 0.259 | +0.098 | 8/8 |
| Unique papers | 21.38 | 21.50 | +0.13 | 4/8 |
| Variant | Core rec. | Avg PR | Rel. dens. |
|---|---|---|---|
| Full graph-aware | 0.373 | 0.630 | 0.259 |
| No overlap | 0.373 | 0.629 | 0.259 |
| No centrality | 0.364 | 0.621 | 0.254 |
| No expansion | 0.219 | 0.486 | 0.161 |
| Flat lexical | 0.216 | 0.481 | 0.162 |
| Budget | Flat rec. | Graph rec. | Wins | |
|---|---|---|---|---|
| top-3 | 0.113 | 0.288 | +0.176 | 7/8 |
| top-5 | 0.216 | 0.373 | +0.157 | 8/8 |
| top-7 | 0.288 | 0.505 | +0.217 | 8/8 |
| Thr. | Keep rel. | Graph rec. | Rec. | Graph dens. |
|---|---|---|---|---|
| 0.5 | 99.0% | 0.373 | +0.157 | 0.259 |
| 0.7 | 80.7% | 0.336 | +0.120 | 0.248 |
| 0.9 | 39.1% | 0.342 | +0.126 | 0.245 |
| Metric | Flat | 1-hop | 2-hop |
|---|---|---|---|
| Core-paper recall | 0.216 | 0.373 | 0.388 |
| Community coverage | 0.758 | 0.863 | 0.817 |
| Average PageRank | 0.481 | 0.630 | 0.659 |
| Relation density | 0.162 | 0.259 | 0.309 |
| Unique papers | 21.38 | 21.50 | 17.00 |
| Expanded nodes | – | 32.02 | 73.78 |
| Variant | Core rec. | Comm. cov. | Avg PR | Rel. dens. |
|---|---|---|---|---|
| Full graph | 0.373 | 0.863 | 0.630 | 0.259 |
| No support | 0.389 | 0.912 | 0.633 | 0.246 |
| Struct.-only | 0.389 | 0.863 | 0.621 | 0.256 |
| Sem.-only | 0.291 | 0.829 | 0.545 | 0.115 |
| Variant vs. default top- | Mean overlap | Median | Range |
| PageRank-heavy | 9.50 | 9.50 | 9–10 |
| Citation-heavy | 9.75 | 10.00 | 9–10 |
| Recency-heavy | 9.75 | 10.00 | 9–10 |
| Degree-heavy | 9.00 | 9.00 | 8–10 |
| Default core-set size | 16.12 | – | – |
| Elbow-selected size | 17.38 | – | – |
| Calibration | Mean | Median | Note |
|---|---|---|---|
| Archived default score | 0.162 | 0.077 | local citation field dormant |
| OpenAlex default weights | 0.189 | 0.125 | same score form, citation proxy repaired |
| LOO tuned | 0.289 | 0.308 | coarse PR/cit/rec/deg grid |
| Run group | Papers | Relations | Rel./paper | Sem. share |
|---|---|---|---|---|
| 50-paper runs | 50.0 | 2314.67 | 46.29 | 10.1% |
| 200-paper runs | 199.2 | 2243.20 | 11.26 | 12.4% |
| In both groups, supports + same_author + shares_architecture account for about 69% of all edges, and structural/meta relation families exceed 87% overall. | ||||
| Relation | Count | Temp. | Cue | Conf. |
|---|---|---|---|---|
| extends | 1276 | 99.8% | 24.4% | 0.673 |
| improves | 366 | 99.7% | 65.8% | 0.675 |
| applies | 449 | 98.0% | 24.7% | 0.620 |
| Control | Edge | Control | Better | |
|---|---|---|---|---|
| Matched non-edge | 0.0748 | 0.0733 | +0.0015 | 1109/2091 |
| Easy non-edge | 0.0748 | 0.0249 | +0.0500 | 2079/2091 |
| Min conf. | Count | Temp. | Cue | Avg conf. |
|---|---|---|---|---|
| 0.5 | 2035 | 99.6% | 32.0% | 0.668 |
| 0.7 | 1078 | 99.7% | 33.2% | 0.754 |
| 0.9 | 52 | 100.0% | 30.8% | 0.907 |
| Relation | Supported | Partial | Unsupported | Total |
|---|---|---|---|---|
| extends | 5 | 6 | 13 | 24 |
| improves | 3 | 14 | 7 | 24 |
| applies | 4 | 12 | 8 | 24 |
| Overall | 12 | 32 | 28 | 72 |
| Rel. | Label | Source | Target | Failure / support mode | Local source snippet |
|---|---|---|---|---|---|
| extends | supported | Adapting While Learning: Grounding LLMs for Scientific Tool Use | Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculum | well-supported protocol case | “Tool Usage Adaptation (TUA)” categorizes problems by model accuracy and trains on adapted tool-use settings. |
| extends | unsupported | Adapting While Learning: Grounding LLMs for Scientific Tool Use | API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs | target-role mismatch | The source discusses tool-use adaptation, but the target is a benchmark resource rather than a precursor method. |
| improves | supported | LaTr: Layout-Aware Transformer for Scene-Text VQA | Spatially Aware Multimodal Transformers for TextVQA | well-supported protocol case | The source explicitly proposes a layout-aware multimodal architecture for scene-text VQA. |
| improves | unsupported | LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding | DocVQA: A Dataset for VQA on Document Images | target-role mismatch | The source describes pre-training objectives, but the target is a dataset paper, so the relation is not a direct method refinement. |
| applies | supported | Advancing SLM Tool-Use Capability using Reinforcement Learning | MetaTool Benchmark for Large Language Models | well-supported protocol case | The source discusses reinforcement learning for tool-augmented agents in a setting tied to tool-use evaluation. |
| applies | partial | CheMatAgent: Enhancing LLMs for Chemistry and Materials | Toolformer: Language Models Can Teach Themselves to Use Tools | lexical-overlap not better than matched control | The source is a domain application of tool-use ideas, but the local evidence is not enough to establish a direct transfer path. |
| Topic | Cand. | Samp. | Anch. | Part. | Weak |
|---|---|---|---|---|---|
| LLM function calling | 242 | 20 | 11 | 7 | 2 |
| Doc. visual QA | 232 | 20 | 13 | 4 | 3 |
| Memory in AI agents | 288 | 20 | 14 | 2 | 4 |
| Embodied QA run 1 | 175 | 20 | 11 | 3 | 6 |
| Text-to-SQL agent | 202 | 20 | 5 | 8 | 7 |
| Embodied QA run 2 | 217 | 20 | 13 | 6 | 1 |
| Lexical label | N | Cited sim. | Neg. sim. | Cited neg. |
|---|---|---|---|---|
| Anchored | 93 | 0.694 | 0.689 | 0.527 |
| Partial anchor | 36 | 0.680 | 0.686 | 0.444 |
| Weak/missing | 31 | 0.651 | 0.672 | 0.226 |
| All | 160 | 0.682 | 0.685 | 0.450 |
| Lexical label | N | Cited ev. | Neg. ev. | Cited neg. |
|---|---|---|---|---|
| Anchored | 92 | 0.220 | 0.160 | 0.717 |
| Partial anchor | 36 | 0.199 | 0.158 | 0.639 |
| Weak/missing | 31 | 0.138 | 0.138 | 0.484 |
| All | 159 | 0.199 | 0.155 | 0.654 |
| Topic | Sec. | Cited | Weak | Prec. | Cites |
|---|---|---|---|---|---|
| LLM function calling | 6 | 6 | 1 | 0.315 | 20.7 |
| Doc. visual QA | 5 | 5 | 0 | 0.272 | 24.4 |
| Memory in AI agents | 5 | 5 | 1 | 0.323 | 25.2 |
| Embodied QA run 1 | 5 | 5 | 1 | 0.290 | 15.8 |
| Text-to-SQL agent | 5 | 5 | 1 | 0.197 | 19.8 |
| Embodied QA run 2 | 5 | 5 | 0 | 0.409 | 22.6 |
| Metric | Base@10 | Base@20 | Recency@20 | PR@20 |
|---|---|---|---|---|
| Mean recall | 0.058 | 0.162 | 0.179 | 0.173 |
| Runs used | 7 | |||
| Mean target size | 12.43 | |||
| Median target size | 13.00 | |||
| Variant | Recall@5 | Prec.@5 | Rel. dens. | Active rel. |
|---|---|---|---|---|
| Full graph | 0.153 | 0.088 | 0.230 | 2265.1 |
| No supports | 0.221 | 0.108 | 0.215 | 1690.3 |
| Structural-only | 0.135 | 0.074 | 0.221 | 2014.0 |
| Semantic-only | 0.266 | 0.139 | 0.091 | 251.1 |
| Mode | R@5 | P@5 | Dens. | Drop |
|---|---|---|---|---|
| Flat lexical | 0.252 | 0.132 | 0.134 | 0.0 |
| Structural-only | 0.135 | 0.074 | 0.221 | 0.0 |
| Semantic-only | 0.266 | 0.139 | 0.091 | 0.0 |
| No supports | 0.221 | 0.108 | 0.215 | 0.0 |
| Full graph | 0.153 | 0.088 | 0.230 | 0.0 |
| Supp./partial gate | 0.151 | 0.089 | 0.224 | 102.6 |
| Mode | Rec. | Prec. | Dens. | Drop |
|---|---|---|---|---|
| Full | 0.164 | 0.105 | 0.165 | 0.0 |
| Supp./partial gate | 0.182 | 0.117 | 0.166 | 103.0 |
| Supp.-only gate | 0.172 | 0.103 | 0.166 | 207.2 |
| Structural | 0.154 | 0.090 | 0.163 | – |
| Relation | Total | Supp. | Partial | Unsupp. | S/P |
|---|---|---|---|---|---|
| extends | 1276 | 159 | 510 | 607 | 0.524 |
| impr. | 366 | 118 | 177 | 71 | 0.806 |
| applies | 449 | 61 | 204 | 184 | 0.590 |
| All | 2091 | 338 | 891 | 862 | 0.588 |
| Relation | N | Supp. | Partial | Unsupp. | S/P |
|---|---|---|---|---|---|
| extends | 36 | 6 | 13 | 17 | 0.528 |
| improves | 36 | 20 | 9 | 7 | 0.806 |
| applies | 36 | 18 | 8 | 10 | 0.722 |
| All | 108 | 44 | 30 | 34 | 0.685 |
| Ranker | Base R | Graph R | Base D | Graph D |
|---|---|---|---|---|
| Existing lexical | 0.252 | 0.153 | 0.134 | 0.230 |
| TF-IDF word | 0.435 | 0.121 | 0.134 | 0.267 |
| TF-IDF char | 0.459 | 0.120 | 0.133 | 0.292 |
| MiniLM | 0.206 | 0.129 | 0.138 | 0.313 |
| BGE-small | 0.247 | 0.160 | 0.133 | 0.293 |
| Base ranker | Safe mode | R | Dens. | Base R |
|---|---|---|---|---|
| BGE-small | preserve 1 | 0.245 | 0.258 | 0.247 |
| MiniLM | preserve 4 | 0.212 | 0.173 | 0.206 |
| Existing lexical | preserve 3 | 0.261 | 0.181 | 0.252 |
| TF-IDF char | base only | 0.459 | 0.133 | 0.459 |
| TF-IDF word | base only | 0.435 | 0.134 | 0.435 |
| Metric | Original | Pruned | |
|---|---|---|---|
| Core-paper recall | 0.373 | 0.395 | +0.022 |
| Relation density | 0.259 | 0.263 | +0.003 |
| Pruned semantic-edge share | – | 0.100 | – |
| Runs used | 8 | ||
| Run / signal | Runtime | Token trace | Cost / note |
|---|---|---|---|
| Archived full mean | 118.91 min | – | mixed historical backends |
| GLM smoke | 55.22 min | 525,681 in / 124,106 out | cost 0.1533 |
| Qwen smoke | 50.47 min | 459,211 in / 137,744 out | cost 0.1515 |
| SciSage partial | 132.07 s | – | live outline-only baseline |
| Group | Runs | Min | Min/100 | Tok./100 |
|---|---|---|---|---|
| 50-paper | 3 | 64.8 | 129.6 | 5.02M |
| 196–200-paper | 5 | 151.4 | 76.0 | 3.31M |
| All | 8 | 118.9 | 96.1 | 3.95M |
| Variant | R@5 | P@5 | Dens. | Sec. |
|---|---|---|---|---|
| Flat lexical | 0.252 | 0.132 | 0.134 | 0.166 |
| Structural-only | 0.135 | 0.074 | 0.221 | 0.177 |
| Full graph | 0.153 | 0.088 | 0.230 | 0.183 |
| Supp./partial gate | 0.151 | 0.089 | 0.224 | 0.177 |
| Supported-only gate | 0.145 | 0.081 | 0.224 | 0.179 |
| Replay stage | Mean sec. | Local share |
| Graph construction + PageRank | 0.066 | 44.7% |
| Semantic evidence gate | 0.010 | 6.5% |
| Graph retrieval replay | 0.072 | 48.7% |
| Local diagnostic replay total | 0.148 | 100.0% |
| Archived generation/reflection | 6601.2 | – |
| Component | Current default / note |
|---|---|
| Core-paper score | ; weights are fixed hand-tuned defaults, not learned parameters. The stored artifact records lack populated citation fields, but the OpenAlex title-matched replay above provides an external citation proxy for the archived analysis; production runs should preserve this enrichment directly. |
| Embeddings | Title + abstract text; HuggingFace-based sentence embeddings with L2 normalization; the repository falls back to a local sentence-transformer-style model when the preferred remote embedding is unavailable. |
| Clustering | HDBSCAN with and , adaptively raised as sample count grows; fallback to KMeans when HDBSCAN is unavailable ( McInnes et al., 2017 ; Reimers and Gurevych, 2019 ) . |
| KMeans fallback | , , . |
| Graph-RAG ranker | Lexical section-to-paper score + category/subcategory overlap + PageRank + 1-hop relation expansion from the top lexical seeds; semantic relations receive a larger expansion bonus than structural ones. |
| SC definition | Fraction of graph communities whose names match at least one section title in the outline; section headings are matched as plain strings, which explains brittleness. |
| Stage | Model | Context | Tokens/run |
|---|---|---|---|
| Paper collection | – | – | 0 |
| Relation inference | GLM-4-Flash | 128K | 1.82M |
| Core-paper scoring | – (rule-based) | – | 0 |
| Outline generation | GLM-4-Flash | 128K | 0.31M |
| Section retrieval | – (local) | – | 0 |
| Narrative synthesis | GLM-4-Flash | 128K | 0.94M |
| Topic | Budget | Enhanced | Exact-topic | |
|---|---|---|---|---|
| LLM-based Multi-Agent | 8 | .333 | .000 | +.333 |
| Evaluation of LLMs | 8 | .250 | .120 | +.130 |
| RAG for LLMs | 8 | 1.000 | .188 | +.812 |
| Multimodal LLMs | 8 | .500 | .357 | +.143 |
| Mean | 8 | .521 | .166 | +.355 |
| System | Available evidence | Main limitation | Use in this paper |
|---|---|---|---|
| MVSS | Published and released structure-first survey-generation results | Different backend and evaluation setting; no live same-backend rerun | Closest multi-view structural baseline; motivates typed semantic diagnostics |
| StructSurvey | Published and released structured retrieval results | Retrieval and generation setup differs from SurveyGraph | Complementary retrieval design; could benefit from typed MEKG edges |
| SurveyForge | Local SurveyBench reference lists and released artifacts | Reference-list probe is not full SurveyGraph generation | Benchmark calibration and collection-stage comparison |
| SciSage | Four same-topic outline-only live partial runs | Outline-only, not full-paper parity | Structural sanity check for outline richness |
Knowledge Graphs
Knowledge graphs represent entities and relationships as structured, queryable networks for integrating data and supporting reasoning.
See all 232 papers