InsightRAG: Aggregating Distributed Scholarly Judgments for Scientific Survey Generation

Yuquan Lin; Wenshuo Zhang; Yilin Ye; Shuchang Xu; Zhao Chen; Weiqi Wang; Huamin Qu

cs.AIOct 20, 2026

InsightRAG: Aggregating Distributed Scholarly Judgments for Scientific Survey Generation

Authors: Yuquan Lin, Wenshuo Zhang, Yilin Ye, Shuchang Xu, Zhao Chen, Weiqi Wang, Huamin Qu

Organizations: The Hong Kong University of Science and Technology · The Hong Kong University of Science and Technology (Guangzhou)

Abstract

The rapid growth of scientific literature makes survey writing increasingly challenging, motivating automatic survey generation. Yet, existing systems must search an open space of papers and an open space of concepts with no trace of how the community itself has navigated them, leading to incomplete scope, weak taxonomies, and unreliable citations. Our core insight is that author-written Related Work sections contain the human judgments needed to address both limitations: authors curate the citations relevant to their contributions and explain how the cited works relate, yielding paths for scope reconstruction and relational evidence for organization. We propose InsightRAG. Its insight-guided retrieval follows author-supported citation relations to reconstruct a verified research scope and structure the resulting relational evidence. It aggregates these judgments under a minimum-conflict objective into a taxonomy that reflects recurring author-written perspectives in the retrieved literature, guides generation, and bounds each section's evidence. Results on ten topics show gains in scope reconstruction, taxonomy quality, citation support, and reference reliability.

Figures & tables

Figure 1: Motivation of InsightRAG. Author-written Related Work sections record how different authors characterize prior work from their own perspectives; aggregating these local views grounds the survey in recurring perspectives within the explored literature.

Figure 2: Overview of InsightRAG. Insight-guided retrieval performs scope-verified forward and backward expansion to construct a verified paper set, a relation-bearing research-scope graph, and per-source Related Work trees. Minimum-conflict taxonomy construction produces and refines the taxonomy, assigns papers to leaves, and validates the resulting structure. Evidence-bounded survey generation then converts taxonomy nodes into writing units with local evidence packages and constrained citation identifiers.

Setting	Method	#Papers	Coverage	Correctness	F1
Online	BM25	1,995	0.1058	0.1014	0.1036
Online	Dense Retrieval	993	0.1027	0.1717	0.1285
Offline	BM25	500	0.0675	0.1578	0.0946
Offline	Dense Retrieval	500	0.0765	0.1900	0.1091
ASG	SurveyGen	246	0.0951	0.4911	0.1593
ASG	SurveyForge	1,063	0.1050	0.1401	0.1200

Table 1: Research-scope reconstruction across retrieval settings and ASG baselines. #Papers totals retrieved papers over ten topics; bold and underlined values mark the best and second-best results.

	Structure			Outline Quality					Taxonomy Alignment
Method	HSR $\uparrow$	CEDS $\uparrow$	Node Ratio $\rightarrow 1$	TU $\uparrow$	SB $\uparrow$	HC $\uparrow$	LO $\uparrow$	Avg. $\uparrow$	RWTA $\uparrow$	PTA $\uparrow$
Vanilla LLM	0.730126	0.448902	1.346384	3.4350	3.2550	3.0500	3.2150	3.2388	0.781515	0.753478
SurveyGen	0.754168	0.526661	1.176255	3.3000	3.5500	3.4000	3.4250	3.4188	0.743593	0.446429
SurveyForge	0.712712	0.478629	1.309190	2.7750	2.6250	2.4750	2.5250	2.6000	0.705402	0.129679
AutoSurvey	0.738040	0.374903	2.290563	2.6850	2.5200	2.4050	2.3750	2.4962	0.731652	0.160142
Deep Research	0.688150	0.382977	0.348203	3.5000	2.8750	3.3750	3.1250	3.2188	0.751199	0.403720

Table 2: Taxonomy evaluation. TU, SB, HC, and LO denote Topic Uniqueness, Structural Balance, Hierarchical Clarity, and Logical Organization. RWTA denotes RW–Taxonomy Alignment, and PTA denotes Paper–Taxonomy Assignment Accuracy. Bold values indicate the best result, and underlined values indicate the second-best result.

Panel of Figure 3 — Figure 3: Research-scope profile distributions for papers retrieved by each method and papers in the golden scope: (a) Venue H-Index and (b) Year Gap. Each violin aggregates papers across all topics; the inner box indicates the median and interquartile range.

Method	SS $\uparrow$	KPR $\uparrow$	Cit. Recall $\uparrow$	Cit. Precision $\uparrow$	#References	VRR $\uparrow$
	Content Correspondence		Citation Support		Reference Reliability
Vanilla LLM	0.705198	53	0.441418	0.406859	254	0.550306
SurveyGen	0.675570	52	0.475770	0.355161	249	0.972690
SurveyForge	0.626670	55	0.178637	0.122507	1,063	0.979134
AutoSurvey	0.619481	63	0.275886	0.255565	2,102	0.950845
Deep Research	0.704412	54	0.301514	0.264325	134	0.898381

Table 3: Content and citation evaluation of generated surveys. SS, KPR, and VRR denote Semantic Similarity, Key-Point Recall, and Verifiable Reference Rate. Bold marks the best result; underlining marks the second-best.

	Structure			Outline Quality					Taxonomy Alignment
Variant	HSR $\uparrow$	CEDS $\uparrow$	Node Ratio $\rightarrow 1$	TU $\uparrow$	SB $\uparrow$	HC $\uparrow$	LO $\uparrow$	Avg. $\uparrow$	RWTA $\uparrow$	PTA $\uparrow$
w/o taxonomy refinement	0.750085	0.299391	3.081808	0.6200	0.5650	0.3500	0.3900	0.4813	0.733383	0.257588
InsightRAG (full)	0.742160	0.408899	2.289222	3.4750	3.7500	3.5000	3.6750	3.6000	0.797152	0.747758

Table 4: Ablation of taxonomy refinement in InsightRAG. The full pipeline is compared with a version without refinement. Metrics follow Table 2 . Bold marks the best result; Node Ratio favors values closer to 1.

Figure 4: Survey quality by topic. Axes average Coverage, Relevance, and Structure across three LLM judges.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

ID	CCF Research Area	Evaluation Topic and Source Survey	RW Trees
0	Interdisciplinary, Comprehensive, and Emerging Areas	Multi-Agent Workflow Automation ( Zhang, 2025 )	25
1	Human-Computer Interaction and Ubiquitous Computing	AI-Powered Video Accessibility for Blind and Low Vision Viewers ( Xu, 2025 )	26
2	Artificial Intelligence	Pose-Driven Dance Video Generation ( Xue et al., 2025 )	22
3	Computer Graphics and Multimedia	Learning-Based 3D Reconstruction Methodologies ( Christodoulides et al., 2025 )	28
4	Theoretical Computer Science	Threshold ECDSA Protocols ( Sedghighadikolaei and Yavuz, 2025 )	20
5	Network and Information Security	Risks and Countermeasures of Fine-Tuning Large Language Models ( Wang et al., 2025a )	22

Appendix

Table 5: Evaluation topics, their source surveys, and the realized numbers of Related Work (RW) Trees.

Method	Candidate source	Retrieval setting	#Papers
BM25–Offline	Frozen corpus (1,064 papers)	Top-50/topic	500
Dense–Offline	Same frozen corpus	Top-50/topic	500
BM25–Online	Semantic Scholar candidate pools	Top-200/topic	1,995
Dense–Online	Semantic Scholar candidate pools	Top-100/topic	993
SurveyGen	Native references	Native	246
SurveyForge	Native references	Native	1,063

Appendix

Table 6: Retrieval configurations for the RQ1 comparison. #Papers totals retrieved papers across the ten topics. BM25 and Dense share a candidate source within each Offline or Online setting, but use different Top-

k

cutoffs Online. Dense retrieval uses BAAI/bge-base-en-v1.5 for ranking.

Variant	Coverage $\uparrow$	Correctness $\uparrow$	F1 $\uparrow$
w/o Forward Expansion	0.0871	0.9507	0.1595
w/o Backward Expansion	0.2210	0.9755	0.3603
w/o Loop Closure & Densification	0.3927	0.9577	0.5570
InsightRAG (Full)	0.4097	0.9521	0.5729

Appendix

Table 7: Ablation of scope-reconstruction components in InsightRAG. Bold indicates the best result in each column.

Multi-Agent Workflow Automation ( Zhang, 2025 )
Role	Seed paper	#Papers	Coverage	Correctness	F1
Original	Multi-agent Architecture Search via Agentic Supernet ( Zhang et al., 2025a )	69	0.6400	0.8841	0.7425
Alternative	FlowReasoner: Reinforcing Query-Level Meta-Agents ( Gao et al., 2025 )	105	0.7200	0.9238	0.8093
Alternative	EvoAgentX: An Automated Framework for Evolving Agentic Workflows ( Wang et al., 2025b )	122	0.8400	0.8361	0.8380
AI-Powered Video Accessibility for Blind and Low Vision Viewers ( Xu, 2025 )
Original	Making Short-Form Videos Accessible with Hierarchical Video Summaries ( Van Daele et al., 2024 )	34	0.6154	0.9706	0.7532

Appendix

Table 8: Seed sensitivity of InsightRAG’s retrieval. Results cover two topics. #Papers counts evaluated retrieval candidates; Coverage, Correctness, and F1 follow the RQ1 definitions.

Metric	Evidence	Value
Validated extra precision	$90/100$	0.9000
False-discovery rate	$10/100$	0.1000
Precision 95% CI	100 samples	$[0.8256,0.9448]$
Projected precision	Extrapolated	0.9005
Projected recall	Extrapolated	0.8915
Projected F1	Extrapolated	0.8959

Appendix

Table 9: Expert validation of InsightRAG papers outside the gold scope.

Method	#Nodes	Node Ratio	Max Depth	Avg. Branch.	Leaf Ratio	Small Leaf %	Papers/Leaf
Vanilla LLM	30.20	1.3464	1.90	2.0528	0.7217	87.69	1.5533
SurveyGen	27.30	1.1763	2.00	3.7833	0.7885	10.00	7.6436
SurveyForge	29.80	1.3092	2.00	4.9600	0.8321	1.60	9.6703
AutoSurvey	52.10	2.2906	2.00	6.4429	0.8655	4.44	9.3235
Deep Research	7.70	0.3482	1.80	2.1500	0.8242	36.17	3.5950
InsightRAG	60.20	2.2892	2.40	3.4767	0.7351	51.79	3.7362

Appendix

Table 10: Structural statistics of the generated taxonomies. These diagnostic metrics describe granularity and do not share a single optimization direction.

	GPT-5.2				Gemini 3.1 Pro				DeepSeek V4 Pro
Method	Cov.	Str.	Rel.	Avg.	Cov.	Str.	Rel.	Avg.	Cov.	Str.	Rel.	Avg.
Vanilla LLM	4.45	4.50	4.50	4.48	5.00	5.00	5.00	5.00	4.50	4.60	4.90	4.67
SurveyGen	3.70	4.05	3.50	3.75	3.75	4.45	3.70	3.97	3.70	3.75	3.75	3.73
SurveyForge	3.15	3.65	3.10	3.30	4.55	3.20	4.10	3.95	3.25	2.90	3.00	3.05
AutoSurvey	3.00	3.85	2.65	3.17	3.85	4.65	3.35	3.95	3.15	3.40	2.85	3.13
Deep Research	4.00	4.05	4.35	4.13	4.90	4.80	5.00	4.90	4.35	4.25	4.85	4.48

Appendix

Table 11: LLM-judged survey quality across three judge models. Cov., Str., and Rel. denote Coverage, Structure, and Relevance, respectively. Bold values indicate the best result in each column.

Method	Auto Supported	Human Supported
Vanilla LLM	11/14 (78.6%)	4/4 (100%)
SurveyGen	11/14 (78.6%)	1/4 (25%)
SurveyForge	4/14 (28.6%)	1/4 (25%)
AutoSurvey	2/14 (14.3%)	1/4 (25%)
Deep Research	10/14 (71.4%)	2/4 (50%)
InsightRAG	13/14 (92.9%)	4/4 (100%)

Appendix

Table 12: Passage-based citation support of generated surveys. Auto scores use 14 claim–citation pairs per method; human scores use four method-blinded pairs per method. Fully and partially supported automatic labels count as Supported; unverifiable references count as Unsupported.

Stage	Model	Main limits	Estimated tokens per topic
Insight-guided retrieval	Qwen-Max	Retrieval depth 4; paper budget 400	1.4M–43.7M input; median 9.1M
Greedy taxonomy construction	Deterministic greedy	Maximum taxonomy depth 4	0 LLM tokens in this step
Global taxonomy refinement	Qwen-Max	Maximum taxonomy depth 4	4.7k–31.1k recorded input
Evidence-bounded survey generation	Qwen-Plus	N/A	14k–84k final-output proxy

Appendix

Table 13: Implementation settings and token estimates for InsightRAG. Input estimates and the final-output proxy have different accounting bases; these figures do not constitute a complete end-to-end token total.

Topic ID	Initial seed paper
0	Multi-agent Architecture Search via Agentic Supernet ( Zhang et al., 2025a )
1	Making Short-Form Videos Accessible with Hierarchical Video Summaries ( Van Daele et al., 2024 )
2	Everybody Dance Now ( Chan et al., 2019 )
3	POCO: Point Convolution for Surface Reconstruction ( Boulch and Marlet, 2022 )
4	Fast Multiparty Threshold ECDSA with Fast Trustless Setup ( Gennaro and Goldfeder, 2018 )
5	Poisoning Language Models During Instruction Tuning ( Wan et al., 2023 )

Appendix

Table 14: Initial seed papers for the ten main-study topics. Topic IDs follow Table 5 .

Retrieval-Augmented Generation

Retrieval-augmented generation connects language models to external knowledge so answers can use current, domain-specific evidence.

See all 792 papers

SurveyAgent-HKA: A multi-agent framework for scientific survey generation with LLMs and human knowledge augmentation
Sep 5, 2026Tong Bao, Mir Tafseer Nayeem, Yi Zhao +2AI-Assisted Scientific Research LLM Agents
Department of Information Management, Nanjing University of Science and Technology, Nanjing, China · Department of Computing Science, University of Alberta, Edmonton, Canada · School of Management, Anhui University, Hefei, China
DeepSurvey: Agent-Oriented Automated Survey Generation with Analytical Depth and Citation Reliability
May 28, 2026Ziyue Yang, Da Ma, Hanqi Li +8AI for Science AI Agents for Scientific Discovery
X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Suzhou Laboratory, Suzhou, China · Shanghai Innovation Institution, Shanghai, China +1
GRASP: Graph-Reasoning Aided Survey Planning for High-Fidelity Related Work Generation
Jul 4, 2026Haoming Li, Jessica OuyangCitation Network Analysis Graph-Based Planning
Department of Computer Science University of Texas at Dallas Richardson, TX 75080

Sep 5, 2026J/K move · Enter open · S save

SurveyAgent-HKA: A multi-agent framework for scientific survey generation with LLMs and human knowledge augmentation

Tong Bao, Mir Tafseer Nayeem, Yi Zhao, Davood Rafiei+1

Department of Information Management, Nanjing University of Science and Technology, Nanjing, China · Department of Computing Science, University of Alberta, Edmonton, Canada · School of Management, Anhui University, Hefei, China

AI-Assisted Scientific Research LLM Agents Retrieval-Augmented Generation

Open paper AskPDF arXiv

Automatic scientific survey generation has become an important task in scientific document processing. The common approach of retrieving literature from a single source (e.g., arXiv) and generating surveys through a one-pass large language model (LLM) call often leads to limited reference coverage and, more importantly, fails to replicate the expert-driven revision process that is crucial for writing high-quality surveys. In this paper, we introduce SurveyAgent-HKA, a multi-agent framework that improves end-to-end scientific survey generation by incorporating knowledge derived from published surveys and peer-review comments. The framework decomposes survey generation into well-defined sub-tasks handled by LLM-powered agent. It first retrieves relevant papers from multiple sources and identifies key topics through clustering to construct an initial outline, which is then refined using outlines from related human-written surveys. Based on the refined outline, topic-focused papers are retrieved and re-ranked to select for drafting a well-grounded survey. Then, we identify common issues raised by experts in peer-review comments from published surveys to guide the revisions and finalize the survey. Experiments on two domains show that our approach outperforms mainstream baselines in citation quality, structural consistency, and content quality. Furthermore, our framework is efficient in both time and cost, making it a practical solution for broader AI-assisted scientific writing applications.

Abstract

Retrieval-Augmented Generation

Explore similar work

SurveyAgent-HKA: A multi-agent framework for scientific survey generation with LLMs and human knowledge augmentation

DeepSurvey: Agent-Oriented Automated Survey Generation with Analytical Depth and Citation Reliability

GRASP: Graph-Reasoning Aided Survey Planning for High-Fidelity Related Work Generation