Navigating Ideation Space: Decomposed Conceptual Representations for Positioning Scientific Ideas

Minqian Liu; Yuexi Shen; Dawei Zhou; Lifu Huang

cs.IRJan 13, 2026

Navigating Ideation Space: Decomposed Conceptual Representations for Positioning Scientific Ideas

Authors: Minqian Liu, Yuexi Shen, Dawei Zhou, Lifu Huang

Organizations: Virginia Tech · University of California, Santa Barbara · University of California, Davis

Abstract

Scientific discovery requires new ideas to be situated within an ever-expanding landscape of existing knowledge. An emerging challenge is how to identify conceptually relevant prior work from rapidly growing literature, and assess how a new idea differentiates from existing research. Current embedding approaches typically conflate distinct conceptual aspects into single representations and cannot support fine-grained retrieval; meanwhile, LLM-based evaluators are subject to sycophancy biases, failing to provide discriminative novelty assessment. We introduce the Ideation Space, a structured representation that decomposes scientific knowledge into three distinct dimensions learned through contrastive training. Our framework enables principled measurement of conceptual distance between ideas and modeling of ideation transitions that capture the logical connections within an idea. Building upon this representation, we propose a Hierarchical Sub-Space Retrieval framework for efficient, targeted literature retrieval, and a Decomposed Novelty Assessment algorithm that identifies which aspects of an idea are novel. Extensive experiments demonstrate substantial improvements, achieving Recall@30 of 0.329 (16.7% over the strongest baseline), and novelty assessment achieves promising correlation with expert judgments, outperforming all open-source LLM baselines up to 70B.

Figures & tables

Figure 1: Illustration of the Ideation Space construction process. Starting from a citation network, we classify citation edges by relationships and decompose the graph into three complementary views. We then use these view-specific subgraphs to construct positive and negative pairs to train specialized encoders via contrastive learning, and produce three distinct vector databases for each sub-space.

Subspace	Model	Pos Mean	Neg Mean	Margin
Research Problem	Base	0.942	0.836	0.106
	Adapter	0.933	0.828	0.105
	Fine-tuned	0.792	0.058	0.735
Method Approach	Base	0.938	0.844	0.095
	Adapter	0.932	0.838	0.093
	Fine-tuned	0.654	0.010	0.644

Table 1: Embedding space statistics on validation set: Positive/Negative means and margins across subspaces.

Model	Recall@K			NDCG@K			Hit Rate@K
Model	$K{=}9$	$K{=}30$	$K{=}90$	$K{=}9$	$K{=}30$	$K{=}90$	$K{=}9$	$K{=}30$	$K{=}90$
BM25	$0.157$	$0.276$	$0.412$	$0.251$	$0.261$	$0.314$	$0.710$	$0.811$	$0.891$
E5 ${}_{\textsc{base}}$ -v2	$0.144$	$0.252$	$0.389$	$0.216$	$0.230$	$0.280$	$0.678$	$0.788$	$0.891$
SPECTER2	$0.160$	$0.282$	$0.423$	$0.242$	$0.257$	$0.310$	$0.667$	$0.811$	$0.893$
SciNCL	$0.157$	$0.281$	$0.427$	$0.233$	$0.250$	$0.305$	$0.697$	$0.788$	$0.886$
Ideation Space (Ours)	$0.169$	0.329	0.483	$0.250$	0.277	0.335	0.702	0.861	0.927

Table 2: Node retrieval performance (Recall, NDCG, Hit Rate) at

K\in\{9,30,90\}

. Baselines are evaluated using the concatenation of title and abstract as input.

Method	Pearson	Spearman
Llama-3.1-8B	0.053	0.089
+ CoT	0.034	0.052
DS-R1-Distill-8B	0.223	0.233
+ CoT	0.065	0.075
+ Few-Shot	0.296	0.241
Qwen3-8B	0.218	0.222

Table 3: Correlation between novelty estimation methods and ground-truth expert scores. Bold : best overall; underline : best among non-proprietary methods.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Input	Recall@K			NDCG@K			Hit Rate@K
Model	Input	$K{=}9$	$K{=}30$	$K{=}90$	$K{=}9$	$K{=}30$	$K{=}90$	$K{=}9$	$K{=}30$	$K{=}90$
BM25	Title	$0.087$	$0.159$	$0.232$	$0.140$	$0.152$	$0.180$	$0.481$	$0.622$	$0.719$
	Abstract	$0.143$	$0.246$	$0.365$	$0.230$	$0.236$	$0.281$	$0.674$	$0.794$	$0.856$
	Concat	$0.157$	$0.276$	$0.412$	$0.251$	$0.261$	$0.314$	$0.710$	$0.811$	$0.891$
E5 ${}_{\textsc{base}}$ -v2	Title	$0.110$	$0.180$	$0.264$	$0.169$	$0.175$	$0.207$	$0.573$	$0.682$	$0.779$
	Abstract	$0.134$	$0.239$	$0.372$	$0.204$	$0.218$	$0.268$	$0.635$	$0.773$	$0.873$

Appendix

Table 4: Full node retrieval performance (Recall, NDCG, Hit Rate) at

K\in\{9,30,90\}

. Each sparse/dense baseline is evaluated on three input variants: title only, abstract only, and their concatenation (Concat). Bold denotes the best result per column.

Model	Recall@K			NDCG@K			Hit Rate@K
Model	$K{=}9$	$K{=}30$	$K{=}90$	$K{=}9$	$K{=}30$	$K{=}90$	$K{=}9$	$K{=}30$	$K{=}90$
BM25	$0.076$	$0.135$	$0.229$	$0.140$	$0.135$	$0.169$	$0.432$	$0.593$	$0.739$
E5 ${}_{\textsc{base}}$ -v2	$0.095$	$0.160$	$0.247$	0.168	$0.160$	$0.192$	0.492	$0.627$	$0.730$
SPECTER2	$0.090$	$0.153$	$0.236$	$0.162$	$0.157$	$0.189$	$0.461$	$0.602$	$0.719$
SciNCL	$0.090$	$0.154$	$0.236$	$0.157$	$0.152$	$0.182$	$0.467$	$0.602$	$0.708$
Transition Space (Ours)	0.099	0.180	0.275	$0.159$	0.166	0.203	$0.465$	0.643	0.762

Appendix

Table 5: Transition retrieval performance at

K\in\{9,30,90\}

Model	Recall@30	NDCG@30	Hit Rate@30
BM25	0.342	0.257	0.798
E5 ${}_{\text{base}}$ -v2	0.322	0.236	0.788
SPECTER2	0.345	0.247	0.804
SciNCL	0.337	0.240	0.796
Ideation Space (Ours)	0.388	0.271	0.820
w/ Subspace: RP	0.360	0.257	0.801

Appendix

Table 6: Node retrieval on the expanded test set with 1,330 papers annotated by Qwen3-4B. All baselines use concatenated title+abstract with

K{=}30

Paper Title	GT	Ideation Space (Ours)	DeepSeek	GPT-5-Nano	GPT-5-Mini	LLaMA-3.1-8B	Qwen3-8B	Qwen3-14B
Adversarial Scenario Extrapolation	5.50	5.53	9	7	6	8	8	8
Probabilistic Opinion Pooling for QA	5.67	9.79	8	7	6	8	8	8

Appendix

Table 7: Case study of novelty evaluation.

Subspace	Retrieved Paper	Sim.	Year
Success: Adversarial Scenario Extrapolation (GT = 5.50, Ours = 5.53)
Research Problem	Model-tuning Via Prompts Makes NLP Models Adversarially Robust	0.817	2023
Method Approach	Don’t Retrain, Just Rewrite: Countering Adversarial Perturbations by Rewriting Text	0.735	2023
Key Findings	Robust Safety Classifier for Large Language Models: Adversarial Prompt Shield	0.772	2023
Failure: Probabilistic Opinion Pooling for QA (GT = 5.67, Ours = 9.79)
Research Problem	Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction	0.760	2020

Appendix

Table 8: Top-1 retrieved papers per subspace for the two novelty evaluation case studies. Similarity (Sim.) denotes cosine similarity in the corresponding ideation sub-space. GT: ground-truth expert novelty score.

Model	Recall @ K			Hit Rate @ K
Model	3	10	30	3	10	30
BM25	0.392	0.592	0.729	0.905	0.939	0.962
E5 ${}_{\text{base}}$ -v2	0.391	0.587	0.723	0.898	0.949	0.970
SPECTER2	0.393	0.609	0.742	0.896	0.955	0.972
SciNCL	0.362	0.557	0.708	0.875	0.934	0.964
Ideation Space	0.373	0.586	0.740	0.904	0.961	0.976

Appendix

Table 9: Zero-shot cross-domain transfer on biomedical domain. Our Ideation Space encoders are trained exclusively on AI/ML papers and applied to biomedical retrieval queries without any fine-tuning. Baselines are off-the-shelf retrievers; only the concat (title + abstract) variant is reported. The best result per column is highlighted in bold.

Metric	Value
N (ideas)	98
Krippendorff’s $\alpha$ (ordinal)	0.236
Krippendorff’s $\alpha$ (interval)	0.258
Mean $\|\text{diff}\|$	1.80
Exact agreement	19.6%
Within-1 agreement	49.1%

Appendix

Table 10: Inter-annotator agreement on expert novelty scores from the AI-Researcher dataset Si et al. [2025] , computed over all 98 ideas.

\alpha

is reported under both ordinal and interval distance metrics.

|\text{diff}|

is the mean absolute pairwise difference between annotators on the 1–10 scale. Exact and Within-1 report the proportion of annotator pairs whose scores match exactly or fall within one point.

Model	Recall @ K			NDCG @ K			Hit Rate @ K
	9	30	90	9	30	90	9	30	90
Jointly Trained	0.172	0.304	0.444	0.251	0.270	0.324	0.695	0.820	0.886
Ideation Space	0.169	0.329	0.483	0.250	0.277	0.335	0.702	0.861	0.927

Appendix

Table 11: Comparison between jointly trained retrieval and Ideation Space retrieval. Bold denotes the best result per column.

Query Paper Details
Title: LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models
Research Problem: This work addresses the gap in existing benchmarks for evaluating hallucinations in multimodal large language models (MLLMs), which rely on simplistic discriminative questions or computationally intensive LLM evaluators .
Method Approach: The paper introduces LongHalQA, an LLM-free benchmark featuring 6K long and complex hallucination texts generated by GPT4V, along with two unified tasks—hallucination discrimination and completion.
Key Findings: Extensive experiments on recent MLLMs reveal significant challenges in handling long and complex hallucination texts , highlighting the necessity for advanced benchmarks like LongHalQA.
Dimension	Retrieved Paper (Title & Text)	Score
Our Multi-Dimensional Retrieval (Top 1 per Dimension)

Appendix

Table 12: Detailed retrieval comparison for LongHalQA . The Query Paper details are presented first. For each dimension, we display the top retrieved paper’s Title and relevant Text. Our retrieval surfaces specific hallucination benchmarks (AMBER, HaELM) while the baseline focuses on general MLLM benchmarks.

Scientific Idea Evaluation

See all 16 papers

Graph2Idea:Retrieval-Augmented Scientific Idea Generation with Graph-Structured Contexts
Jun 8, 2026Xu Li, Hanzhe Tu, Xun HanGraph Retrieval-Augmented Generation Scientific Hypothesis Generation
Southwest Petroleum University, Chengdu, Sichuan, China · Sichuan Police College, Luzhou, Sichuan, China
Generating Interesting Scientific Ideas using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaders
May 27, 2024Xuemei Gu, Mario KrennAI-Assisted Scientific Research Scientific Hypothesis Generation
Max Planck Institute for the Science of Light, Staudtstrasse 2, 91058 Erlangen, Germany
Is this Idea Novel? An Automated Benchmark for Judgment of Research Ideas
Mar 11, 2026Tim Schopf, Michael FärberAutomated Evaluation LLM-as-a-Judge
TU Dresden & ScaDS.AI Dresden/Leipzig, Germany · National Institute of Informatics, Tokyo, Japan

Jun 8, 2026J/K move · Enter open · S save

Graph2Idea:Retrieval-Augmented Scientific Idea Generation with Graph-Structured Contexts

Xu Li, Hanzhe Tu, Xun Han

Southwest Petroleum University, Chengdu, Sichuan, China · Sichuan Police College, Luzhou, Sichuan, China

Graph Retrieval-Augmented Generation Scientific Hypothesis Generation Research Idea Generation Large Language Model-Guided Scientific Discovery

Open paper AskPDF arXiv

Generating novel, feasible, and high-quality research ideas is an important yet challenging task in scientific discovery. Recent Large Language Model (LLM)-based methods often ground idea generation with retrieved literature, but the retrieved evidence is usually provided as flat text, such as titles, abstracts, or summaries. Such flat contexts may contain redundant or weakly relevant information, while making cross-paper relations among problems, methods, mechanisms, and findings difficult to identify and trace. To address this challenge, we propose Graph2Idea, a knowledge graph-guided framework for retrieval-augmented scientific idea generation.Graph2Idea first retrieves papers according to the input topic, transforms them into structured knowledge triples, and dynamically constructs a target-centered knowledge graph to make literature relations explicit. It then extracts compact graph-derived contexts that retain target-relevant relational evidence while reducing noisy textual input. Based on these contexts, a two-stage generation process first identifies promising research directions and then guides the LLM to synthesize candidate ideas from graph-grounded evidence. Experiments on a scientific idea generation benchmark show that Graph2Idea outperforms representative baselines under the automatic evaluation protocol. Compared with the strongest baseline scores, it improves Novelty from 0.45 to 0.52, Quality from 0.24 to 0.29, and Feasibility from 0.22 to 0.28. These results suggest that graph-structured evidence helps LLMs generate research ideas through more explicit, compact, and traceable recombination of prior scientific knowledge.

Abstract

Scientific Idea Evaluation

Explore similar work

Graph2Idea:Retrieval-Augmented Scientific Idea Generation with Graph-Structured Contexts

Generating Interesting Scientific Ideas using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaders

Is this Idea Novel? An Automated Benchmark for Judgment of Research Ideas