Doc2LoRA Provides Decodable Representations of Scientific Ideas

Chand Sahil Mansuri; Joel Zachariah; Sadamori Kojaku

cs.CLSep 29, 2026

Doc2LoRA Provides Decodable Representations of Scientific Ideas

Authors: Chand Sahil Mansuri, Joel Zachariah, Sadamori Kojaku

Organizations: School of Systems Science and Industrial Engineering Binghamton University, Binghamton, NY, USA

Abstract

Representing scientific papers as points in a space lets us search for similar papers and inquire about how fields relate to one another and drive innovation. Beyond search, the vector space of papers invites generation: mixing papers through simple vector operations creates new points, mirroring combinatorial novelty, the recombination of existing ideas into new ones. However, a mixed point often represents an idea no paper has yet realized, with no papers nearby to identify the idea. We propose representing each paper by a LoRA adapter generated by the Doc-to-LoRA hypernetwork. Every point in the space, including mixtures, thus represents a large language model (LLM) open to questions and instructions in natural language. On papers from the American Physical Society (APS), we instruct the LLM at the average of each subfield to name the field in a few words and obtain labels closer to the official names than the labels of five baselines, as judged by word overlap and a panel of five LLM judges. We also ask the LLMs at points between two APS papers to write an abstract and obtain descriptions shifting from one paper to the other in step with the mixing weight. While Doc-to-LoRA is trained for generation, a small invertible transform makes the embeddings competitive for search, on par with SPECTER2 and EmbeddingGemma and close to SBERT. Because the transform is invertible, every point in the transformed space still maps back to an LLM. The embeddings thus serve both search and generation, enabling researchers to question the idea at any point in the space as a starting point for generating new ideas.

Figures & tables

Figure 1: Cluster labeling and mixing with D2L . (a) Fuzzy token-set overlap between each decoded label and official 28 PACS labels. Each point is a node, with distribution represented by kernel density estimation (KDE). (b) Pairwise judge win rates, showing the fraction of PACS labels where the generated label from the row method is judged closer to the official label than that from the column. Color runs blue (row loses), grey (tie), to orange (row wins), row means at right. (c) Cluster centroids, decoded per PACS field/division/subdivision. Horizontal axis is vector length, circle area is the number of papers. Decoded labels (orange) and official PACS names (grey, if different) are shown. (d) Specificity as a function of vector length: longer vectors yield specific, shorter more general concepts. For each Wikipedia seed, lines show continuity of label, grey bands mark collapse to base-model prior. (e) Decoded abstract position (SBERT space) as a function of B weight, A $\to$ B, for 50 near pairs (papers from the same PACS subtopic). D2L and ICAE track the target, in-context stays near midpoint, T2L flat. (f) Verbatim copy rate by method, measured as the fraction of output tokens present in sources. ICAE is highest, and D2L and in-context prompting are comparable.

	Ideal mixing weight on paper B
Method	$3/12$	$5/12$	$9/12$
D2L	This research investigates the learning of real-valued functions in neural networks using a statistical-mechanical framework, focusing on the generalization error of trained networks. The problem involves understanding how well a network can generalize from a finite training set to unseen data, particularly…	This research investigates the learning of quantum gates in a unitary model of quantum computation , where a quantum system evolves under a unitary transformation and is trained using a quantum neural network …	This research topic investigates the existence and properties of quantum error-correcting codes , which are essential for protecting quantum information from decoherence . The problem is to determine whether, given a quantum system of qubits , there exists a unitary encoding (quantum map) that transforms…
ICAE	This study investigates the learning of statistical mechanics from feedforward neural networks , focusing on the existence of stochastic, Gibbs -free energy-based models. The study assumes a temperature-dependent distribution of networks and considers both realizable and unrealizable rules…	This research explores the existence of quantum error-correcting codes in statistical mechanics , specifically in the context of learning from examples. The study focuses on the existence of a unitary random code, which is a type of error-correcting code that can be defined…	This research explores the existence of quantum error-correcting codes ( QECCs ) in the context of quantum computing . It defines a unitary mapping called a quantum error-correcting code ( QECC ) that maps qubits to a subspace of the quantum state space…
in-context	This research investigates the statistical-mechanical behavior of quantum neural networks under the influence of both stochastic learning dynamics and quantum decoherence , blending the statistical mechanics of learning from examples (A:75%) with the principles of quantum error correction (B:25%)…	This research investigates the statistical-mechanical behavior of quantum neural networks under the influence of both stochastic training and quantum decoherence , blending the framework of learning from examples (A) with the principles of quantum error correction (B) in a 58:42 proportion…	This research investigates the emergence of quantum spin-glass phases in neural network architectures under the influence of quantum decoherence , blending statistical mechanics of learning (A) with the theory of quantum error correction (B)…

Table 1: One near pair (two papers from the same PACS subtopic) decoded along the interpolation path at

3/12

5/12

, and

9/12

weight on paper B, under the prompt quoted in the text.

5/12

is the grid point where the D2L decode uses the terms of both papers. Paper A is “Statistical mechanics of learning from examples” (PhysRevA 45.6056) and paper B is “Good quantum error-correcting codes exist” (PhysRevA 54.1098). Each cell is the opening of the decoded abstract, cut at

42

words. Terms from paper A are in blue and terms from paper B in orange .

		D2L		text encoders
Task	Field / dataset	raw	adapted	SBERT	GTE	Emb.Gemma	Instructor	SPECTER2
Next-paper (AUC)	Economics	.813 $\pm$ .001	.910 $\pm$ .001	.937 $\pm$ .001	.910 $\pm$ .001	.906 $\pm$ .001	.877 $\pm$ .001	.903 $\pm$ .001
	Psychology	.799 $\pm$ .001	.923 $\pm$ .001	.937 $\pm$ .001	.912 $\pm$ .001	.910 $\pm$ .001	.888 $\pm$ .001	.908 $\pm$ .001
	Physics	.880 $\pm$ .001	.958 $\pm$ .001	.956 $\pm$ .001	.962 $\pm$ .001	.947 $\pm$ .001	.908 $\pm$ .001	.941 $\pm$ .001
Topic (macro-F1)	Economics	.242 $\pm$ .015	.374 $\pm$ .022	.418 $\pm$ .022	.398 $\pm$ .023	.362 $\pm$ .021	.379 $\pm$ .021	.344 $\pm$ .021
	Psychology	.217 $\pm$ .010	.340 $\pm$ .020	.366 $\pm$ .023	.350 $\pm$ .019	.358 $\pm$ .020	.348 $\pm$ .026	.319 $\pm$ .020

Table 2: Similarity benchmarks, absolute scores. Each row is one task on one field or dataset; raw is the D2L embedding and adapted applies

g_{\theta}

, the linear transform trained on OpenAlex. Text encoders are ordered by mean rank. Cells are the bootstrap mean

\pm

its standard deviation over

1{,}000

resamples. Bold and shaded indicates the top method per row; the second-best is underlined. For disambiguation, SPECTER replaces SPECTER2 . Full protocol: Appendix G .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Operates on	#docs	Native output (scored as-is)
D2L	embedding (mean adapter)	up to $2{,}000$	field name (2–3 words)
ICAE	embedding (mean memory slots)	$8$	field name (2–3 words)
T2L	embedding (mean TaskEncoder output)	up to $2{,}000$	field name (2–3 words)
vec2text	embedding (mean GTR vector, inverted)	$20$	reconstruction (no prompt)
KeyLLM	text (medoid document)	$1$	keyword list
BERTopic	text (members, c-TF-IDF)	up to $2{,}000$	keyword list

Appendix

Table 3: Cluster-labeling protocols comparing embedding-arithmetic (top) and text-based (bottom) methods without downstream language model post-processing: “Operates on” denotes the representation averaged and labeled, “#docs” indicates consumed member documents per node, and “Native output” defines directly evaluated raw outputs.

Method	Words	Fuzzy overlap	Pairwise
Ground truth (control)	$4.0$	$1.000$	—
D2L	$2.4$	$\mathbf{.642\pm.055}$	$\mathbf{.755\pm.032}$
ICAE	$2.4$	$.573\pm.058$	$.678\pm.039$
KeyLLM	$8.8$	$.311\pm.019$	$.474\pm.034$
vec2text	$15.5$	$.302\pm.026$	$.215\pm.021$
BERTopic (top $10$ )	$10.0$	$.296\pm.020$	$.696\pm.033$

Appendix

Table 4: Cluster-label quality over

28

PACS nodes. Each method’s native output is scored against the true node label. Fuzzy overlap: deterministic token-set ratio with the reference. Pairwise: win rate from a five-judge round robin over the seven methods below; ties score

0.5

and require both label orders to agree.

\pm

is the bootstrap s.d. over

1{,}000

node resamples. Words: mean label length ( PACS:

4.0

). The ground-truth control has no win rate. Best (excluding control) in bold. Full pairwise matrix: Fig. 1 b.

cluster set	$n$	belongs=yes	rejected
PACS node (real)	30	73.3%	3.3%
cross-chapter control	60	0%	100%
cross-chapter, norm-matched	60	0%	96.7%

Appendix

Table 5: Scores from the blinded five-judge panel on the incoherent-cluster control experiment. PACS node (real) denotes actual clusters, while the control rows consist of clusters of the same size sampled across different PACS chapters and decoded either at their native centroid norm or rescaled to the real nodes’ norm. belongs=yes and rejected indicate the proportion of clusters whose label the panel accepted as valid for the members or rejected entirely. The rest were judged partial.

decode family	units	$K$	mean pairwise sim.	sim. to the reported prompt	identical strings
PACS node labels (2–3 words)	30	8	0.787	0.841	51%
two-paper midpoint descriptions	40	8	0.705	0.705	0%

Appendix

Table 6: Prompt sensitivity.

K

is the number of semantically equivalent prompts per point, the first being the one the paper reports. mean pairwise sim. averages SBERT cosine over prompt pairs for the same point; sim. to the reported prompt compares the other seven against it. identical strings is the fraction of those prompt pairs that emit the same text after lowercasing. Two different PACS nodes decoded under one prompt have mean cosine

.463

. Two different midpoints decoded under one prompt have mean cosine

.244

Figure 2: Decoded abstracts along the A–B interpolation under paraphrase, near pairs (left) and far pairs (right). Solid lines are the wording reported in Section 3.2 and dashed lines its three paraphrases. Top , where the decoded abstract lands on the A

\to

B axis in SBERT space against the requested mix weight, with the diagonal marking proportional tracking. D2L and ICAE sit near a source paper at the endpoints and blend through the middle, while in-context prompting stays near the midpoint across the whole interior. Bottom , verbatim copy rate. ICAE sits above the other two everywhere, and D2L and in-context prompting overlap. Paraphrase moves every curve by less than the gap between methods.

space	mean $\Delta$	range	better	worse
ICAE	+.178	[-.004, +.549]	13/14	1/14
D2L (Qwen3-4B)	+.061	[-.009, +.133]	11/14	1/14
Instructor	+.009	[-.020, +.040]	7/14	2/14
GTE	+.007	[-.022, +.029]	8/14	2/14
EmbeddingGemma	+.001	[-.044, +.025]	6/14	3/14
SPECTER2 / SPECTER	+.001	[-.034, +.023]	5/14	4/14

Appendix

Table 7: Change in benchmark score when the same transform

g_{\theta}

is trained in each embedding space. Each row summarizes one space over the

14

benchmarks of Table 2 . mean

\Delta

: score with

g_{\theta}

minus score without, averaged over the benchmarks. range : smallest and largest change. better and worse : number of benchmarks with a significant gain or loss, i.e., a

95\%

bootstrap interval of the change entirely above or below zero. Each bootstrap sample scores the space with and without

g_{\theta}

on the same resampled evaluation units. This pairing yields a narrower interval for the change than the separate intervals of Table 2 .

Figure 3: Gain from the shared transform

g_{\theta}

in every space on the

14

benchmarks of Table 2 . For each benchmark and space we take the natural logarithm of the score with

g_{\theta}

divided by the score without it. Each circle averages this log ratio over the benchmarks of a panel. The first four panels are collaboration prediction, next-paper prediction and topic classification on Physics, Economics and Psychology, and author-name disambiguation (B 3 F1) on the five S2AND datasets. The last panel averages all

14

. Whiskers are

95\%

intervals of a two-level bootstrap that resamples the panel’s benchmarks and redraws each score with and without

g_{\theta}

from its own bootstrap distribution. EmbGemma is EmbeddingGemma , and SPECTER is SPECTER2 on the field benchmarks and SPECTER on S2AND, each with its own transform.

space	Collab. (AUC)	Next paper (AUC)	Topic (F1)	Disamb. (B 3 F1)
D2L (Qwen3-4B)	.663	.823	.358	.760
ICAE (Mistral-7B)	.660	.677	.227	.617
SBERT	.726	.941	.443	.827
EmbeddingGemma	.712	.919	.433	.813
Instructor	.708	.887	.425	.800
SPECTER2 / SPECTER	.708	.915	.417	.784

Appendix

Table 8: Scores of every space without

g_{\theta}

, averaged over the benchmarks of each task (three fields for collaboration, next-paper and topic, five S2AND datasets for disambiguation). Per-benchmark scores of D2L and the text encoders, with bootstrap standard deviations, are in Table 2 .

Encoder	Field	Next-paper (AUC)	Topic (F1)	Collaboration (AUC)
Gemma-2-2b	Economics	.792 / .888	.217 / .317	.592 / .617
	Psychology	.786 / .910	.180 / .319	.593 / .673
	Physics	.864 / .946	.569 / .582	.827 / .823
Qwen3-4B	Economics	.813 / .910	.242 / .374	.596 / .629
	Psychology	.799 / .923	.217 / .340	.602 / .687
	Physics	.880 / .958	.590 / .590	.792 / .795

Appendix

Table 9: Per-encoder field-task matrix. Each cell reports the two D2L variants as raw / +

g_{\theta}

(the frozen embedding and the single general citation transform), with metrics, evaluation pools, and bootstrap protocol as in Table 2 (standard deviations omitted). Text-baseline scores are encoder-independent and given in Table 2 .

method	Physics			Economics				Psychology
	2000	2008	pooled	2008	2012	2016	pooled	2008	2012	2016	pooled
D2L (raw)	.786	.848	.792	.646	.556	.579	.596	.581	.610	.627	.602
$+g_{\theta}$	.784	.849	.795	.680	.594	.603	.628	.672	.721	.678	.688
SPECTER2	.788	.857	.826	.672	.593	.621	.632	.651	.679	.676	.665
SBERT	.826	.887	.832	.696	.626	.637	.655	.691	.718	.667	.691
Instructor	.831	.843	.854	.676	.577	.597	.620	.645	.657	.648	.649

Appendix

Table 10: AUC-ROC of collaboration prediction for each anchor year. At an anchor year, each author is represented by the mean embedding of their papers from the previous four years. Candidate pairs are authors two steps apart in the coauthorship network who have not published together. A candidate pair is positive if the two authors coauthor a paper within the three years after the anchor year. Each column scores the candidate pairs of one anchor year. The pooled column scores the candidate pairs of all anchor years together and is the number reported in Table 2 . D2L is the Qwen3-4B embedding with and without

g_{\theta}

(Appendix G ).

$\alpha$	Physics	Biochemistry	Fruit	Person	Patent (LED)
$1.15$	is the fractional quantum effect of accumulating electrons in certain atoms. …	(abbreviated CY), commonly ATP, is an enzyme cycle for the production …	a banana is an edible plantation fruit, traditionally specializing in long, …	Nikola Tesla, an American engineer from Serbia, was an 1860s American …	comprised an ion-doped light-emitting semiconductor forming a nitride structure, nitrid
$0.9$	quantum fraction is a function of the Hall effect in physics: …	enzyme cycle is the metabolic use of ATP for generating cryptic …	fruit is a elongated banana. This is due to the variety …	Nikola Tesla was a Serbian engineer and American engineer, known for …	structure comprised a light-emitting compound of a double-type nitride semiconductor (e.g.
$0.7$	quantum mechanics is the fraction of the Hall effect which is …	acids used for the production of glycogen. The Krebs cycle is …	plant species is a banana. The name derives from a linguistic …	engineering genius Nikola Tesla is a Serbian engineer who contributed to …	structure of a double-layered nickel semiconductor (indicated in the introduction of …
$0.55$	equation for the physics of fractional heating is a contribution to …	the CYCY process of carbohydrates is a way of generating ATP, …	botanical designation of a plant. A banana is a combination of …	American mechanical engineer. Nikola Tesla is credited for the production of …	layer of a g-type nickel plating (indicated in the following table: …
$0.42$	equation for quantum mechanics is a result of the publication of …	the metabolic cycle of enzymes is a contribution to the aforementioned …	botanical name for a banana. This is a variant of the …	American electrical engineer. Nikola Tesla is credited for the contribution of …	a GLTI semiconductor pairing of the following structures: (a) (initiated) (a) …
$0.3$	a number of awards in the field of physics	biology is a key component of the Crypt of the Chemical …	biology of plants is a classic in the discipline of banana …	American electrical engineer who was a major contributor to the creation …	a layered semiconductor for the neo-Neo-Geo-Neo-Neo-Dai

Appendix

Table 11: vec2text magnitude sweep on the same five seed documents. vec2text is an inverter and takes no prompt, so we read back its reconstruction directly. The reconstruction stays on topic at the same surface specificity for larger

\alpha

and degrades into off-topic fragments at small

\alpha

, never abstracting toward the broader field. Entries truncated to leading terms.

	Native output (as scored)
PACS node	D2L	KeyLLM	ICAE	vec2text
0 General	Quantum field theory	partial decoherence, thermalization, time-domain …	Quantum many-body systems	coupled-dissonance correlations in one-way systems. The …
03 Quantum mechanics, QFT	Quantum optics	quantum state transfer, entanglement, four qubits …	Quantum optics	coupled entanglement of one-state quantums, where the …
03.65 Quantum mechanics	Quantum mechanics	quantum decoherence, single qubit, random matrix theory …	Quantum mechanics	as an empirical measure of the interpolation of one-state …
03.67 Quantum information	Quantum information	Measurement-based quantum computing, AKLT state, spin-1 …	Quantum information	entanglement of single-state quantum computations, so …
05 Statistical physics	Statistical physics	Disturbance spreading, Incommensurate systems …	Statistical mechanics	diffuse-phase dynamics in univariate quantization. This …

Appendix

Table 12: Cluster labels for every node of a four-branch PACS slice. Each cell is the method’s native output, exactly the string scored in Table 4 , truncated at a word boundary for the page but scored in full. D2L and ICAE answer the field prompt; KeyLLM returns a keyword list and vec2text a reconstruction, which is why only the first two columns read as field names. The first column gives the PACS code and a short name, indented by tree depth, with field rows in bold.

Explore similar work

Jun 8, 2026cs.AI

Graph2Idea:Retrieval-Augmented Scientific Idea Generation with Graph-Structured Contexts

Generating novel, feasible, and high-quality research ideas is an important yet challenging task in scientific discovery. Recent Large Language Model (LLM)-based methods often ground idea generation with retrieved literature, but the retrieved evidence is usually provided as flat text, such as titles, abstracts, or summaries. Such flat contexts may contain redundant or weakly relevant information, while making cross-paper relations among problems, methods, mechanisms, and findings difficult to identify and trace. To address this challenge, we propose Graph2Idea, a knowledge graph-guided framework for retrieval-augmented scientific idea generation.Graph2Idea first retrieves papers according to the input topic, transforms them into structured knowledge triples, and dynamically constructs a target-centered knowledge graph to make literature relations explicit. It then extracts compact graph-derived contexts that retain target-relevant relational evidence while reducing noisy textual input. Based on these contexts, a two-stage generation process first identifies promising research directions and then guides the LLM to synthesize candidate ideas from graph-grounded evidence. Experiments on a scientific idea generation benchmark show that Graph2Idea outperforms representative baselines under the automatic evaluation protocol. Compared with the strongest baseline scores, it improves Novelty from 0.45 to 0.52, Quality from 0.24 to 0.29, and Feasibility from 0.22 to 0.28. These results suggest that graph-structured evidence helps LLMs generate research ideas through more explicit, compact, and traceable recombination of prior scientific knowledge.

Xu Li, Hanzhe Tu, Xun Han

Southwest Petroleum University, Chengdu, Sichuan, China · Sichuan Police College, Luzhou, Sichuan, China

Scientific Ideation Graph-Based Retrieval Augmented Generation Rdf Code Generation

Ask PDF

May 12, 2026cs.AI

Read, Grep, and Synthesize: Diagnosing Cross-Domain Seed Exposure for LLM Research Ideation

The discovery of novel methodologies for emerging problems is a continuing cycle in ML, often driven by the migration of techniques across domains. Building on this observation, we ask whether current LLM ideation systems benefit from targeted cross-domain retrieval or simply from exposure to diverse mechanisms. We study this question through PaperGym, a three-stage pipeline: (1) tool-augmented seed extraction via read, grep, and bash over an isolated paper environment, (2) cross-domain seed retrieval via paraphrasing across seven ML domains, and (3) method synthesis from retrieved seeds, each scored by rubric-based judges. Tool-augmented extraction improves specificity, and paraphrase-based retrieval broadens domain coverage. In synthesis, cross-domain retrieval receives more pairwise novelty wins than no-retrieval and same-domain baselines, but shows no significant difference from a random diverse-seed control. These findings suggest LLM ideation systems benefit from diverse seed exposure, but do not yet reliably exploit the semantic reason particular seeds were retrieved. We release the seed library, rubric prompts, and run scripts at https://github.com/yunjoochoi/PaperGym

Yunju Choi, Min Song

1Yonsei University, Seoul, Republic of Korea · 2OnomaAI, Seoul, Republic of Korea.

Topic pending

Ask PDF

May 27, 2024cs.AI

Generating Interesting Scientific Ideas using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaders

The rapid growth of scientific literature makes it increasingly challenging for researchers to identify novel and impactful ideas, especially across disciplines. Modern artificial intelligence (AI) systems offer new opportunities for scientific ideation, but how compelling are AI-generated ideas, and how can their quality be improved? Here, we introduce SciMuse, which generates personalized research ideas using a knowledge graph of 58 million papers and a large language model (LLM). A central focus of this work is to understand how interesting these ideas are. Therefore, we conducted a large-scale evaluation in which more than 100 research group leaders -- spanning the natural sciences to the humanities -- rated over 4,400 personalized ideas according to their level of interest. Overall, expert ratings were modest (mean 2.40 on a 5-point scale, most common rating 1), while 24.9% of ideas were rated 4 or 5. We find that supplying concept pairs selected using the knowledge graph does not improve expert-rated interest over a titles-only GPT baseline. High-citation-predicted pairs even showed a weak tendency (1.94

σ

) toward lower interest than random pairs. Nevertheless, graph features can be used to control properties of ideas, and, using this unique evaluation dataset, we show that idea interest can be predicted with both a supervised neural network based on graph features and a zero-shot ranking approach based on an LLM. Our work provides an AI methodology for generating scientific ideas and a large-scale interdisciplinary expert evaluation, paving the way to study and improve difficult-to-measure metrics such as expert-perceived scientific interestingness.

Xuemei Gu, Mario Krenn

Max Planck Institute for the Science of Light, Staudtstrasse 2, 91058 Erlangen, Germany

Scientific Ideation Knowledge Graphs

Ask PDF