LitCurate: A Configuration-Driven AI-Assisted Framework for Scientific Database Construction with an Application to Lower-Mantle Equation-of-State Data

Abin Shakya; Wilson Samuels; Dominica Wilson; Gioia A. Marchi; Israa Draz; Chenxing Luo; Renata M. Wentzcovitch

cs.IRAug 27, 2026

LitCurate: A Configuration-Driven AI-Assisted Framework for Scientific Database Construction with an Application to Lower-Mantle Equation-of-State Data

Authors: Abin Shakya, Wilson Samuels, Dominica Wilson, Gioia A. Marchi, Israa Draz, Chenxing Luo, Renata M. Wentzcovitch

Organizations: Lamont–Doherty Earth Observatory, Columbia University, Palisades, NY, 10964, United States · Department of Electrical and Computer Engineering, Tuskegee University, Tuskegee, AL, 36088, United States · Department of Industrial Engineering and Operations Research, Columbia University, New York, NY, 10027, United States · Department of Applied Physics and Applied Mathematics, Columbia University, New York, NY, 10027, United States ·

Abstract

The growing scientific literature contains decades of experimental and computational results that could support data-driven and physics-based modeling, yet much of this infor- mation remains locked in publications and is not readily usable for large-scale analysis or sci- entific software. Building structured databases from the literature is particularly challenging whenrelevantstudiesmustfirstbediscoveredamonglargecollectionsofpapersandreported quantities must be extracted with enough scientific context to remain usable. We present LitCurate, an open-source framework for building scientific databases from the literature using large language models within an auditable, stage-wise curation workflow. LitCurate integratesliteraturediscovery, relevancescreening, full-textprocessing, andstructuredinfor- mation extraction while retaining intermediate results and provenance, allowing researchers to inspect and revise individual stages rather than treating automated curation as a black- box process. We apply LitCurate to construct an equation-of-state database of lower-mantle and lower-mantle-relevant high-pressure mineral phases from experimental and theoretical studies, comprising 1,334 entries from 205 papers. The resulting dataset links reported equation-of-state parameters to mineral phases, compositions, equation formulations, meth- ods, and parameter constraints, and labels values as source-reported or citation-reported when provenance can be determined. The records are available through a searchable web application. By connecting scientific literature to traceable, machine-readable data, LitCu- rate provides a reusable approach for transforming accumulated literature into resources for scientific analysis and computational modeling.

Figures & tables

Figure 1: LitCurate architecture. Pipeline advances through ten stages (A–J), from LLM-assisted query planning and literature discovery through screening, full-text preparation, and schema-guided extraction, to export of a merged structured database. Each stage writes intermediate artifacts and records its status in a per-run ledger (bottom layer), so individual stages can be inspected, re-run, or resumed. Domain knowledge (research goals, filter criteria, extraction schemas, and prompts) is supplied via YAML configuration and associated files (top-left), allowing stage machinery to be reused across disciplines.

Figure 2: Schema-guided extraction (stage I). Cleaned markdown is combined with a task prompt and a declared record schema; LLM returns structured JSON that is validated against that schema and stored as an envelope, then merged into exported EOS database.

Stage	Outcome	Papers retained
A Query planning	20 queries generated from the research goal	–
B Literature search	12,000 query-result records (20 queries $\times$ 12 year slices $\times$ 50 results)	–
C Merge and rank	deduplicated, gated, and ranked candidates	1,846
D Abstract screening	1,030 kept (56%)	1,030
E PDF acquisition	HTTP-accessible PDFs retrieved automatically; others ingested manually	918
F Conversion and cleanup	917 cleaned markdown documents (1 conversion failure)	917

Table 1: Stage-by-stage outcome of lower-mantle EOS curation run. Stage letters refer to Fig. 1 .

Model	Detection	Unmatched	Accuracy	Precision	Recall	F1	Cost (USD/paper)
	Record level (%)		Field level, all fields (%)
Claude Fable 5	89.1	10.0	95.7	95.9	99.4	97.6	$\sim$ 0.32
Gemini 3.1 Pro Preview	91.8	7.3	91.7	96.6	94.0	95.2	$\sim$ 0.10
GPT-5.6 Sol High	95.5	23.6	95.3	96.6	98.2	97.4	$\sim$ 0.21
Qwen3.6 27B	53.6	1.4	89.5	91.4	97.3	94.3	0

Table 2: Evaluation of extraction stage against a manually curated ground truth of 220 EOS records from 50 papers. Detection is share of ground-truth records recovered; unmatched-record rate is share of extracted records without a ground-truth counterpart, relative to ground-truth count. Field-level metrics aggregate six extracted fields over all matched records. Cost is mean API charge (USD) per paper for a single extraction pass over these 50 papers; commercial models were run as reasoning models, which increases token use relative to non-reasoning chat models. Local Qwen3.6 27B deployment incurs no API charge.

Figure 3: Field-level validation of production extraction model (Claude Fable 5): precision, recall, and F1 per extracted field, computed over matched records.

Figure 4: Composition of curated EOS database of lower-mantle and lower-mantle-relevant high-pressure mineral phases (1,334 entries from 205 papers). (a) Entries for ten most frequent phases, divided into source-reported values (new determinations by extracting paper) and citation-reported values (values a paper quotes from earlier literature); four entries of unknown provenance are not shown separately. (b) Entries per publication year, divided by method class assigned from reported method description (experimental, computational, or unspecified).

Figure 5: Distributions of extracted EOS parameters for pure endmember bridgmanite (MgSiO 3 ), periclase (MgO), davemaoite (CaSiO 3 ), and post-perovskite (MgSiO 3 ). Solid-solution compositions, citation-reported values, high-temperature fits ( $T_{\mathrm{ref}}>310$ K), assumed or fixed parameters, and adiabatic moduli are excluded. Experimental entries (left, blue) are room-temperature static-compression values; computational entries (right, orange) include 0 K static-lattice fits and other calculations with $T_{\mathrm{ref}}\leq 310$ K, with no thermal adjustment between these states. Rows are $V_{0}$ on a common molar-volume basis, $K_{0}$ , and fitted $K^{\prime}$ , with $K^{\prime}=4$ marked (dashed line).

Figure S1: Detailed view of literature discovery (stages A–C of main-text architecture figure). Research goal is expanded into a portfolio of search queries (A), each query is executed against OpenAlex over publication-year slices (B), and overlapping results are merged, quality-gated, and ranked into a single table of candidate papers (C).

Figure S2: Detailed view of screening and full-text preparation (stages D–H of main-text architecture figure). Candidates are screened at abstract level (D), PDFs are acquired (E) and converted to cleaned markdown (F), a regular-expression pre-extraction gate skips documents with no EOS-parameter signals (G), and remaining papers are screened at full-text level (H).

Field	Description
doi	Digital Object Identifier of the source paper.
title	Paper title.
year	Publication year.
phase	Mineral or high-pressure phase name. Unspecified if the paper does not name a phase.
composition	Chemical composition as reported.
structure	Crystal structure, when stated.

Table S1: Column dictionary for Zenodo xlsx deposit.

LLM Information Extraction

Large language model information extraction converts unstructured documents into structured entities, relations, events, and records.

See all 238 papers

Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models
Aug 19, 2026Valentin Romanov, Monique Bax, Steven NiedererLLM Information Extraction LLM Reliability
National Heart & Lung Institute, Faculty of Medicine, Imperial College London, United Kingdom · Cambridge Stem Cell Institute, JeDrey Cheah Biomedical Centre, Cambridge Biomedical Campus, University of Cambridge, Cambridge, United Kingdom · Department of Medicine, University of Cambridge, Cambridge, United Kingdom +1
SciCurateBench: Curating Experimental Data from Heterogeneous Scientific Literature with Verbal-Action Agents
Oct 9, 2026Minuk Ma, Kaiyun Guo, Jiarui DingAI-Assisted Scientific Research AI Agent Benchmarks
University of British Columbia
Material Database Agent: A Multimodal Agentic Framework for Scientific Literature Mining
May 5, 2026Achuth Chandrasekhar, Omid Barati Farimani, Radheesh Sharma Meda +1Multi-Agent LLM Systems Document Information Extraction
Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA · Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA · Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA +2

Aug 19, 2026J/K move · Enter open · S save

Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models

Valentin Romanov, Monique Bax, Steven Niederer

National Heart & Lung Institute, Faculty of Medicine, Imperial College London, United Kingdom · Cambridge Stem Cell Institute, JeDrey Cheah Biomedical Centre, Cambridge Biomedical Campus, University of Cambridge, Cambridge, United Kingdom · Department of Medicine, University of Cambridge, Cambridge, United Kingdom+1

LLM Information Extraction LLM Reliability LLM Prompting AI Agents for Scientific Discovery

Open paper AskPDF arXiv

Accurately extracting nuanced, contextualized data from research articles is laborious and time intensive. Here, we investigate the performance of frontier, browser-based large language models (LLMs) to extract highly contextualized information. We demonstrate four escalating workflows, 1) given an expert curated prompt and research articles, most frontier LLMs perform well at data extraction, however can struggle with interpreting scientific context and nuance, 2) given simple instructions, LLMs can author their own prompts which were almost as eNective as expert-written prompts, 3) autonomous discovery of research literature was diNicult, agents either missed or hallucinated references, and 4) LLMs can create new datasets from published guidelines that closely match human-expert judges, but still require a human-in-the-loop. Together, these findings define an auditable division of labour in which experts specify the evidence standard, models cross-check repeated extractions and researchers resolve disputed cases, providing a practical route to scaling scientific data curation without relinquishing expert oversight.

Abstract

LLM Information Extraction

Explore similar work

Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models

SciCurateBench: Curating Experimental Data from Heterogeneous Scientific Literature with Verbal-Action Agents

Material Database Agent: A Multimodal Agentic Framework for Scientific Literature Mining