LitCurate: A Configuration-Driven AI-Assisted Framework for Scientific Database Construction with an Application to Lower-Mantle Equation-of-State Data
Authors: Abin Shakya, Wilson Samuels, Dominica Wilson, Gioia A. Marchi, Israa Draz, Chenxing Luo, Renata M. Wentzcovitch
Organizations: Lamont–Doherty Earth Observatory, Columbia University, Palisades, NY, 10964, United States · Department of Electrical and Computer Engineering, Tuskegee University, Tuskegee, AL, 36088, United States · Department of Industrial Engineering and Operations Research, Columbia University, New York, NY, 10027, United States · Department of Applied Physics and Applied Mathematics, Columbia University, New York, NY, 10027, United States · Department of Earth and Environmental Sciences, Columbia University, New York, NY, 10027, United States
The growing scientific literature contains decades of experimental and computational results that could support data-driven and physics-based modeling, yet much of this infor- mation remains locked in publications and is not readily usable for large-scale analysis or sci- entific software. Building structured databases from the literature is particularly challenging whenrelevantstudiesmustfirstbediscoveredamonglargecollectionsofpapersandreported quantities must be extracted with enough scientific context to remain usable. We present LitCurate, an open-source framework for building scientific databases from the literature using large language models within an auditable, stage-wise curation workflow. LitCurate integratesliteraturediscovery, relevancescreening, full-textprocessing, andstructuredinfor- mation extraction while retaining intermediate results and provenance, allowing researchers to inspect and revise individual stages rather than treating automated curation as a black- box process. We apply LitCurate to construct an equation-of-state database of lower-mantle and lower-mantle-relevant high-pressure mineral phases from experimental and theoretical studies, comprising 1,334 entries from 205 papers. The resulting dataset links reported equation-of-state parameters to mineral phases, compositions, equation formulations, meth- ods, and parameter constraints, and labels values as source-reported or citation-reported when provenance can be determined. The records are available through a searchable web application. By connecting scientific literature to traceable, machine-readable data, LitCu- rate provides a reusable approach for transforming accumulated literature into resources for scientific analysis and computational modeling.
Figures & tables
Figure 1: LitCurate architecture. Pipeline advances through ten stages (A–J), from LLM-assisted query planning and literature discovery through screening, full-text preparation, and schema-guided extraction, to export of a merged structured database. Each stage writes intermediate artifacts and records its status in a per-run ledger (bottom layer), so individual stages can be inspected, re-run, or resumed. Domain knowledge (research goals, filter criteria, extraction schemas, and prompts) is supplied via YAML configuration and associated files (top-left), allowing stage machinery to be reused across disciplines.
Figure 2: Schema-guided extraction (stage I). Cleaned markdown is combined with a task prompt and a declared record schema; LLM returns structured JSON that is validated against that schema and stored as an envelope, then merged into exported EOS database.
Stage
Outcome
Papers retained
A Query planning
20 queries generated from the research goal
–
B Literature search
12,000 query-result records (20 queries × 12 year slices × 50 results)
Table 1: Stage-by-stage outcome of lower-mantle EOS curation run. Stage letters refer to Fig. 1 .
Record level (%)
Field level, all fields (%)
Model
Detection
Unmatched
Accuracy
Precision
Recall
F1
Cost (USD/paper)
Claude Fable 5
89.1
10.0
95.7
95.9
99.4
97.6
∼ 0.32
Gemini 3.1 Pro Preview
91.8
7.3
91.7
96.6
94.0
95.2
∼ 0.10
GPT-5.6 Sol High
95.5
23.6
95.3
96.6
98.2
97.4
∼ 0.21
Qwen3.6 27B
53.6
1.4
89.5
91.4
97.3
94.3
0
Table 2: Evaluation of extraction stage against a manually curated ground truth of 220 EOS records from 50 papers. Detection is share of ground-truth records recovered; unmatched-record rate is share of extracted records without a ground-truth counterpart, relative to ground-truth count. Field-level metrics aggregate six extracted fields over all matched records. Cost is mean API charge (USD) per paper for a single extraction pass over these 50 papers; commercial models were run as reasoning models, which increases token use relative to non-reasoning chat models. Local Qwen3.6 27B deployment incurs no API charge.
Figure 3: Field-level validation of production extraction model (Claude Fable 5): precision, recall, and F1 per extracted field, computed over matched records.
Figure 4: Composition of curated EOS database of lower-mantle and lower-mantle-relevant high-pressure mineral phases (1,334 entries from 205 papers). (a) Entries for ten most frequent phases, divided into source-reported values (new determinations by extracting paper) and citation-reported values (values a paper quotes from earlier literature); four entries of unknown provenance are not shown separately. (b) Entries per publication year, divided by method class assigned from reported method description (experimental, computational, or unspecified).
Figure 5: Distributions of extracted EOS parameters for pure endmember bridgmanite (MgSiO 3 ), periclase (MgO), davemaoite (CaSiO 3 ), and post-perovskite (MgSiO 3 ). Solid-solution compositions, citation-reported values, high-temperature fits ( Tref>310 K), assumed or fixed parameters, and adiabatic moduli are excluded. Experimental entries (left, blue) are room-temperature static-compression values; computational entries (right, orange) include 0 K static-lattice fits and other calculations with Tref≤310 K, with no thermal adjustment between these states. Rows are V0 on a common molar-volume basis, K0 , and fitted K′ , with K′=4 marked (dashed line).
Figure S1: Detailed view of literature discovery (stages A–C of main-text architecture figure). Research goal is expanded into a portfolio of search queries (A), each query is executed against OpenAlex over publication-year slices (B), and overlapping results are merged, quality-gated, and ranked into a single table of candidate papers (C).
Figure S2: Detailed view of screening and full-text preparation (stages D–H of main-text architecture figure). Candidates are screened at abstract level (D), PDFs are acquired (E) and converted to cleaned markdown (F), a regular-expression pre-extraction gate skips documents with no EOS-parameter signals (G), and remaining papers are screened at full-text level (H).
Field
Description
doi
Digital Object Identifier of the source paper.
title
Paper title.
year
Publication year.
phase
Mineral or high-pressure phase name. Unspecified if the paper does not name a phase.
composition
Chemical composition as reported.
structure
Crystal structure, when stated.
Table S1: Column dictionary for Zenodo xlsx deposit.
Accurately extracting nuanced, contextualized data from research articles is laborious and time intensive. Here, we investigate the performance of frontier, browser-based large language models (LLMs) to extract highly contextualized information. We demonstrate four escalating workflows, 1) given an expert curated prompt and research articles, most frontier LLMs perform well at data extraction, however can struggle with interpreting scientific context and nuance, 2) given simple instructions, LLMs can author their own prompts which were almost as eNective as expert-written prompts, 3) autonomous discovery of research literature was diNicult, agents either missed or hallucinated references, and 4) LLMs can create new datasets from published guidelines that closely match human-expert judges, but still require a human-in-the-loop. Together, these findings define an auditable division of labour in which experts specify the evidence standard, models cross-check repeated extractions and researchers resolve disputed cases, providing a practical route to scaling scientific data curation without relinquishing expert oversight.
Valentin Romanov, Monique Bax, Steven Niederer
National Heart & Lung Institute, Faculty of Medicine, Imperial College London, United Kingdom · Cambridge Stem Cell Institute, JeDrey Cheah Biomedical Centre, Cambridge Biomedical Campus, University of Cambridge, Cambridge, United Kingdom · Department of Medicine, University of Cambridge, Cambridge, United Kingdom +1
Scientific research increasingly depends on curating experimental data from literature. Scientific data curation requires verifying exact experimental conditions (e.g., specific assays, clinical outcomes, and data availability) across long, heterogeneous files, which is a highly time-consuming process. Existing benchmarks evaluate only broad relevance on short queries, failing to capture this complexity. We introduce SciCurateBench, a benchmark of 200 expert-annotated papers from major life-science journals that challenges systems to verify conjunctive biological experimental criteria from raw, unprocessed scientific literature. We show that plain LLMs and popular reasoning, tool-using, and feedback-based agents all struggle with this task. To study what makes the task tractable, we further propose LangPoli, an integrative system in which a policy agent generates natural language action guidelines that actor agents convert into executable tools. LangPoli showed strong performance on both Paper Classification (83.16% F1 vs. 58.22% for its base LLM) and Structured Information Retrieval, and also improves re-ranking on the public benchmarks of LitSearch, BEIR, and TREC. We further explore curriculum transfer, a method that distills generalizable strategies from solved tasks to accelerate the solving of related problems. We hope this work facilitates research in truly practical scientific AI assistance. The anonymized pre-release can be found from https://anonymous.4open. science/r/scicuratebench-D885/.
Materials science workflows rely on structured and unstructured data from the vast body of available scientific literature. However, most of the experimental details remain buried in text, tables, graphs and figures. Thus, constructing databases that incorporate this data is a manual, time-consuming, and hard-to-scale process. Multimodal large language models have made it feasible to extract information from text and scientific figures with high speed and accuracy. This opens the possibility of an AI system that can create production-scale material databases. Material Database Agent (MDA) is a modular, multi-agent system architecture for converting research literature into structured databases. MDA accepts article PDFs as input, which are subsequently processed in parallel into markdown files and figures. Multiple sub-agents read these markdown files and figures in parallel to assemble sub-databases for each paper. These sub-databases are then compiled into a single tabular database by an agent. As opposed to using either a rule-based approach or a single-pass pipeline for extracting information, MDA is a specialized architecture for transforming the literature into a database in the field of materials science. More generally, this study provides a basis for positioning multimodal agentic information extraction as a viable means for constructing next-generation scientific databases from the primary literature.
Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA · Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA · Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA +2