Organizations: DP Technology · Peking University · The Hong Kong University of Science and Technology · School of Mathematical Sciences, Peking University
Reaction data underpin machine learning for reaction prediction, condition recommendation, and retrosynthetic planning, yet extracting these data from literature figures still requires substantial manual effort. Many figures encode multiple experiments through shared templates, substituent tables, and product grids. Recovering concrete reaction records therefore requires molecular recognition, association of evidence with individual experimental entries, and reconstruction of indirectly specified participants. We present RxnMiner, a multimodal agent system that combines an expert-informed extraction procedure with a chemistry toolbox to coordinate these operations. The agent reads molecular structures, resolves shared and entry-specific evidence, and instantiates participants from templates to assemble source-linked reaction records. We also introduce ChemRxn-Bench, a benchmark of 603 reaction figures with 8,547 manually annotated and reviewed reaction records, evaluated under a common protocol for exact molecular identity and reactant-product pairing. On this benchmark, with Qwen3.8-Flash-Next as the backbone, RxnMiner raises stereochemistry-aware reaction macro-F1 from 21.0% under direct prediction to 71.0%, compared with 25.0% for the strongest existing extraction system. Across five backbones, it also outperforms the same base agent without the extraction procedure and chemistry toolbox by 9.4 to 25.5 points. Applied without human intervention to 10,107 literature figures, RxnMiner produces a silver corpus of 88,202 source-linked reaction records.
Figures & tables
Figure 1: From compositional evidence to concrete reaction records. An optimization table specifies eight experimental entries through a shared reaction scheme, catalyst definitions, table rows, and footnotes. The numbered steps indicate recognition, association, and reconstruction. Entry 7 is expanded to illustrate the target record, including its molecular participants, conditions, outcomes, and JSON representation.
Figure 2: Composition of ChemRxn-Bench . (a) Difficulty tiers defined by Qwen3.8-Flash-Next performance. (b) Distribution of concrete reaction records per figure. (c) Coverage of eight reaction types with representative transformations. Percentages use all figures as the denominator; multiple tags may apply to each figure. X denotes a heteroatom. (d) Representative layout examples. Tier definitions and source attributions, including licenses and modifications to the examples in (d), are provided in Appendix A.2 .
Figure 3: Workflow of RxnMiner . The agent combines visual reasoning and chemical tools for recognition, entry-level association, and template-based reconstruction. Recovered participants are assembled into source-linked reaction records, with stereochemical checks and visual review supporting validation.
System / Configuration
Precision
Recall
F1
F1 w/o stereochemistry
Existing extraction systems
ChemEAGLE
25.1
49.7
25.0
35.0
OpenChemIE
9.8
8.1
6.0
9.0
MERMaid
5.8
15.2
6.0
8.0
RxnScribe
6.1
3.4
2.0
3.0
MinerU.Chem
7.1
2.6
2.0
3.0
Table 1: End-to-end reaction extraction on ChemRxn-Bench . Scores are percentages, macro-averaged over all 603 figures. Precision, Recall, and F1 include stereochemistry; the final column omits it. The best results are highlighted in bold . The vanilla LLM reference is our primary locally deployed backbone.
Reaction F1 (%)
Δ Reaction F1 (%)
Backbone
Vanilla LLM
Bare Pi
RxnMiner
ΔPi
ΔVanilla
Qwen3.8-Flash-Next
21.0
59.6
71.0
+11.4
+50.0
DeepSeek V4.1 Flash
16.0
54.6
64.0
+9.4
+48.0
GLM 5.3 Flash
31.0
15.5
41.0
+25.5
+10.0
Kimi K3
40.0
57.7
69.0
+11.3
+29.0
Doubao Seed 2.1 Pro
34.0
26.5
47.0
+20.5
+13.0
Table 2: Backbone-matched comparison of Vanilla LLM, Bare Pi, and RxnMiner . Scores are stereochemistry-aware macro-F1 (%) on all 603 figures. The best results are highlighted in bold . ΔPi and ΔVanilla denote gains over Bare Pi and Vanilla LLM, respectively, in percentage points.
Figure 4: Recovering reaction information at scale. (a) Applying RxnMiner to the RxnCaption-11k training split produces source-linked concrete reaction records with molecular SMILES and provenance. (b) Corpus attributes, record-field coverage, and the distribution of extracted reactions per figure. (c) An example of instantiating a shared template, R-group assignments, and reaction conditions into explicit reaction records, with interfaces for source-based inspection.
Appendix figures & tables30 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Provenance of the 603 benchmark figures. Journal figures (398, 66.0%) span 25 journals, led by Synfacts; the ChemEAGLE (145, 24.0%) and OpenChemIE (60, 10.0%) collections supply the rest. The right panel lists the 19 journals grouped as other journals (75 figures).
File
Contents
Visible
instruction.md
Task statement, output schema, and scoring rules
Yes
environment/
Container definition: the shared base image plus the figure
Yes
task.toml
Metadata (source, layout, reaction types, record count, difficulty) and resource limits
No
tests/
Reference record and verifier script
No; mounted after the run
solution/
Reference answer, used to check that the verifier returns full credit
No
Appendix
Table 3: Contents of one ChemRxn-Bench task in the Harbor format. The last column states whether the file is visible to the system under evaluation.
Axis
Classes, figures (share)
Layout
Markush template with examples 425 (70.5%); table 161 (26.7%); single-step reaction 6 (1.0%); complex figure 6 (1.0%); multistep and total synthesis 5 (0.8%)
Reaction type
C–C bond formation 230 (38.1%); cyclization 211 (35.0%); C–X or X–X bond formation 197 (32.7%); other 74 (12.3%); C=C or C ≡ C bond formation 42 (7.0%); oxidation 41 (6.8%); rearrangement 39 (6.5%); reduction 39 (6.5%)
Table 4: Coverage of the 603 benchmark figures. Counts are figures, with shares of 603. Reaction types are multi-label.
Figure 6: Extraction operations illustrated through compositional reaction figures. (a) An optimization table requires associating a selected entry with the shared reaction scheme, catalyst definitions, and footnotes, and instantiating concrete catalysts from a scaffold and aryl fragments. (b) A product scope requires recognizing molecular connectivity, stereochemical cues, and abbreviated groups, and combining each product with the shared transformation to recover its undrawn reactants. Together, the examples illustrate recognition, association, and reconstruction rather than a measured agent run. Adapted from Nature Communications (2018), doi:10.1038/s41467-018-05375-2, and Nature Communications (2024), doi:10.1038/s41467-024-45687-0, under CC BY.
Figure 7: System answers, common adaptation, and retained evidence. Left: an example JSON answer contains reactant and product SMILES, conditions, and yield. Center: the adaptation, canonicalization, and matching path shared by every system (Section 3.3 ). Right: the richer record retains identifiers, template inheritance, substituent bindings, and molecular provenance. The rule-based score evaluates molecular participants and their pairing; conditions and outcomes are assessed only by the secondary judge, and the other retained fields are not evaluated.
Content
Stored information
Scored
Reactants
Molecular structures as SMILES
Yes, with pairing
Products
Molecular structures as SMILES
Yes, with pairing
Conditions
Attached reaction conditions
Judge only
Outcomes
Metrics and their roles, including yields
Judge only
Identifiers
Printed compound or entry identifiers
No
Provenance
Source boxes and template bindings
No
Appendix
Table 5: Semantic contents of the record representation. Names describe field meanings rather than literal JSON keys. Scoring refers to the rule-based reaction metrics and the secondary LLM judge (Section 3.3 ).
Stage
Tool
Function
Evidence
detect
Locate and read every drawn structure in the figure; return Extended SMILES, boxes, and confidences with an overlay for inspection
ocsr_detect
Read the drawn structures in a given image, such as a crop the full-figure pass missed
Structure
reverse_rgroup
Recover the substituent bindings that map a template onto one concrete product
fill_rgroup
Instantiate a template from a substituent map, preserving tetrahedral and double-bond stereochemistry
reverse_assemble
Derive every concrete reactant of one entry from its product and the reactant templates
Verification
verify_smiles
Audit a structure’s validity, stereochemistry, and Extended-SMILES labels, with an RDKit rendering for comparison
Appendix
Table 6: The chemistry toolbox of RxnMiner . Tools are grouped by the workflow stage that calls them. All tools are deterministic given their inputs except the two recognition tools, which call the MolParser OCSR model ( Fang et al., 2025 ) .
Model
Reactants
Products
Conditions
Yield
Selectivity
RP
RR
RF1
Overall
Direct MLLM prediction
Claude Opus 5
88.95
83.79
91.82
93.96
81.33
61.38 (81.91)
68.66 (86.29)
60.98 (81.64)
88.32
GPT-5.6-Sol
79.11
66.94
93.13
94.59
88.31
43.60 (62.32)
51.29 (68.98)
43.09 (61.51)
81.11
Gemini 3.1 Pro
75.06
59.97
91.12
94.20
88.27
36.44 (56.52)
43.87 (62.19)
35.84 (56.07)
77.04
Qwen3.8-Flash-Next
64.77
49.89
91.35
94.72
91.33
24.31 (38.46)
36.27 (51.67)
24.55 (38.67)
71.31
DeepSeek V4.1 Flash
34.33
27.34
44.88
46.09
35.27
20.06 (25.44)
24.07 (27.29)
19.57 (24.77)
36.50
Appendix
Table 7: LLM-judge evaluation of eight direct, seven RxnMiner , and one existing extraction system configuration. Scores are percentages; parentheses give stereochemistry-blind values. Reactants, Products, Conditions, Yield, and Selectivity are judge image-level category scores; Conditions is the mean of condition-molecule and condition-text scores. RP is judge reaction precision, RR is rule-based reaction recall, RF1 is macro-F1 of per-figure judge reaction F1, and Overall is the judge composite.
Model
RP
RR
RF1
Qwen3.8-Flash-Next
58.6 (65.0)
79.7 (83.2)
59.6 (65.9)
DeepSeek V4.1 Flash
54.1 (59.4)
70.7 (73.1)
54.6 (59.8)
GLM 5.3 Flash
14.9 (19.6)
26.4 (31.7)
15.5 (20.4)
Kimi K3
57.2 (68.3)
76.1 (84.0)
57.7 (68.8)
Doubao Seed 2.1 Pro
26.0 (42.8)
38.0 (54.3)
26.5 (43.5)
Appendix
Table 8: Bare Pi results for five backbones on all 603 figures. RP and RR are macro-averaged rule-based reaction precision and recall; RF1 is macro-F1, the arithmetic mean of per-figure F1. Percentages show strict stereochemistry, with stereochemistry-blind values in parentheses.
All
OpenChemIE
ChemEAGLE
Molecules compared
4,675
2,811
1,864
Both annotations agree
3,446 (73.7%)
2,162 (76.9%)
1,284 (68.9%)
Stereo-only difference
624 (13.3%)
377 (13.4%)
247 (13.3%)
Molecule absent in GT
605 (12.9%)
272 (9.7%)
333 (17.9%)
Skeleton mis-drawn in GT a
98 (2.1%)
26 (0.9%)
72 (3.9%)
Stereo-only differences by type
Appendix
Table 9: Molecule-level audit of the ChemEAGLE ground truth (4,675 molecules).
Figure 8: Representative annotation errors. Blue bonds mark the erroneous substructure; red circles in the GT panels mark the corresponding annotation errors; grey tags name the source image.
Figure 9: Multiple errors within a single scheme ( ejoc.202300707 example_2 ). Six of the fourteen reactions are mis-annotated by ChemEAGLE: six side products (15e/g/i/k/m and 16) are omitted entirely, and one reactant (3i) is given as the wrong enantiomer.
Figure 10: Corrections on the same source figure and backbone. Source crops (left) and products rendered from the output JSON (middle and right). (a) Opus omits an aryl chlorine; RxnMiner retains it. The omission affects both participants in all three table rows (direct: 0/3 matched pairs; RxnMiner : 3/3). (b) Qwen predicts the opposite product configuration ( R instead of S ); RxnMiner agrees with the source. The highlighted atoms locate the two corrections; conditions and outcomes are assessed separately.
Figure 11: Inspecting a complex extraction in the RxnMiner workspace. The source figure, molecular boxes, rendered structures, and condition fields are linked in the blue-violet interface.
Figure 12: A dense scope with complex molecular scaffolds. Opus-RxnMiner recovers 51/51 reference molecular pairs. Four enlarged reactions illustrate reconstructed boron-containing reactants and functionalized products.
Figure 13: Local stereochemistry in a spiro-fused scaffold. Source: ChemEAGLE collection. The boxed example (66% yield) and enlarged crop locate the selected entry. The highlighted phenyl-bearing center is S in the GT and R in the extracted product; connectivity and the remaining stereocenters agree.
Figure 14: Local stereochemistry in a triterpene scaffold. Source: ChemEAGLE collection. The comparison highlights the PivO-bearing C3 center of product 3aq (55% yield), which is S in the GT and R in the extracted product; connectivity and the remaining eight stereocenters agree.
Figure 15: Preserving unspecified stereochemistry. Source: ChemEAGLE collection. The boxed entry 3t provides the source context. The final product agrees with the GT, while the reactant comparison highlights an additional S assignment at a center left unspecified in the GT.
Table 12: Checklist summary for the same real case from ChemEAGLE benchmark. ✓ correct; △ partially correct; × not correctly recovered. SMILES ratios use the final-release molecule-set scorer.
Figure 16: RDE2 output for the selected case.
Figure 17: RxnScribe output for the selected case.
Figure 18: RxnCaption output for the selected case.
Figure 19: RxnID output for the selected case.
Figure 20: MinerU.Chem output for the selected case.
Figure 21: OpenChemIE output for the selected case.
Figure 22: ChemEAGLE output for the selected case.
Screen Tables and Substrate Scopes are information-dense figures widely used in synthetic chemistry literature, yet automated extraction of structured reaction data from them remains challenging. We introduce RxnExBench, a multimodal benchmark for fine-grained reaction-level evaluation. Built from high-impact chemistry journals, it contains 3,565 expert-annotated reactions and 2,361 molecular structures. The dataset adopts a unified schema with 15 reaction roles and 26 core fields, supporting downstream AI-for-Chemistry tasks such as retrosynthetic planning, yield prediction, and reaction condition recommendation. RxnExBench further provides a reasoning-driven difficulty taxonomy, a modular annotation pipeline, and the Oracle-MolRec protocol, which disentangles molecular recognition from higher-level reasoning. Evaluation of 9 state-of-the-art VLMs shows that performance degrades substantially as schema complexity and reasoning demands increase. Under the Oracle-MolRec setting, results further reveal that accurate molecular recognition alone is insufficient, with multimodal integration and chemistry-aware reasoning remaining the primary bottlenecks. We release all data and code to facilitate future research in chemistry-aware multimodal understanding and AI-for-Chemistry dataset construction. The dataset and code will be released at https://github.com/opendatalab/RxnExBench.
Jiaxing Sun, Junyuan Gao, Ruikang Zhang +16
Shanghai Artificial Intelligence Laboratory · Peking University · South China Normal University +7
High-quality structured organic reaction data are essential for developing artificial intelligence for chemistry (AI4Chem), yet much of this knowledge remains dispersed across patent text, images, and reaction schemes. We present DianShi-RxnDB, a large-scale, fine-grained organic reaction data platform built via a fully automated extraction and normalization pipeline integrating patent text, images, and reaction schemes. Its corpus covers organic synthesis patents from the USPTO and EPO published between 1976 and 2025, yielding approximately 24 million reaction instances, of which approximately 14.8 million (61.7%) pass automated qualification checks. Each instance represents a specific single-step experiment recording participants, roles, quantities, temperatures, reaction times, yields, experimental procedures, and provenance links to source patents. In a manual evaluation of 1,300 sampled qualified instances, the micro-averaged field-level accuracy was 92.95%. A matched comparison with Pistachio further indicated advantages in deduplicated record counts, representation granularity, and field-level exact agreement. The platform provides a Web research workbench for searching, filtering, comparing, and source-verifying records, and a Model Context Protocol (MCP) service offering AI agents composable structured retrieval tools. DianShi-RxnDB is available at https://dianshi.opendatalab.org.cn/ .
In organic chemistry papers and patents, molecular structures, reaction schemes, and experimental conditions are often presented as molecular structure depictions, reaction diagrams, and complex tables or figures. Such information is difficult for general-purpose document parsing systems to directly convert into machine-readable data. This limits data production for organic chemistry knowledge base construction and for AI for Chemistry tasks such as reaction prediction, retrosynthesis, condition recommendation, molecular property prediction, and drug molecule design. This report introduces MinerU-Chem, a document parsing system for organic chemistry literature integrated into the MinerU online platform. Built on top of MinerU's general document parsing pipeline, MinerU-Chem adds five chemistry-specific modules: chemistry relevance filtering, molecular structure detection, molecule identifier extraction, molecular structure recognition, and reaction scheme parsing. Together, these modules convert organic-chemistry-related image regions in documents into a Molecule Summary List and a Reaction Summary List. For molecular structure recognition, MinerU-Chem uses CARBON (Complex Atomic Representation and Bonding Object Notation) as its core representation. CARBON enables recognition results to preserve both the visual layout of the original image and complex chemical semantics, while supporting the export of standard downstream formats such as MolFile and SMILES. On the SMILES-evaluable subset of MolRecBench-Wild (N=2,392), MinerU-Chem's molecular structure recognition module achieves a SMILES exact-match accuracy of 93.02%, outperforming the best evaluated comparison system, GPT-5.6-Sol (74.87%), by 18.15 percentage points. The system has been integrated into the MinerU online platform and is available at https://mineru.net/OpenSourceTools/Extractor .