Cloudra: Geometric Reasoning and Scientific Validation for Autonomous Reaction Mechanism Exploration

Chengyi Yang; Qianxin Zhou; Yang Li

cs.MANov 2, 2026

Cloudra: Geometric Reasoning and Scientific Validation for Autonomous Reaction Mechanism Exploration

Authors: Chengyi Yang, Qianxin Zhou, Yang Li

Organizations: College of Chemistry and Pharmaceutical Engineering, Changsha University of Science and Technology, Changsha 410114, China · State Key Laboratory of Photocatalysis on Energy and Environment, College of Chemistry, Fuzhou University, Fuzhou 350002, China

Abstract

Autonomous reaction-mechanism exploration requires proposing reaction steps, preparing three-dimensional structures, and determining which reactant and product a computed saddle point connects. Large language model (LLM) agents can misjudge spatial relationships and confuse a computed structure with the species or step they intended to study. We present Cloudra, a multi-agent framework that constrains mechanistic output through a Delivery DAG of chemical objects. Species nodes carry geometries and bonding topology; elementary-step edges carry transition states and reaction connections between species. Cheminformatics algorithms establish molecular identities and atom correspondences. Deterministic routines measure and transform structures to prepare candidate geometries. Frequency and IRC calculations determine the stationary points found and the species they connect. Agents use these objects to test proposed steps, compare pathways, and revise the mechanism. CloudraBench evaluates the framework's end-to-end mechanism-calculation capability across 50 tasks covering elementary reactions, multistep mechanisms, stereochemical competition, and computational method limitations. Across 200 runs under four workflow configurations, the primary Guided workflow achieves a mean composite score of 0.805, compared with 0.584 for the same-model Standard baseline. On Hard-tier tasks, the Guided mean increases from 0.689 under Standard to 0.958. Under Guided, the self-consistency component reaches its maximum score in 24 of the 40 non-Impossible tasks, compared with seven under Standard. Behavioral analyses examine how the agents use bond and dihedral measurements, optimized structures, and IRC endpoints to revise local mechanistic assignments.

Figures & tables

Figure 1: Physical targets in a structured literature sample of 85 chemical AI agent systems: 39 materials/surface, 23 finite-molecule/complex, 12 soft-matter/biomolecular, and 11 cross-class systems.

Figure 2: Documented tasks in the finite-molecule/complex class (panel 1 of 2; 23 chemical AI agent systems in total). T: structure construction/editing; E: static energy/properties; O: geometry optimization; F: vibrations/thermochemistry; R: explicit pathways/TS; D: molecular/quantum dynamics. Filled cells denote positive evidence; blanks leave capability coverage unspecified.

Figure 3: Chemical AI agent systems in the finite-molecule/complex class, continued (panel 2 of 2), using the codes in Figure 2 . Cloudra (this work) is benchmarked at the bottom.

Figure 4: Cloudra’s mechanism exploration workflow. Atom mapping and conformer sampling prepare reactant and product structures; ComputeDB and the scheduler manage the calculations. Mode diagnostics measure motion on monitored atom pairs. Bidirectional IRCs start from the same transition state, and their outer frames undergo endpoint optimization and frequency analysis. The resulting structures are compared with the species declared in the Delivery DAG. The Supervisor reviews the graph and identifies calculations to complete and assignments to resolve.

Figure 5: CloudraBench task composition and research themes. (a) Counts and evaluation focus of the four task tiers. (b) Two-dimensional PCA, computed with scikit-learn [ 36 ] , of BAAI/bge-m3 embeddings [ 35 ] of all 50 original task names. E1, NB1, HA1, and IA1 are the four tasks used in the repeated-run experiment. Colors and marker shapes encode the predefined task tiers.

Figure 6: Two requested steps in the archived Guided/NI1 study: methylamine addition to formaldehyde and carbinolamine dehydration. Structures and electronic energies come from this run’s B3LYP-D3(BJ)/def2-SVP gas-phase calculations. Each saddle has one imaginary frequency, and its short IRC branches lead through endpoint optimization/frequency analysis to minima with zero imaginary frequencies. Both energy panels use RC as the common reference.

Figure 7: P2 benchmark outcomes under the archive-adjudicated track. (a) Guided and the same-backbone Standard baseline across all 50 tasks and each tier. (b) All 50 paired task outcomes. (c) Mean paired score differences with 95% percentile intervals from 100,000 task-level bootstrap resamples. Each task has one run per configuration; Easy contains 20 tasks and the other tiers contain 10 each.

Figure 8: CloudraBench score layers across all 200 organized runs. (a–c) Means of Graph, Self-consistency, and Accuracy in each arm and tier, using 50 tasks per arm overall, 20 Easy tasks, and ten tasks in each other tier. (d) Guided–Backbone mean differences decomposed with the fixed weights 0.1/0.3/0.6, using all 50 pairs, ten Normal pairs, and ten Hard pairs. The component values come from each run’s final evaluation; their weighted sum gives the published composite score to the reported precision.

Figure 9: Runtime graph delivery and final scientific support in P2. (a) Original root mechanism graphs across 50 tasks per arm: Guided 50, Backbone 43, Record 50, and Brief 50. The hatched segment marks seven additional Backbone graphs assembled offline from the runs’ own research materials; the organized set contains 50 graphs per arm. (b) Final continuous self-consistency (SC) scores for all 40 non-Impossible tasks per arm, separating SC = 1, partial positive credit, and SC = 0. Full-score counts are 24, 7, 16, and 20; Backbone has four zero-score tasks. (c) Organized composite scores for the seven Backbone tasks whose root graphs were reconstructed offline, ranging from 0.0333 to 0.3425 (mean 0.1537).

Figure 10: Research elapsed time and computational investment. (a) Twelve P1b four-task batches at concurrency two, three per arm; lines mark means. (b) Cumulative QM process durations for the 15 P2 tasks shared by all arms. (c) Mean completed, failed, and cancelled jobs in that cohort. (d) Same-task Backbone-to-Guided changes in investment and final score; diamonds mark offline-organized Backbone graphs. Job-hours sum process walltime; panel a measures batch latency. Backbone/M1 effort is reconstructed from native CPU time divided by four.

Figure 11: Complete CloudraBench results for all 50 tasks and four configurations. The aligned blocks show composite, Graph, Self-consistency, and Accuracy on the same 0–1 scale; all 800 cells are present. Values are rounded to two decimals; the source table retains full precision.

Figure 12: Energy references and pathway assignments in matched Guided–Backbone tasks. (a) HA8 barriers use a stable-minimum or reactive-conformer reference and are compared with campaign Gold. (b) Both HH1 runs report 23.48 kcal/mol with similar accuracy but different self-consistency scores. (c) NU6’s recorded energy difference is 18.836772 kcal/mol: Guided reports the cost of reaching the concerted saddle region and leaves the target ring-closure saddle unresolved; Backbone reports 18.837 as the second-step ring-closure barrier.

Figure 13: Impossible-tier outcomes after offline organization and scoring. (a–c) Composite score, self-consistency (SC), and accuracy for ten tasks and four arms, using a shared 0–1 color scale. Backbone denotes Standard. Brief averages 0.603 and Guided 0.505. (d) Guided scores higher on 5 tasks, Brief on 4, and 1 tie. (e) The +0.09788 mean difference comprises accuracy +0.12000, Graph $-0.00661$ , and SC $-0.01550$ after weighting. The paired task-bootstrap 95% interval is $-0.160$ to $+0.361$ .

Figure 14: Molecular-identity judgments under changes in molecular representation and option order. Each of the 16 molecular pairs has four input conditions. Circles and diamonds indicate correct and incorrect judgments, respectively; the letters retain the model’s actual answers. Shaded regions denote identical-string conditions. The right panel shows the pair-14 connectivities CC1CC=C1 and C=C1CCC1 , both C 5 H 8 , and the meanings of the answer options.

Panel of Figure 17 — Figure 17: Target-specific geometry operations in E1. Newman projections retain the directions of the original candidate coordinates viewed down C2 $\to$ C3. H7 is the specified transferring atom. Candidate A and candidate B give target angles of 59.4° and 179.4°, respectively. The operation sequence is shown alongside the projections.

Figure 20: Intervention on a single reported distance. A one-digit edit changes the C2–C4 distance from 2.029 to 1.029 Å while O3–H8 remains 1.248 Å. The original-input residual state at token position 53 is inserted into the changed-input run at a specified layer. Token positions are zero-based.

Figure 25: Completion requests and DAG feedback in the original IA1 trajectory. A report with unresolved evidence is followed by a completion request, a response identifying missing short_irc proof, feedback to the model, and a revised partial-completion request.

Workflow	MAE	RMSE	Median $\|\Delta\|$
Guided	1.17	2.86	0.057
Standard	1.54	4.64	0.057
Record	1.58	5.71	0.073
Brief-errors	1.13	2.31	0.099

Table 1: Uncapped electronic-barrier errors for the same 26 reference edges in all four configurations, in kcal/mol.

Appendix figures & tables34 assets

Supplementary material from the paper’s appendix.

Appendix

Design	Repeated-task study (P1b)	Full-benchmark study (P2)
Task set	E1, NB1, HA1, IA1	All 50 CloudraBench tasks
Backbone	DeepSeek-v4.1-flash (1017)	DeepSeek-v4.1-flash (1017); official DeepSeek chat-completions API
Configurations	Guided, Standard, Record, Brief-errors	Guided, Standard, Record, Brief-errors
Repetitions	Three per task and configuration	One retained run per task and configuration
Task-level runs	$4\times 4\times 3=48$	$50\times 4=200$
Elapsed-time unit	Four-task batch at concurrency two; three batches per configuration	Computational effort on the common 15-task subset

Appendix

Table S1: Experimental units in the two studies.

Configuration	Delivery-graph interface	Required physical validation	Diagnostic presentation
Guided	Explicit	Retained	Detailed
Standard	Explicit DAG tools hidden	Removed	Neutral computation feedback
Record	Available tools and records retained	Removed	Recording retained
Brief-errors	Retained	Retained	Shortened

Appendix

Table S2: Operational definition of the four workflow configurations. Brief-errors is abbreviated as Brief in figures and tables; Standard is also labelled Backbone.

Item	Recorded value
Deployed model	DeepSeek-v4.1-flash (1017)
Provider and API alias	DeepSeek; deepseek-v4-flash in all eight snapshots and their request/completion telemetry
Endpoint and transport	https://api.deepseek.com ; streaming chat completions
Role slots	Coordinator and follower use the same provider, model identifier, and request settings
Reasoning setting	Requested reasoning_effort=minimal
Temperature	1.0 in the archived run-start settings

Appendix

Table S3: Frozen P2 configuration and campaign metadata. Request settings describe the harness configuration; telemetry counts are provided in the accompanying p2_model_metadata.json .

Evidence stage	Contribution
Verified first-order saddle: exactly one imaginary frequency	0.35
IRC starting geometry consistent with the selected saddle	0.15
Logs linking both actual IRC terminal frames through Opt/Freq to the declared minima	0.50

Appendix

Table S4: Graded self-consistency contribution for an accessible transition-state step. The endpoint contribution is 0.25 for one verified side and 0.50 for both sides.

ID	Chemical task	Stated conditions and task-specific research target
D1	butadiene + ethylene Diels-Alder (parent DA) Charge 0; multiplicity 1.	Conditions: gas phase, 487–648 ${}^{\circ}\mathrm{C}$ kinetics. Target: Cycloaddition pathway, stationary points, and electronic activation barrier.
D3	cyclopentadiene dimerization (self Diels-Alder) Charge 0; multiplicity 1.	Conditions: gas/liquid phase, slow at room temperature. Target: Dimerization pathway, stationary points, and electronic activation barrier.
D4	cyclopentadiene + acrolein Diels-Alder Charge 0; multiplicity 1.	Conditions: gas phase or hydrocarbon solvent, heated. Target: Cycloaddition pathway, stationary points, and electronic activation barrier.
E1	F- + CH3CH2Cl elimination -¿ ethylene + HF + Cl- Charge -1; multiplicity 1.	Conditions: gas phase. Target: Elimination pathway from the pre-reaction complex, stationary points, and electronic activation barrier.
F1	BH4- + formaldehyde -¿ borane-methoxide adduct [CH3OBH3]- Charge -1; multiplicity 1.	Conditions: gas phase. Target: Hydride-transfer pathway to the borane–methoxide adduct, stationary points, and electronic barrier.
F4	BH3 + formaldehyde -¿ methoxyborane via Lewis adduct Charge 0; multiplicity 1.	Conditions: gas phase. Target: Lewis-adduct-mediated conversion to methoxyborane, with the investigated species and elementary steps.

Appendix

Table S5: Easy-tier task catalogue. Method set A applies to every task in this table.

ID	Chemical task	Stated conditions and task-specific research target
NB1	malonic acid thermal decarboxylation: malonic acid -¿ acetic acid + CO2 Charge 0; multiplicity 1.	Conditions: gas-phase thermolysis. Target: Identify intermediates and preserve elementary-step continuity through decarboxylation to the final products.
NI1	imine formation: formaldehyde + methylamine -¿ N-methylmethanimine + water Charge 0; multiplicity 1.	Conditions: gas phase; optimal around pH 4-6 (bell-shaped rate profile). Target: Identify the addition/condensation intermediates and connecting elementary steps leading to the imine and water.
NI5	imine formation (aqueous): formaldehyde + methylamine -¿ N-methylmethanimine + water Charge 0; multiplicity 1.	Conditions: aqueous solution, weakly acidic (pH 4-6, bell-shaped rate profile). Target: Resolve the aqueous imine-formation network and relevant proton-transfer steps under the stated conditions.
NQ1	aromatic Claisen rearrangement: allyl phenyl ether -¿ 2-allylphenol Charge 0; multiplicity 1.	Conditions: gas phase; thermal, $\sim$ 200 ${}^{\circ}\mathrm{C}$ . Target: Resolve rearrangement and subsequent conversion to 2-allylphenol, retaining the intermediate network.
NQ5	oxy-Cope rearrangement: 1,5-hexadien-3-ol -¿ hex-5-enal ([3,3] then enol-keto tautomerization) Charge 0; multiplicity 1.	Conditions: gas phase, heated. Target: Resolve the [3,3] rearrangement and enol–keto tautomerization as the investigated network.
NR1	formamide dehydration: formamide -¿ hydrogen cyanide + water Charge 0; multiplicity 1.	Conditions: Gas phase; isolated neutral, closed-shell formamide; no added water, acid, or catalyst. Target: Investigate formimidic-acid intermediates and conformational transitions in the neutral dehydration network.

Appendix

Table S6: Normal-tier task catalogue. Method set A applies to every task in this table.

ID	Chemical task	Stated conditions and task-specific research target
HA1	3-methylcyclobutene ring opening: determine the stereochemical outcome Charge 0; multiplicity 1.	Conditions: gas phase or inert solvent, $\sim$ 150–200 ${}^{\circ}\mathrm{C}$ , thermal. Target: Determine stereochemical products and channel identities; report channel barriers and the requested relative preference.
HA6	trans-3,4-dimethylcyclobutene ring opening: determine the stereochemical outcome Charge 0; multiplicity 1.	Conditions: gas phase or inert solvent, heating, thermal. Target: Determine stereochemical products and channel identities; report channel barriers and the requested relative preference.
HA8	(2E,4Z,6E)-octa-2,4,6-triene thermal 6-pi electrocyclization: determine the stereochemical outcome Charge 0; multiplicity 1.	Conditions: Gas phase; first thermal ground-state electrocyclization only; no subsequent hydrogen shifts or photochemistry. Target: Resolve the product stereochemistry of the first thermal electrocyclization; report investigated channel identities and barriers.
HB1	cyclopentadiene + maleic anhydride Diels-Alder: determine endo/exo outcome Charge 0; multiplicity 1.	Conditions: benzene, room temperature or mild heating. Target: Determine endo/exo products, channel barriers, and relative preference.
HB10	cyclopentadiene + acrylonitrile Diels-Alder reaction: determine both stereochemical pathways Charge 0; multiplicity 1.	Conditions: Gas phase; concerted addition; common reactant-complex reference for the two channels. Target: Compute both endo/exo pathways using a common reactant-complex reference; report identities and barriers.
HD2	cyclohexene oxide + hydroxide: determine the stereochemical outcome of the epoxide opening Charge -1; multiplicity 1.	Conditions: aqueous NaOH, heat. Target: Determine the stereochemistry of the epoxide-opening product from the computed structures.

Appendix

Table S7: Hard-tier task catalogue. Method set A applies to every task in this table.

ID	Chemical task	Stated conditions and task-specific research target
IA1	cyclobutadiene D4h automerization Charge 0; multiplicity 1.	Conditions: gas phase, low temperature (matrix isolation regime). Target: Thermal automerization of rectangular cyclobutadiene through the square transition structure: mechanism and activation barrier.
IA5	cyclobutene disrotatory ring opening Charge 0; multiplicity 1.	Conditions: gas phase, thermal. Target: Thermal disrotatory ring opening of cyclobutene to butadiene: mechanism and activation barrier.
IA7	Bergman cyclization of (Z)-hex-3-ene-1,5-diyne Charge 0; multiplicity 1.	Conditions: gas phase, thermal. Target: Bergman cyclization to p-benzyne: mechanism and activation barrier.
IA9	1,2-dioxetane thermolysis to two formaldehyde Charge 0; multiplicity 1.	Conditions: gas phase, thermal. Target: Thermolysis of 1,2-dioxetane to two formaldehyde molecules: mechanism and activation barrier.
IB9	diazomethane and ethylene to cyclopropane via 1-pyrazoline Charge 0; multiplicity 1.	Conditions: gas-phase thermolysis. Target: Conversion of diazomethane and ethylene to cyclopropane and nitrogen through 1-pyrazoline.
IC3	phenylnitrene open-shell singlet state Charge 0; multiplicity 3.	Conditions: gas phase. Target: Singlet–triplet energy gap and the electronic character of the lowest phenylnitrene singlet state.

Appendix

Table S8: Impossible-tier task catalogue. Method set B applies to every task in this table.

Object	$E$ / Ha	$\Delta E$ / kcal mol -1	$n_{\mathrm{imag}}$	Imaginary frequency / cm -1
Reactant complex $R$	$-210.218065844$	0.00000	0	—
Addition saddle TS1	$-210.173256653$	28.11819	1	$-1481.0579$
Lowest carbinolamine $I$	$-210.237884930$	$-12.43666$	0	—
IRC-side carbinolamine $I^{\prime}$	$-210.236755311$	$-11.72782$	0	—
Dehydration saddle TS2	$-210.145578627$	45.48642	1	$-1796.8431$
Lowest imine–water $P$	$-210.223036629$	$-3.11921$	0	—

Appendix

Table S9: Stationary points in Guided/NI1. Relative electronic energies use the reactant complex as zero;

n_{\mathrm{imag}}

counts imaginary frequencies.

I^{\prime}

and

P^{\prime}

are the actual optimized endpoints of the second step’s IRC.

Workflow	MAE	RMSE	Median absolute error	Maximum absolute error
Guided	1.1676	2.8566	0.0573	10.7826
Standard	1.5423	4.6374	0.0574	21.4713
Record	1.5763	5.7082	0.0731	28.6373
Brief	1.1310	2.3080	0.0985	9.0486

Appendix

Table S10: Uncapped electronic-barrier differences on the 26 reference edges shared by all four workflows, covering 23 positive tasks. All entries are in kcal mol -1 ; every workflow uses the same edge set.

Workflow	MAE	RMSE	Median absolute error	Maximum absolute error
Guided	1.0105	2.7118	0.0388	10.7826
Standard	0.8015	2.2167	0.0424	9.0486
Record	0.3934	0.8609	0.0424	3.0918
Brief	1.2811	2.6213	0.0744	9.0486

Appendix

Table S11: Uncapped differences for the 19 common reference edges with additional corroboration from retained selected Gold input files, covering 17 positive tasks. Units and statistics follow Table S10 .

Tier	Workflow	$n$	Composite	Graph	SC	Accuracy
All	Guided	50	0.8050	0.8851	0.9233	0.7324
	Standard	50	0.5836	0.7774	0.5303	0.5779
	Record	50	0.7038	0.7813	0.7177	0.6839
	Brief	50	0.7781	0.8493	0.8468	0.7320
Easy	Guided	20	0.9125	0.9778	0.9250	0.8954
	Standard	20	0.7044	0.8849	0.5858	0.7336

Appendix

Table S12: Composite and component means for each workflow and task tier.

Comparison	Mean difference	95% interval	W/L/T
Guided–Record	+0.1012	[+0.0396, +0.1695]	28/7/15
Guided–Standard	+0.2214	[+0.1349, +0.3124]	35/8/7
Guided–Brief	+0.0268	[-0.0486, +0.0996]	26/11/13
Record–Standard	+0.1202	[+0.0224, +0.2197]	28/13/9
Record–Brief	-0.0743	[-0.1455, -0.0066]	14/24/12
Standard–Brief	-0.1946	[-0.2990, -0.0914]	11/34/5

Appendix

Table S13: All pairwise composite-score comparisons across the 50 tasks.

Scope	$n$	Composite	$0.1\Delta G$	$0.3\Delta\mathrm{SC}$	$0.6\Delta A$
All	50	+0.221366	+0.010776	+0.117875	+0.092717
Easy	20	+0.208115	+0.009287	+0.101750	+0.097082
Normal	10	+0.163970	+0.009823	+0.114125	+0.040033
Hard	10	+0.268900	+0.006259	+0.135250	+0.127391
Impossible	10	+0.257730	+0.019226	+0.136500	+0.102000
Normal + Hard	20	+0.216435	+0.008041	+0.124688	+0.083712

Appendix

Table S14: Weighted decomposition of the Guided–Standard composite difference.

Workflow	SC $=1$	$0<$ SC $<1$	SC $=0$	Mean SC
Guided	24	16	0	0.9266
Standard	7	29	4	0.5492
Record	16	24	0	0.7633
Brief	20	20	0	0.8439

Appendix

Table S15: Self-consistency distribution over all 40 non-Impossible tasks.

Workflow	Batch 1	Batch 2	Batch 3	Mean
Guided	4.2383	3.8669	5.2489	4.4514
Standard	5.3381	2.5044	4.0306	3.9577
Record	5.4189	2.6203	3.6433	3.8942
Brief	7.2689	3.6475	3.5656	4.8273

Appendix

Table S16: All 12 elapsed-time observations in the repeated-task study, in hours.

Workflow	Job-h total	Job-h/task	Jobs	Failed	Cancelled	Mean $S$
Guided	166.19	11.079	789	114	94	0.8124
Standard	113.32	7.555	419	45	50	0.6495
Record	118.16	7.877	431	62	38	0.7360
Brief	233.85	15.590	883	167	120	0.7273

Appendix

Table S17: Recorded quantum-chemical effort on the 15 tasks shared by all configurations.

Task	Workflow	Composite	Graph	SC	Accuracy
HA8	Guided	0.9625	1.0000	0.8750	1.0000
	Standard	0.6250	1.0000	0.3500	0.7000
HH1	Guided	0.9838	1.0000	1.0000	0.9731
	Standard	0.8338	1.0000	0.5000	0.9731
HI1	Guided	0.9999	0.9992	1.0000	1.0000
	Standard	0.7032	0.3325	0.2333	1.0000

Appendix

Table S18: Complete score profiles for the four scientific-attribution cases.

Task	Guided C	Brief C	Brief $-$ Guided	Guided A	Brief A
IA1	0.1717	1.0000	+0.8283	0.0	1.0
IA5	1.0000	0.9667	-0.0333	1.0	1.0
IA7	0.2583	0.2583	+0.0000	0.0	0.0
IA9	0.3667	0.9333	+0.5666	0.0	1.0
IB9	0.8200	0.7700	-0.0500	0.7	0.7
IC3	0.4000	1.0000	+0.6000	0.0	1.0

Appendix

Table S19: Task-resolved comparison of Brief and Guided in the Impossible tier.

Task	Workflow	Graph	$\mathrm{SC}_{\mathrm{all}}$	$\mathrm{SC}_{\mathrm{Gold}}$	Accuracy	Composite
F4	Guided	0.9167	1.0000	1.0000	1.0000	0.9917
	Standard	0.5176	0.7500	0.0000	0.0000	0.2768
	Record	0.6009	0.7500	0.0000	0.0000	0.2851
	Brief	0.9167	1.0000	1.0000	1.0000	0.9917
G2	Guided	0.8889	0.5000	0.5000	0.0000	0.2389
	Standard	1.0000	0.5000	0.5000	0.8782	0.7769

Appendix

Table S20: Final component scores for the five inspected Easy tasks, rounded to four decimal places. Brief denotes Brief-errors. Accuracy is the final component after reference matching and the specified numerical penalty cap.

Track	Guided	Standard	Record	Brief
Official evaluation values	0.50506	0.24733	0.29250	0.60294

Appendix

Table S21: Impossible-tier mean composite scores after the official target-correspondence Accuracy overlay.

Task	Configuration	$S$	$G$	$\mathrm{SC}$	$A$
D1	Guided	0.8200	1.000000	1.000000	0.700000
D1	Standard	0.8291	0.791667	0.500000	0.999960
D1	Record	0.7992	0.791667	1.000000	0.700000
D1	Brief	0.7450	1.000000	0.750000	0.700000
D3	Guided	0.7450	1.000000	0.750000	0.700000
D3	Standard	0.9247	1.000000	0.750000	0.999496

Appendix

Table S22: Complete Easy-tier results for all four configurations.

S

: composite score;

G

: graph structure;

\mathrm{SC}

: scientific self-consistency score;

A

: accuracy.

Task	Configuration	$S$	$G$	$\mathrm{SC}$	$A$
NB1	Guided	0.9628	0.628571	1.000000	0.999972
NB1	Standard	0.9128	0.628571	0.833333	0.999970
NB1	Record	0.8625	0.600000	0.675000	0.999972
NB1	Brief	0.9900	0.900000	1.000000	0.999972
NI1	Guided	0.9958	1.000000	1.000000	0.993046
NI1	Standard	0.6196	0.946119	0.350000	0.700000

Appendix

Table S23: Complete Normal-tier results for all four configurations.

S

: composite score;

G

: graph structure;

\mathrm{SC}

: scientific self-consistency score;

A

: accuracy.

Task	Configuration	$S$	$G$	$\mathrm{SC}$	$A$
HA1	Guided	0.9625	1.000000	0.875000	1.000000
HA1	Standard	0.1000	1.000000	0.000000	0.000000
HA1	Record	0.9250	1.000000	0.750000	1.000000
HA1	Brief	0.9250	1.000000	0.750000	1.000000
HA6	Guided	0.9750	1.000000	0.916667	1.000000
HA6	Standard	0.9625	1.000000	0.875000	1.000000

Appendix

Table S24: Complete Hard-tier results for all four configurations.

S

: composite score;

G

: graph structure;

\mathrm{SC}

: scientific self-consistency score;

A

: accuracy.

Task	Configuration	$S$	$G$	$\mathrm{SC}$	$A$
IA1	Guided	0.1717	0.666667	0.350000	0.000000
IA1	Standard	0.1717	0.666667	0.350000	0.000000
IA1	Record	0.1717	0.666667	0.350000	0.000000
IA1	Brief	1.0000	1.000000	1.000000	1.000000
IA5	Guided	1.0000	1.000000	1.000000	1.000000
IA5	Standard	0.1417	0.666667	0.250000	0.000000

Appendix

Table S25: Complete Impossible-tier results for all four configurations.

S

: composite score;

G

: graph structure;

\mathrm{SC}

: scientific self-consistency score;

A

: accuracy.

Figure S1: All Easy and Normal task scores. Each task has four configuration rows and four score layers: composite, graph, scientific self-consistency, and accuracy. Cells are rounded to two decimals; the preceding tables and companion CSV retain greater precision. Backbone denotes Standard.

Figure S2: Scores for all 80 Hard and Impossible runs, with the same component and configuration ordering as Figure S1 .

Study	Frozen inputs and experimental factors	Calls
Chemical identity	16 pairs $\times$ 2 representations $\times$ 2 answer orders	64
Distance judgment	4 geometries $\times$ 2 information conditions $\times$ 2 answer orders $\times$ 2 thinking settings	32
Target-angle feedback	2 information conditions; 3 cyclic next-action permutations and 2 sufficiency permutations per condition	10
Evidence associations	4 chains $\times$ 2 association conditions $\times$ 2 presentations $\times$ 2 answer orders	32
Native diagnostic follow-up	20 identity inputs and 16 association inputs, each under 2 thinking settings	72
Outstanding requirements	3 state questions $\times$ 2 context conditions $\times$ 2 answer orders	12

Appendix

Table S26: Experimental units and crossed conditions of the local diagnostic studies.

Source workflow	Pair 1	$d_{1}$ (Å)	Pair 2	$d_{2}$ (Å)
Guided	C2–C4	2.029	O3–H8	1.248
Brief	C4–H11	2.197	O7–H11	0.981
Brief	C4–H11	1.388	O7–H11	1.362
Brief	C4–H11	2.386	O7–H11	0.968

Appendix

Table S27: Distance comparisons derived from the four archived NB1 geometries.

Thinking	Information	Calls	Correct	Valid wrong	Budget-truncated
Off	XYZ coordinates	8	4	4	0
Off	Measured distances	8	8	0	0
On	XYZ coordinates	8	7	0	1
On	Measured distances	8	8	0	0

Appendix

Table S28: Complete outcomes of the native distance-judgment controls.

Candidate	Atom quadruplet	Signed angle ( ∘ )	Distance to $180^{\circ}$
A	H5–C2–C3–Cl4	$-55.7831$	124.2169
A	H7–C2–C3–Cl4	59.3752	120.6248
B	H5–C2–C3–Cl4	64.2169	115.7831
B	H7–C2–C3–Cl4	179.3752	0.6248

Appendix

Table S29: Dihedral measurements and circular residuals for the saved E1 candidates.

Input / intervention	Forward: original answer B	Reversed: original answer A
Original input	0.954032	0.996932
Edited input	0.008694	0.073626
Transfer at layer 7	0.029099	0.955908
Transfer at layer 15	0.015033	0.717660
Transfer at layer 31	0.007961	0.072117
Transfer at layer 47	0.008763	0.069810

Appendix

Table S30: Probability of the original-fact answer under single-position activation replacement.

Rendering / mode	Family	Calls	Correct	Valid wrong	Budget-truncated
First-stage raw	Identity	20	14	6	0
First-stage raw	Association	16	14	2	0
Native, thinking off	Identity	20	14	6	0
Native, thinking off	Association	16	14	2	0
Native, thinking on	Identity	20	8	0	12
Native, thinking on	Association	16	16	0	0

Appendix

Table S31: Complete outcomes for the selected native diagnostic follow-up.

No.	System/workflow	Demonstrated object class	Task codes	Original sources
1	Buehler molecular-DFT multi-agent demonstration	Molecules/complexes	T E	L01 , a
2	ProtAgents	Soft/biomolecular	T F	L02 , a
3	LLaMP	Materials/surfaces	T D	L03
4	ChemReasoner	Materials/surfaces	E O T	L04
5	AtomAgents	Materials/surfaces	T E O R	L05
6	Adsorb-Agent	Materials/surfaces	T E O	L06

Appendix

Table S32: Complete source-linked register of the 85 chemical AI agent systems/workflows used in the literature sample. Task codes are defined in the text. L01–L85 link to the primary records; lettered links give additional sources.

Chemical Reasoning in LLMs

See all 20 papers

Oct 9, 2025

oMeBench: Towards Robust Benchmarking of LLMs in Organic Mechanism Elucidation and Reasoning

Organic reaction mechanisms describe the step-wise elementary processes by which reactants transform into intermediates and products, and are fundamental to understanding chemical reactivity and guiding molecular and reaction de-sign. While large language models (LLMs) have shown promise on chemical tasks such as synthesis design, it remains unclear to what extent this reflects genuine chemical reasoning capabilities: the ability to generate chemically valid intermediates, maintain consistency across reaction steps, and follow logically coherent multi-step pathways. To investigate this, we introduce oMeBench, the first large-scale, expert-curated benchmark for organic mechanism reasoning, comprising over 10,000 annotated mechanistic steps with reaction type labels, intermediate structures, and difficulty ratings. To enable fine-grained evaluation, we further propose oMeS, a dynamic scoring framework that jointly assesses step-level logical consistency and chemical structural similarity. Systematic evaluation of state-of-the-art LLMs reveals that while current models exhibit promising chemical intuition, they struggle to produce correct and consistent reasoning across multi-step mechanisms. Notably, combining prompting strategies with fine-tuning enables smaller-scale models to achieve performance comparable to closed-source frontier models. We hope oMeBench will serve as a rigorous foundation for advancing AI systems toward genuine chemical reasoning.

Ruiling Xu, Yifan Zhang

University of Illinois Urbana-Champaign

LLM Evaluation Benchmark Design LLM Reasoning

Ask PDF

Sep 10, 2026

Autonomous Chemical Mechanistic Discovery through Agentic Reasoning and Validation

Unraveling reaction mechanisms is central to modern chemistry, yet automating these investigations remains challenging because computational workflows still rely heavily on expert intervention. Here we introduce ARCHE, an autonomous agentic system that integrates a general-purpose reasoning model, a domain-specialized computational chemistry model, and a structured tool registry to transform mechanistic inquiry into a scalable, self-validating process. ARCHE interprets scientific questions, generates and prioritizes mechanistic hypotheses, orchestrates computational workflows, and iteratively refines conclusions based on computed evidence within a closed loop. We validate its capabilities across three increasingly demanding scenarios: reconstructing stereocontrolling transition states and validating the corresponding reaction mechanism in a previously reported asymmetric catalytic reaction; proposing and validating a plausible radical pathway through iterative hypothesis refinement for a recently discovered but unpublished

α

-iodoboronate C-I cleavage reaction; and identifying a chemically interpretable descriptor that governs selectivity in nickel-catalysed migratory cross-coupling reactions. By coupling agentic reasoning with rigorous computational validation, ARCHE advances autonomous mechanistic discovery and establishes a foundation for broader machine-assisted chemical research. The code for ARCHE is publicly available at https://github.com/JetAstra/Arche-Harness.

Dong Li, Sixuan Mi, Zihao Ye +10

Shanghai Artificial Intelligence Laboratory, Shanghai, China. · Institute for Advanced Study in Mathematics, Harbin Institute of Technology, Harbin, China. · School of Chemistry and Molecular Engineering, East China Normal University, Shanghai, China. +4

Agentic Reasoning Autonomous Scientific Discovery

Ask PDF

Jul 14, 2026

Learning Mechanistic Reasoning for Chemical Reactions with Large Language Models

Reaction mechanisms consist of the step-by-step sequences of elementary reactions that explain chemical transformations. Learning the mechanism logic is therefore essential for enhancing the fundamental chemical intelligence of large language models (LLMs). The stepwise deduction of reaction mechanism aligns naturally with the reasoning paradigms of reasoning LLMs. However, current chemical LLMs primarily emphasize coarse-grained name reactions for product prediction and retrosynthesis, often leading to physical inconsistencies and hallucinations. In contrast, specialized small-scale generative models for mechanism inference typically suffer from restricted generalization capacity across diverse chemical spaces. To overcome these limitations, we built a novel, large-scale reasoning dataset of reaction mechanisms. Furthermore, we established the FukuyamaBench, a difficult benchmark derived from Fukuyama's Advanced Organic Reaction Mechanism book, to rigorously evaluate model performance on hierarchical mechanism reasoning. Our fine-tuned Qwen3-30B-A3B achieves 8.3% exact pathway match on FukuyamaBench Set~A, surpassing the specialized FlowER model (5.1%), demonstrating that mechanism-aware training substantially enhances chemical reasoning in language models.

Xingyu Dang, Haocheng Tang, Junmei Wang +1

Department of Computer Science, Princeton University, Princeton, NJ, 08540, United States · School of Pharmacy, University of Pittsburgh, Pittsburgh, PA, 15261, United States · Khoury College of Computer Science, Northeastern University, Boston, MA, 02115, United States +2

LLM Reasoning

Ask PDF

Abstract

Chemical Reasoning in LLMs

Explore similar work

oMeBench: Towards Robust Benchmarking of LLMs in Organic Mechanism Elucidation and Reasoning

Autonomous Chemical Mechanistic Discovery through Agentic Reasoning and Validation

Learning Mechanistic Reasoning for Chemical Reactions with Large Language Models