Grounding LLMs in Scientific Discovery via Embodied Actions

Bo Zhang; Jinfeng Zhou; Yuxuan Chen; Jianing Yin; Minlie Huang; Hongning Wang

cs.AIFeb 24, 2026

Grounding LLMs in Scientific Discovery via Embodied Actions

Authors: Bo Zhang, Jinfeng Zhou, Yuxuan Chen, Jianing Yin, Minlie Huang, Hongning Wang

Organizations: Tsinghua University, Beijing, China.

Abstract

Large Language Models (LLMs) have shown significant potential in scientific discovery but struggle to bridge the gap between theoretical reasoning and verifiable physical simulation. Existing solutions operate in a passive "execute-then-response" loop and thus lacks runtime perception, obscuring agents to transient anomalies (e.g., numerical instability or diverging oscillations). To address this limitation, we propose EmbodiedAct, a framework that transforms established scientific software into active embodied agents by grounding LLMs in embodied actions with a tight perception-execution loop. We instantiate EmbodiedAct within MATLAB and evaluate it on complex engineering design and scientific modeling tasks. Extensive experiments show that EmbodiedAct significantly outperforms existing baselines, achieving SOTA performance by ensuring satisfactory reliability and stability in long-horizon simulations and enhanced accuracy in scientific modeling.

Figures & tables

Figure 1: Comparison of EmbodiedAct with existing paradigms. EmbodiedAct integrates executable simulation primitives and continuous runtime perception, endowing the agent with the capacity for embodied action within physical simulation environments.

Figure 2: Overview of EmbodiedAct, which bridges the LLM agent and simulation environment via the Asynchronous State Sync Protocol. EmbodiedAct orchestrates a fast inner loop driven by the Runtime Perception Engine to trigger immediate Hot-Fixes, and a slow outer loop driven by the Reflective Decision Maker to guide Re-planning.

Models	$\mathbb{CS}$	$\mathbb{SE}$	$\mathbb{SP}$	$\mathbb{ROB}$	$\mathbb{PP}$	$\mathbb{AD}$	$\mathbb{CV}$	$\mathbb{DD}$	$\mathbb{MIS}$	Overall
Models	Core/Ext	Core/Ext	Core/Ext	Core/Ext	Core/Ext	Core/Ext	Core/Ext	Core/Ext	Core/Ext	Core/Ext
Generative Models
Qwen3-8B-VL-Instruct	18.0 / 18.0	14.2 / 18.3	80.0 / 24.9	13.3 / 22.2	45.4 / 45.4	13.3 / 13.3	15.0 / 40.1	– / 44.3	26.7 / 27.5	21.7 / 25.9
Qwen3-235B-Instruct	33.0 / 33.0	28.5 / 30.5	80.0 / 28.9	13.3 / 32.6	68.8 / 68.8	13.3 / 13.3	53.3 / 55.2	– / 56.7	78.9 / 53.1	39.6 / 42.4
GPT-5-mini	38.3 / 38.3	56.5 / 36.3	80.0 / 36.4	20.0 / 26.7	25.0 / 25.0	40.0 / 40.0	76.7 / 82.6	– / 85.2	81.1 / 41.0	42.0 / 46.9
Claude-Sonnet-4.5	38.2 / 38.2	25.7 / 23.4	80.0 / 40.0	13.3 / 32.3	68.3 / 68.3	33.3 / 33.3	40.0 / 63.3	– / 65.9	72.2 / 50.1	42.1 / 44.5

Table 1: Average Score on the EngDesign benchmark. “Core” set consists of 45 tasks requiring MATLAB simulation. “Extended” (Ext) set expands this to 92 tasks, employing an execution backend compatible with open-source simulation software beyond just MATLAB.

Models	Chemistry					Physics				Math				Avg.
Models	atkins	chemmc	quan	matter	avg.	fund	class	thermo	avg.	diff	stat	calc	avg.	Avg.
Generative Models
Qwen3-8B-VL-Instruct	12.50	25.00	12.50	0.00	12.50	12.50	9.52	10.00	10.25	6.25	10.00	20.00	11.11	11.21
Gemini-3-Flash	12.50	25.00	25.00	12.50	18.75	12.50	19.05	20.00	17.95	25.00	40.00	70.00	41.67	26.17
Qwen3-235B-Instruct	12.50	62.50	25.00	0.00	25.00	25.00	14.29	30.00	20.52	18.75	60.00	50.00	38.89	28.04
GPT-5-mini	25.00	50.00	50.00	25.00	37.50	37.50	19.05	50.00	30.77	31.25	70.00	80.00	55.56	41.12

Table 2: Accuracy on the SciBench-107 (%). The detailed textbook categories (e.g., atkins ) of each subject are shown in Appendix B.1 .

Figure 3: Results (%) of the average pass rate (Pass@3) on the EngDesign benchmark. A shorter error bar indicates greater reliability.

Figure 4: Analysis of Stability. The consistency of the minimum (

x

-axis) versus maximum (

y

-axis) scores (on the EngDesign, Extended set) across three independent runs. Ideally, results cluster along the diagonal (

y=x

), indicating perfect stability.

Models	GPT-5.2		GPT-5-mini
Models	EngDesign	SciBench-107	EngDesign	SciBench-107
Generative Models	48.0	44.9	42.0	41.1
CodeAct	55.4	49.5	52.9	48.6
EmbodiedAct	70.6	54.2	59.3	49.5
w/o $\mathcal{M}_{ref}$	65.0	51.0	55.0	48.0
w/o $\mathcal{M}_{perc}$	56.5	50.0	48.5	46.5

Table 3: Ablation results of average score on EngDesign (Core set) and accuracy (%) on SciBench-107. w/o is the ablated variant.

Figure 5: Case Study: PID Controller Design for Magnetic Levitation. (Left) Traditional static methods (e.g., Ziegler-Nichols tuning on FOPDT approximation). (Right) EmbodiedAct succeeds via a dual-loop cognitive architecture : the Outer Loop (Top Right) performs structural replanning to resolve opacity, while the Inner Loop (Bottom Right) executes physics-informed parameter tuning.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Module	Output	Key Prompt Constraints
$\mathcal{M}_{plan}$	$\mathcal{P}_{global},\mathcal{P}_{local}$	Hierarchical decomposition; toolbox-grounded steps; first-principles derivation; validation checkpoints
$\mathcal{M}_{code}$	$a_{t}$	Executable primitives only; intermediate variable logging; ASCII-only syntax; scope-safe closures
$\mathcal{M}_{perc}$	$z_{t}\in\{$ N, E, W $\}$	Tri-state classification (Normal/Error/Warning); streaming observation parsing; Hot-Fix trigger
$\mathcal{M}_{ref}$	Feedback t , $p^{\prime}$	LLM-as-judge verification; local adjustment vs. global re-planning; physics-informed error analysis

Appendix

Table 4: Four-module prompt architecture aligned with §3.2 formalism. N/E/W denotes Normal/Error/Warning states.

Control Variable	CodeAct	EmbodiedAct
LLM backbone	Identical (GPT-5.2 / Claude-4.5 / Qwen3)
Max interaction turns	5	5
Problem input	Identical statements, no ground-truth
Task-specific tuning	None	None
MATLAB toolbox access	Equivalent (Python bridge / native)
Differentiator	Single-prompt ReAct	Four-module architecture

Appendix

Table 5: Controlled experimental setup. Both methods access identical MATLAB capabilities; the architectural difference—streaming perception and modular cognition—is the sole variable under evaluation.

Textbook	Domain	Original	Subset
class	Classical Mechanics	56	21
diff	Differential Equations	50	16
thermo	Thermodynamics	66	10
stat	Statistics	72	10
calculus	Calculus	42	10
quan	Quantum Mechanics	33	10

Appendix

Table 6: SciBench subject distribution showing original benchmark and our curated subset.

Subject	Direct	ODE/	Root-	Conserv.	Numer.	Other	Total
	Formula	IVP	Finding	Laws	Methods
Classical Mech.	8	5	3	3	1	1	21
Diff. Equations	4	7	2	0	2	1	16
Thermodynamics	6	0	1	1	1	1	10
Statistics	7	0	0	0	2	1	10
Calculus	5	0	1	0	3	1	10

Appendix

Table 7: SciBench-107 subset balance across subject categories and computation patterns. The subset preserves diversity in both dimensions while concentrating on challenging cases.

Field	Type	Description
id	string	Unique message identifier (UUID)
type	enum	Message type (see Table 9 )
payload	object	Type-specific data payload
timestamp	float	Unix timestamp of message creation
session_id	string	Session identifier for routing
operation_id	string	Links related messages in a flow

Appendix

Table 8: EnhancedMessage Schema Fields

Category	Message Type	Direction
Operation Lifecycle	operation_request	$\rightarrow$
	operation_ack	$\leftarrow$
	operation_start	$\leftarrow$
	operation_progress	$\leftarrow$
	operation_complete	$\leftarrow$
	operation_failed	$\leftarrow$

Appendix

Table 9: WebSocket Message Types

Panel of Figure 8 — Figure 8: Streaming monitoring sequence showing the full cognitive loop. Message types (gray labels below arrows) correspond to Table 9 . $\mathcal{M}_{code}$ generates action $a_{t}$ , while $\mathcal{M}_{perc}$ continuously inspects the observation stream $o_{t}$ during $t\in[t_{start},t_{end}]$ to infer intermediate states $s_{t^{\prime}}$ and compute $z_{t}$ .

Panel of Figure 9 — Figure 9: Active interruption with Hot-Fix Loop. When $\mathcal{M}_{perc}$ detects $z_{t}=\text{Warning}$ from system events $\Psi_{sys}$ , it alerts $\mathcal{M}_{code}$ , which generates interrupt action $a_{stop}$ (red arrows). The Hot-Fix Loop then computes repair action $a^{\prime}_{t}\leftarrow\text{Repair}(a_{t},o_{t})$ (blue arrows) to recover.

Panel of Figure 10 — Figure 10: Operation status state machine. Normal flow: PENDING $\rightarrow$ ACKNOWLEDGED $\rightarrow$ STARTED $\rightarrow$ IN_PROGRESS $\rightarrow$ COMPLETED. Errors or timeouts transition to FAILED.

Category	Case	Mechanism	Outcome
1. Runtime Error Recovery	1.1	I/O mismatch $\rightarrow$ Local Adjustment	✓
	1.2	Undefined function $\rightarrow$ Local Adjustment	✓
	1.3	Errors fixed, semantic extraction wrong	×
2. Intermediate Monitoring	2.1	5% threshold verification	✓
	2.2	Grid search + reflective refinement	✓
	2.3	Script structure errors persisted	×

Appendix

Table 10: Summary of 9 case studies demonstrating EmbodiedAct’s self-correction mechanisms.

Method	Total Tokens	Ratio	Avg Score	Pass Rate
Baseline (Direct)	104,889	1.0 $\times$	53.3	33.3%
CodeAct	657,141	6.3 $\times$	44.0	15.6%
EmbodiedAct	1,115,740	10.6 $\times$	61.6	51.1%

Appendix

Table 11: Token usage comparison on multi-step reasoning tasks. All ratios are relative to Baseline.

Module	Symbol	Input	Output	Total	%
Strategic Planner	$\mathcal{M}_{plan}$	124,764	9,264	134,028	12%
Primitive Generator	$\mathcal{M}_{code}$	601,858	44,778	646,636	58%
Runtime Perception	$\mathcal{M}_{perc}$	187,146	13,894	201,040	18%
Reflective Decision	$\mathcal{M}_{ref}$	123,772	10,264	134,036	12%
Total	–	1,037,540	78,200	1,115,740	100%

Appendix

Table 12: Token distribution by module.

\mathcal{M}_{plan}

and

\mathcal{M}_{ref}

show similar consumption due to symmetric input contexts and comparable output schemas.

Explore similar work

AI scientists produce results without reasoning scientifically
Apr 20, 2026Martiño Ríos-García, Nawaf Alampara, Chandan Gupta +5AI Agent Reliability Scientific Reasoning
Laboratory of Organic and Macromolecular Chemistry (IOMC), Friedrich Schiller University Jena, Humboldstr. 10, 07743 Jena, Germany · Department of Civil Engineering, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India · School of Interdisciplinary Research, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India +3
Embodied Science: Closing the Discovery Loop with Agentic Embodied AI
Mar 20, 2026Xiang Zhuang, Chenyi Zhou, Kehua Feng +10AI Agents for Scientific Discovery Autonomous Experimentation
Zhejiang University, Hangzhou, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Tongji University, Shanghai, China
LLM Agents Perform Controlled Experiments Using Simulation Models
Aug 22, 2026Yuchen Xia, Michael Weyrich, Nasser Jazdi +6Multi-Agent LLM Systems Large Language Model-Guided Scientific Discovery
Institute for Industrial Automation and Software Engineering University of Stuttgart, Stuttgart, Germany · Sustainable Innovation & Transformational Excellence (xSITE), Pharmaceutical Technology & Development, Operations AstraZeneca Macclesfield, UK · Chemical Development, Pharmaceutical Technology & Development, Operations AstraZeneca Macclesfield, UK +1

Apr 20, 2026J/K move · Enter open · S save

AI scientists produce results without reasoning scientifically

Martiño Ríos-García, Nawaf Alampara, Chandan Gupta, Indrajeet Mandal+4

Laboratory of Organic and Macromolecular Chemistry (IOMC), Friedrich Schiller University Jena, Humboldstr. 10, 07743 Jena, Germany · Department of Civil Engineering, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India · School of Interdisciplinary Research, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India+3

AI Agent Reliability Scientific Reasoning LLM Agent Evaluation LLM Reliability Autonomous Scientific Discovery Scientific Reasoning in Language Models+2

Open paper AskPDF arXiv

Large language model (LLM)-based systems are increasingly deployed to conduct scientific research autonomously, yet whether their reasoning adheres to the epistemic norms that make scientific inquiry self-correcting is poorly understood. Here, we evaluate LLM-based scientific agents across eight domains, spanning workflow execution to hypothesis-driven inquiry, through more than 25,000 agent runs and two complementary lenses: (i) a systematic performance analysis that decomposes the contributions of the base model and the agent scaffold, and (ii) a behavioral analysis of the epistemological structure of agent reasoning. We observe that the base model is the primary determinant of both performance and behavior, accounting for 41.4% of explained variance versus 1.5% for the scaffold. Across all configurations, evidence is ignored in 68% of traces, refutation-driven belief revision occurs in 26%, and convergent multi-test evidence is rare. The same reasoning pattern appears whether the agent executes a computational workflow or conducts hypothesis-driven inquiry. They persist even when agents receive near-complete successful reasoning trajectories as context, and the resulting unreliability compounds across repeated trials in epistemically demanding domains. Thus, current LLM-based agents execute scientific workflows but do not exhibit the epistemic patterns that characterize scientific reasoning. Outcome-based evaluation cannot detect these failures, and scaffold engineering alone cannot repair them. Until reasoning itself becomes a training target, the scientific knowledge produced by such agents cannot be justified by the process that generated it.

Abstract

Explore similar work

AI scientists produce results without reasoning scientifically

Embodied Science: Closing the Discovery Loop with Agentic Embodied AI

LLM Agents Perform Controlled Experiments Using Simulation Models