Probing Materials Knowledge in LLMs: From Latent Embeddings to Reliable Predictions

Vineeth Venugopal; Soroush Mahjoubi; Elsa Olivetti

cond-mat.mtrl-sciMar 2, 2026

Probing Materials Knowledge in LLMs: From Latent Embeddings to Reliable Predictions

Authors: Vineeth Venugopal, Soroush Mahjoubi, Elsa Olivetti

Organizations: Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, 02139, MA, USA.

Abstract

Large language models are increasingly applied to materials science, yet fundamental questions remain about their reliability and knowledge encoding. Evaluating 25 LLMs across four materials science tasks -- over 200 base and fine-tuned configurations -- we find that output modality fundamentally determines model behavior. For symbolic tasks, fine-tuning converges to consistent, verifiable answers with reduced response entropy, while for numerical tasks, fine-tuning improves prediction accuracy but models remain inconsistent across repeated inference runs, limiting their reliability as quantitative predictors. For numerical regression, we find that better performance can be obtained by extracting embeddings directly from intermediate transformer layers than from model text output, revealing an ``LLM head bottleneck,'' though this effect is property- and dataset-dependent. Finally, we present a longitudinal study of GPT model performance in materials science, tracking four models over 18 months and observing 9--43% performance variation that poses reproducibility challenges for scientific applications.

Figures & tables

	Bandgap	Dielectric	Crystal System	MatKG
	(RMSE, eV)	(RMSE)	(Acc., %)	(Top-1, %)
1	Llama-3-70B (0.70)	GPT-4o (1.8)	GPT-4.1-mini (66)	GPT-4.1-mini (68)
2	GPT-4.1-mini (0.71)	GPT-4.1-mini (1.9)	Llama-3-70B (63)	GPT-4.1 (66)
3	GPT-4o-mini (0.73)	Llama-3-70B (2.1)	GPT-4o-mini (63)	GPT-4o (65)

Table 1: Top three fine-tuned models per task. RMSE (lower is better) for regression; accuracy (higher is better) for symbolic tasks. See Methods for metric definitions and Supplementary Tables S5.1–S5.4 for all 25 models.

Figure 1: Performance versus response entropy for symbolic tasks. MatKG link prediction (left) and crystal system classification (right). Circles indicate base models; squares indicate fine-tuned models. Marker size is proportional to model parameter count, and colors denote model families. Dashed lines show mean performance and entropy for base models. Fine-tuning dramatically improves accuracy while reducing response entropy across all model families.

Figure 2: Performance versus response entropy for numerical regression tasks. Bandgap prediction (left) and dielectric constant prediction (right). Visualization follows the same conventions as Figure 1 . Lower RMSE indicates better performance. Fine-tuning generally reduces prediction error, with variable effects on entropy across models.

Figure 3: Cross-task transfer matrix. Each cell shows the effect of fine-tuning on a source task (columns) when evaluated on a target task (rows), compared to base model performance. Color coding: dark green (+2) indicates statistically significant improvement; light green (+1) indicates improvement within noise; light red (–1) indicates degradation within noise; dark red (–2) indicates significant degradation; grey (0) indicates no data. Statistical significance is determined by comparison to base model standard deviation across inference runs.

Figure 4: Factors determining MatKG link prediction accuracy. (a) Accuracy binned by subject frequency (wide bars) and object answer frequency (narrow bars) for both base and fine-tuned models. The fine-tuned model shows strong monotonic dependence on object answer frequency while subject frequency has no effect; the base model remains flat at $\sim$ 4–5% regardless of frequency. (b) Performance breakdown by relation target category, showing that Descriptors (DSC) and Applications (APL) are easiest to predict ( $\sim$ 70% accuracy) while Symmetry Phase Labels (SPL), Synthesis Methods (SMT) and Materials (CHM) are harder ( $\sim$ 55%).

Figure 5: Layer-wise embedding probes for property prediction. Test RMSE as a function of transformer layer for bandgap (top row) and dielectric constant (bottom row) across three model families, shown for ridge regression (left) and two-layer neural network (right). All three probe architectures (ridge, single-layer NN, two-layer NN) were tested with consistent results; the single-layer NN is omitted for clarity. Horizontal dashed lines indicate fine-tuned LLM text generation performance. For bandgap, intermediate layer embeddings match or exceed text output performance, suggesting a verbalization bottleneck. For dielectric constant, embeddings consistently underperform text generation by approximately 3 $\times$ , indicating property-dependent knowledge encoding.

Figure 6: Dielectric prediction error varies with property range across all model families. RMSE ratio (embedding probe / fine-tuned LLM) as a function of evaluation range for Llama-3-8B (left), Mistral-7B (center), and Llama-2-7B (right). All three models show consistent degradation as the evaluation range widens. Remarkably, Llama-2-7B embeddings outperform fine-tuned text generation at narrow band (0.9 $\times$ FT-LLM at $\mu\pm 0.5\sigma$ ), while Mistral-7B achieves 1.3 $\times$ and Llama-3-8B reaches 1.6 $\times$ . Performance degrades to 2.4–3.8 $\times$ when including extreme values ( $\mu\pm 3\sigma$ ). All models use Layer 17 embeddings and exhibit the same monotonic degradation pattern despite different FT-LLM baselines (2.4, 2.7, and 3.9 RMSE respectively).

Model	Mean RMSE (eV)	Max Variation (%)
GPT-FT	0.84	13
GPT-4	1.3	9
GPT-4o	1.9	43
GPT-3.5	2.4	19

Table 2: GPT model performance variance over 18 months. Maximum variation represents the largest percentage deviation from mean RMSE observed across evaluation timepoints. Data through December 2025.

Figure 7: Temporal variation in GPT model performance. RMSE for bandgap prediction across evaluation timepoints spanning 18 months. GPT-4o shows the largest variation (42.8%), with a notable performance shift following the November 2024 endpoint update. Even fine-tuned models exhibit measurable drift over time.

LLM Reliability

Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.

See all 757 papers

OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields
May 28, 2026Wanhao Liu, Jiaqing Xie, Qian Tan +10Benchmark Design Scientific QA
University of Science and Technology of China · Shanghai Artificial Intelligence Laboratory · Fudan University +4
From Knowledge to Action: Outcomes of the 2025 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry
May 4, 2026Aritra Roy, Kevin Shen, Andrew MacBride +350Multi-Agent LLM Systems AI Agents for Scientific Discovery
From Blind Guess to Informed Judgment: Teaching LLMs to Evaluate Materials by Building Knowledge-Augmented Preference Signals
May 28, 2026Yeyong Yu, Wenya Hu, Xing Wu +1LLM Evaluation Pairwise Preference Evaluation
School of Computer Engineering & Science, Shanghai University, Shanghai, 200444, China. · Center of Materials Informatics and Data Science, Materials Genome2026 Institute, Shanghai University, Shanghai, 200444, China. · Key Laboratory of Silicate Cultural Relics Conservation (Shanghai University), Ministry of Education, China. +1

May 28, 2026J/K move · Enter open · S save

OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields

Wanhao Liu, Jiaqing Xie, Qian Tan, Weida Wang+9

University of Science and Technology of China · Shanghai Artificial Intelligence Laboratory · Fudan University+4

Benchmark Design Scientific QA Multimodal Reasoning Materials Science Multimodal Model Evaluation

Open paper AskPDF arXiv

As multimodal language models play an increasingly important role in scientific research, materials science offers a critical testbed due to its interdisciplinary, multimodal, and application-driven nature. However, existing materials benchmarks mainly focus on property prediction, knowledge QA, or characterization understanding, leaving the broader reasoning process from materials knowledge to application underexplored. To fill this gap, we present OmniMatBench, a human-calibrated multimodal reasoning benchmark for materials science. OmniMatBench contains 3,171 expert-curated QA and calculation problems across 19 materials-science subfields, spanning fundamental materials knowledge, structural and engineering materials, materials processing and manufacturing, and functional and applied materials. We evaluate 13 open-source and closed-source MLLMs and find that the best model achieves only a 0.372 overall score, revealing a substantial gap in current materials-science reasoning. Further analysis shows strong variation across subfields, fixed reasoning heuristics, uneven materials knowledge, and limited high-level knowledge application under formula-, retrieval-, and code-assisted settings. OmniMatBench provides crucial insights into the capabilities and limitations of current MLLMs and establishes a foundation for reliable AI assistants in materials-science research.

Abstract

LLM Reliability

Explore similar work

OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields

From Knowledge to Action: Outcomes of the 2025 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry

From Blind Guess to Informed Judgment: Teaching LLMs to Evaluate Materials by Building Knowledge-Augmented Preference Signals