cs.AINov 9, 2026

MOTIF: LLM-Defined Material Taxonomies for OOD Property Prediction via Invariant Feature Selection

Authors: Gwangseon Jang, Sieun Yun, Dong Ho Lee, Kyong-Ha Lee, Chanuk Lim, Jin Woo Choi, Han Seul Kim

Organizations: Large-scale AI Research Center, Korea Institute of Science and Technology Information (KISTI) · Department of Advanced Materials Engineering, Chungbuk National University · Department of Data Information and Physics, Kongju National University

Abstract

Out-of-distribution (OOD) generalization remains a central challenge in materials property prediction, where models must extrapolate to novel chemistries beyond the training distribution. While large language models (LLMs) have demonstrated strong scientific reasoning, their direct use for quantitative prediction remains limited. To address this, we systematically evaluate multiple roles for LLM assistance, finding that only taxonomy-based grouping consistently improves OOD generalization over a strong kNN+GBT baseline. Building on this, we here propose MOTIF, a framework that leverages LLMs not as predictors, but as generators of candidate taxonomies that partition materials into chemically meaningful environments. These taxonomies are evaluated via training-only pseudo-OOD validation, and only those that improve generalization are retained. Invariant feature selection is then applied within each accepted taxonomy, followed by gated aggregation across taxonomies. We evaluate on 2D halide perovskites, where bandgap prediction must generalize across distinct organic spacer families. On this benchmark, MOTIF reduces mean OOD MAE from 0.122 to 0.111 eV. Under a frozen dev/test protocol, the deployable configuration further improves errors from 0.123 to 0.117 eV (electronic bandgap) and from 0.181 to 0.173 eV (optical bandgap). Beyond this primary task, we validate transferability across additional materials datasets, including 3D perovskite solar cell efficiencies and Matbench benchmarks with structure-based splits. Across domains, MOTIF consistently improves robustness under distribution shift, outperforming expert-defined groupings, rule-based heuristics, and clustering baselines, while remaining competitive with strong classical selection methods. These results demonstrate that LLMs are most effective not as predictors, but as tools for discovering structure in data.

Figures & tables

Appendix figures & tables41 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

May 3, 2025

BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models

Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel molecules. Discovering novel molecules requires accurate out-of-distribution (OOD) predictions, but ML models struggle to generalize OOD. Currently, no systematic benchmarks exist for molecular OOD prediction tasks. We present BOOM\mathbf{BOOM}, b\mathbf{b}enchmarks for o\mathbf{o}ut-o\mathbf{o}f-distribution m\mathbf{m}olecular property predictions: a chemically-informed benchmark for OOD performance on common molecular property prediction tasks. We evaluate over 150 model-task combinations to benchmark deep learning models on OOD performance. Overall, we find that no existing model achieves strong generalization across all tasks: even the top-performing model exhibited an average OOD error 3x higher than in-distribution. Current chemical foundation models do not show strong OOD extrapolation, while models with high inductive bias can perform well on OOD tasks with simple, specific properties. We perform extensive ablation experiments, highlighting how data generation, pre-training, hyperparameter optimization, model architecture, and molecular representation impact OOD performance. Developing models with strong OOD generalization is a new frontier challenge in chemical ML. This open-source benchmark is available at https://github.com/FLASK-LLNL/BOOM
Nov 2, 2026

Quantifying out-of-distribution data severity with Wasserstein distances: benchmarking large language models for alloy property prediction

General-purpose Large Language Models (LLMs) increasingly rival domain-specific pipelines on scientific prediction, but their performance under distribution shift is untested. Across six alloy and crystal-structure datasets, we benchmark LLMs against traditional, BERT-based, TabPFN, and text-regression baselines under random cross-validation and six out-of-distribution (OOD) protocols shifting input and target distributions, with severity quantified via a standardized Wasserstein distance. We isolate the LLM's advantage over k-nearest-neighbor (k-NN) retrieval: it beats k-NN on all eight tasks at a fixed 5% reference budget, on seven of eight when independently optimized, and in 72.5% of 120 OOD task--protocol configurations against an optimized baseline. No model family is uniformly best; the LLM is often strongest under severe shift, and error increases with target-space but not representation-space severity. Self-reported LLM confidence tracks both shift and error, more consistently than a density-only control from the same retrieved samples, and a matched-cohort check finds no substantial evidence of memorization.
Nov 9, 2026

MOOD: Benchmarking Post-Hoc OOD Detection for Materials Property Prediction

Out-of-distribution (OOD) detection is essential for deploying machine learning models in high stakes scientific applications, yet methods that flag OOD inputs without retraining the model have been validated almost exclusively on image classification. We introduce MOOD (Materials Out-of-Distribution Detection), a benchmark for evaluating whether these post-hoc OOD detectors transfer to graph neural networks (GNNs) on materials property prediction. MOOD evaluates 15 post-hoc detectors and 2 supervised diagnostic probes across 4 GNN architectures, 5 MatBench tasks, and 15 physically motivated compositional and structural shifts, yielding 264 evaluation settings and over 13,000 AUROC measurements across three encoder training seeds. Our evaluation reveals three key findings. First, current post-hoc detectors perform near chance; all 15 detectors achieve an AUROC around 0.5. Second, detector failures decompose into two regimes: method bottlenecks and encoder bottlenecks. Method bottlenecks arise when supervised probes can separate in-distribution (ID) from OOD samples but unsupervised detectors fail, meaning that the embeddings contain OOD signal that current detectors do not extract. Encoder bottlenecks arise when even supervised probes fail to separate ID from OOD samples, indicating that the relevant shift is not accessible in the frozen GNN representation. Third, when OOD prediction error exceeds ID error, the AUROC of distance-based and density-based detectors correlates with the size of that gap. Collectively, these findings indicate that reliable OOD detection in scientific domains requires representation learning strategies that explicitly preserve OOD relevant signal, particularly for tasks where current GNN architectures encode no extractable signal at all.