MOTIF: LLM-Defined Material Taxonomies for OOD Property Prediction via Invariant Feature Selection
Organizations: Large-scale AI Research Center, Korea Institute of Science and Technology Information (KISTI) · Department of Advanced Materials Engineering, Chungbuk National University · Department of Data Information and Physics, Kongju National University
Abstract
Out-of-distribution (OOD) generalization remains a central challenge in materials property prediction, where models must extrapolate to novel chemistries beyond the training distribution. While large language models (LLMs) have demonstrated strong scientific reasoning, their direct use for quantitative prediction remains limited. To address this, we systematically evaluate multiple roles for LLM assistance, finding that only taxonomy-based grouping consistently improves OOD generalization over a strong kNN+GBT baseline. Building on this, we here propose MOTIF, a framework that leverages LLMs not as predictors, but as generators of candidate taxonomies that partition materials into chemically meaningful environments. These taxonomies are evaluated via training-only pseudo-OOD validation, and only those that improve generalization are retained. Invariant feature selection is then applied within each accepted taxonomy, followed by gated aggregation across taxonomies. We evaluate on 2D halide perovskites, where bandgap prediction must generalize across distinct organic spacer families. On this benchmark, MOTIF reduces mean OOD MAE from 0.122 to 0.111 eV. Under a frozen dev/test protocol, the deployable configuration further improves errors from 0.123 to 0.117 eV (electronic bandgap) and from 0.181 to 0.173 eV (optical bandgap). Beyond this primary task, we validate transferability across additional materials datasets, including 3D perovskite solar cell efficiencies and Matbench benchmarks with structure-based splits. Across domains, MOTIF consistently improves robustness under distribution shift, outperforming expert-defined groupings, rule-based heuristics, and clustering baselines, while remaining competitive with strong classical selection methods. These results demonstrate that LLMs are most effective not as predictors, but as tools for discovering structure in data.
Figures & tables
| Method | Alkyl-S | Alkyl-L | Phenyl | Hetero | Cyclo | Special | Mean | |
| kNN (30-dim) | 0.113 | 0.075 | 0.142 | 0.092 | 0.183 | 0.165 | 0.128 | — |
| GBT (30-dim) | 0.125 | 0.104 | 0.207 | 0.058 | 0.164 | 0.148 | 0.134 | — |
| kNN+GBT Blend (baseline) | 0.123 | 0.136 | 0.119 | 0.110 | 0.137 | 0.105 | 0.122 | — |
| Self-Refine [ 12 ] (best LLM) | 0.425 | 0.282 | 0.644 | 0.419 | 0.658 | 0.335 | 0.461 | |
| Motif (gated median) | 0.128 | 0.129 | 0.095 | 0.099 | 0.098 | 0.119 | 0.111 | |
| Motif (best single axis; post hoc) | 0.118 | 0.130 | 0.109 | 0.094 | 0.118 | 0.094 | 0.110 |
| Method | Alkyl-S | Alkyl-L | Phenyl | Hetero | Cyclo | Special |
| Motif | 0.128 | 0.129 | 0.095 | 0.099 | 0.098 | 0.119 |
| kNN+GBT Blend | 0.123 | 0.136 | 0.119 | 0.110 | 0.137 | 0.105 |
| kNN | 0.113 | 0.075 | 0.142 | 0.092 | 0.183 | 0.165 |
| GBT | 0.125 | 0.104 | 0.207 | 0.058 | 0.164 | 0.148 |
| Self-Refine | 0.425 | 0.282 | 0.644 | 0.419 | 0.658 | 0.335 |
| Category | Method | Eval. | MAE | vs. BL | Ref. |
| (A) Frozen-protocol environment controls (bandgap) | |||||
| — | Baseline blend (kNN+GBT, 30-dim) | Frozen BG | 0.1233 | — | — |
| Motif | LLM tax. ( gpt-4o ) + Safe blend † | Frozen BG | 0.1174 | Ours | |
| Motif | LLM tax. ( gpt-4o ) + Top-1 CV | Frozen BG | 0.1168 | Ours | |
| Control | Expert-family tax. + Safe blend | Frozen BG | 0.1190 | — | |
| Control | Rule-based chemistry axes | Frozen BG | 0.1244 | — | |
| Dataset | Target | Folds | BL MAE | Gated | Safe | Best | |
| Matbench Dielec. | dielectric | 4,764 | 7 | 0.608 | 0.584 | 0.571 | |
| Matbench 2D Exfol. | exfol. en. | 636 | 6 | 72.2 | 70.1 | 68.7 | |
| OQMD ABX3 | form. en. | 4,000 | 7 | 0.435 | 0.427 | 0.420 | |
| MP ABX3 | bandgap | 4,557 | 7 | 0.845 | 0.847 | 0.817 |
| Method | OOD MAE (eV) | Cost (753 queries) | Hardware |
| Motif (after caching) | 0.111 | $0.00 | CPU only |
| Motif (incl. taxonomy gen.) | 0.111 | $0.03–0.37 | CPU only |
| Self-Refine (best LLM) | 0.461 | $45.18 | API |
| LATS | 0.567 | $112.95 | API |
Appendix figures & tables41 assets
Supplementary material from the paper’s appendix.
Appendix
| Fold (Held-Out Family) | Held-Out | Train | Mean BG (eV) | Std BG (eV) |
| Alkyl-short | 232 | 521 | 2.41 | 0.40 |
| Alkyl-long | 79 | 674 | 2.31 | 0.26 |
| Phenyl | 166 | 587 | 2.45 | 0.44 |
| Heterocyclic | 128 | 625 | 2.38 | 0.36 |
| Cycloalkyl | 37 | 716 | 2.70 | 0.44 |
| Special | 111 | 642 | 2.33 | 0.34 |
| Method | Alkyl-S | Alkyl-L | Phenyl | Hetero | Cyclo | Special | Mean | Cat. |
| Motif (oracle) | 0.117 | 0.118 | 0.092 | 0.086 | 0.097 | 0.089 | 0.100 | Tax. |
| Motif (best single) | 0.118 | 0.130 | 0.109 | 0.094 | 0.118 | 0.094 | 0.110 | Tax. |
| Motif (gated median) | 0.128 | 0.129 | 0.095 | 0.099 | 0.098 | 0.119 | 0.111 | Tax. |
| kNN+GBT Blend | 0.123 | 0.136 | 0.119 | 0.110 | 0.137 | 0.105 | 0.122 | ML |
| kNN (30-dim) | 0.113 | 0.075 | 0.142 | 0.092 | 0.183 | 0.165 | 0.128 | ML |
| Random Forest | 0.131 | 0.136 | 0.128 | 0.104 | 0.131 | 0.109 | 0.123 | ML |
| Method | Mean MAE | Cat. |
| Evidence Wt. Mean | 0.304 | RAG |
| Step-Back | 0.463 | Prompt |
| Self-Consistency | 0.492 | Prompt |
| Self-Discover | 0.506 | Prompt |
| LLM Process | 0.511 | Reason |
| CoVe | 0.513 | Reason |
| Method | MAE (eV) | vs. Baseline | Description |
| Zero-Shot | 0.552 | Direct question, no examples | |
| Chain-of-Thought [ 24 ] | 0.563 | Step-by-step reasoning | |
| Self-Consistency | 0.492 | Majority vote over 5 CoT samples | |
| Self-Refine | 0.461 | Iterative refinement (3 rounds) | |
| Step-Back Abstraction | 0.463 | Abstract principle then predict | |
| Analogical Reasoning | 0.542 | Find similar compounds then predict |
| Method | MAE (eV) | vs. Baseline | Description |
| BM25-RAG | 2.427 | BM25 text retrieval + LLM | |
| Embedding-RAG | 2.430 | Embedding similarity retrieval + LLM | |
| Evidence Weighted Mean | 0.304 | Weighted mean of retrieved bandgaps | |
| CRAG | 0.680 | Corrective RAG with self-grading |
| Variant | Alkyl-S | Alkyl-L | Phenyl | Hetero | Cyclo | Special | Mean |
| Full Motif | 0.128 | 0.129 | 0.095 | 0.099 | 0.098 | 0.119 | 0.111 |
| w/o invariant kNN | 0.130 | 0.134 | 0.098 | 0.103 | 0.106 | 0.120 | 0.115 |
| w/o CV gate | 0.131 | 0.130 | 0.096 | 0.100 | 0.100 | 0.120 | 0.113 |
| w/o balance filter | 0.134 | 0.135 | 0.101 | 0.106 | 0.108 | 0.125 | 0.118 |
| w/o strict | 0.132 | 0.131 | 0.098 | 0.103 | 0.104 | 0.121 | 0.115 |
| Single taxonomy | 0.130 | 0.131 | 0.096 | 0.101 | 0.102 | 0.119 | 0.113 |
| Pilot axis | # Groups | Max Group % | Status | Problem |
| Ionic character | 3 | 85% | Filtered | Most spacers classified as “ionic” |
| Structural motif | 3 | 89% | Filtered | “chain” dominates over “ring” and “cage” |
| Electronic coupling | 3 | 72% | Passed | Marginal; weak group produces unreliable Chow test |
| Confinement | 2 | 100% | Filtered | All 2D perovskites are “quantum confined” |
| Stability class | 3 | 99% | Filtered | Nearly all classified as “air-stable” |
| V2 Axis | # Groups | Max Group % | Balance Assessment |
| Spacer size | 4 | 50% | Excellent (4 roughly equal groups) |
| Conjugation | 3 | 56% | Good |
| Polarity | 3 | 57% | Good |
| Steric profile | 4 | 42% | Excellent (most balanced axis) |
| H-bonding | 3 | 57% | Good |
| Flexibility | 3 | 37% | Excellent |
| Axis | # Grp | Group 1 | Group 2 | Group 3 | Group 4 | Max % |
| Spacer size | 4 | 287 (tiny) | 231 (small) | 156 (medium) | 79 (large) | 38.1% |
| Conjugation | 3 | 198 (saturated) | 247 (partial) | 308 (full) | — | 40.9% |
| Polarity | 3 | 312 (nonpolar) | 289 (polar) | 152 (amphiphilic) | — | 41.4% |
| Steric profile | 4 | 186 (linear) | 134 (branched) | 209 (cyclic) | 224 (planar) | 29.7% |
| H-bonding | 3 | 267 (strong) | 312 (weak) | 174 (bifunctional) | — | 41.4% |
| Flexibility | 3 | 278 (rigid) | 256 (semi-flex) | 219 (flexible) | — | 36.9% |
| Structural | B-site | Halide | Spacer | A-site | ||||||||||||||||||||||||||
| Axis | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 |
| Size | ||||||||||||||||||||||||||||||
| Conj. | ||||||||||||||||||||||||||||||
| Polar. | ||||||||||||||||||||||||||||||
| Steric | ||||||||||||||||||||||||||||||
| H-bond | ||||||||||||||||||||||||||||||
| # Inv | Alkyl-S | Alkyl-L | Phenyl | Hetero | Cyclo | Special | Mean (Best) | Mean (Gated) | Std | |
| 0.001 | 18.2 | 0.121 | 0.132 | 0.098 | 0.096 | 0.101 | 0.098 | 0.108 | 0.114 | 0.0147 |
| 0.005 | 20.6 | 0.119 | 0.130 | 0.096 | 0.095 | 0.099 | 0.096 | 0.106 | 0.112 | 0.0138 |
| 0.01 | 22.1 | 0.118 | 0.130 | 0.095 | 0.094 | 0.098 | 0.094 | 0.105 | 0.111 | 0.0144 |
| 0.02 | 23.8 | 0.120 | 0.131 | 0.098 | 0.098 | 0.102 | 0.098 | 0.108 | 0.115 | 0.0122 |
| 0.05 | 25.4 | 0.122 | 0.133 | 0.104 | 0.103 | 0.108 | 0.103 | 0.112 | 0.115 | 0.0108 |
| Alkyl-S | Alkyl-L | Phenyl | Hetero | Cyclo | Special | Mean | Std | |
| 0.001 | 0.131 | 0.133 | 0.099 | 0.103 | 0.104 | 0.112 | 0.1130 | 0.0147 |
| 0.005 | 0.130 | 0.131 | 0.097 | 0.101 | 0.101 | 0.112 | 0.1120 | 0.0138 |
| 0.01 | 0.128 | 0.129 | 0.095 | 0.099 | 0.098 | 0.119 | 0.1113 | 0.0144 |
| 0.02 | 0.129 | 0.131 | 0.101 | 0.102 | 0.103 | 0.123 | 0.1148 | 0.0122 |
| 0.05 | 0.130 | 0.133 | 0.106 | 0.105 | 0.108 | 0.109 | 0.1152 | 0.0108 |
| Strategy | Alkyl-S | Alkyl-L | Phenyl | Hetero | Cyclo | Special | Mean |
| Simple Average | 0.128 | 0.131 | 0.098 | 0.101 | 0.102 | 0.114 | 0.1123 |
| Median | 0.129 | 0.132 | 0.097 | 0.100 | 0.103 | 0.115 | 0.1127 |
| Trimmed Mean | 0.129 | 0.131 | 0.097 | 0.101 | 0.103 | 0.115 | 0.1127 |
| CV-weighted | 0.128 | 0.131 | 0.097 | 0.100 | 0.102 | 0.115 | 0.1122 |
| Top-1 CV Select | 0.127 | 0.132 | 0.098 | 0.100 | 0.101 | 0.115 | 0.1122 |
| Diversity-weighted | 0.128 | 0.131 | 0.098 | 0.101 | 0.102 | 0.114 | 0.1123 |
| LLM | Bandgap Top-1 | Bandgap Safe | Optical Top-1 | Optical Safe | One-time Cost |
| gpt-4o | 0.1168 | 0.1174 | 0.2056 | 0.1732 | $0.37 |
| o3-mini | 0.1173 | 0.1170 | 0.2042 | 0.1823 | — |
| o3 | 0.1148 | 0.1159 | 0.2298 | 0.1859 | $5.22 |
| o1 | 0.1225 | 0.1188 | 0.1942 | 0.1762 | $134.02 |
| Fold | Gated | Oracle | Gap | Best Axis | # Inv Features | |
| Alkyl-short | 0.128 | 0.117 | 0.011 | electron_effect | 0.01 | 24 |
| Alkyl-long | 0.129 | 0.118 | 0.011 | steric | 0.01 | 22 |
| Phenyl | 0.095 | 0.092 | 0.003 | flexibility | 0.01 | 24 |
| Heterocyclic | 0.099 | 0.086 | 0.013 | spacer_size | 0.005 | 20 |
| Cycloalkyl | 0.098 | 0.097 | 0.001 | spacer_size | 0.01 | 22 |
| Special | 0.119 | 0.089 | 0.030 | spacer_size | 0.01 | 22 |
| Variant | Alkyl-S | Alkyl-L | Phenyl | Hetero | Cyclo | Special | Mean |
| R1_gated (gated ensemble) | 0.1239 | 0.1323 | 0.1114 | 0.1042 | 0.1207 | 0.1054 | 0.1163 |
| R2_hybrid (env+standard) | 0.1176 | 0.1257 | 0.1120 | 0.1002 | 0.1185 | 0.1051 | 0.1132 |
| R3_alpha (per-fold ) | 0.1243 | 0.1351 | 0.1098 | 0.1053 | 0.1284 | 0.1261 | 0.1215 |
| R4_strict (strict inv.) | 0.1176 | 0.1295 | 0.1095 | 0.1014 | 0.1170 | 0.0940 | 0.1115 |
| R5_knn_env (env-aware kNN) | 0.1187 | 0.1324 | 0.1083 | 0.1012 | 0.1140 | 0.1041 | 0.1131 |
| R6_best (best single tax.) | 0.1192 | 0.1341 | 0.1109 | 0.1038 | 0.1192 | 0.1072 | 0.1157 |
| Compound | True BG | BL Error | Motif Error | Change | Fold |
| Largest improvements (Motif reduces error) | |||||
| (CHA) 2 PbBr 4 | 3.41 | 0.41 | 0.12 | Cyclo | |
| (PyrA) 2 SnI 4 | 1.83 | 0.35 | 0.09 | Hetero | |
| (4-FPEA) 2 PbBr 4 | 3.52 | 0.38 | 0.14 | Phenyl | |
| (ThMA) 2 PbI 4 | 2.27 | 0.29 | 0.08 | Hetero | |
| (CyPA) 2 PbI 4 | 2.35 | 0.32 | 0.11 | Cyclo | |
| Component | Time | API Cost | Hardware | Notes |
| Motif Pipeline | ||||
| Taxonomy generation ( gpt-4o-mini ) | 3 min | $0.03 | CPU | One-time, cached |
| Taxonomy generation ( gpt-4o ) | 5 min | $0.37 | CPU | One-time, cached |
| Taxonomy generation ( o3-mini ) | — | — | CPU | First cached run predates usage logging |
| Taxonomy generation ( o3 ) | — | $5.22 | CPU | One-time, cached |
| Taxonomy generation ( o1 ) | — | $134.02 | CPU | One-time, cached |
| Subcategory | # Configs | Best MAE | vs. BL |
| Feature augmentation | 12 | 0.127 | |
| Feature imputation | 10 | 0.128 | |
| LLM embeddings | 8+ | 0.131 |
| Subcategory | # Configs | Best MAE | vs. BL |
| Importance weighting | 8 | 0.126 | |
| Sample reranking | 7 | 0.127 | |
| Selective correction | 5+ | 0.130 |
| Subcategory | # Configs | Best MAE | vs. BL |
| GP prior | 5 | 0.132 | |
| Physics constraints | 6 | 0.134 | |
| Bayesian blending | 4+ | 0.133 |
| Category | # Configs | Best MAE (eV) | Best |
| (iii) LLM-generated features | 30+ | 0.127 | |
| (iv) LLM-guided sample selection | 20+ | 0.126 | |
| (v) LLM as prior/constraint | 15+ | 0.132 | |
| Total | 65+ | 0.126 |
| Comparison | Backbone | MAE | 95% CI | |
| BL vs Safe Blend | gpt-4o | 0.0041 | [0.0014, 0.0068] | 0.002 |
| BL vs Top-1 CV | gpt-4o | 0.0021 | [ 0.0030, 0.0072] | 0.206 |
| BL vs Safe Blend | o3 | 0.0058 | [0.0031, 0.0086] | 0.001 |
| BL vs Top-1 CV | o3 | 0.0042 | [ 0.0008, 0.0094] | 0.051 |
| BL vs Safe Blend | gpt-5.2 | 0.0052 | [0.0026, 0.0079] | 0.001 |
| BL vs Top-1 CV | gpt-5.2 | 0.0045 | [ 0.0003, 0.0099] | 0.036 |
| Fold | BL MAE | Motif MAE | [95% CI] | ||
| Alkyl-short | 232 | 0.130 | 0.128 | 0.002 [ 0.003, 0.007] | 0.201 |
| Alkyl-long | 79 | 0.124 | 0.120 | 0.004 [ 0.001, 0.009] | 0.060 |
| Phenyl | 166 | 0.133 | 0.123 | 0.010 [0.003, 0.017] | 0.004 |
| Heterocyclic | 128 | 0.111 | 0.108 | 0.002 [ 0.004, 0.009] | 0.220 |
| Cycloalkyl | 37 | 0.137 | 0.117 | 0.020 [0.004, 0.037] | 0.006 |
| Special | 111 | 0.106 | 0.110 | 0.003 [ 0.008, 0.001] | 0.926 |
| Fold | Role | BL MAE | Motif MAE | |
| Alkyl-short | Dev | 0.130 | 0.128 | |
| Phenyl | Dev | 0.133 | 0.123 | |
| Alkyl-long | Test | 0.124 | 0.120 | |
| Heterocyclic | Test | 0.111 | 0.108 | |
| Cycloalkyl | Test | 0.137 | 0.117 | |
| Special | Test | 0.106 | 0.110 |
| Fold | Test Spacers | Train Spacers | Overlap | Test-Only |
| Alkyl-short | 92 | 197 | 0 | 92 |
| Alkyl-long | 20 | 269 | 0 | 20 |
| Phenyl | 73 | 216 | 0 | 73 |
| Heterocyclic | 55 | 234 | 0 | 55 |
| Cycloalkyl | 14 | 275 | 0 | 14 |
| Special | 48 | 241 | 0 | 48 |
| Selector | Mean MAE (eV) | vs. Baseline |
| Chow test ( ) + forward sel. | 0.1144 | |
| Bonferroni Chow ( ) | 0.1233 | |
| Permutation test (200 perms) | 0.1233 | |
| MI-stability ( ) | 0.1244 | |
| Baseline (no invariance filter) | 0.1233 | — |
Materials Property Prediction
Materials property prediction uses data and physical models to estimate how composition, structure, and processing determine a material’s behavior.
See all 200 papers