cs.LGNov 3, 2026

SteelAgent: LLM-Orchestrated, Physics-Informed Steel Property Prediction with Audited Generalization and Calibrated Uncertainty

Authors: Aleksandr Volkov, Roman Sultimov, Mikhail Kuzin, Yury Maximov

Abstract

Steel property models can perform well on random test splits even when they extrapolate poorly to unfamiliar alloys. SteelAgent couples metallurgical descriptors and machine-learning models with a language-model interface for property prediction, uncertainty reporting and inverse design. We examine how the choice of held-out unit changes the conclusions drawn from SteelBench tensile-strength evaluations. For gradient boosting, pooled out-of-fold R2R^2 fell from 0.859 on a random split to 0.113 when steel families were held out. Empirical coverage of nominal 90% prediction intervals ranged from 91.0% under random splitting to 37.6% on held-out sources. The effect of physics-derived features depended on the evaluation. They increased gradient-boosting R2R^2 by 0.2534 in a 17-family holdout analysis and decreased it by 0.358 in a separate four-grade laboratory analysis. We report group-level errors and a synthetic regression experiment illustrating how calibration can deteriorate on held-out groups. These results motivate reporting the evaluation population alongside a property estimate or compliance score.

Figures & tables

Explore similar work

CardsList
  1. Quantifying out-of-distribution data severity with Wasserstein distances: benchmarking large language models for alloy property prediction

    Nov 2, 2026YuJie Liu, Dezhen Xue, Jun Sun +3Materials Property Prediction

  2. Distribution-Free Conformal Prediction for Steel Fatigue Strength: Marginal Validity Is Not Enough

    Aug 5, 2026Irene BoruahConformal PredictionMaterials Property Prediction

  3. Multitask-Informed Prior for In-Context Learning on Tabular Data: Application to Steel Property Prediction

    Mar 24, 2026Dimitrios Sinodinos, Bahareh Nikpour, Jack Yi Wei +5Multi-Task LearningTabular Foundation Models