Confidence-Based Materials Data Quality Assessment via Domain Knowledge and Graph-Structured Learning
Organizations: Computer Network Information Center, Chinese Academy of Sciences, Beijing, 100083, China · University of Chinese Academy of Sciences, Beijing, 101408, China · School of Materials and New Energy, Ningxia University, Yinchuan, 750021, China
Abstract
The rapid development of materials machine learning relies on large-scale, heterogeneous datasets, yet model reliability is often compromised by imperfect data from diverse experiments, simulations, and database integration. Real-world materials databases may contain measurement errors, failed calculations, missing or duplicate records, and hidden violations of physicochemical constraints. Such problems are difficult to identify because corrupted samples may remain statistically plausible rather than appearing as explicit outliers. In this paper, we propose a confidence-based materials data quality assessment framework that integrates domain knowledge with graph-structured learning. Data quality is formulated as a continuous confidence estimation problem rather than a binary anomaly identification task, enabling sample-level assessment and historical-reference-based batch comparison. A nine-dimensional feature-quality prior and duplicate-aware sample initialization are calibrated using materials domain rules and statistical evidence. Confidence is further refined through sample-feature bipartite graph propagation and weakly supervised graph attention, while a Memory-GAT module provides historical reference scoring. Experiments on 23 ODDS benchmark datasets show that the generic module achieves an average AUC-ROC of 0.866. In five repeated mixed-contamination experiments on mp_20, rule calibration improves HistGradientBoostingRegressor test from to . Graph refinement increases average precision from to 0.953\pm0.004R^2R^20.590\pm0.0530.880\pm0.027$ after rule-based screening.
Figures & tables
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Name / interpretation | Formula in this reference experiment | Operational meaning |
|---|---|---|
| provenance / documentation availability | . | Documentation completeness from available metadata. |
| dataset_characteristics / range–center agreement | If an endpoint lies outside the configured range, ; otherwise . Set and . | Measures range agreement and distance from the configured range center. |
| uniformity / numeric compatibility | Fraction of nonmissing observations convertible to a number; zero for an empty field. | Numerical conversion rate; constant for the numerical fields used here. |
| metadata_coupling / lexical match | difflib.SequenceMatcher ratio between lowercased field name and description. | Lexical agreement proxy for field documentation. |
| non_missing_cells / observed-cell fraction | Fraction nonmissing at scoring time. | Adapted completeness; equals one after imputation here. |
| non_duplicate_rows / duplicate handling | ; exact duplicate detection acts through instead. | Fixed at one; sample duplicates are handled by the sample prior. |
| Setting | AP | F1 | |
|---|---|---|---|
| accuracy_x0.0 | 0.9032 0.0056 | 0.7851 0.2689 | 0.5857 0.0425 |
| accuracy_x0.8 | 0.9025 0.0054 | 0.7726 0.2766 | 0.5828 0.0430 |
| accuracy_x1.2 | 0.9023 0.0053 | 0.7683 0.2807 | 0.5858 0.0371 |
| dataset_characteristics_x0.0 | 0.9067 0.0049 | 0.6881 0.2692 | 0.5656 0.0392 |
| dataset_characteristics_x0.8 | 0.9040 0.0052 | 0.7657 0.2816 | 0.5828 0.0414 |
| dataset_characteristics_x1.2 | 0.9009 0.0055 | 0.7837 0.2726 | 0.5866 0.0427 |
| Availability | Category | Fields | Definition |
|---|---|---|---|
| Both | Sample identifier | Alloy_Name (185-record table); No. (315-record table) | Alloy-composition label or numerical record identifier used to distinguish individual observations. |
| Both | Elemental composition | Co , Cr , Fe , Mn , Ni , Al , Ti , V , Cu , W , Mo , Zr , Nb , Ta | Contents of the 14 constituent elements. The values describe the nominal elemental fractions recorded for each alloy, typically in atomic percent. |
| 185 only | Testing condition | temperature | Temperature at which the reported UTS and FE values were measured. |
| 315 only | Homogenization | Hom-T , Hom-t | Homogenization temperature and holding time, respectively. |
| 315 only | Deformation | Def-T , Def-r | Deformation temperature and deformation or reduction ratio, respectively. |
| 315 only | First annealing stage | AnnI-T , AnnI-t | Temperature and holding time of the first annealing treatment. |
Materials Science
Materials science studies how composition, structure, processing, and defects determine material properties and how to design useful materials.
See all 333 papers