Measurable Does Not Imply Transportable: A Prospective Audit of Model Explanations in Multi-Year Field Trials
Organizations: University of California, Davis
Abstract
Model explanations are increasingly used as scientific evidence, yet it remains unclear what they license scientists to conclude beyond the data on which they were constructed. When a fine-grained explanation fails to replicate, the failure may be attributed to noisy measurement of its target, raising a fundamental question: does reliable measurement imply a reproducible explanation? We test this directly. We formalize prospective explanation transportability: the extent to which an explanation learned from past data is recovered in a genuinely future context. This property is distinct from feature importance and measurement reliability. Using multi-year Genomes-to-Fields maize trials, where each new year is a real future context rather than a random resample, we evaluate 16 agronomically interpretable environmental concepts across five forward splits and one prospectively locked test year. Coarse environment-level explanations transport reliably (median cosine 0.83); their locked 2024 value, 0.811, fell within a prespecified 80% bootstrap percentile interval. For grain yield, genotype-conditioned explanations have a historical median cosine of 0.30 and range from 0.006 to 0.822 across years. Reliability does not order these outcomes: the year with the highest split-half reliability had the lowest explanation transport. Within-context resampling showed that GxE explanations were reproducible within a fixed context (0.77-0.95) while exhibiting lower prospective alignment across the 2020-2023 transitions (0.01-0.36). A second phenotype transports strongly at both resolutions, and non-temporal validation inflates apparent interaction-level transport by +0.22. These results show that target reliability and within-context explanation reproducibility do not by themselves establish when a model explanation can support a reproducible scientific claim.
Figures & tables
| Quantity | Historical | 2024 |
|---|---|---|
| Main-effect cosine | median | |
| G E cosine | median | |
| Main–G E gap | ||
| Main forecast | – | |
| Main bootstrap percentile interval | – | |
| Main forecast outcome | – | inside interval |
| Stress test | Coarse/Main | Fine/G E | Difference |
|---|---|---|---|
| Yield, XGBoost/SHAP | |||
| Yield, Ridge | |||
| Grain Moisture | |||
| Random-Env validation | – | ||
| Strict-Forward validation | – | ||
| Validation optimism gap | – |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Target year | Main (mean SD) | G E (mean SD) | Main G E |
|---|---|---|---|
| 2019 | |||
| 2020 | |||
| 2021 | |||
| 2022 | |||
| 2023 | |||
| 2024 |
| Metric | Main | G E |
|---|---|---|
| Unweighted cosine | ||
| Spearman correlation (absolute importance) | ||
| Top-5 Jaccard (absolute importance) | ||
| SHAP-weighted sign agreement |
Explainability Evaluation
Explainability evaluation tests whether AI explanations are faithful, stable, understandable, useful, and reliable across models and settings.
See all 350 papers