cs.LGNov 4, 2026

TrustMix: Graceful Degradation for Pretrained Models under Distribution Shift

Authors: Juliusz Ziomek, Maike Osborne, Yee Whye Teh

Organizations: Machine Learning Research Group, University of Oxford, Oxford OX1 2JD, United Kingdom · Department of Statistics, University of Oxford, Oxford OX1 2JD, United Kingdom

Abstract

Predicting beyond the training domain necessarily requires assumptions about how the ground truth extrapolates, yet in most neural networks these assumptions are made only implicit through the architecture. We propose TrustMix, a lightweight post-hoc method that makes a pretrained model's extrapolative behaviour explicit and controllable. TrustMix combines the predictive distribution produced by a pretrained model with a user-specified fallback distribution p0(y∣x)p_0(y|x), such that as the out-of-distribution (OOD) deviation statistic grows, we collapse towards p0(y∣x)p_0(y|x). TrustMix provides graceful degradation: the predictive distribution converges toward the fallback distribution as the OOD signal grows. The fallback distribution p0(y∣x)p_0(y|x) can reflect domain knowledge, if available, or rely on a domain-agnostic stationarity assumption. TrustMix does not require retraining of the base model and can be applied to both regression and classification problems. We evaluated TrustMix on several synthetic and real-world distribution shifts, including tabular, image, and natural language processing tasks. We tested TrustMix on datasets such as CIFAR-10/100 and ImageNet and on numerous architectures including Convolutional Neural Networks (CNNs) and Transformers. TrustMix remains competitive in-distribution while improving predictive uncertainty under distribution shift, with gains in predictive likelihood for regression and calibration for classification. Finally, we showed that TrustMix can provide useful predictive distributions for decision-making applications by utilizing it as a surrogate for Bayesian Optimisation.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Audited Conformal Prediction for Classification under Unknown Distribution Shift

    Jun 12, 2026Yanfei Zhou, Rizal Fathony, Nam H. Nguyen +1Distribution Shift RobustnessUncertainty Quantification

  2. Partially Performative Prediction

    Jun 5, 2026Jaewook Lee, Tijana ZrnicDistribution ShiftPerformative Prediction