Overview

The field studies the consequences of using models that omit variables, impose incorrect structure, or fail to capture changes across environments. Approaches include robust estimation such as median-of-means methods, posterior predictive calibration, residual and identifiability diagnostics, and discrepancy correction; applications span forecasting, physics-informed learning, simulation-based inference, and reinforcement learning. A central challenge is distinguishing genuine misspecification from noise, weak data, or poor optimization, then correcting it without erasing useful uncertainty or real effects. Because no diagnostic can establish that a model is fully correct, reliable practice often combines complementary checks with iterative model revision.

Latest papers

35 papers4 papers in the last four weeks, down 56% on the four weeks before. 0.0% of all new papers.

Feb 18, 2026

Omitted Variable Bias in Language Models Under Distribution Shift

Despite their impressive performance on a wide variety of tasks, modern language models remain susceptible to distribution shifts, exhibiting brittle behavior when evaluated on data that differs in distribution from their training data. In this paper, we describe how distribution shifts in language models can be separated into observable and unobservable components, and we discuss how established approaches for dealing with distribution shift address only the former. Importantly, we identify that the resulting omitted variable bias from unobserved variables can compromise both evaluation and optimization in language models. To address this challenge, we introduce a framework that maps the strength of the omitted variables to bounds on the worst-case generalization performance of language models under distribution shift. In empirical experiments, we show that using these bounds directly in language model evaluation and optimization provides more principled measures of out-of-distribution performance, improves true out-of-distribution performance relative to standard distribution shift adjustment methods, and further enables inference about the strength of the omitted variables when target distribution labels are available.
Sep 6, 2025

Robust Amortized Simulation-Based Inference via Learned Error Models

Recent advances in neural density estimation have enabled amortized Bayesian inference for complex stochastic simulators. However, these methods rely on simulators accurately reflecting the true data-generating process and can degrade significantly under model misspecification. We consider the setting where multiple unlabeled observations are available and introduce robust variational neural posterior estimation (RVNP), an amortized Bayesian inference method that uses an importance-weighted autoencoder to jointly learn a misspecification-robust posterior proxy and an explicit error model that captures the simulation-to-reality gap. Our results show that RVNP can recover robust posterior inference in a data-driven and interpretable manner, outperforming previous methods on multiple metrics across misspecified benchmarks.
Mar 1, 2025

Dissecting the Impact of Model Misspecification in Data-driven Optimization

Data-driven optimization aims to translate a machine learning model into decision-making by optimizing decisions on estimated costs. Such a pipeline can be conducted by fitting a distributional model which is then plugged into the target optimization problem. While this fitting can utilize traditional methods such as maximum likelihood, a more recent approach uses estimation-optimization integration that minimizes decision error instead of estimation error. Although intuitive, the statistical benefit of the latter approach is not well understood yet is important to guide the prescriptive usage of machine learning. In this paper, we dissect the performance comparisons between these approaches in terms of the amount of model misspecification. In particular, we show how the integrated approach offers a ``universal double benefit'' on the top two dominating terms of regret when the underlying model is misspecified, while the traditional approach can be advantageous when the model is nearly well-specified. Our comparison is powered by finite-sample tail regret bounds that are derived via new higher-order expansions of regrets and the leveraging of a recent Berry-Esseen theorem.
Oct 18, 2024

Predictive variational inference: Learn the predictively optimal posterior distribution

Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density such that the resulting posterior predictive distribution is as close to the true data generating process as possible, while this closeness is measured by multiple scoring rules. By optimizing the objective, the predictive variational inference is generally not the same as, or even attempting to approximate, the Bayesian posterior, even asymptotically. Rather, we interpret it as implicit hierarchical expansion. Further, the learned posterior uncertainty detects heterogeneity of parameters among the population, enabling automatic model diagnosis. This framework applies to both likelihood-exact and likelihood-free models. We demonstrate its application in real data examples.
Jan 6, 2022

Robust Linear Predictions: Analyses of Uniform Concentration, Fast Rates and Model Misspecification

The problem of linear predictions has been extensively studied for the past century under pretty generalized frameworks. Recent advances in the robust statistics literature allow us to analyze robust versions of classical linear models through the prism of Median of Means (MoM). Combining these approaches in a piecemeal way might lead to ad-hoc procedures, and the restricted theoretical conclusions that underpin each individual contribution may no longer be valid. To meet these challenges coherently, in this study, we offer a unified robust framework that includes a broad variety of linear prediction problems on a Hilbert space, coupled with a generic class of loss functions. Notably, we do not require any assumptions on the distribution of the outlying data points (O\mathcal{O}) nor the compactness of the support of the inlying ones (I\mathcal{I}). Under mild conditions on the dual norm, we show that for misspecification level εε, these estimators achieve an error rate of O(max⁡{∣O∣1/2n−1/2,∣I∣1/2n−1}+ε)O(\max\left\{|\mathcal{O}|^{1/2}n^{-1/2}, |\mathcal{I}|^{1/2}n^{-1} \right\}+ε), matching the best-known rates in literature. This rate is slightly slower than the classical rates of O(n−1/2)O(n^{-1/2}), indicating that we need to pay a price in terms of error rates to obtain robust estimates. Additionally, we show that this rate can be improved to achieve so-called "fast rates" under additional assumptions.