Overview

Robust regression studies how to estimate relationships between variables when data contain outliers, contamination, heavy-tailed noise, or distribution shifts. Common approaches include robust losses such as Huber and Welsch, trimming and median-of-means estimators, sparse thresholding, and distributionally robust optimization. The field also develops guarantees for settings such as adversarial corruption, missing features, and multi-task learning, alongside practical methods for uncertainty estimates and nonlinear models. A central tension is balancing protection against bad data with statistical efficiency and computational cost, since robust methods do not always outperform ordinary least squares when its assumptions are adequate.

Latest papers

34 papers3 papers in the last four weeks, down 25% on the four weeks before. 0.0% of all new papers.

Oct 30, 2025

Outlier-Robust Multi-Output Gaussian Processes

Multi-output Gaussian process (MOGP) regression allows modelling dependencies among multiple correlated response variables. Similarly to standard Gaussian processes (GPs), MOGPs are sensitive to model misspecification and outliers, which can distort predictions within individual outputs. In the multi-output case, this situation can be further exacerbated as anomalous observations in one response can adversely influence predictions for other correlated outputs. To handle this situation, we propose R-MOGP, an MOGP that is provably robust to outliers in any individual response. Our approach preserves conjugacy, achieving robustness without sacrificing computational efficiency, and integrates naturally with sparse variational inference for scalability. Empirically, we show that R-MOGP achieves performance comparable to existing state-of-the-art robust MOGPs at a fraction of their computational costs.
Sep 23, 2025

Linear Regression under Missing or Corrupted Coordinates

We study multivariate linear regression under Gaussian covariates in two settings, where data may be erased or corrupted by an adversary under a coordinate-wise budget. In the incomplete data setting, an adversary may inspect the dataset and delete entries in up to an ηη-fraction of samples per coordinate; a strong form of the Missing Not At Random model. In the corrupted data setting, the adversary instead replaces values arbitrarily, and the corruption locations are unknown to the learner. Despite substantial work on missing data, linear regression under such adversarial missingness remains poorly understood, even information-theoretically. Unlike the clean setting, where estimation error vanishes with more samples, here the optimal error remains a positive function of the problem parameters. Our main contribution is to characterize this error up to constant factors across essentially the entire parameter range. Specifically, we establish novel information-theoretic lower bounds on the achievable error that match the error of (computationally efficient) algorithms. A key implication is that, perhaps surprisingly, the optimal error in the missing data setting matches that in the corruption setting-so knowing the corruption locations offers no general advantage.
May 29, 2025

DRO: A Python Library for Distributionally Robust Optimization in Machine Learning

We introduce dro, an open-source Python library for distributionally robust optimization (DRO) for regression and classification problems. The library implements 15 DRO formulations and 9 backbone models, enabling 85 distinct DRO methods. Furthermore, dro is compatible with both scikit-learn and PyTorch through a unified interface, and uses approximation techniques to reduce runtime for some computationally demanding formulations. Comprehensive documentation is available at https://python-dro.org.
Jan 6, 2022

Robust Linear Predictions: Analyses of Uniform Concentration, Fast Rates and Model Misspecification

The problem of linear predictions has been extensively studied for the past century under pretty generalized frameworks. Recent advances in the robust statistics literature allow us to analyze robust versions of classical linear models through the prism of Median of Means (MoM). Combining these approaches in a piecemeal way might lead to ad-hoc procedures, and the restricted theoretical conclusions that underpin each individual contribution may no longer be valid. To meet these challenges coherently, in this study, we offer a unified robust framework that includes a broad variety of linear prediction problems on a Hilbert space, coupled with a generic class of loss functions. Notably, we do not require any assumptions on the distribution of the outlying data points (O\mathcal{O}) nor the compactness of the support of the inlying ones (I\mathcal{I}). Under mild conditions on the dual norm, we show that for misspecification level εε, these estimators achieve an error rate of O(max⁡{∣O∣1/2n−1/2,∣I∣1/2n−1}+ε)O(\max\left\{|\mathcal{O}|^{1/2}n^{-1/2}, |\mathcal{I}|^{1/2}n^{-1} \right\}+ε), matching the best-known rates in literature. This rate is slightly slower than the classical rates of O(n−1/2)O(n^{-1/2}), indicating that we need to pay a price in terms of error rates to obtain robust estimates. Additionally, we show that this rate can be improved to achieve so-called "fast rates" under additional assumptions.