cs.LGMay 29, 2025
SaveDRO: A Python Library for Distributionally Robust Optimization in Machine Learning
Abstract
We introduce dro, an open-source Python library for distributionally robust optimization (DRO) for regression and classification problems. The library implements 15 DRO formulations and 9 backbone models, enabling 85 distinct DRO methods. Furthermore, dro is compatible with both scikit-learn and PyTorch through a unified interface, and uses approximation techniques to reduce runtime for some computationally demanding formulations. Comprehensive documentation is available at https://python-dro.org.
Figures & tables
Figure 1: Overview of the dro library.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Exact Optimization | Approximate Optimization | ||||||||
| LAD | OLS | SVM | Logistic | Kernel | Personal | Tree-based | NN | Personal | |
| WDRO | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
| RS-WDRO | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |||
| -DRO | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| KL-DRO | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||
| Bayesian-DRO | ✓ | ✓ | ✓ | ✓ | |||||
Table 1: Different DRO methods supported in dro package.
| Function | Source | Description |
| classification_basic | Custom | Multi-class Gaussian blobs on a sphere; baseline data generator. |
| classification_DN21 | Duchi and Namkoong (2021) (Sec 3.1.1) | Linear decision boundary with controlled label noise. |
| classification_SNVD20 | Sinha et al. (2018) (Sec 5.1) | Ring-shaped classification with uncertain margin regions. |
| classification_LWLC | Liu et al. (2022) (Sec 4.1) | High-dimensional mixed features with geometric scrambling. |
Table 2: Summary of synthetic classification datasets and their sources.
| Function | Source | Description |
| regression_basic | Custom | Standard linear regression with Gaussian noise; serves as a baseline. |
| regression_DN20_1 | Duchi and Namkoong (2021) (Sec 3.1.2) | Linear model with heteroscedastic noise based on covariate threshold. |
| regression_DN20_2 | Duchi and Namkoong (2021) (Sec 3.1.3) | Mixture of linear models simulating group shifts. |
| regression_DN20_3 | Duchi and Namkoong (2021) (Sec 3.3) | Real-world UCI crime dataset with pre-processing. |
| regression_LWLC | Liu et al. (2022) (Sec 4.1) | Complex high-dimensional regression with group imbalance and spurious correlation. |
Table 3: Summary of regression datasets and their sources.
| Samples | Features | Radius | Base sec | Acc. sec | Acc. ratio | (pp) |
| 1,000 | 50 | 0.1 | 13.317 0.065 | 0.263 0.026 | 0.77 0.39 | |
| 1,000 | 50 | 0.5 | 13.303 0.064 | 0.2401 0.0057 | 2.06 0.88 | |
| 1,000 | 50 | 1 | 13.262 0.070 | 0.2348 0.0027 | 2.98 1.07 | |
| 1,000 | 50 | 2 | 13.270 0.069 | 0.2341 0.0027 | 3.03 1.06 | |
| 1,000 | 100 | 0.1 | 13.91 0.15 | 0.465 0.017 | 0.49 0.06 | |
| 1,000 | 100 | 0.5 | 13.83 0.15 | 0.4472 0.0065 | 0.78 0.42 |
Table 4: Marginal-DRO acceleration comparison, where the values following denote the standard errors of the means computed across 50 random seeds.
| Samples | Features | Radius | Base sec | Acc. sec | Acc. ratio | (pp) |
| 1,000 | 50 | 0.0005 | 1276 18 | 98.53 0.66 | 0.54 0.08 | |
| 1,000 | 50 | 0.001 | 1252 25 | 94.01 0.84 | 0.66 0.10 | |
| 1,000 | 50 | 0.002 | 1240 15 | 98.5 1.5 | 0.70 0.10 | |
| 1,000 | 50 | 0.005 | 1389 33 | 95.50 0.64 | 1.30 0.18 | |
| 1,000 | 100 | 0.0005 | 1333 27 | 106.0 1.3 | 0.74 0.14 | |
| 1,000 | 100 | 0.001 | 1298 19 | 105.12 0.88 | 0.82 0.16 |
Table 5: MMD-DRO acceleration comparison with 1,000 Nyström components, where the values following denote the standard errors of the means computed across 50 random seeds.
More in this topic
Distributionally Robust Optimization
Distributionally robust optimization (DRO) minimizes risk or cost against a set of plausible probability distributions rather than a single estimated one.
See all 96 papersWe study distributionally robust optimization (DRO) for robust inference when the worst-case distribution is continuous, leading to significant computational challenges due to the infinite-dimensional nature of the optimization problem. Unlike traditional discrete DRO approaches, which often suffer from scalability issues, limited generalization, and costly worst-case inference, our framework exploits Brenier's theorem to characterize the least favorable distribution as the pushforward of a transport map from a continuous reference measure. This characterization motivates our study of the minimax problem in Wasserstein space. We propose an iterative algorithmic framework with multiple variants and establish global convergence guarantees under mild assumptions, deriving complexity bounds in terms of subgradient evaluations and inexact Jordan-Kinderlehrer-Otto updates. Numerical results with neural network-based transport maps demonstrate that the proposed method enables both stable training of robust classifiers and effective worst-case inference for classification tasks.
Distributionally robust linear regression through the lens of adversarial training
Distributionally robust optimization (DRO) studies parameter estimation under uncertainty in the underlying probability distribution and has emerged as a principled framework for analyzing robustness and generalization. In particular, Wasserstein DRO, with distributional uncertainty induced by the Wasserstein distance, generalizes several popular regularizers. This paper studies Wasserstein DRO linear regression, unifying square-root Lasso and adversarial linear regression as important special cases. We prove that many properties of these two special cases carry over to this general method. In particular, we show (i) deterministic and non-asymptotic in-sample error bounds in general and under design matrix and sparsity conditions; (ii) insensitivity to the noise level, also known as the pivotal property; and (iii) solution equivalences for small and large ambiguity sets. The key proof step is to recast the method into a quadratic form, mimicking adversarial linear regression. We also show that the method can be solved efficiently, and we validate our findings through numerical simulations.
Regret-Optimal Wasserstein-Robust Linear Regression
Distributionally robust optimization (DRO) minimizes worst-case risk under distributional uncertainty, but its decisions can be suboptimal in terms of their excess risk, or regret. We study this gap for multivariate least-squares regression, assuming that the observed source law and unknown target law differ by an adversarial 2-Wasserstein perturbation of size . We first identify the asymptotic complexity of robust decision-making in the limit, characterized by a problem-specific condition number . Here, a wide spectrum of simple algorithms, including DRO, achieve the instance-optimal rate of . For fixed , we prove that DRO still achieves the minimax rate if the problem has bounded residual effective rank. On the other hand, we identify significant failure modes where DRO is provably suboptimal by dimension-dependent factors. Fortunately, we prove that two alternative procedures, minimax regret optimization (MRO) and spectral-norm regularization (SNR), attain the minimax rate. We derive an exact convex reformulation of MRO and an accelerated proximal algorithm for SNR, along with high-probability finite-sample guarantees. We support these theoretical results with numerical experiments separating the performance of MRO and SNR from DRO.