Adaptive Scenario Generation for KL-DRO Around a Learned Density Model

Aron Brenner; Saurabh Amin

eess.SYOct 29, 2026

Adaptive Scenario Generation for KL-DRO Around a Learned Density Model

Authors: Aron Brenner, Saurabh Amin

Organizations: Laboratory for Information & Decision Systems Massachusetts Institute of Technology Cambridge, MA 02139, USA

Abstract

Distributionally robust optimization (DRO) hedges data-driven two-stage stochastic programs against error in the distribution estimated from a finite record, and standard ambiguity sets differ in how they treat realizations absent from that record. A divergence ball around the empirical distribution contains only reweightings of the record, whereas a transport ball also contains distributions supported off the record, including on realizations the data-generating process cannot produce, against which the decision is then hedged as well. We instead propose a Kullback-Leibler (KL) ball centered at a density model fitted to the record, the learned nominal, which lets the adversary up-weight costly unobserved realizations in proportion to the probability the model assigns them. Solving the resulting problem poses a second difficulty, because it can accommodate only a few scenarios, and an independent sample of that size is likely to omit the rare realizations on which the worst case concentrates. We therefore build the scenario set adaptively, alternating between solving the finite problem and drawing new realizations from the learned nominal, steered toward the worst case at the current decision, with importance weights that correct the draws back to the model. On a multi-item newsvendor problem and a capacity expansion problem on realistic grid data, centering the ball at the learned nominal rather than at the record lowers cost under distribution shift, at comparable in-distribution cost, when the record is scarce and the model class captures structure that lets it extrapolate beyond the record. The advantage fades as the record grows or as the shift departs from the structure the model has learned.

Figures & tables

	(a) OOD cost at SAA’s ID budget								(b) ID cost to halve SAA’s OOD cost
	$N=16$				$N=128$				$N=16$				$N=128$
Method	L	T	S	C	L	T	S	C	L	T	S	C	L	T	S	C
SAA	643	154	434	244	129	88	98	39	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$
KL-Rec	647	155	435	227	141	89	111	39	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$
W-DRO	80	39	92	154	24	45	30	18	55	68	55	97	12	42	12	20
KL-OT	88	43	99	159	45	57	44	25	57	64	57	97	12	29	12	$\cdot$

Table 1: Newsvendor: cost above the optimum (%, median over records) under the level (L), tail (T), support (S), and coincidence (C) shifts. (a) Out-of-distribution cost at each method’s selection rule. (b) In-distribution cost needed to halve SAA’s out-of-distribution cost;

\cdot

: never reached; † : lowest in under 80% of bootstrap resamples. Bold: lowest among the rows shown.

Figure 2: Capacity expansion at $k=\tfrac{1}{2}$ (panels defined in App. B.6 ). (a) OOD against ID cost (bn$/yr) of each setting, with Pareto frontiers; KL-Rec, CSSC, and SAA lie outside the range. (b) Zonal wind on the most severe OOD day and admitted scenarios (top), and their spatial coherence (bottom). (c) Severity against demand-rescaled distance to the nearest real day, for the most severe admitted scenarios. (d) Demand relative to its 2001–2010 maximum against system wind; stars: the 20 most severe OOD days.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: The four newsvendor shift families: joint densities of log demand for products 1 and 3 (1 and 2 in (c)) under the records’ distribution (top) and the shifted distribution (bottom), with contours spanning 3.5 orders of magnitude so that the shocks are visible. (a) Level: the density translates. (b) Tail: the shock spreads into a wide, faint band. (c) Support: a new component appears where the records’ distribution has almost no mass. (d) Coincidence: shocks to one product at a time become joint shocks, with unchanged marginals.

Figure 4: What each nominal can represent: 4,000 draws from each nominal fitted to one record of $N=16$ (top) and one of $N=128$ (bottom), shown for products 1 and 3 (black points: the record; dashed box: its range). The copula’s draws stay within the record’s range, while the other nominals extrapolate beyond it. Here, cross-validation keeps the flow at its log-normal initialization, and the heavy-tailed flow selects a log- $t$ with 30 degrees of freedom.

Method	Scenarios in the master	Weights	Radius
SAA	the $N$ record points	uniform	$0$
KL-Rec	the $N$ record points	uniform	grid
KL-Gen-P, oracle	$n=100$ draws of the sampler	$\varphi/r$	grid
KL-Gen	356 draws of the nominal, resampled 100 times by the tilt	$c_{i}/(356\,\tau_{i})$	grid
CSSC	$K=100$ medoids of 1,024 draws of the nominal	cluster mass	grid
W-DRO	the $N$ record points, moved	uniform	$0$

Appendix

Table 2: Newsvendor methods. KL-Gen-P and KL-Gen are run with each nominal of Figure 4 , and CSSC with the log-normal and the log-normal mixture. Weights are normalized to sum to one.

Figure 5: Newsvendor frontiers: out-of-distribution against in-distribution cost above the optimum (%, median over records) as each method’s settings vary. Each line is a method’s Pareto frontier (lower left is better). Rows are record sizes and columns shifts. The open square is SAA, and the dashed line is half of SAA’s out-of-distribution cost, the target of Table 1 (b).

Gaussian
	(a) OOD cost at SAA’s ID budget				ID cost	(b) ID cost to halve SAA’s OOD
Method	L	T	S	C	$\rho=0$	L	T	S	C
SAA	643 $\pm$ 59	154 $\pm$ 8	434 $\pm$ 39	244 $\pm$ 21	184	$\cdot$	$\cdot$	$\cdot$	$\cdot$
KL-Rec	647 $\pm$ 55	155 $\pm$ 7	435 $\pm$ 36	227 $\pm$ 20	184	$\cdot$	$\cdot$	$\cdot$	$\cdot$
W-DRO	80 $\pm$ 10	39 $\pm$ 4	92 $\pm$ 10	154 $\pm$ 20	–	55	68	55	97
KL-OT	88 $\pm$ 11	43 $\pm$ 4	99 $\pm$ 11	159 $\pm$ 20	–	57	64	57	97

Appendix

Table 3: Newsvendor, every method,

N=16

(300 records): cost above the optimum (%) under the level (L), tail (T), support (S), and coincidence (C) shifts. (a) Median out-of-distribution cost (

\pm

2 SE) at each method’s selection rule. ID cost at

\rho=0

: median in-distribution cost of the unhedged fit (none for W-DRO and KL-OT). (b) Lowest median in-distribution cost at which a setting halves SAA’s median out-of-distribution cost;

\cdot

: never. Bold: lowest per column, excluding the oracle; † : lowest in fewer than 80% of bootstrap resamples.

Gaussian
	(a) OOD cost at SAA’s ID budget				ID cost	(b) ID cost to halve SAA’s OOD
Method	L	T	S	C	$\rho=0$	L	T	S	C
SAA	412 $\pm$ 41	126 $\pm$ 5	273 $\pm$ 26	149 $\pm$ 14	87	$\cdot$	$\cdot$	$\cdot$	$\cdot$
KL-Rec	412 $\pm$ 41	126 $\pm$ 5	284 $\pm$ 27	143 $\pm$ 13	87	$\cdot$	$\cdot$	$\cdot$	$\cdot$
W-DRO	46 $\pm$ 5	23 $\pm$ 3	57 $\pm$ 6	85 $\pm$ 11	–	26	43	26	56
KL-OT	46 $\pm$ 5	24 $\pm$ 3	61 $\pm$ 6	98 $\pm$ 12	–	24	37	24	56

Appendix

Table 4: Newsvendor, every method,

N=32

(300 records); as Table 3 .

Gaussian
	(a) OOD cost at SAA’s ID budget				ID cost	(b) ID cost to halve SAA’s OOD
Method	L	T	S	C	$\rho=0$	L	T	S	C
SAA	245 $\pm$ 30	107 $\pm$ 4	169 $\pm$ 18	76 $\pm$ 10	38	$\cdot$	$\cdot$	$\cdot$	$\cdot$
KL-Rec	264 $\pm$ 30	108 $\pm$ 4	182 $\pm$ 18	78 $\pm$ 8	38	$\cdot$	$\cdot$	$\cdot$	$\cdot$
W-DRO	31 $\pm$ 4	34 $\pm$ 3	44 $\pm$ 5	37 $\pm$ 4	–	16	32	16	36
KL-OT	39 $\pm$ 8	45 $\pm$ 4	48 $\pm$ 6	51 $\pm$ 7	–	16	29	16	36

Appendix

Table 5: Newsvendor, every method,

N=64

(300 records); as Table 3 .

Gaussian
	(a) OOD cost at SAA’s ID budget				ID cost	(b) ID cost to halve SAA’s OOD
Method	L	T	S	C	$\rho=0$	L	T	S	C
SAA	129 $\pm$ 17	88 $\pm$ 3	98 $\pm$ 11	39 $\pm$ 5	15	$\cdot$	$\cdot$	$\cdot$	$\cdot$
KL-Rec	141 $\pm$ 19	89 $\pm$ 3	111 $\pm$ 13	39 $\pm$ 4	15	$\cdot$	$\cdot$	$\cdot$	$\cdot$
W-DRO	24 $\pm$ 5	45 $\pm$ 3	30 $\pm$ 3	18 $\pm$ 3	–	12	42	12	20
KL-OT	45 $\pm$ 8	57 $\pm$ 4	44 $\pm$ 6	25 $\pm$ 3	–	12	29	12	$\cdot$

Appendix

Table 6: Newsvendor, every method,

N=128

(300 records); as Table 3 .

Figure 6: The 20 zones of the capacity expansion model. Counties of the ERCOT region are shaded by zone (other Texas counties in gray) and zone boundaries are drawn in dark gray. Lines join zone centroids along the 40 candidate transmission corridors, and centroid markers are sized by the zone’s share of 2010 demand.

Technology	Capital	Fixed O&M	VOM	Heat rate	Lifetime
	($/kW)	($/kW-yr)	($/MWh)	(MMBtu/MWh)	(yr)
Wind	1494	31.0	0.0	–	30
Solar	1266	19.1	0.0	–	30
Gas, combined cycle	1489	33.7	2.2	6.4	30
Gas, combustion turbine	1348	26.5	7.4	9.5	30
Battery 4h	1540	34.5	0.0	–	15

Appendix

Table 7: Cost and technology parameters, in real 2024 dollars. Generation costs are from the 2024 Annual Technology Baseline of the National Renewable Energy Laboratory ( Mirletz et al., 2024 ) (Moderate scenario, 2030, converted from 2022 dollars by

\times 1.061

), and heat rates are assumed. The gas price and emissions rate are from the Energy Information Administration (Annual Energy Outlook 2024, 2030). Capital costs are annualized with the capital recovery factor

\mathrm{CRF}(r,L)=r(1+r)^{L}/((1+r)^{L}-1)

over the lifetime

L

, plus fixed operation and maintenance (O&M). VOM is variable O&M.

Period	Years	Days	Role
Record	2001–2010	$10\times 364$	records and fitted nominals
In distribution (ID)	2011–2014	1,458	evaluation
Out of distribution (OOD)	2020	364	evaluation

Appendix

Table 8: Data periods. Records A to J are the years 2010, 2009, 2007, 2008, 2005, 2001, 2002, 2003, 2004, and 2006. The years 2015–2019 are used only to place 2020 among the other years.

Method	Candidates per round	Scenarios in the master	Weights
KL-Rec	256 record days	at most 48, four per round	KL worst case at $\rho$
KL-Gen	256 draws of the flow	same	same
KL-Gen-P	192 draws from a fitted proposal	same	same
KL-Gen (copula)	256 copula draws	same	same
KL-Gen-P (copula)	192 draws from a proposal fitted to the copula	same	same
KL-OT ( $\delta$ )	256 record days, moved from the second round	same	same

Appendix

Table 9: Capacity expansion methods. All use the objective in equation 12 with

\beta=0.25

\alpha=0.9

, and the budget

B

Figure 7: The two radii act as designed (capacity expansion,

k=\tfrac{1}{2}

). Boxes span the interquartile range over the ten records (each the median over five seeds; in (a), seeds are pooled), and whiskers the full range. (a) Share of admitted scenarios above the record’s 99th percentile of peak net load, at

\rho=0

(dotted: the record’s own 1 %). Raising

\rho_{\mathrm{ref}}

admits more severe scenarios, in 37 of 40 (method, record) pairs. (b) Effective sample size of the worst-case weights, as a share of the admitted scenarios, at

\rho_{\mathrm{ref}}=0.2

. It falls as

\rho

grows, and the realized divergence equals

\rho

in every run. (c) Capacity added relative to

\rho=0

(GW) at

\rho_{\mathrm{ref}}=0.2

, paired within record and seed. Raising

\rho

moves capital toward the budget and changes the mix.

Figure 8: KL-Gen’s advantage across record lengths (capacity expansion). Paired cost difference (bn

/yr; negative: KL-Gen is cheaper) against (a) KL-Rec, (b) KL-OT (

\delta=1

), and (c) KL-OT (

\delta=3

), out of distribution (filled) and in distribution (hatched). Each record contributes the median, over the radius pairs of Table 10 at which both methods were run (all five; for

\delta=3

, three at

k\leq 1

and two at

k=5

), of its median over seeds. Boxes span the interquartile range over records and whiskers the full range; the numbers are two-sided Wilcoxon signed-rank

-values over records (with five records, the smallest attainable is 0.0625). The out-of-distribution gap to KL-Rec and to KL-OT (

\delta=1

) shrinks as the record grows; KL-OT (

\delta=3

) narrows it at

k=\tfrac{1}{2}$ at a higher in-distribution cost.

$k=\frac{1}{2}$ (10 record years)
		In-distribution					Out-of-distribution
$\rho_{\mathrm{ref}}$	$\rho$	KL-Rec	KL-OT	KL-Gen	KL-Gen-P	KL-Gen (copula)	KL-Rec	KL-OT	KL-Gen	KL-Gen-P	KL-Gen (copula)
0.2	0	33.44	30.35	30.34	30.71	30.39	56.96	38.42	33.91	33.35	38.79
	0.05	32.67	30.43	30.89	30.65	31.09	53.62	35.97	33.07	32.90	33.51
	0.2	32.44	30.73	31.14	31.00	31.08	50.19	38.86	33.19	33.17	33.66
0.4	0	32.31	30.29	30.53	30.48	30.34	55.71	40.57	33.58	33.60	34.30

Appendix

Table 10: Capacity expansion: in-distribution (2011–2014) and out-of-distribution (2020) cost (bn

/yr) at the five radius pairs

(\rho_{\mathrm{ref}},\rho)

run at every

, and of SAA, CSSC, and W-DRO, which have no KL radii (median over records of the median over five seeds). Standard errors over records are 0.02–1.6 in distribution and 0.05–6.5 out of distribution for the methods with radii, and up to 6.7 and 15.6 for the others. Table 11 gives paired differences. Bold: on the Pareto front at that

. KL-OT and W-DRO use

\delta=1$ .

$k=\frac{1}{2}$ (10 record years)
		In-distribution				Out-of-distribution
$\rho_{\mathrm{ref}}$	$\rho$	KL-Gen $-$ KL-Rec	KL-Gen $-$ KL-OT	KL-Gen-P $-$ KL-Gen	KL-Gen $-$ KL-Gen (copula)	KL-Gen $-$ KL-Rec	KL-Gen $-$ KL-OT	KL-Gen-P $-$ KL-Gen	KL-Gen $-$ KL-Gen (copula)
0.2	0	$-2.92$ (1.02) ∗	$+0.14$ (0.30)	$-0.07$ (0.12)	$+0.30$ (0.15)	$-22.21$ (4.64) ∗∗	$-2.65$ (2.22) ∗	$-0.27$ (0.75)	$-1.44$ (0.87)
	0.05	$-1.72$ (1.06)	$+0.28$ (0.30)	$-0.05$ (0.14)	$+0.19$ (0.16)	$-18.29$ (5.13) ∗∗	$-2.41$ (2.85)	$-0.07$ (0.30)	$-0.55$ (0.90)
	0.2	$-1.19$ (0.78) ∗	$+0.31$ (0.15)	$+0.20$ (0.15)	$-0.06$ (0.25)	$-14.95$ (3.86) ∗∗	$-2.28$ (1.48) ∗∗	$-0.10$ (0.58)	$+0.05$ (0.58)
0.4	0	$-1.93$ (1.14) ∗	$+0.12$ (0.45)	$+0.04$ (0.18)	$+0.16$ (0.15)	$-17.66$ (4.45) ∗∗	$-6.51$ (2.96) ∗	$-0.13$ (1.36)	$-0.03$ (1.10)

Appendix

Table 11: Capacity expansion: paired cost differences (bn

/yr; negative: the first method is cheaper) at each radius pair of Table 10 : the median over records (standard error) of each record’s median over seeds. ∗ / ∗∗ : two-sided Wilcoxon signed-rank

p<0.05

/

0.01

over records. With five records (

k=1

) or blocks (

k=5

), the smallest attainable

is

0.0625$ .

Distributionally Robust Optimization

Distributionally robust optimization (DRO) minimizes risk or cost against a set of plausible probability distributions rather than a single estimated one.

See all 89 papers

Jul 27, 2026

Generative Distributionally Robust Optimization

Generative models are increasingly adopted in distributionally robust optimization (DRO), but existing approaches trade off model compatibility and adversarial structure: methods that accept arbitrary samplers do not restrict worst-case laws to a generator family, while generator-parameterized adversaries rely on model-specific access such as likelihoods, scores, or training data. We propose Generative Distributionally Robust Optimization (GDRO), a principled framework that accepts any sampleable conditional generator as the nominal model and restricts worst-case laws to a chosen conditional generator family. The key is the sampler-Sinkhorn pairing: samplers represent the conditional laws exactly, while Sinkhorn divergence compares their induced distributions without likelihood access and can be estimated from samples alone. The resulting population problem admits a direct finite-sample approximation and differentiable primal-dual implementation at the active decision context. For Lipschitz losses, the population Sinkhorn radius bounds downstream degradation. Across explicit and implicit generators, our method reduces rare-context inventory regret by 60% and SocialGAN navigation collisions by 50% relative to nominal decisions.

Ziwei Zhang, Jonathan Yu-Meng Li, Zhihao Jin

1Telfer School of Management, University of Ottawa · Department of Electrical and Computer Engineering, Western University

Sinkhorn Divergence Distributionally Robust Optimization

Ask PDF

Aug 13, 2026

Statistical Properties of Robust Learning under Distributional Shifts

Distributional shifts arise when the target deployment environment differs from the source environment that generated the training data. Robust learning frameworks such as Distributionally Robust Optimization (DRO) and Robust Satisficing (RS) aim to address this challenge, yet their finite-sample guarantees under such shifts, and their systematic comparison, remain underexplored: existing analyses typically establish guarantees either in the source environment or for adversarial worst-case performance over an ambiguity set. This paper instead studies generalization error in the target environment---the excess loss under the shifted target distribution. Our contributions are threefold. First, we derive finite-sample generalization error bounds in the shifted target environment for both DRO and RS. These bounds explicitly characterize the trade-off between reduced sensitivity to shift and the regularization penalty induced by each method's robustness hyperparameter, and they avoid the curse of dimensionality associated with Wasserstein empirical concentration. Second, when partial shift information such as shift magnitude or direction is available, we propose information-directed hyperparameter calibrations and compare the two methods given the same information. Under these calibrations, and in the partial-information regimes we study, DRO and RS exhibit complementary theoretical and empirical behavior. Finally, we apply the framework to a network lot-sizing problem, using it to interpret how robust policies respond to positive shifts in the demand distribution. Together, these results fill a gap in understanding the statistical properties of robust learning methods under distributional shifts and provide a principled basis for comparing DRO and RS.

Zhiyi Li, Xiaojie Mao, Yunbei Xu +1

National University of Singapore · Tsinghua University · University College London

Distribution Shift Robustness Distributionally Robust Optimization Robust Optimization Statistical Learning Theory

Ask PDF

Oct 7, 2026

Calibrating Ambiguity Set via Diagnostic Transport for Distributionally Robust Optimization

Distributionally robust optimization (DRO) protects decisions against distributional uncertainty by optimizing over an ambiguity set, but poorly aligned set geometry can require large radii and yield overly conservative decisions. We introduce diagnostic-transport DRO (DT-DRO), which uses held-out calibration data to adapt the ambiguity-set geometry to observed predictive errors. DT-DRO uses the conditional probability integral transform cumulative distribution function to diagnose systematic probability misallocation and translates this information into an outcome-level transport that jointly adjusts the ambiguity-set center and ground cost. The resulting formulation admits a computationally tractable dual reformulation. Theoretically, we derive valid ambiguity radii and decision-risk guarantees that tighten as estimation and approximation errors vanish, and show that DT-DRO can eliminate the nonvanishing robustness floor caused by model misspecification. Synthetic experiments and a power-outage application demonstrate improved decision quality, particularly under structural and tail misspecification.

Wenbin Zhou, Elizabeth Cucuzzella, Shixiang Zhu

Heinz College of Information Systems and Public Policy, Carnegie Mellon University · Machine Learning Department, Carnegie Mellon University · Department of Statistics and Data Science, Carnegie Mellon University

Distributionally Robust Optimization Wasserstein Distributionally Robust Optimization Uncertainty Calibration

Ask PDF

Abstract

Distributionally Robust Optimization

Explore similar work

Generative Distributionally Robust Optimization

Statistical Properties of Robust Learning under Distributional Shifts

Calibrating Ambiguity Set via Diagnostic Transport for Distributionally Robust Optimization