Adaptive Scenario Generation for KL-DRO Around a Learned Density Model

Aron Brenner; Saurabh Amin

eess.SYOct 29, 2026

Adaptive Scenario Generation for KL-DRO Around a Learned Density Model

Authors: Aron Brenner, Saurabh Amin

Organizations: Laboratory for Information & Decision Systems Massachusetts Institute of Technology Cambridge, MA 02139, USA

Abstract

Distributionally robust optimization (DRO) hedges data-driven two-stage stochastic programs against error in the distribution estimated from a finite record, and standard ambiguity sets differ in how they treat realizations absent from that record. A divergence ball around the empirical distribution contains only reweightings of the record, whereas a transport ball also contains distributions supported off the record, including on realizations the data-generating process cannot produce, against which the decision is then hedged as well. We instead propose a Kullback-Leibler (KL) ball centered at a density model fitted to the record, the learned nominal, which lets the adversary up-weight costly unobserved realizations in proportion to the probability the model assigns them. Solving the resulting problem poses a second difficulty, because it can accommodate only a few scenarios, and an independent sample of that size is likely to omit the rare realizations on which the worst case concentrates. We therefore build the scenario set adaptively, alternating between solving the finite problem and drawing new realizations from the learned nominal, steered toward the worst case at the current decision, with importance weights that correct the draws back to the model. On a multi-item newsvendor problem and a capacity expansion problem on realistic grid data, centering the ball at the learned nominal rather than at the record lowers cost under distribution shift, at comparable in-distribution cost, when the record is scarce and the model class captures structure that lets it extrapolate beyond the record. The advantage fades as the record grows or as the shift departs from the structure the model has learned.

Figures & tables

	(a) OOD cost at SAA’s ID budget								(b) ID cost to halve SAA’s OOD cost
	$N=16$				$N=128$				$N=16$				$N=128$
Method	L	T	S	C	L	T	S	C	L	T	S	C	L	T	S	C
SAA	643	154	434	244	129	88	98	39	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$
KL-Rec	647	155	435	227	141	89	111	39	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$
W-DRO	80	39	92	154	24	45	30	18	55	68	55	97	12	42	12	20
KL-OT	88	43	99	159	45	57	44	25	57	64	57	97	12	29	12	$\cdot$

Table 1: Newsvendor: cost above the optimum (%, median over records) under the level (L), tail (T), support (S), and coincidence (C) shifts. (a) Out-of-distribution cost at each method’s selection rule. (b) In-distribution cost needed to halve SAA’s out-of-distribution cost;

\cdot

: never reached; † : lowest in under 80% of bootstrap resamples. Bold: lowest among the rows shown.

Figure 2: Capacity expansion at $k=\tfrac{1}{2}$ (panels defined in App. B.6 ). (a) OOD against ID cost (bn$/yr) of each setting, with Pareto frontiers; KL-Rec, CSSC, and SAA lie outside the range. (b) Zonal wind on the most severe OOD day and admitted scenarios (top), and their spatial coherence (bottom). (c) Severity against demand-rescaled distance to the nearest real day, for the most severe admitted scenarios. (d) Demand relative to its 2001–2010 maximum against system wind; stars: the 20 most severe OOD days.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: The four newsvendor shift families: joint densities of log demand for products 1 and 3 (1 and 2 in (c)) under the records’ distribution (top) and the shifted distribution (bottom), with contours spanning 3.5 orders of magnitude so that the shocks are visible. (a) Level: the density translates. (b) Tail: the shock spreads into a wide, faint band. (c) Support: a new component appears where the records’ distribution has almost no mass. (d) Coincidence: shocks to one product at a time become joint shocks, with unchanged marginals.

Figure 4: What each nominal can represent: 4,000 draws from each nominal fitted to one record of $N=16$ (top) and one of $N=128$ (bottom), shown for products 1 and 3 (black points: the record; dashed box: its range). The copula’s draws stay within the record’s range, while the other nominals extrapolate beyond it. Here, cross-validation keeps the flow at its log-normal initialization, and the heavy-tailed flow selects a log- $t$ with 30 degrees of freedom.

Method	Scenarios in the master	Weights	Radius
SAA	the $N$ record points	uniform	$0$
KL-Rec	the $N$ record points	uniform	grid
KL-Gen-P, oracle	$n=100$ draws of the sampler	$\varphi/r$	grid
KL-Gen	356 draws of the nominal, resampled 100 times by the tilt	$c_{i}/(356\,\tau_{i})$	grid
CSSC	$K=100$ medoids of 1,024 draws of the nominal	cluster mass	grid
W-DRO	the $N$ record points, moved	uniform	$0$

Appendix

Table 2: Newsvendor methods. KL-Gen-P and KL-Gen are run with each nominal of Figure 4 , and CSSC with the log-normal and the log-normal mixture. Weights are normalized to sum to one.

Figure 5: Newsvendor frontiers: out-of-distribution against in-distribution cost above the optimum (%, median over records) as each method’s settings vary. Each line is a method’s Pareto frontier (lower left is better). Rows are record sizes and columns shifts. The open square is SAA, and the dashed line is half of SAA’s out-of-distribution cost, the target of Table 1 (b).

Gaussian
	(a) OOD cost at SAA’s ID budget				ID cost	(b) ID cost to halve SAA’s OOD
Method	L	T	S	C	$\rho=0$	L	T	S	C
SAA	643 $\pm$ 59	154 $\pm$ 8	434 $\pm$ 39	244 $\pm$ 21	184	$\cdot$	$\cdot$	$\cdot$	$\cdot$
KL-Rec	647 $\pm$ 55	155 $\pm$ 7	435 $\pm$ 36	227 $\pm$ 20	184	$\cdot$	$\cdot$	$\cdot$	$\cdot$
W-DRO	80 $\pm$ 10	39 $\pm$ 4	92 $\pm$ 10	154 $\pm$ 20	–	55	68	55	97
KL-OT	88 $\pm$ 11	43 $\pm$ 4	99 $\pm$ 11	159 $\pm$ 20	–	57	64	57	97

Appendix

Table 3: Newsvendor, every method,

N=16

(300 records): cost above the optimum (%) under the level (L), tail (T), support (S), and coincidence (C) shifts. (a) Median out-of-distribution cost (

\pm

2 SE) at each method’s selection rule. ID cost at

\rho=0

: median in-distribution cost of the unhedged fit (none for W-DRO and KL-OT). (b) Lowest median in-distribution cost at which a setting halves SAA’s median out-of-distribution cost;

\cdot

: never. Bold: lowest per column, excluding the oracle; † : lowest in fewer than 80% of bootstrap resamples.

Gaussian
	(a) OOD cost at SAA’s ID budget				ID cost	(b) ID cost to halve SAA’s OOD
Method	L	T	S	C	$\rho=0$	L	T	S	C
SAA	412 $\pm$ 41	126 $\pm$ 5	273 $\pm$ 26	149 $\pm$ 14	87	$\cdot$	$\cdot$	$\cdot$	$\cdot$
KL-Rec	412 $\pm$ 41	126 $\pm$ 5	284 $\pm$ 27	143 $\pm$ 13	87	$\cdot$	$\cdot$	$\cdot$	$\cdot$
W-DRO	46 $\pm$ 5	23 $\pm$ 3	57 $\pm$ 6	85 $\pm$ 11	–	26	43	26	56
KL-OT	46 $\pm$ 5	24 $\pm$ 3	61 $\pm$ 6	98 $\pm$ 12	–	24	37	24	56

Appendix

Table 4: Newsvendor, every method,

N=32

(300 records); as Table 3 .

Gaussian
	(a) OOD cost at SAA’s ID budget				ID cost	(b) ID cost to halve SAA’s OOD
Method	L	T	S	C	$\rho=0$	L	T	S	C
SAA	245 $\pm$ 30	107 $\pm$ 4	169 $\pm$ 18	76 $\pm$ 10	38	$\cdot$	$\cdot$	$\cdot$	$\cdot$
KL-Rec	264 $\pm$ 30	108 $\pm$ 4	182 $\pm$ 18	78 $\pm$ 8	38	$\cdot$	$\cdot$	$\cdot$	$\cdot$
W-DRO	31 $\pm$ 4	34 $\pm$ 3	44 $\pm$ 5	37 $\pm$ 4	–	16	32	16	36
KL-OT	39 $\pm$ 8	45 $\pm$ 4	48 $\pm$ 6	51 $\pm$ 7	–	16	29	16	36

Appendix

Table 5: Newsvendor, every method,

N=64

(300 records); as Table 3 .

Gaussian
	(a) OOD cost at SAA’s ID budget				ID cost	(b) ID cost to halve SAA’s OOD
Method	L	T	S	C	$\rho=0$	L	T	S	C
SAA	129 $\pm$ 17	88 $\pm$ 3	98 $\pm$ 11	39 $\pm$ 5	15	$\cdot$	$\cdot$	$\cdot$	$\cdot$
KL-Rec	141 $\pm$ 19	89 $\pm$ 3	111 $\pm$ 13	39 $\pm$ 4	15	$\cdot$	$\cdot$	$\cdot$	$\cdot$
W-DRO	24 $\pm$ 5	45 $\pm$ 3	30 $\pm$ 3	18 $\pm$ 3	–	12	42	12	20
KL-OT	45 $\pm$ 8	57 $\pm$ 4	44 $\pm$ 6	25 $\pm$ 3	–	12	29	12	$\cdot$

Appendix

Table 6: Newsvendor, every method,

N=128

(300 records); as Table 3 .

Figure 6: The 20 zones of the capacity expansion model. Counties of the ERCOT region are shaded by zone (other Texas counties in gray) and zone boundaries are drawn in dark gray. Lines join zone centroids along the 40 candidate transmission corridors, and centroid markers are sized by the zone’s share of 2010 demand.

Technology	Capital	Fixed O&M	VOM	Heat rate	Lifetime
	($/kW)	($/kW-yr)	($/MWh)	(MMBtu/MWh)	(yr)
Wind	1494	31.0	0.0	–	30
Solar	1266	19.1	0.0	–	30
Gas, combined cycle	1489	33.7	2.2	6.4	30
Gas, combustion turbine	1348	26.5	7.4	9.5	30
Battery 4h	1540	34.5	0.0	–	15

Appendix

Table 7: Cost and technology parameters, in real 2024 dollars. Generation costs are from the 2024 Annual Technology Baseline of the National Renewable Energy Laboratory ( Mirletz et al., 2024 ) (Moderate scenario, 2030, converted from 2022 dollars by

\times 1.061

), and heat rates are assumed. The gas price and emissions rate are from the Energy Information Administration (Annual Energy Outlook 2024, 2030). Capital costs are annualized with the capital recovery factor

\mathrm{CRF}(r,L)=r(1+r)^{L}/((1+r)^{L}-1)

over the lifetime

L

, plus fixed operation and maintenance (O&M). VOM is variable O&M.

Period	Years	Days	Role
Record	2001–2010	$10\times 364$	records and fitted nominals
In distribution (ID)	2011–2014	1,458	evaluation
Out of distribution (OOD)	2020	364	evaluation

Appendix

Table 8: Data periods. Records A to J are the years 2010, 2009, 2007, 2008, 2005, 2001, 2002, 2003, 2004, and 2006. The years 2015–2019 are used only to place 2020 among the other years.

Method	Candidates per round	Scenarios in the master	Weights
KL-Rec	256 record days	at most 48, four per round	KL worst case at $\rho$
KL-Gen	256 draws of the flow	same	same
KL-Gen-P	192 draws from a fitted proposal	same	same
KL-Gen (copula)	256 copula draws	same	same
KL-Gen-P (copula)	192 draws from a proposal fitted to the copula	same	same
KL-OT ( $\delta$ )	256 record days, moved from the second round	same	same

Appendix

Table 9: Capacity expansion methods. All use the objective in equation 12 with

\beta=0.25

\alpha=0.9

, and the budget

B

Figure 7: The two radii act as designed (capacity expansion,

k=\tfrac{1}{2}

). Boxes span the interquartile range over the ten records (each the median over five seeds; in (a), seeds are pooled), and whiskers the full range. (a) Share of admitted scenarios above the record’s 99th percentile of peak net load, at

\rho=0

(dotted: the record’s own 1 %). Raising

\rho_{\mathrm{ref}}

admits more severe scenarios, in 37 of 40 (method, record) pairs. (b) Effective sample size of the worst-case weights, as a share of the admitted scenarios, at

\rho_{\mathrm{ref}}=0.2

. It falls as

\rho

grows, and the realized divergence equals

\rho

in every run. (c) Capacity added relative to

\rho=0

(GW) at

\rho_{\mathrm{ref}}=0.2

, paired within record and seed. Raising

\rho

moves capital toward the budget and changes the mix.

Figure 8: KL-Gen’s advantage across record lengths (capacity expansion). Paired cost difference (bn

/yr; negative: KL-Gen is cheaper) against (a) KL-Rec, (b) KL-OT (

\delta=1

), and (c) KL-OT (

\delta=3

), out of distribution (filled) and in distribution (hatched). Each record contributes the median, over the radius pairs of Table 10 at which both methods were run (all five; for

\delta=3

, three at

k\leq 1

and two at

k=5

), of its median over seeds. Boxes span the interquartile range over records and whiskers the full range; the numbers are two-sided Wilcoxon signed-rank

-values over records (with five records, the smallest attainable is 0.0625). The out-of-distribution gap to KL-Rec and to KL-OT (

\delta=1

) shrinks as the record grows; KL-OT (

\delta=3

) narrows it at

k=\tfrac{1}{2}$ at a higher in-distribution cost.

$k=\frac{1}{2}$ (10 record years)
		In-distribution					Out-of-distribution
$\rho_{\mathrm{ref}}$	$\rho$	KL-Rec	KL-OT	KL-Gen	KL-Gen-P	KL-Gen (copula)	KL-Rec	KL-OT	KL-Gen	KL-Gen-P	KL-Gen (copula)
0.2	0	33.44	30.35	30.34	30.71	30.39	56.96	38.42	33.91	33.35	38.79
	0.05	32.67	30.43	30.89	30.65	31.09	53.62	35.97	33.07	32.90	33.51
	0.2	32.44	30.73	31.14	31.00	31.08	50.19	38.86	33.19	33.17	33.66
0.4	0	32.31	30.29	30.53	30.48	30.34	55.71	40.57	33.58	33.60	34.30

Appendix

Table 10: Capacity expansion: in-distribution (2011–2014) and out-of-distribution (2020) cost (bn

/yr) at the five radius pairs

(\rho_{\mathrm{ref}},\rho)

run at every

, and of SAA, CSSC, and W-DRO, which have no KL radii (median over records of the median over five seeds). Standard errors over records are 0.02–1.6 in distribution and 0.05–6.5 out of distribution for the methods with radii, and up to 6.7 and 15.6 for the others. Table 11 gives paired differences. Bold: on the Pareto front at that

. KL-OT and W-DRO use

\delta=1$ .

$k=\frac{1}{2}$ (10 record years)
		In-distribution				Out-of-distribution
$\rho_{\mathrm{ref}}$	$\rho$	KL-Gen $-$ KL-Rec	KL-Gen $-$ KL-OT	KL-Gen-P $-$ KL-Gen	KL-Gen $-$ KL-Gen (copula)	KL-Gen $-$ KL-Rec	KL-Gen $-$ KL-OT	KL-Gen-P $-$ KL-Gen	KL-Gen $-$ KL-Gen (copula)
0.2	0	$-2.92$ (1.02) ∗	$+0.14$ (0.30)	$-0.07$ (0.12)	$+0.30$ (0.15)	$-22.21$ (4.64) ∗∗	$-2.65$ (2.22) ∗	$-0.27$ (0.75)	$-1.44$ (0.87)
	0.05	$-1.72$ (1.06)	$+0.28$ (0.30)	$-0.05$ (0.14)	$+0.19$ (0.16)	$-18.29$ (5.13) ∗∗	$-2.41$ (2.85)	$-0.07$ (0.30)	$-0.55$ (0.90)
	0.2	$-1.19$ (0.78) ∗	$+0.31$ (0.15)	$+0.20$ (0.15)	$-0.06$ (0.25)	$-14.95$ (3.86) ∗∗	$-2.28$ (1.48) ∗∗	$-0.10$ (0.58)	$+0.05$ (0.58)
0.4	0	$-1.93$ (1.14) ∗	$+0.12$ (0.45)	$+0.04$ (0.18)	$+0.16$ (0.15)	$-17.66$ (4.45) ∗∗	$-6.51$ (2.96) ∗	$-0.13$ (1.36)	$-0.03$ (1.10)

Appendix

Table 11: Capacity expansion: paired cost differences (bn

/yr; negative: the first method is cheaper) at each radius pair of Table 10 : the median over records (standard error) of each record’s median over seeds. ∗ / ∗∗ : two-sided Wilcoxon signed-rank

p<0.05

/

0.01

over records. With five records (

k=1

) or blocks (

k=5

), the smallest attainable

is

0.0625$ .

Distributionally Robust Optimization

Distributionally robust optimization (DRO) minimizes risk or cost against a set of plausible probability distributions rather than a single estimated one.

See all 89 papers

Generative Distributionally Robust Optimization
Jul 27, 2026Ziwei Zhang, Jonathan Yu-Meng Li, Zhihao JinSinkhorn Divergence Distributionally Robust Optimization
1Telfer School of Management, University of Ottawa · Department of Electrical and Computer Engineering, Western University
Statistical Properties of Robust Learning under Distributional Shifts
Aug 13, 2026Zhiyi Li, Xiaojie Mao, Yunbei Xu +1Distribution Shift Robustness Distributionally Robust Optimization
National University of Singapore · Tsinghua University · University College London
Calibrating Ambiguity Set via Diagnostic Transport for Distributionally Robust Optimization
Oct 7, 2026Wenbin Zhou, Elizabeth Cucuzzella, Shixiang ZhuDistributionally Robust Optimization Wasserstein Distributionally Robust Optimization
Heinz College of Information Systems and Public Policy, Carnegie Mellon University · Machine Learning Department, Carnegie Mellon University · Department of Statistics and Data Science, Carnegie Mellon University

Jul 27, 2026J/K move · Enter open · S save

Generative Distributionally Robust Optimization

Ziwei Zhang, Jonathan Yu-Meng Li, Zhihao Jin

1Telfer School of Management, University of Ottawa · Department of Electrical and Computer Engineering, Western University

Sinkhorn Divergence Distributionally Robust Optimization

Open paper AskPDF arXiv

Generative models are increasingly adopted in distributionally robust optimization (DRO), but existing approaches trade off model compatibility and adversarial structure: methods that accept arbitrary samplers do not restrict worst-case laws to a generator family, while generator-parameterized adversaries rely on model-specific access such as likelihoods, scores, or training data. We propose Generative Distributionally Robust Optimization (GDRO), a principled framework that accepts any sampleable conditional generator as the nominal model and restricts worst-case laws to a chosen conditional generator family. The key is the sampler-Sinkhorn pairing: samplers represent the conditional laws exactly, while Sinkhorn divergence compares their induced distributions without likelihood access and can be estimated from samples alone. The resulting population problem admits a direct finite-sample approximation and differentiable primal-dual implementation at the active decision context. For Lipschitz losses, the population Sinkhorn radius bounds downstream degradation. Across explicit and implicit generators, our method reduces rare-context inventory regret by 60% and SocialGAN navigation collisions by 50% relative to nominal decisions.

Abstract

Distributionally Robust Optimization

Explore similar work

Generative Distributionally Robust Optimization

Statistical Properties of Robust Learning under Distributional Shifts

Calibrating Ambiguity Set via Diagnostic Transport for Distributionally Robust Optimization