The Revenue Cost of Inference in Online Assortment Optimization
Organizations: Department of Management Science and Engineering, Tsinghua University, Beijing, China. · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore.
Abstract
A seller running an assortment experiment must earn revenue while collecting data to compare future offers. For a regular finite-dimensional parametric choice model and a finite assortment family, we characterize the oracle frontier between expected revenue loss and the largest local first-order variance constant across prespecified revenue comparisons. The oracle benchmark remains finite at zero cost exactly when revenue-maximizing assortments provide information in every target-gradient direction. When costly offers can supply the missing information, we derive the exact leading coefficient of the inverse-budget variance law and the limiting optimal spending shares. A pilot-based procedure reserves reference offers to preserve information and tightens the estimated budget constraint to account for uncertainty. We give conditions for the implemented design's variance criterion to converge to the stabilized oracle value at a fixed budget and reference share. Simulations with synthetic instances and Expedia-calibrated markets show that allocating offers across the full assortment family can improve the precision of revenue comparisons under a common opportunity-cost budget. Additional experiments assess pilot spending, budget protection, and sensitivity to choice-model misspecification.
Figures & tables
| Design | ||
|---|---|---|
| Balanced singletons | 1.1543 | 0.3276 |
| Optimized singletons | 0.6582 | 0.2747 |
| Target-plus-reference | 0.4109 | 0.1968 |
| Full SDP | 0.2864 | 0.06649 |
| Market | Shared pilot | Full | Restricted | Equal pool | Optimized pool | D-optimal |
|---|---|---|---|---|---|---|
| 4562 | 2,320.84 | 2.388 / 158.15 | 3.144 / 196.05 | 3.955 / 195.25 | 3.461 / 195.52 | 2.772 / 121.53 |
| 8192 | 2,065.57 | 2.266 / 129.23 | 2.917 / 141.48 | 3.274 / 140.98 | 2.893 / 141.35 | 2.813 / 153.77 |
| 9402 | 2,575.86 | 3.308 / 84.67 | 4.180 / 212.90 | 4.971 / 229.58 | 4.370 / 229.24 | 3.736 / 132.13 |
| 13870 | 2,021.69 | 2.697 / 72.62 | 4.506 / 123.35 | 3.864 / 121.57 | 3.386 / 122.05 | 3.468 / 126.20 |
| Test log loss | ||||||||
|---|---|---|---|---|---|---|---|---|
| Market | Anchor | Retained | Calib. | Valid. | Test | Hotel-specific | Shared | Difference |
| 4562 | 53494 | 2501 | 1539 | 497 | 465 | 0.5820 | 0.5879 | 0.0059 |
| 8192 | 116942 | 4691 | 2870 | 916 | 905 | 0.8759 | 0.8812 | 0.0053 |
| 9402 | 35679 | 1976 | 1172 | 409 | 395 | 0.6366 | 0.6462 | 0.0095 |
| 13870 | 106230 | 1720 | 1055 | 323 | 342 | 0.7132 | 0.7156 | 0.0024 |
| Post-pilot estimation | Empirical all-planned refit | |||||
|---|---|---|---|---|---|---|
| Market | Reduction | 95% interval | Holm | Reduction | 95% interval | Holm |
| 4562 | 22.19 | [18.43, 25.58] | 4/4 | 7.29 | [3.56, 10.59] | 0/4 |
| 8192 | 24.96 | [21.37, 28.21] | 4/4 | 12.72 | [8.61, 16.31] | 1/4 |
| 9402 | 28.18 | [24.65, 31.46] | 4/4 | 14.28 | [10.66, 17.65] | 3/4 |
| 13870 | 21.85 | [18.17, 25.26] | 4/4 | 11.19 | [7.35, 14.65] | 3/4 |
| All sixteen targets | 24.29 | [22.48, 25.86] | 16/16 | 11.37 | [9.47, 13.03] | 7/16 |
| Pilot-best +reference | Pilot-matched mixture | D-optimal (pilot-tightened) | D-optimal 95% interval | |
|---|---|---|---|---|
| 0 | 17.35 | 18.10 | 12.26 | [10.38, 13.98] |
| 0.1 | 17.25 | 18.02 | 12.22 | [10.34, 13.94] |
| 1 | 16.45 | 17.41 | 11.97 | [10.10, 13.69] |
| Panel A. Zero-floor frontier | ||||||
|---|---|---|---|---|---|---|
| Market | Hotel | Regime | Line | Optimal paid offer | ||
| 4562 | 61529 | Free | Solid | 0.035327 | — | — |
| 4562 | 79740 | Free | Dashed | 0.033463 | — | — |
| 4562 | 21155 | Costly | Dashed | 0.313835 | ||
| 4562 | 14536 | Costly | Solid | 1.356274 | ||
| 8192 | 77089 | Costly | Solid | 1.017735 | ||
| Contract | Comparator | Post gain | 95% interval | All gain | 95% interval |
|---|---|---|---|---|---|
| Ceiling | Restricted | 26.84 | [8.98, 41.14] | 10.34 | [-10.96, 27.56] |
| Ceiling | Equal pool | 33.51 | [17.12, 46.61] | 10.85 | [-9.97, 27.76] |
| Ceiling | Optimized pool | 24.34 | [6.14, 38.99] | 10.03 | [-11.74, 27.51] |
| Ceiling | D-optimal | 16.74 | [-3.58, 33.06] | 11.30 | [-9.74, 28.30] |
| Match | Restricted a | 41.98 | [27.71, 53.36] | 8.04 | [-13.63, 25.58] |
| Match | Equal pool | 26.96 | [8.64, 41.48] | 7.08 | [-14.89, 24.84] |
| Comparator | Post gain | 95% interval | All gain | 95% interval |
|---|---|---|---|---|
| Restricted | 30.25 | [13.36, 43.68] | 18.98 | [0.12, 34.28] |
| Optimized pool | 14.74 | [-5.30, 30.84] | 12.05 | [-8.26, 28.49] |
| D-optimal | 7.10 | [-14.93, 24.76] | 5.99 | [-15.61, 23.54] |
| Full post-pilot | Full all-planned | Direct | Total cost | |||||
| MAE / bias a | Coverage (%) | MAE / bias a | Coverage (%) | MAE / bias a | Coverage (%) | Full | Direct | |
| Panel A. 20k/5k, Lipschitz certificate | ||||||||
| 0 | 2.665 / 0.937 | 95.6 | 1.803 / 0.338 | 94.6 | 4.570 / 0.424 | 95.0 | 2,357.16 | 1,528.43 |
| 0.5 | 2.771 / 1.099 | 95.8 | 1.833 / 0.460 | 92.8 | 4.732 / 0.380 | 94.6 | 2,372.63 | 1,546.08 |
| 1.5 | 4.371 / 3.834 | 84.0 | 3.032 / 2.614 | 56.0 | 5.091 / 0.361 | 94.0 | 2,655.79 | 1,852.29 |
| Panel B. 5k/20k, box certificate | ||||||||
| Comparator | Common ceiling | Matched cost | Independent 5k/20k |
|---|---|---|---|
| Restricted | |||
| Equal pool | — | ||
| Optimized pool | |||
| D-optimal |
Assortment Optimization
Assortment optimization selects and positions products to maximize revenue or platform value under customer-choice models and operational constraints.
See all 15 papers