Learning to bid and bundle: a posterior-sampling agent for online retail
Organizations: Utrecht University · aDepartment of Methodology, Statistics & Data Science, Utrecht University, Padualaan 14, 3584 CH Utrecht, The Netherlands
Abstract
An online retailer launching a perishable product bids for sponsored-search clicks, discounts a bundle with an always-stocked staple and displays its remaining stock, which raises willingness to pay. It knows neither how clicks respond to its bid nor how strong and dispersed its market is, and learns both over repeated seasons from data its own decisions censor: a zero bid reveals nothing about paid clicks, and choices follow only clicks. We design a posterior-sampling agent for it. Each period it draws a market from an exact posterior and posts the bid and discount of that market's exactly solved season plan, so every action is one a fully informed seller could take. In a simulated online florist it has the lowest regret averaged over markets. In weak markets it gives up 12% to 23% of one season's revenue over 500 seasons, four to seven times less than drawing once per season. A planner acting on point estimates can stop bidding and never learn that bidding pays. The agent escapes after about three seasons on average, against fifty with one draw per season. Where displayed scarcity keeps willingness to pay above the posted price, the discount needs no learning. Customers waiting for the discount to return to its recent level look like lost sales, so frequent waiting makes the agent underrate its market. One discount per season removes waiting at a cost of 5.9% of clairvoyant revenue in a weak market and none in a strong one.
Figures & tables
| Study | Decisions | Learned | Feedback | Customers |
|---|---|---|---|---|
| Rana & Oliveira (2015a) ; Qiao et al. (2024a) | prices of perishable products | a policy or value function, model-free | sales | myopic |
| Boer & Zwart (2015a) | price of a finite stock over repeated seasons | demand parameters | sales | myopic |
| Ferreira et al. (2018a) | prices against finite stocks | demand parameters | sales | myopic |
| Gurkan et al. (2026a) | price, advertising level, inventory | demand and advertising response | sales | myopic |
| Zhang & Jasin (2022a) | prices | valuations and patience | purchases over time | patient |
| This paper | per-click bid and bundle discount for a fixed stock | click curve, market strength and spread | clicks, informative only at positive bids; choices only after a click | myopic, or waiting for a better discount |
| Offline | Online | ||||
|---|---|---|---|---|---|
| Tables (s) | Plans (s) | MiB | Decision ( s) | Update ( s) | |
| 27 | 1.5 | 0.007 | 0.03 | 2.7 | 0.64 |
| 125 | 4.2 | 0.033 | 0.16 | 2.9 | 0.63 |
| 405 | 13.6 | 0.106 | 0.52 | 3.7 | 0.68 |
| 891 | 16.6 | 0.237 | 1.14 | 4.1 | 0.70 |
| Season | , , ; , controls on -grid |
|---|---|
| Prices, tastes | , ; , ; |
| Click curve | , , ; unknown |
| Grid | , |
| Truths | baseline ; low mean ; low dispersion |
| Replicates | , paired; for the four main learners in the main settings, in robustness and scaling; seeds |
| Seasons | headline, weak markets ; trap ; waiting ; robustness |
| Baseline truth | Bayes mode | ||
|---|---|---|---|
| Policy | 10 seasons | 500 seasons (s.e.) | 500 seasons (s.e.) |
| Agent ( ts-update ) | 22.1 | 25.4 (1.0) | 33.6 (1.6) |
| One draw per season ( psrl ) | 33.0 | 37.4 (0.6) | 109.4 (4.5) |
| Certainty equivalence | 9.5 | 11.4 (0.3) | 74.0 (5.6) |
| -greedy | 47.7 | 721.5 (3.5) | 887.8 (13.7) |
| Explore-then-commit | 1,169.6 | 1,175.4 (0.6) | 1,574.2 (80.8) |
| Rule | Signal to watch | Action | Evidence here |
|---|---|---|---|
| Draw every period | always | draw a market from the posterior in each period and post its plan’s bid and discount | regret 91.3 and 160.7 in the weak markets against 680.5 and 712.5 with one draw per season; 33.6 against 109.4 averaged over markets (Sections 6.2 and 6.4 ) |
| Keep a prior floor on bidding | before launch | put prior mass on click curves whose plans bid | bids within three seasons with probability (Proposition 3 ); without bidding, $464.9 is lost every season |
| Distrust a plan to stop advertising | the posterior-mean plan bids zero everywhere | keep sampling until a bid has moved the click-curve posterior | a zero bid reveals nothing about paid clicks (Lemma 1 ) |
| Freeze the discount where scarcity carries the price | the estimated share of full-stock shoppers who would not buy the product at its list price is below a small | post the harvest discount ( \5.40$ here) and learn only the bid | the loss is at most of order (Proposition 1 ); tuning buys nothing there |
| Commit the discount when shoppers wait | bundle purchases without a click rise after the discount falls | hold the discount drawn at the start of the season and keep drawing the bid every period | of clairvoyant revenue at every patience level, against and at patience and ; the bid still leaves the trap after seasons; costs points when nobody waits, nothing in the strong market (Section 7.2 ) |
| Refine the grid when learning stalls | the posterior settles between neighbouring candidates | add candidates near the posterior mode and re-solve offline | a sevenfold finer grid cuts the late regret from to per season (Table S.2) |
Automated Bidding
Automated bidding studies algorithms that choose auction bids over time while balancing value, budgets, and performance constraints.
See all 40 papers