econ.GNApr 15, 2026

A Comparative Study of Dynamic Programming and Reinforcement Learning in Finite Horizon Dynamic Pricing

Authors: Lev Razumovskiy, Nikolay Karenin

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper provides a systematic comparison between Fitted Dynamic Programming (DP), where demand is estimated from data, and Reinforcement Learning (RL) methods in finite-horizon dynamic pricing problems. We analyze their performance across environments of increasing structural complexity, ranging from a single typology benchmark to multi-typology settings with heterogeneous demand and inter-temporal revenue constraints. Unlike simplified comparisons that restrict DP to low-dimensional settings, we apply dynamic programming in richer, multi-dimensional environments with multiple product types and constraints. We evaluate revenue performance, stability, constraint satisfaction behavior, and computational scaling, highlighting the trade-offs between explicit expectation-based optimization and trajectory-based learning.

Figures & tables

Figure 1: Environment 1
Figure 2: Environment 2: Two identical typologies
Figure 3: Distribution of cumulative revenue before the penalty step ( t=7t=7 ) for A2C, PPO, DQN, and Fitted DP in Environment 2. The dashed line denotes the revenue target.
Figure 4: Environment 3: Two different typologies
Figure 5: Distribution of cumulative revenue before the penalty step for A2C, PPO, DQN and Fitted DP in Environment 3 (two different typologies). The dashed line denotes the revenue target.
Figure 6: Environment 4: Single typology with constraint
 DPDQNA2CPPO
Env 10.24616
Env 212085822
Env 312115822
Env 41784616
Table 1: Training time at 20002000 episodes (in seconds)
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Environment 5: single typology with non-linear demand. Mean revenue and one standard deviation band over 1010 runs.
More in this topic

Dynamic Pricing

Dynamic pricing studies how sellers and platforms adjust prices as demand, competition, inventory, and resource constraints change.

See all 50 papers

Explore similar work

CardsList

  1. Multi-Agent Reinforcement Learning for Dynamic Pricing: Balancing Profitability,Stability and Fairness

    Feb 28, 2026Krishna Kumar Neelakanta Pillai Santha Kumari AmmaDynamic PricingReinforcement Learning

  2. Relating Reinforcement Learning to Dynamic Programming-Based Planning

    Mar 8, 2026Filip V. Georgiev, Kalle G. Timperi, Başak Sakçak +1Markov Decision ProcessesDynamic Programming

    Center for Applied Computing, Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland · Dept. of Advanced Computing Sciences, Maastricht University, the Netherlands

  3. On Non-Stationary Dynamic Pricing: Adaptivity and Optimality

    Jul 27, 2026Feiyu Jiang, Zifeng ZhaoNon-Stationary BanditsDynamic Pricing

    School of Management, Fudan University · Mendoza College of Business, University of Notre Dame