From Individual Learning to Market Equilibrium: Correcting Structural and Parametric Biases in RL Simulations of Economic Models
Organizations: Department of Economics, Nagoya University, Japan · Institute of Neural Information Processing, Ulm University, Germany
Abstract
Applying Reinforcement Learning (RL) to economic modeling reveals a fundamental tension: standard RL agents do not behave as competitive price-takers. While macroeconomic models typically assume agents take aggregate conditions as given, a naive single-agent RL algorithm learns to internalize its impact on the environment, effectively exerting market power. We illustrate this divergence in a search-and-matching model with concave production, showing that a standard RL agent converges to a monopsonistic allocation rather than the competitive equilibrium. Furthermore, we show that RL's approach to time discounting distorts standard intertemporal economic trade-offs. To resolve these issues, we propose a Calibrated Mean-Field Reinforcement Learning framework that restricts the agent to optimize against an exogenous aggregate state to restore price-taking behavior, and recalibrates the RL reward structure to reflect true economic opportunity costs. By iteratively updating the aggregate state, our algorithm converges to a stationary fixed point that recovers the competitive equilibrium. This approach provides a method for embedding learning algorithms into dynamic general equilibrium models.
Figures & tables
| Symbol | Description | Value |
|---|---|---|
| Productivity | ||
| Matching efficiency | ||
| Parameter in production function | ||
| Separation rate | ||
| Worker’s bargaining power | ||
| Vacancy cost |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Reinforcement Learning
Reinforcement learning studies how agents learn sequential decisions from rewards and interaction, balancing exploration, performance, safety, and adaptation.
See all 1,847 papers