econ.GNJul 24, 2025

From Individual Learning to Market Equilibrium: Correcting Structural and Parametric Biases in RL Simulations of Economic Models

Authors: Ruxin Chen, Zeqiang Zhang

Organizations: Department of Economics, Nagoya University, Japan · Institute of Neural Information Processing, Ulm University, Germany

Abstract

Applying Reinforcement Learning (RL) to economic modeling reveals a fundamental tension: standard RL agents do not behave as competitive price-takers. While macroeconomic models typically assume agents take aggregate conditions as given, a naive single-agent RL algorithm learns to internalize its impact on the environment, effectively exerting market power. We illustrate this divergence in a search-and-matching model with concave production, showing that a standard RL agent converges to a monopsonistic allocation rather than the competitive equilibrium. Furthermore, we show that RL's approach to time discounting distorts standard intertemporal economic trade-offs. To resolve these issues, we propose a Calibrated Mean-Field Reinforcement Learning framework that restricts the agent to optimize against an exogenous aggregate state to restore price-taking behavior, and recalibrates the RL reward structure to reflect true economic opportunity costs. By iteratively updating the aggregate state, our algorithm converges to a stationary fixed point that recovers the competitive equilibrium. This approach provides a method for embedding learning algorithms into dynamic general equilibrium models.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. A Survey of Reinforcement Learning For Economics

    Mar 9, 2026Pranjal RawatReinforcement Learning

  2. JaxMARL-BC: Multi-Agent Reinforcement Learning for Macroeconomic Modelling, in Seconds

    Oct 20, 2026Daniele Montagnani, Andrea Coletta, Antonietta Mira +1Multi-Agent Reinforcement LearningReinforcement Learning

  3. Can Reinforcement Learning Efficiently Discover Price Manipulation?

    Jul 7, 2026Ioanna-Yvonni Tsaknaki, Andrea Macrì, Fabrizio LilloModel-Free RLAlgorithmic Trading