cs.MAJun 3, 2026

Failure Modes of Deep Multi-Agent RL in Asynchronous Pricing: Reproducible Triggers, Trace Diagnostics, and a Partial Fix

Authors: Shree Murthy, Rohan Pandey

Organizations: DigitalOcean, USA

Abstract

We study two reproducible failure modes of deep multi-agent reinforcement learning in continuous-time pricing markets: (i) tacit cartel formation between competing DDPG agents, and (ii) actor--critic instability at high event rates. We instantiate both inside a single CT-MARL benchmark (Poisson-clocked price updates, observation latency δδ, interior-optimum logit demand), show that synchronous DDPG agents reliably trigger Failure Mode 1 with collusion index Δ=0.69±0.11Δ= 0.69 \pm 0.11, and quantify a partial microstructure fix: asynchrony alone cuts collusion by 48% and adding latency drives it to a minimum of Δ=0.28Δ= 0.28. The fix has clearly documented costs: it is partial (ΔΔ remains supra-Bertrand), it is non-monotone in δδ, and it does not survive Failure Mode 2, which emerges as DDPG critic divergence at λ=5λ= 5 and corrupts the phase-diagram cell at (λ=5,δ=1)(λ{=}5, δ{=}1). We accompany the scalar collusion index with trajectory-level trace diagnostics that expose the within-episode signalling collapse and the post-shock non-recovery.

Explore similar work

CardsList