Concentrated liquidity provision in decentralized exchanges presents a fundamental Impulse Control problem. Liquidity Providers (LPs) face a non-trivial trade-off between maximizing fee accrual through tight price-range concentration and minimizing the friction costs of rebalancing, including gas fees and swap slippage. Existing methods typically employ heuristic or threshold strategies that fail to account for market dynamics. This paper formulates liquidity management as an optimal control problem and derives the corresponding Hamilton-Jacobi-Bellman quasi-variational inequality (HJB-QVI). We present an approximate solution RAmmStein, a Deep Reinforcement Learning method that incorporates the mean-reversion speed (theta) of an Ornstein-Uhlenbeck process among other features as input to the model. We demonstrate that the agent learns to separate the state space into regions of action and inaction. We further extend the framework with RAmmStein-Width, which jointly optimizes rebalancing timing and position width via a 6-action DDQN. We evaluate the framework using high-frequency 1Hz Coinbase trade data comprising over 6.8M trades on a realistic environment (10M TVL, 1% default width). Experimental results show that RAmmStein achieves a net ROI of 1.60%, the highest among all realistic (non-omniscient) strategies, while greedy strategies lose up to -8.4% to gas costs. Notably, the agent reduces rebalancing frequency by 85% compared to greedy rebalancing. RAmmStein-Width discovers extreme parsimony on its own, executing only 9 rebalances and $40 in gas, and degrades more slowly than all active strategies at elevated gas costs. Our results demonstrate that regime-aware laziness can significantly improve capital efficiency by preserving the returns that would otherwise be eroded by the operational costs.
Figures & tables
Parameter
Value
Learning rate
10−4
Discount factor γ
0.99
Replay buffer size
100,000
Batch size
128
Target update frequency
100 steps
ϵ start / end / decay
1.0 / 0.05 / 0.9998
TABLE I: Training Hyperparameters
Parameter
Value
Range width
1% (100 bps)
Pool fee tier ( ϕ )
0.05%
Gas cost ( G )
$2.00 USD
Initial capital ( K )
$10,000 USD
Pool TVL ( Lpool )
$10,000,000 USD
DEX/CEX volume ratio ( α )
10%
TABLE II: Environment Parameters
Strategy
Active%
Rebal.
Fees ($)
Gas ($)
Net ROI
Set & Forget (Oracle)
100.0%
1
198
4.5
1.94%
Fixed Passive
19.7%
1
117
4.5
1.13%
Fixed Active (Greedy)
100.0%
344
644
1,483
− 8.40%
LSTM Predictor
95.1%
237
580
1,036
− 4.55%
RAmmStein
57.7%
51
389
228
1.60%
RAmmStein-Width
9.3%
9
75
40
0.35%
TABLE III: Strategy Comparison on Test Data (10MTVL,12 gas)
Fig. 1: Learned decision boundary with actual rebalance events (blue dots, n=50 ). The heatmap shows Q(rebalance)−Q(hold) ; green = hold preferred, red = rebalance preferred. White dashed lines mark the range edges ( dedge=±1 ). All rebalances occur outside the range at low θ .
Gas (USD)
Fixed Active
RAmmStein
RAmmStein-Width
$1.00
− 4.96%
1.86%
0.44%
$2.00
− 8.40%
1.60%
0.35%
$5.00
− 18.72%
0.83%
0.08%
$10.00
− 35.94%
− 0.50%
− 0.36%
$20.00
− 70.38%
− 3.16%
− 1.24%
TABLE IV: Net ROI vs. Gas Cost
Fig. 2: RAmmStein-Width: Q(best rebalance)−Q(hold) landscape with actual rebalance events (left), and per-event dedge with chosen width annotation (right). All 8 rebalances are downside exits at θ≈0 ; 7 of 8 select 1% width.