Power Setpoint Tracking for Large-Scale EV Charging: Transformer-Constrained Charging Policy Learning with Self-Tuning Constraint Prices
Organizations: Graduate School of Bio-Applications and Systems Engineering, Tokyo University of Agriculture and Technology, Koganei, Tokyo, Japan · Institute of Science Tokyo, Yokohama 226-8501, Japan · Norwegian University of Science and Technology, 7491 Trondheim, Norway · School of Technology and Innovations, University of Vaasa, Wolffintie 32, 65200 Vaasa, Finland
Abstract
In power setpoint tracking (PST), an operator follows a purchased charging profile without overloading shared transformers or undercharging vehicles. PST is cast as a constrained Markov decision process that minimizes tracking error subject to overload and mean user-satisfaction constraints. Transformer-Constrained Charging Policy Learning (TC-CPL) extends soft actor-critic with proportional-integral Lagrange-multiplier updates from measured violations, a residual on the incumbent dispatcher that is warm-started from its demonstrations, and a transformer-aware encoder shared by all ports. It observes no departure times or states of charge. Simulations (five seeds, 100 paired days) compare it with the incumbent, Round-Robin. On a 600-port, 122-substation district, raw overload falls from 3.59 to 0.0017 kWh/day (overloaded seed-days from 77% to 0.4%), and absolute tracking error falls by 27%. Delivered energy rises by 16%, and satisfaction rises from 0.959 to 0.989. On a 42-kW transformer with household load, overload falls from 1.86 to 0.0014 kWh/day and satisfaction rises from 0.974 to 0.985, but absolute tracking error is 22% higher. Against hand-set fixed multipliers, self-tuning is statistically indistinguishable on the district. On the single transformer, it raises satisfaction by 0.011 at similar overload, but a short sweep found a better-serving fixed pair. The overload multipliers stay below their cap.
Figures & tables
| Reference | Algorithm | Real-time | Constraint handling | Dep. | SoC | Transf. limit | Warm start | V2G | Open-source |
|---|---|---|---|---|---|---|---|---|---|
| [ 23 ] | CPO | ✓ | full charge at departure | ✓ | n/r | ✓ | |||
| [ 24 ] | constrained SAC | ✓ | Lagrangian + safety filter | n/r | n/r | n/r | n/r | ✓ | |
| [ 25 ] | SAC | ✓ | aug. Lagrangian, dual step per update | ✓ | n/r | ✓ | |||
| [ 26 ] | SAC | ✓ | bus-voltage Lagrangian | ✓ | ✓ | n/r | ✓ | ||
| [ 27 ] | multi-agent SAC | ✓ | overload penalty | n/r | n/r | ✓ penalty | n/r | n/r | |
| [ 2 ] | multi-agent TD3 | ✓ | fixed-weight aging + user costs | ✓ | ✓ | ✓ aging | curriculum | ✓ |
| (a) Per scenario | Transformers | Daily demand | Day | Learner | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Scenario | Ports (stations) | (with ports) | Ports each | Limit outside DR, (kW) | Lowest in DR (kW) b | Non-EV peak / limit | EV sessions per day | Setpoint / EV energy | Start | Days | (kW) | (kWh) | |
| PublicPST-S | ( ) | ( ) | – | – | – | 05:00 | weekdays | ||||||
| cityNL | ( ) | ( ) | – | – a | – | – | – | 00:00 | all days | ||||
| Controller | Information | Action; network | Constraint handling | Warm start; replay | Budget; tuning |
|---|---|---|---|---|---|
| CAFAP | occupancy | on all ports | none | – | rule |
| Round-Robin | , port status | EVs in turn | none | M2 incumbent | rule |
| DDPG-PST | PST state, normalized | direct; MLP - / - | none (tracking reward only) | uniform, | ; as [ 14 ] |
| PPO-PST | PST state, normalized | direct; MLP - | none (tracking reward only) | on-policy | |
| CPO | (no ), normalized | direct; MLP - - | trust region; raw overload | on-policy | |
| Self-tuning TC-CPL | residual, ; M3, load-aware | above , at ; PI law (M1) from | Round-Robin steps (M2); PER | ; Table II |
| Tracking | Service | Overload | |||||||
| Controller | Unmet | EVs | Seed-days | Max excess | |||||
| kWh/d | kWh/d | % | % | kWh/d | % | kW | |||
| CAFAP | 5,727 | 0.997 | 11,919 | 0.14 | 1.9 | 3.9807 | 76.0 | 41.54 | |
| Round-Robin | 3,411 | 0.959 | 9,891 | 3.94 | 23.9 | 3.5932 | 77.0 | 34.20 | |
| DDPG-PST | 7,756 240 | 0.823 0.012 | 6,370 402 | 17.61 1.23 | 47.6 3.1 | 1.3342 0.3913 | 54.8 8.6 | 23.92 4.44 | |
| PPO-PST | 7,502 73 | 0.845 0.002 | 5,846 78 | 15.41 0.22 | 58.5 0.8 | 0.9819 0.1133 | 59.0 1.2 | 17.44 3.93 | |
| Variant | OV | ||||||
| kWh/d | kWh/d | kWh/d | days % | final | |||
| Self-tuning TC-CPL | 3 | 123.6 | 0.970 | 114.7 | 0.003 | 0.7 | 54 |
| Round-Robin (incumbent) | – | 100.0 ∗ | 0.974 | 122.7 ∗ | 1.858 ∗ | 49.0 ∗ | – |
| M1 off, prices frozen at | |||||||
| 2 | 116.9 ∗ | 0.978 ∗ | 119.9 ∗ | 0.011 | 4.0 | 10 | |
| 2 | 116.5 ∗ | 0.974 ∗ | 118.3 ∗ | 1.051 | 27.0 | 10 | |
| (a) Deployment checks | Self-tuning TC-CPL | Fixed-price | ||||
| Observation error | Seed- days | Max excess | Seed- days | |||
| ( steps, five seeds) | kWh/d | % | kW | kWh/d | % | |
| None (absolute values) | 0.0014 | 0.6 | 0.48 | 0.985 | 0.0004 | 0.4 |
| Status flipped, 10% | 13.4 | 8.46 | 12.8 | |||
| Status hidden, 10% | 13.8 | 9.64 | 13.2 | |||
| Port data 15 min late | 22.6 | 7.15 | 22.0 | |||
Electric Vehicle Charging
Electric vehicle charging research studies how to forecast, plan, control, and secure charging while coordinating vehicles, stations, and power grids.
See all 44 papers