Skip to main content
Back to timeline
arXivSource publication:

Grid imbalance moves to agent dispatch: MADWMPG holds cumulative violation cost at 0.006 on an IEEE 123-bus feeder

Synopsis

For multiphase active distribution networks with phase-changing soft open points (PCSOPs, power-electronics devices that move active power directly between phases), MADWMPG hands three-phase voltage and feeder-head power imbalance dispatch to multi-agent safe reinforcement learning. A wide-margin classifier switches between behavior and recourse policies, and noncrossing quantile critics track the return distribution. On a modified IEEE 123-bus test feeder its cumulative risk-adjusted constraint violation score is 0.006, against 35.010 for DDPG, 4.571 for TD3PG, 5.103 for RCPO and 0.076 for PID-Lagrangian; average test step reward is 0.957, below the 0.989 of the ACOPF expert.

Source-provided article image: Safely Mitigating Grid Imbalance: A Hybrid Distributional Wide-Margin Recourse Approach
arXiv

Interpretation

Real-time dispatch for three-phase imbalance no longer rests on reinforcement learning driven by a single expected return; a set of behavior policies and a set of recourse policies jointly decide each phase-changing soft open point's output. Earlier deep reinforcement learning dispatch optimized only the scalar expectation of cumulative discounted return, and safe reinforcement learning mostly offered safety in converged policies while training exploration still violated limits frequently. On the modified IEEE 123-bus test feeder, MADWMPG's cumulative risk-adjusted constraint violation score is 0.006, against 35.010 for DDPG, 4.571 for TD3PG, 5.103 for RCPO and 0.076 for PID-Lagrangian on the same test.

Each phase-changing soft open point action is first produced by a physics-based predictor-corrector, then a centralized wide-margin classifier judges whether the joint action is safe and switches to the recourse policy when it is not. The classifier uses a hinge loss to push safe and unsafe actions apart, and recourse policies learn a state-dependent remapping conditioned on the behavior action, so the same state yields different corrections depending on which action drives the violation. Training and testing both run on a modified IEEE 123-bus feeder with three distributed phase-changing soft open points, using load data from the Norwegian 124-bus distribution dataset and PV data from the UNISOLAR open dataset, split roughly 87:13 into training and test sets.

Returns are no longer collapsed into one expectation; a noncrossing quantile regression critic estimates the whole return distribution. Quantiles are forced to stay monotone, avoiding value-estimation bias when the behavior and recourse action distributions diverge. The test set spans 8 days drawn across the year, 3840 dispatch instances in total, at a 3-minute dispatch interval.

Perspective

The demonstrated scope is a modified multiphase IEEE 123-bus test feeder with three distributed phase-changing soft open points, load and PV profiles taken from public datasets, and a 3-minute dispatch interval; inference averages about 6 milliseconds in decentralized execution and about 11 milliseconds in centralized execution that retains the classifier and recourse policies.

The safety guarantee is a probabilistic soft bound that assumes the recourse policy's proposed action has a non-negative predicted safety score; the feeder-head power imbalance reward component shows a wider spread, which the authors attribute to the agent seeing only bus net-injection features without topological context.