Understanding the limits of snapshot-based fault diagnosis in distribution networks
Organizations: Faculty of Electrical Engineering, University of Science and Technology, The University of Danang, 54 Nguyen Luong Bang Street, Da Nang 550000, Vietnam · R&D Department, PowerMore Ltd., Da Nang 550000, Vietnam
Abstract
Inverter-based resources weaken fault currents in distribution networks, and snapshot-based diagnosis, which classifies a fault from voltage phasors at a few buses at one instant, offers a cheap first-stage screen. Its published accuracies, however, rest on different noise assumptions and do not show whether the error comes from the classifier, the features, the measurement error, or the fault resistance. This paper separates these sources on three IEEE test feeders and a public data set with distributed generation, using four feature sets, thirteen classifiers, and measurement error defined by total vector error classes. The classifier is not the limiting factor, because the full phasor snapshot classifies every fault correctly without noise, and the remaining error comes from measurement error on high-impedance faults. Three voltage sensors classify 90-96% of faults at the 1% accuracy class and 96-99% with distribution micro phasor measurement units (micro-PMUs). A discriminability-margin placement selects them from simulations alone in under one second and matches the best of twelve placement methods. The results show when snapshot-based diagnosis can be trusted.
Figures & tables
| Measurement input | Test conditions | Method | |||||
| Study | Snapshot | Measured | Standard | max | Diagnostic | Error | Reject |
| input | buses | noise | ( ) | placement | decomp. | option | |
| Kaliuzhnyi [ 17 ] | – | 1–3 | ✓ | 10 | ✓ | partly | – |
| Ngo [ 2 ] | – | 13–51 | – | 10 | – | – | – |
| Chanda [ 20 ] | ✓ | all (128) | – | 500 | partly | – | – |
| Sun [ 21 ] | ✓ | 10–23 | ✓ | 100 | – | – | – |
| Level | (%) | ( ∘ ) | Median TVE | 99th pct. TVE | TVE |
|---|---|---|---|---|---|
| Micro-PMU (measured [ 30 ] ) | 0.02 | 0.002 | 0.01 | 0.05 | 0 |
| 1% TVE accuracy class | 0.33 | 0.19 | 0.39 | 1.00 | 1 |
| 3% TVE accuracy class | 0.99 | 0.57 | 1.16 | 3.00 | 60 |
| Common reading of 1% TVE | 1.00 | 0.50 | 1.10 | 2.86 | 56 |
| All 14 buses monitored | Three monitored buses (figure 1 ) | |||||||
|---|---|---|---|---|---|---|---|---|
| Classifier | AGG | PPA | SEQ | FPS | AGG | PPA | SEQ | FPS |
| RF | 83.7 1.2 | 90.5 0.7 | 90.4 0.6 | 92.6 0.7 | 83.5 0.9 | 88.3 0.8 | 88.5 0.6 | 91.2 0.8 |
| XGBoost | 83.6 0.9 | 90.6 0.8 | 90.8 0.6 | 92.6 0.6 | 83.5 1.0 | 88.5 0.8 | 88.5 0.6 | 90.6 0.7 |
| LightGBM | 83.3 1.1 | 90.6 0.8 | 90.5 0.8 | 92.7 0.7 | 83.4 0.8 | 88.2 0.7 | 88.5 0.5 | 90.9 0.7 |
| MLP | 78.7 1.3 | 85.2 1.0 | 85.9 1.8 | 89.9 2.2 | 80.8 1.2 | 85.5 1.4 | 85.4 1.0 | 87.6 1.6 |
| SVM | 79.2 1.3 | 80.2 1.1 | 85.6 1.2 | 85.4 1.1 | 81.0 1.0 | 80.8 1.1 | 85.1 1.1 | 83.3 0.9 |
| 1% TVE | PMU (0.02%, 0.002 ∘ ) | |||||
|---|---|---|---|---|---|---|
| Network | AGG | SEQ | FPS | AGG | SEQ | FPS |
| IEEE 13-node | 83.4 1.0 | 88.4 0.6 | 91.0 0.6 | 89.1 1.1 | 95.3 0.9 | 97.6 0.7 |
| IEEE 34-node | 93.0 0.4 | 95.8 0.4 | 96.3 0.3 | 93.9 0.6 | 97.8 0.4 | 99.3 0.1 |
| IEEE 123-node | 72.2 1.0 | 74.1 1.3 | 85.9 0.9 | 78.2 0.6 | 84.2 0.7 | 95.3 0.4 |
| Target accuracy | IEEE 13-node | IEEE 34-node | IEEE 123-node |
|---|---|---|---|
| 90% | 1.0 | 3.0 | 0.5 |
| 95% | 0.1 | 3.0 | 0.02 |
| 99% | 0.005 | 0.02 | not reached |
| Accuracy at the 1% accuracy class | 91.7 0.9 | 96.5 0.4 | 89.9 0.6 |
| Accuracy at the 3% accuracy class | 87.0 1.0 | 95.3 0.3 | 84.6 0.7 |
| Accuracy, common reading of 1% TVE | 87.3 1.0 | 95.4 0.3 | 85.2 0.9 |
| IEEE 13-node | IEEE 34-node | IEEE 123-node | |||||
| Family | Classifier | 1% TVE | PMU | 1% TVE | PMU | 1% TVE | PMU |
| Tree ensemble | RF | 91.1 1.0 | 97.7 0.7 | 96.5 0.3 | 99.4 0.1 | 85.6 1.2 | 94.9 0.6 |
| ExtraTrees | 91.4 0.9 | 97.6 0.7 | 96.7 0.3 | 99.5 0.1 | 85.9 0.7 | 95.4 0.5 | |
| XGBoost | 90.7 0.8 | 97.7 0.5 | 96.3 0.5 | 99.3 0.1 | 86.0 0.8 | 95.2 0.2 | |
| LightGBM | 91.0 0.6 | 97.6 0.6 | 96.3 0.3 | 99.3 0.1 | 85.9 0.8 | 95.3 0.3 | |
| Stacking | 91.2 1.0 | 97.6 0.6 | 96.4 0.2 | 99.4 0.1 | 86.4 0.9 | 95.4 0.4 | |
| IEEE 13-node | IEEE 34-node | IEEE 123-node | ||||
|---|---|---|---|---|---|---|
| Method | Acc. | HIF | Acc. | HIF | Acc. | HIF |
| DMP (proposed) | 91.9 0.8 | 79.7 | 96.4 0.4 | 91.4 | 89.8 0.7 | 77.1 |
| DMP N-1 (proposed) | 91.6 0.7 | 79.9 | 96.3 0.4 | 91.0 | 89.8 0.7 | 77.2 |
| MRA ILP | 91.2 0.8 | 75.6 | 96.4 0.3 | 91.2 | 90.1 0.7 | 78.3 |
| QR pivot | 91.5 0.8 | 78.5 | 96.3 0.4 | 91.2 | 89.5 0.6 | 76.7 |
| D-optimal | 91.5 0.7 | 78.8 | 96.6 0.3 | 92.0 | 89.6 0.8 | 76.7 |
| IEEE 34-node, 4 sensors | IEEE 123-node, 9 sensors | |||||
|---|---|---|---|---|---|---|
| Sensor set | No noise, | PMU, | 1% TVE, | No noise, | PMU, | 1% TVE, |
| all | all | all | all | |||
| FSNR-SVM [ 18 ] | 99.97 0.06 | 99.27 0.09 | 96.68 0.24 | 99.98 0.04 | 95.79 0.49 | 90.82 0.83 |
| DMP | 99.94 0.09 | 99.37 0.12 | 96.63 0.35 | 99.92 0.09 | 95.73 0.36 | 91.06 0.73 |
| Random (5 sets) | 99.96 0.06 | 99.35 0.17 | 96.65 0.28 | 99.93 0.08 | 95.80 0.41 | 90.41 0.63 |
| Method | Trained classifier | IEEE 13-node | IEEE 34-node | IEEE 123-node |
|---|---|---|---|---|
| DMP (proposed) | no | 0.01 | 0.02 | 0.19 |
| DMP N-1 (proposed) | no | 0.01 | 0.03 | 1.66 |
| MRA ILP | no | 0.01 | 0.04 | 0.17 |
| QR pivot | no | 0.01 | 0.01 | 0.13 |
| D-optimal | no | 0.01 | 0.03 | 0.32 |
| SA coverage | no | 0.14 | 0.15 | 0.24 |
| Trained and tested on S1–S6 | LightGBM, by scenario | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Set | RF | XGBoost | LightGBM | MLP | SVM | kNN | LR | S1–S6 | S1 S2–S6 |
| AGG | 87.7 0.2 | 87.2 0.6 | 87.3 0.4 | 81.0 1.1 | 79.6 0.3 | 87.5 0.4 | 58.6 1.0 | 86.3–87.8 | 80.6–83.7 |
| PPA | 92.8 0.1 | 92.0 0.3 | 92.0 0.2 | 90.2 0.7 | 84.5 0.2 | 91.9 0.2 | 69.1 0.4 | – | – |
| SEQ | 91.9 0.2 | 91.3 0.2 | 91.3 0.2 | 89.7 0.8 | 88.1 0.6 | 90.9 0.3 | 87.8 0.6 | 90.4–91.9 | 89.4–90.1 |
| FPS | 87.9 0.4 | 88.1 0.4 | 88.4 0.2 | 87.8 0.4 | 87.4 0.5 | 87.5 0.6 | 82.3 0.6 | 88.0–89.3 | 86.2–87.3 |
| Condition | 13-node | 34-node | 123-node |
|---|---|---|---|
| Reference | 92.0 0.4 | 96.5 0.5 | 90.0 0.4 |
| Load shift (low high) | 89.7 0.2 | 95.3 0.2 | 87.2 0.4 |
| Trained PMU, tested 1% TVE | 88.0 0.2 | 95.8 0.6 | 86.2 0.4 |
| N-1 (mean over lost PMU) | 86.3 2.0 | 93.5 2.2 | 81.7 0.5 |
| N-2 (mean over lost pairs) | 70.7 2.5 | 80.0 5.3 | 67.7 1.1 |
| N-2 (worst pair) | 62.2 6.3 | 66.8 5.3 | 54.6 3.5 |
| IEEE 13-node | IEEE 34-node | IEEE 123-node | |||||
|---|---|---|---|---|---|---|---|
| Zones | Chance | 1% TVE | PMU | 1% TVE | PMU | 1% TVE | PMU |
| 2 | 50.0 | 90.7 0.6 | 97.7 0.3 | 96.0 0.4 | 99.9 0.1 | 91.0 0.6 | 94.6 0.3 |
| 3 | 33.3 | 84.1 0.5 | 90.4 0.5 | 84.8 0.6 | 94.6 0.3 | 84.6 0.9 | 89.8 0.8 |
| 4 | 25.0 | 79.3 0.7 | 87.5 0.6 | 80.8 0.5 | 93.8 0.3 | 77.7 0.6 | 83.1 0.8 |
| 5 | 20.0 | 75.6 0.6 | 85.4 0.4 | 76.5 0.9 | 88.8 0.6 | 74.7 0.9 | 80.5 0.6 |
| IEEE 13-node | IEEE 34-node | IEEE 123-node | ||||
|---|---|---|---|---|---|---|
| P | M | P | M | P | M | |
| Accuracy without rejection | 91.9 0.9 | 96.5 0.4 | 89.8 0.7 | |||
| Share classified at 99% accuracy | 87.1 1.2 | 86.3 1.2 | 96.0 0.6 | 95.8 0.5 | 84.1 1.2 | 83.2 0.9 |
| Errors among the 10% rejected | 73.8 5.5 | 69.1 5.2 | 100 0.0 | 100 0.0 | 61.2 4.5 | 56.0 5.0 |
| HIF among the 10% rejected | 54.2 3.2 | 54.6 3.4 | 43.0 3.8 | 44.5 4.0 | 44.6 4.9 | 47.6 3.6 |
| HIF in the whole data set | 20.1 | 20.3 | 20.4 | |||
| Condition | Suitability | Recommendation |
|---|---|---|
| Measurement noise | ||
| Micro-PMU noise (0.02%, 0.002 ∘ ) | Reliable | Preferred measurement-device accuracy class for snapshot diagnosis |
| 1% TVE, 34-node feeder | Reliable | Usable without further measures |
| 1% TVE, 13- and 123-node feeders | Screening only | Confirm by protection or time-domain analysis |
| Service noise above training noise | Screening only | Train at the highest noise level expected in service |
| Fault resistance | ||