Non-intrusive appliance state detection in unseen homes using classical machine learning
Organizations: School of Computing and Engineering Sciences, Strathmore University, Nairobi, Kenya
Abstract
Non-intrusive load monitoring (NILM) infers, from the aggregate signal of a single smart meter, when each appliance in a home is ON or OFF, offering significant potential to enhance energy efficiency. Classical machine-learning detectors report F1 scores (the harmonic mean of precision and recall) from 0.53 to 0.98, but the causes of this spread are unseparated, so it is unclear which design choices matter. Real-world deployments, however, require detectors that work in houses never seen during training, whereas most studies test on houses used for training. To address this challenge, we propose a factorial analysis of variance design that separates the effects of classifier, feature set and class-imbalance strategy across five appliances (270 configurations), and a leave-house-out protocol with house-grouped model selection. Unlike studies that vary one factor at a time or pool houses, the approach attributes performance to each factor and measures it on unseen homes. Extensive experiments on REFIT, a public dataset of 20 UK households, show that the appliance explains 38 percent of F1 variance, the feature set 13 percent, the classifier 8 percent and the imbalance strategy 2 percent. Pooled testing overstates support-weighted F1 (0.548 versus 0.447 in unseen houses), class weighting gains grow from 32 percent to 69 percent, and feature engineering gives the same 52 percent gain in both protocols. House-grouped selection raises F1 for three of five appliances, by up to 0.110, and a convolutional neural network scores 0.450 (0.496 calibrated) against 0.447 for classical models. The findings guide appliance-level monitoring in smart-metering and energy-management systems.
Figures & tables
| Ref. | Approach and data | Limitation | Contribution of this work |
| [ 4 ] | Multi-label classifiers (RAkEL, MLkNN); REDD | Single dataset; no feature or imbalance ablation | Features and imbalance strategies varied jointly under one protocol |
| [ 5 ] | k-NN, Random Forest, Extra-Trees and other classifiers; one REDD house | Accuracy under strong imbalance; not comparable with F1 | ON-class F1 as primary metric; accuracy bias quantified |
| [ 6 ] | Multi-label sparse representation classification; REDD, Pecan Street | Single model family and feature representation | Seven models from five algorithm families |
| [ 7 ] | Random Forest with high-frequency electrical features; BLUED (one house) | Single house, temporal split; training-set size not reported | Training-set size reported and its effect tested |
| [ 24 , 25 , 26 , 27 , 28 ] | LR, DT, k-NN, SVM and gradient boosting; smart-meter data, REFIT, REDD, PLAID, WHITED, PRECON | Each study fixes its model set, features and dataset | Joint sweep on REFIT with the appliances of the deep-learning benchmarks |
| [ 10 , 11 , 12 ] | Recurrent, convolutional and autoencoder-based deep models; REFIT | High computational cost; class-imbalance handling not examined | Classical reference on the same dataset, including four imbalance strategies |
| Appliance | Houses | Median ON time (%) | ON share in pooled data (%) | Median ON power (W) | ON threshold (W) |
| Fridge | 18 | 33.6 | 31.5 | 94 | 50 |
| Washing machine | 19 | 2.46 | 3.14 | 551 | 20 |
| Dishwasher | 15 | 2.08 | 4.73 | 836 | 10 |
| Kettle | 16 | 0.71 | 0.62 | 2729 | 2000 |
| Microwave | 16 | 0.27 | 0.38 | 1281 | 200 |
| Feature | Definition | FS-A | FS-B |
| Aggregate power | P t (W) | Yes | Yes |
| Hour of day | 0 to 23 | Yes | Yes |
| Day of week | 0 to 6 | No | Yes |
| Rolling mean and standard deviation | Mean and standard deviation of P | No | Yes |
| Rolling minimum and maximum | Minimum and maximum of P | No | Yes |
| Rolling skewness and kurtosis | Third and fourth standardised moments of P | No | Yes |
| Model | Family | Hyperparameter grid |
| LR | Linear | Inverse regularisation strength C {0.1, 1, 10}; standardised inputs |
| DT | Single tree | Maximum depth {6, 12, unlimited}; minimum samples per leaf {1, 5} |
| k-NN | Instance-based | Number of neighbours k {3, 5, 11}; standardised inputs |
| SVM-RBF | Kernel | C {1, 10}; kernel coefficient {scale, auto}; standardised inputs |
| RF | Bagging ensemble | Number of trees {100, 300}; maximum depth {12, unlimited} in the first pass and {12} in the final run |
| XGBoost | Boosting ensemble | Number of trees {100, 300}; maximum depth {4, 8} |
| Model | First pass | Final run |
| LR, DT, k-NN | Full pooled data | Full pooled data (not re-run) |
| SVM-RBF | 30,000 samples, uniform random | 100,000 samples, stratified |
| RF | 60,000 samples, uniform random | Full pooled data; maximum depth fixed at 12 |
| XGBoost | 100,000 samples, uniform random | Full pooled data |
| LightGBM | 100,000 samples, uniform random | Full pooled data |
| Appliance | No correction | Class weighting | SMOTE | ADASYN |
| Fridge | 0.330 | 0.550 | 0.504 | 0.502 |
| Washing machine | 0.487 | 0.428 | 0.435 | 0.409 |
| Dishwasher | 0.500 | 0.344 | 0.336 | 0.313 |
| Kettle | 0.537 | 0.412 | 0.334 | 0.299 |
| Microwave | 0.202 | 0.228 | 0.198 | 0.194 |
| Appliance | Configuration | F1 | SD | Precision | Recall | Accuracy | ON samples |
| Fridge | RF, FS-B, CW | 0.556 | 0.002 | 0.444 | 0.746 | 0.626 | 339,857 |
| Washing machine | XGBoost, FS-B, none | 0.532 | 0.006 | 0.690 | 0.433 | 0.976 | 35,826 |
| Dishwasher | XGBoost, FS-B, none | 0.512 | 0.008 | 0.790 | 0.378 | 0.966 | 42,545 |
| Kettle | XGBoost, FS-B, none | 0.570 | 0.021 | 0.779 | 0.450 | 0.996 | 5,979 |
| Microwave | XGBoost, FS-B, CW | 0.324 | 0.040 | 0.276 | 0.444 | 0.992 | 3,632 |
| Weighted F1 | 0.548 | 427,839 |
| Appliance | First pass | F1 | Final run | F1 | Change |
| Fridge | LightGBM, FS-B, CW | 0.548 | RF, FS-B, CW | 0.556 | +0.008 |
| Washing machine | XGBoost, FS-B, CW | 0.512 | XGBoost, FS-B, none | 0.532 | +0.021 |
| Dishwasher | DT, FS-B, none | 0.500 | XGBoost, FS-B, none | 0.512 | +0.012 |
| Kettle | k-NN, FS-B, none | 0.551 | XGBoost, FS-B, none | 0.570 | +0.020 |
| Microwave | XGBoost, FS-B, CW | 0.256 | XGBoost, FS-B, CW | 0.324 | +0.069 |
| Appliance | F1, pooled | F1, unseen houses | SD | Range over folds | Precision | Recall | Change |
| Fridge | 0.556 | 0.474 | 0.112 | 0.292 to 0.580 | 0.380 | 0.692 | 0.082 |
| Washing machine | 0.532 | 0.474 | 0.086 | 0.330 to 0.537 | 0.619 | 0.392 | 0.058 |
| Dishwasher | 0.512 | 0.224 | 0.082 | 0.099 to 0.295 | 0.457 | 0.156 | 0.288 |
| Kettle | 0.570 | 0.421 | 0.160 | 0.229 to 0.591 | 0.511 | 0.366 | 0.149 |
| Microwave | 0.324 | 0.224 | 0.069 | 0.126 to 0.307 | 0.199 | 0.279 | 0.100 |
| Weighted F1 | 0.548 | 0.447 | 0.101 |
| Appliance | Selected by pooled CV | F1, unseen houses | Best on unseen houses | F1, unseen houses | F1 lost |
| Fridge | RF, FS-B, CW | 0.474 | SVM-RBF, FS-A, ADASYN | 0.477 | +0.003 |
| Washing machine | XGBoost, FS-B, no correction | 0.474 | LightGBM, FS-B, SMOTE | 0.500 | +0.026 |
| Dishwasher | XGBoost, FS-B, no correction | 0.224 | XGBoost, FS-B, SMOTE | 0.374 | +0.150 |
| Kettle | XGBoost, FS-B, no correction | 0.421 | LightGBM, FS-B, SMOTE | 0.479 | +0.058 |
| Microwave | XGBoost, FS-B, CW | 0.224 | LightGBM, FS-B, SMOTE | 0.243 | +0.019 |
| Appliance | CNN, uncalibrated | CNN, calibrated, centred | CNN, calibrated, causal | Classical, selected by pooled CV | Classical, best on unseen houses |
| Fridge | 0.468 | 0.492 | 0.497 | 0.474 | 0.477 |
| Washing machine | 0.490 | 0.663 | 0.632 | 0.474 | 0.500 |
| Dishwasher | 0.344 | 0.441 | 0.390 | 0.224 | 0.374 |
| Kettle | 0.150 | 0.390 | 0.533 | 0.421 | 0.479 |
| Microwave | 0.080 | 0.160 | 0.249 | 0.224 | 0.243 |
| Weighted F1 | 0.450 | 0.497 | 0.496 | 0.447 | 0.467 |
| Study | Approach | Dataset | Metric | Score |
| Tabatabaei et al. [ 4 ] | Multi-label classifiers | REDD | F1 | 0.528 |
| Singh and Majumdar [ 6 ] | Sparse representation | REDD, Pecan Street | Micro F1 | 0.68 to 0.70 |
| Wu et al. [ 7 ] | Random Forest, high-frequency features | BLUED (one house) | F-score | 0.98 |
| Rehmani et al. [ 5 ] | k-NN, RF, Extra-Trees | REDD (one house) | Accuracy | up to 0.98 |
| Langevin et al. [ 11 ] | Autoencoder-based disaggregation | REFIT | F1; weighted F1 [ 12 ] | 0.76; 0.78 |
| Tanoni et al. [ 10 ] | Convolutional recurrent network | REFIT, held-out houses; 80:20 split | Micro F1; weighted F1 [ 12 ] | 0.63; 0.83 |
| Item | Evidence from this study |
| 1. Report the F1 score (or another minority-class metric) of the ON class, not accuracy | Accuracy exceeded F1 in all 270 configurations; models with F1 = 0 reached an accuracy of 0.996 (Section 4.1) |
| 2. Validate on houses unseen during training and state the split | Weighted F1 0.101 lower on unseen houses, and up to 0.288 lower for one appliance (Section 4.8) |
| 3. Select models, features and imbalance strategies with house-grouped validation | In a nested comparison, house-grouped selection improved F1 on unseen houses by up to 0.110; the pooled choice was not the best on unseen houses for four or five appliances in each of four partitions (Sections 4.9 and 4.10) |
| 4. Report results per appliance, with variability over folds or houses | The appliance explained 38% of the variance; test houses explained six times more variance than the model (Sections 4.7 and 4.8) |
| 5. Report the imbalance strategy for each appliance | Appliance-by-strategy interaction of 14% of the variance; the best strategy differed between appliances (Sections 4.4 and 4.7) |
| 6. State the training-set size of every model and test its effect when subsampling | Removing a training-set cap changed the best configuration for four of five appliances, three of them near-ties (Section 4.6) |
Non-Intrusive Load Monitoring
Non-intrusive load monitoring estimates individual appliance use from a household’s combined electricity measurements.
See all 13 papers