Correctness and Predictive Uncertainty in Bayesian TSK Fuzzy Regression: A Reproducible Evaluation on Tabular and Building-Meter Data
Organizations: Guangdong University of Finance, Guangzhou, China
Abstract
Uncertainty estimates from fuzzy regression are informative only when the implementation is correct and the comparison protocol makes the sources of variation visible. We revisit a first-order Takagi--Sugeno--Kang (TSK) regression implementation and rebuild its dense, conjugate Bayesian, BIC-based, and rule-level spike-and-slab estimators using corrected code. The corrected pipeline freezes the Gaussian membership partition at training time, uses a scaled conjugate prior, evaluates firing strengths in the log domain, and reports direct posterior-predictive quantiles for Bayesian methods. Across 30 outer 80/20 splits on three targets from two UCI regression datasets, corrected TSK-LS obtains , , and on Energy-Heating, Energy-Cooling, and Concrete, respectively; rule-level Gibbs sampling with Bayesian model averaging obtains , , and . We then evaluate the same workflow on a bounded chronological sample of the Building Data Genome 2 meter data that covers 158 buildings across 17 sites. In a common-unit evaluation, every method is scored on the same 47,400 test rows and every trained model uses 94,800 train--validation rows. Persistence has RMSE , compared with for both TSK-LS and Bayesian TSK; the pre-specified building-level mean RMSE difference between Bayesian TSK and persistence is with a 95% site-cluster bootstrap interval of . The Bayesian TSK 90% intervals have PICP and MPIW . Taken together, the evidence supports a narrower conclusion: spike-and-slab model averaging is a reproducible implementation option, but it does not provide a demonstrated accuracy or calibration advantage across the tested tabular and building-meter settings.
Figures & tables
| Item | Frozen choice |
|---|---|
| Dataset | BDG2 v1.0; cleaned electricity meter |
| Cohort | Existing outcome-independent rule; 158 buildings across 17 sites |
| Target | Next-hour electricity; unit as supplied by the dataset |
| Features | 8 causal lag, rolling and calendar features; primary weather excluded |
| Temporal split | Train 2016; validation 2017-01–2017-06; test 2017-07–2017-12 |
| Sampling | 300 evenly spaced valid rows per building and split; 47,400 common test rows |
| Energy-Heating | Energy-Cooling | Concrete | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Method | RMSE | PICP | RMSE | PICP | RMSE | PICP | |||
| TSK-LS | 0.957 | 2.025 | – | 0.923 | 2.599 | – | 0.786 | 7.694 | – |
| Bayesian-TSK | 0.957 | 2.030 | 0.940 | 0.923 | 2.602 | 0.932 | 0.786 | 7.696 | 0.938 |
| TSK-SpikeSlab-BIC | 0.956 | 2.044 | 0.940 | 0.922 | 2.614 | 0.934 | 0.786 | 7.695 | 0.938 |
| TSK-SpikeSlab-Gibbs | 0.957 | 2.033 | 0.940 | 0.922 | 2.607 | 0.934 | 0.784 | 7.732 | 0.938 |
| Random Forest | 0.998 | 0.488 | – | 0.969 | 1.663 | – | 0.903 | 5.180 | – |
| Method | RMSE | MAE | PICP 90 | MPIW 90 | |
|---|---|---|---|---|---|
| Persistence | 27.710 | 7.321 | 0.995 | – | – |
| Seasonal naive (24 h) | 86.360 | 16.654 | 0.947 | – | – |
| Seasonal naive (168 h) | 110.895 | 18.165 | 0.913 | – | – |
| Ridge | 30.463 | 9.187 | 0.993 | – | – |
| HistGradientBoosting | 52.042 | 10.480 | 0.981 | – | – |
| TSK-LS | 29.083 | 8.676 | 0.994 | – | – |
| Fold | Held-out sites | RMSE | PICP 90 | |
| 0 | Bear, Fox, Mouse, Shrew | 10800 | 13.444 | 0.987 |
| 1 | Bobcat, Gator, Panther, Wolf | 9600 | 9.995 | 0.998 |
| 2 | Bull, Hog, Peacock | 9000 | 77.032 | 0.969 |
| 3 | Cockatoo, Lamb, Rat | 9000 | 24.239 | 0.984 |
| 4 | Eagle, Moose, Robin | 9000 | 36.700 | 0.947 |
| Source: site_holdout_results.json ; MPIW 90 and proper scores remain in the machine-readable result ledger. | ||||
| Target | Method | PICP | MPIW | WIS | CRPS |
|---|---|---|---|---|---|
| Energy-Heating | Bayesian-TSK | 0.940 | 8.164 | 0.938 | 1.129 |
| Energy-Heating | TSK-SpikeSlab-BIC | 0.940 | 8.151 | 0.944 | 1.138 |
| Energy-Heating | TSK-SpikeSlab-Gibbs | 0.940 | 8.180 | 0.939 | 1.132 |
| Energy-Heating | Gaussian Process | 0.931 | 1.748 | 0.210 | 0.252 |
| Energy-Cooling | Bayesian-TSK | 0.932 | 10.270 | 1.203 | 1.442 |
| Energy-Cooling | TSK-SpikeSlab-BIC | 0.934 | 10.248 | 1.205 | 1.445 |
| Target | PICP | MPIW | |
|---|---|---|---|
| Energy-Heating | 0.956 | 0.971 | 9.433 |
| Energy-Cooling | 0.922 | 0.964 | 12.524 |
| Concrete | 0.787 | 0.957 | 32.932 |