Correctness and Predictive Uncertainty in Bayesian TSK Fuzzy Regression: A Reproducible Evaluation on Tabular and Building-Meter Data

Zhiren Xiao

cs.LGNov 1, 2026

Correctness and Predictive Uncertainty in Bayesian TSK Fuzzy Regression: A Reproducible Evaluation on Tabular and Building-Meter Data

Authors: Zhiren Xiao

Organizations: Guangdong University of Finance, Guangzhou, China

Abstract

Uncertainty estimates from fuzzy regression are informative only when the implementation is correct and the comparison protocol makes the sources of variation visible. We revisit a first-order Takagi--Sugeno--Kang (TSK) regression implementation and rebuild its dense, conjugate Bayesian, BIC-based, and rule-level spike-and-slab estimators using corrected code. The corrected pipeline freezes the Gaussian membership partition at training time, uses a scaled conjugate prior, evaluates firing strengths in the log domain, and reports direct posterior-predictive quantiles for Bayesian methods. Across 30 outer 80/20 splits on three targets from two UCI regression datasets, corrected TSK-LS obtains $R^2=0.957$ , $0.923$ , and $0.786$ on Energy-Heating, Energy-Cooling, and Concrete, respectively; rule-level Gibbs sampling with Bayesian model averaging obtains $0.957$ , $0.922$ , and $0.784$ . We then evaluate the same workflow on a bounded chronological sample of the Building Data Genome 2 meter data that covers 158 buildings across 17 sites. In a common-unit evaluation, every method is scored on the same 47,400 test rows and every trained model uses 94,800 train--validation rows. Persistence has RMSE $27.710$ , compared with $29.083$ for both TSK-LS and Bayesian TSK; the pre-specified building-level mean RMSE difference between Bayesian TSK and persistence is $+0.0447$ with a 95% site-cluster bootstrap interval of $[-1.5536,1.5665]$ . The Bayesian TSK 90% intervals have PICP $0.980$ and MPIW $116.469$ . Taken together, the evidence supports a narrower conclusion: spike-and-slab model averaging is a reproducible implementation option, but it does not provide a demonstrated accuracy or calibration advantage across the tested tabular and building-meter settings.

Figures & tables

Item	Frozen choice
Dataset	BDG2 v1.0; cleaned electricity meter
Cohort	Existing outcome-independent rule; 158 buildings across 17 sites
Target	Next-hour electricity; unit as supplied by the dataset
Features	8 causal lag, rolling and calendar features; primary weather excluded
Temporal split	Train 2016; validation 2017-01–2017-06; test 2017-07–2017-12
Sampling	300 evenly spaced valid rows per building and split; 47,400 common test rows

Table 1: Frozen Route C protocol used for the BDG2 common-unit evaluation.

	Energy-Heating			Energy-Cooling			Concrete
Method	$R^{2}$	RMSE	PICP	$R^{2}$	RMSE	PICP	$R^{2}$	RMSE	PICP
TSK-LS	0.957	2.025	–	0.923	2.599	–	0.786	7.694	–
Bayesian-TSK	0.957	2.030	0.940	0.923	2.602	0.932	0.786	7.696	0.938
TSK-SpikeSlab-BIC	0.956	2.044	0.940	0.922	2.614	0.934	0.786	7.695	0.938
TSK-SpikeSlab-Gibbs	0.957	2.033	0.940	0.922	2.607	0.934	0.784	7.732	0.938
Random Forest	0.998	0.488	–	0.969	1.663	–	0.903	5.180	–

Table 2: Main comparison. Values are means over the 30 outer 80/20 splits. RMSE is on the original target scale; PICP is the empirical coverage of the 95% posterior-predictive interval when available.

Method	RMSE $\downarrow$	MAE $\downarrow$	$R^{2}$ $\uparrow$	PICP 90	MPIW 90
Persistence	27.710	7.321	0.995	–	–
Seasonal naive (24 h)	86.360	16.654	0.947	–	–
Seasonal naive (168 h)	110.895	18.165	0.913	–	–
Ridge	30.463	9.187	0.993	–	–
HistGradientBoosting	52.042	10.480	0.981	–	–
TSK-LS	29.083	8.676	0.994	–	–

Table 3: BDG2 Route C common-unit comparison. All methods use the same 47,400 sampled test rows; trained models use the same 94,800 train-plus-validation rows.

Fold	Held-out sites	$n$	RMSE $\downarrow$	PICP 90
0	Bear, Fox, Mouse, Shrew	10800	13.444	0.987
1	Bobcat, Gator, Panther, Wolf	9600	9.995	0.998
2	Bull, Hog, Peacock	9000	77.032	0.969
3	Cockatoo, Lamb, Rat	9000	24.239	0.984
4	Eagle, Moose, Robin	9000	36.700	0.947
Source: site_holdout_results.json ; MPIW 90 and proper scores remain in the machine-readable result ledger.

Table 4: Complementary stored warm-start site-held-out behavior of Bayesian TSK on BDG2. Each fold holds out sites while retaining explicitly available historical lag features; this table is separate from the Route C common-unit primary estimand and is not a cold-start test.

Target	Method	PICP	MPIW	WIS	CRPS
Energy-Heating	Bayesian-TSK	0.940	8.164	0.938	1.129
Energy-Heating	TSK-SpikeSlab-BIC	0.940	8.151	0.944	1.138
Energy-Heating	TSK-SpikeSlab-Gibbs	0.940	8.180	0.939	1.132
Energy-Heating	Gaussian Process	0.931	1.748	0.210	0.252
Energy-Cooling	Bayesian-TSK	0.932	10.270	1.203	1.442
Energy-Cooling	TSK-SpikeSlab-BIC	0.934	10.248	1.205	1.445

Table 5: Posterior-predictive metrics for methods that produce probabilistic intervals. WIS and CRPS are unavailable for the conformal baseline because it does not define a posterior-predictive distribution.

Target	$R^{2}$	PICP	MPIW
Energy-Heating	0.956	0.971	9.433
Energy-Cooling	0.922	0.964	12.524
Concrete	0.787	0.957	32.932

Table 6: Split-conformal TSK-LS results. Calibration is performed inside each outer training split, so this row is not a posterior-predictive baseline under the same protocol as Table 5 .

Regression Uncertainty Estimation

See all 26 papers

Apr 4, 2026

Sparse Regression under Correlation and Weak Signals: A Reproducible Benchmark of Classical and Bayesian Methods

Choosing between classical and Bayesian sparse regression methods involves a real trade-off: penalized estimators like Lasso run in milliseconds but give no uncertainty estimates,while Horseshoe and Spike-and-Slab priors produce full posteriors but need MCMC chains that take minutes per fit.Surprisingly few studies compare these two families head-to-head under the conditions that actually make sparse regression hard -- correlated features, weak signals, and growing dimensionality. We benchmark six methods (OLS, Ridge,Lasso, Elastic Net, Horseshoe, Spike-and-Slab) on synthetic data with three covariance structures (rho up to 0.9), four SNR levels, and p in {20, 50, 100}, plus the Diabetes dataset,totalling over 2,600 experiments. The results are clear on some points and nuanced on others. Bayesian methods win on prediction error (MSE 72 vs. 108-267), and the Horseshoe delivers near-nominal 95% coverage (94.8%). But Spike-and-Slab,despite narrower intervals, under-covers at 91.9% -- its continuous relaxation likely plays a role. For variable selection, Lasso and Spike-and-Slab tie at F1 ~ 0.47, making Lasso the practical default when posteriors are not needed. Code and data are available at https://github.com/xiao98/sparse-bayesian-regression-bench.

Hao Xiao

ATLAS AI Lab

Sparse Recovery Linear Regression Feature Selection Regression Prediction Intervals

Ask PDF

Jul 22, 2026

Interpretable Fuzzy Rule-Based Regression Extension for Ex-Fuzzy Library

Machine learning models achieve high predictive accuracy in regression tasks, but their deployment in safety-critical and regulated domains requires interpretability. While fuzzy rule-based systems offer transparent, linguistically explicit interpretable models, Mamdani-style fuzzy regression remains underrepresented in modern machine learning software libraries. This paper presents an interpretable regression extension for the Ex-Fuzzy library, enabling Mamdani fuzzy inference with scalar consequents learned directly from data. For this, a target-aware partition initialisation strategy based on Fuzzy C-Means clustering is introduced, in which linguistic variables are derived from an augmented input-output space to emphasise output-relevant regions of the feature space. The proposed extension is evaluated on ten regression datasets from the KEEL repository, comparing Gaussian and trapezoidal partition strategies against standard baselines including linear regression, multilayer perceptron, and random forests. Experimental results show that Gaussian partitions consistently outperform uniform trapezoidal partitions, achieving a mean coefficient of determination of approximately 0.86 while producing compact rule bases of 10-15 human-readable rules. The proposed implementation provides a transparent and competitive alternative to black-box regression models, supporting practical interpretability with competitive predictive performance.

Cayan Deniz Kucuktopana, Javier Fumanal-Idocin, Richard Pitts +1

School of Computer Science and Electronic Engineering University of Essex Colchester, United Kingdom

Interpretable ML Fuzzy Logic Tabular Regression

Ask PDF

Jul 5, 2026

Robust Bayes-Assisted Conformal Prediction

Bayes-assisted conformal prediction combines the strengths of Bayesian modelling with exact, distribution-free frequentist coverage guarantees. Although conformal validity is preserved even when the Bayesian working model (BWM) is misspecified, the size of the resulting prediction sets can degrade substantially when the prior is poorly aligned with the observed data. We address this limitation by introducing RoBAS (Robust Bayes-Assisted Shrinkage): a Bayes-assisted framework for constructing robust nonconformity scores, with two instantiations: one induced by a heavy-tailed BWM, and a closed-form empirical Bayes shrinkage score. The resulting scores adapt to the quality of the working information encoded in the prior: when this information is reliable, they exploit it to produce efficient prediction sets; when it is weak or inaccurate, they revert to the Distance-To-Average (DTA) score, a robust non-informative baseline. We evaluate the proposed scores on tabular and image regression tasks where the training distribution may differ from the calibration and test distributions, while the calibration and test data themselves remain exchangeable. We find that they are competitive with widely used scores in the absence of such shift, while substantially reducing interval widths in shifted settings.

Kianoosh Ashouritaklimi, Stefano Cortinovis, François Caron

Department of Statistics, University of Oxford

Distribution Shift Robustness Conformal Prediction Regression Prediction Intervals

Ask PDF

Abstract

Regression Uncertainty Estimation

Explore similar work

Sparse Regression under Correlation and Weak Signals: A Reproducible Benchmark of Classical and Bayesian Methods

Interpretable Fuzzy Rule-Based Regression Extension for Ex-Fuzzy Library

Robust Bayes-Assisted Conformal Prediction