Learning Interpretable Differentiable Logic Networks for Tabular Regression

Chang Yue; Niraj K. Jha

cs.LGMay 29, 2025

Learning Interpretable Differentiable Logic Networks for Tabular Regression

Authors: Chang Yue, Niraj K. Jha

Organizations: Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ 08544, USA

Abstract

Neural networks (NNs) achieve outstanding performance in many domains; however, their decision processes are often opaque and their inference can be computationally expensive in resource-constrained environments. We recently proposed Differentiable Logic Networks (DLNs) to address these issues for tabular classification based on relaxing discrete logic into a differentiable form, thereby enabling gradient-based learning of networks built from binary logic operations. DLNs offer interpretable reasoning and substantially lower inference cost. We extend the DLN framework to supervised tabular regression. We first redesign the final output layer (the SumLayer) to support continuous targets. More critically, we find the original two-phase training procedure used for classification is suboptimal for regression, and thus develop a unified, single-stage optimization procedure. We also demonstrate that temperature annealing of the network's differentiable relaxations is decisive for achieving stable convergence and high accuracy. We evaluate the resulting model on 15 public regression benchmarks, comparing it with modern neural networks and classical regression baselines. Regression DLNs match or exceed baseline accuracy while preserving interpretability and fast inference. Our results show that DLNs are a viable, cost-effective alternative for regression tasks, especially where model transparency and computational efficiency are important.

Figures & tables

Figure 1: A simplified regression DLN example. Continuous input features are first binarized by a ThresholdLayer, producing a binary vector. This vector is then processed by successive LogicLayers composed of two-input Boolean operators. The activations of the final LogicLayer are combined by a SumLayer that computes a weighted sum, yielding the real-valued prediction.

Figure 2: Training workflow for a regression DLN.

	ThresholdLayer	LogicLayer	SumLayer
Trainable parameters	bias $\mathbf{b}\in\mathbb{R}^{\text{in\_dim}}$ slope $\mathbf{s}\in\mathbb{R}^{\text{in\_dim}}$	logic fn weight $\mathbf{W}\in\mathbb{R}^{\text{out\_dim}\times 16}$ link $a$ weight $\mathbf{U}\in\mathbb{R}^{\text{out\_dim}\times\text{in\_dim}}$ link $b$ weight $\mathbf{V}\in\mathbb{R}^{\text{out\_dim}\times\text{in\_dim}}$	link weight $\mathbf{S}\in\mathbb{R}^{\text{in\_dim}}$ coefficient $\mathbf{C}\in\mathbb{R}^{\text{in\_dim}}$
Training	$\mathbf{y}_{i}=\mathrm{Sigmoid}\bigl(\mathbf{s}_{i}\cdot(\mathbf{x}_{i}-\mathbf{b}_{i})/\tau\bigr)$	$\mathbf{y}_{i}=\sum_{k=0}^{15}P_{k}\,\mathrm{SoftLogic}_{k}(a,b)$ $P=\mathrm{Softmax}(\mathbf{W}_{i,:}/\tau)$ $a=\sum_{j=0}^{\text{in\_dim}-1}[\mathrm{Softmax}(\mathbf{U}_{i,:}/\tau)]_{j}\cdot\mathbf{x}_{j}$ $b=\sum_{j=0}^{\text{in\_dim}-1}[\mathrm{Softmax}(\mathbf{V}_{i,:}/\tau)]_{j}\cdot\mathbf{x}_{j}$	$y=\sum_{j=0}^{\text{in\_dim}-1}\mathrm{Sigmoid}\bigl(\mathbf{S}_{j}/\tau\bigr)\cdot\mathbf{C}_{j}\cdot\mathbf{x}_{j}$
Inference	$\mathbf{y}_{i}=\mathrm{Heaviside}\bigl(\mathbf{s}_{i}\cdot(\mathbf{x}_{i}-\mathbf{b}_{i})\bigr)$	$\mathbf{y}_{i}=\text{HardLogic}_{k}(\mathbf{x}_{a_{i}},\mathbf{x}_{b_{i}})$ $k=\text{arg max}_{j}\mathbf{W}_{i,j}$ $a_{i}=\text{arg max}_{j}\mathbf{U}_{i,j}$ $b_{i}=\text{arg max}_{j}\mathbf{V}_{i,j}$	$y=\sum_{j=0}^{\text{in\_dim}-1}1_{\{\mathrm{Sigmoid}(\mathbf{S}_{j}/\tau)\,\geq\,\theta_{\mathrm{sum\text{-}th}}\}}\cdot\mathbf{C}_{j}\cdot\mathbf{x}_{j}$

Table 1: Summary of DLN trainable parameters and feedforward functions during unified training and during inference. We use

\mathbf{x}

to denote input and

\mathbf{y}

to denote the output of each layer.

Figure 3: Illustration of the training process. The network learns neuron functions and connections simultaneously: Key details are highlighted for clarity.

Dataset	# Train	# Test	# Cont.	# Cate.	Source	ID
Abalone	3132	1045	7	2	UCI	1
Airfoil	1127	376	5	0	UCI	291
Bike	13032	4345	6	14	UCI	275
CCPP	7145	2382	4	0	UCI	294
Concrete	753	252	8	0	UCI	165
Electrical	7500	2500	12	0	UCI	471

Table 2: Characteristics of the datasets after preprocessing

Figure 4: Comparison of

R^{2}

and the number of operations required for inference across models. We plot results for 32, 64, and 128 hyperparameter-search trials and draw the Pareto frontier for each case. DLN is Pareto-optimal in all three. XGBoost is the best-performing model in terms of accuracy. The vanilla DLN ensembles extend the Pareto frontier, outperforming random forest, a model that uses a similar ensembling framework, on both accuracy and inference cost.

	Linear	Ridge	Lasso	KNN	DT	AB	RF	XGB	SVR	MLP	DLN
Abalone	0.534 $\pm$ 0.024	0.534 $\pm$ 0.024	0.534 $\pm$ 0.024	0.517 $\pm$ 0.013	0.475 $\pm$ 0.018	0.463 $\pm$ 0.027	0.547 $\pm$ 0.019	0.554 $\pm$ 0.018	0.551 $\pm$ 0.018	0.570 $\pm$ 0.024	0.526 $\pm$ 0.026
Airfoil	0.516 $\pm$ 0.034	0.516 $\pm$ 0.033	0.516 $\pm$ 0.033	0.874 $\pm$ 0.026	0.793 $\pm$ 0.052	0.743 $\pm$ 0.020	0.868 $\pm$ 0.016	0.952 $\pm$ 7.4e-03	0.822 $\pm$ 0.024	0.943 $\pm$ 5.7e-03	0.889 $\pm$ 0.013
Bike	0.403 $\pm$ 0.012	0.403 $\pm$ 0.012	0.403 $\pm$ 0.012	0.659 $\pm$ 0.012	0.904 $\pm$ 0.011	0.660 $\pm$ 0.017	0.870 $\pm$ 7.1e-03	0.956 $\pm$ 3.2e-03	0.645 $\pm$ 0.014	0.939 $\pm$ 3.0e-03	0.916 $\pm$ 6.4e-03
CCPP	0.929 $\pm$ 2.4e-03	0.929 $\pm$ 2.4e-03	0.929 $\pm$ 2.4e-03	0.957 $\pm$ 2.0e-03	0.942 $\pm$ 2.8e-03	0.920 $\pm$ 3.8e-03	0.947 $\pm$ 4.6e-03	0.966 $\pm$ 1.3e-03	0.946 $\pm$ 2.2e-03	0.948 $\pm$ 2.4e-03	0.943 $\pm$ 2.3e-03
Concrete	0.593 $\pm$ 0.034	0.593 $\pm$ 0.034	0.593 $\pm$ 0.034	0.709 $\pm$ 0.038	0.795 $\pm$ 0.029	0.791 $\pm$ 0.015	0.877 $\pm$ 0.018	0.931 $\pm$ 0.010	0.869 $\pm$ 0.014	0.882 $\pm$ 0.022	0.888 $\pm$ 0.020
Electrical	0.647 $\pm$ 0.014	0.647 $\pm$ 0.014	0.647 $\pm$ 0.013	0.798 $\pm$ 4.0e-03	0.755 $\pm$ 0.012	0.826 $\pm$ 8.4e-03	0.843 $\pm$ 0.013	0.955 $\pm$ 2.7e-03	0.962 $\pm$ 1.7e-03	0.968 $\pm$ 2.0e-03	0.936 $\pm$ 3.6e-03

Table 3: Average test

R^{2}

across 10 random seeds and 128 hyperparameter trials per model

Figure 5: Distribution of

R^{2}

across datasets for each model, averaged over 10 random seeds and 128 hyperparameter-search trials.

Figure 6: Pearson correlation matrix of model

R^{2}

scores, averaged over 10 random seeds and 128 hyperparameter-search trials.

	32 trials	64 trials	128 trials
Linear	0.614	0.614 $\rightarrow$	0.614 $\rightarrow$
Ridge	0.615	0.615 $\rightarrow$	0.615 $\rightarrow$
Lasso	0.615	0.615 $\rightarrow$	0.615 $\rightarrow$
KNN	0.726	0.727 $\uparrow$	0.727 $\rightarrow$
DT	0.755	0.771 $\uparrow$	0.777 $\uparrow$
AB	0.722	0.723 $\uparrow$	0.725 $\uparrow$

Table 4: Mean

R^{2}

scores across HPO budgets

	Linear	Ridge	Lasso	KNN	DT	AB	RF	XGB	SVR	MLP	DLN
Abalone	11.2K	11.2K	10.7K	38.5M	1.19K	206K	240K	432K	383M	829K	41.2K
Airfoil	6.22K	6.22K	6.22K	9.66M	2.12K	295K	174K	853K	109M	1.71M	32.6K
Bike	24.9K	24.9K	21.9K	271M	2.85K	158K	177K	1.01M	1.30G	12.9M	50.0K
CCPP	4.98K	4.98K	4.98K	38.9M	2.01K	199K	201K	960K	504M	1.22M	22.5K
Concrete	9.95K	9.95K	8.71K	9.46M	2.05K	293K	161K	678K	67.6M	2.57M	60.6K
Electrical	14.9K	14.9K	13.3K	89.5M	2.36K	296K	191K	840K	1.03G	4.65M	91.1K

Table 5: Average number of basic gate-level logic operations required for inference across 10 random seeds and 128 hyperparameter trials, assuming FP16 for floating-point and INT16 for integer arithmetic

Batch Size	XGB	MLP	DLN
Batch = 1	227 $\mu\text{s}$	39.8 $\mu\text{s}$	12.6 $\mu\text{s}$
Batch = 32	13.8 $\mu\text{s}$	1.62 $\mu\text{s}$	1.11 $\mu\text{s}$

Table 6: Per-sample inference latency (in microseconds) on a single-threaded CPU. Values are the geometric mean (across 15 datasets) of the per-dataset average latencies (mean over 10 seeds). Lower is better.

Figure 7: Decision process learned by a DLN on the Insurance dataset. The model attains a test

R^{2}

of 0.866 and relies on two continuous features and two one-hot encoded categorical features, selected from an original set of two continuous and 12 one-hot features.

Figure 8: Decision process learned by a DLN on the Estate dataset. The model attains a test

R^{2}

of 0.758 and utilizes all six continuous input features.

Figure 9: Decision process learned by a DLN on the Yacht dataset. The model attains a test

R^{2}

of 0.997 and selects three of the six available continuous features.

	Orig	No $\tau$ sched.	Two phases	Subspace 16	Subspace 4	No concat
Abalone	0.525 $\pm$ 0.024	0.524 $\pm$ 0.024	0.526 $\pm$ 0.027	0.522 $\pm$ 0.025	0.520 $\pm$ 0.024	0.514 $\pm$ 0.015
Airfoil	0.871 $\pm$ 0.021	0.830 $\pm$ 0.024	0.864 $\pm$ 0.016	0.874 $\pm$ 0.014	0.866 $\pm$ 0.015	0.859 $\pm$ 0.018
Bike	0.902 $\pm$ 0.016	0.856 $\pm$ 0.037	0.900 $\pm$ 0.017	0.911 $\pm$ 0.016	0.880 $\pm$ 0.017	0.854 $\pm$ 0.039
CCPP	0.942 $\pm$ 2.5e-03	0.939 $\pm$ 2.8e-03	0.941 $\pm$ 3.1e-03	0.941 $\pm$ 3.0e-03	0.940 $\pm$ 2.9e-03	0.938 $\pm$ 2.9e-03
Concrete	0.887 $\pm$ 0.011	0.866 $\pm$ 0.023	0.876 $\pm$ 0.016	0.882 $\pm$ 0.012	0.885 $\pm$ 0.016	0.873 $\pm$ 0.028
Electrical	0.926 $\pm$ 7.2e-03	0.913 $\pm$ 0.015	0.932 $\pm$ 5.1e-03	0.930 $\pm$ 8.6e-03	0.919 $\pm$ 8.0e-03	0.918 $\pm$ 5.3e-03

Table 7: Average test

R^{2}

for the ablation studies across 10 random seeds and 32 hyperparameter trials

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

	Linear	Ridge	Lasso	KNN	DT	AB	RF	XGB	SVR	MLP	DLN
Abalone	2.16	2.16	2.16	2.20	2.29	2.32	2.13	2.11	2.12	2.07	2.18
Airfoil	4.77	4.78	4.78	2.42	3.10	3.48	2.49	1.50	2.89	1.64	2.28
Bike	140	140	140	106	55.9	105	65.3	37.9	108	44.6	52.5
CCPP	4.54	4.54	4.54	3.52	4.11	4.80	3.90	3.11	3.94	3.86	4.06
Concrete	10.3	10.3	10.3	8.70	7.31	7.38	5.65	4.23	5.84	5.53	5.40
Electrical	0.0220	0.0220	0.0220	0.0166	0.0183	0.0154	0.0147	7.86e-03	7.19e-03	6.57e-03	9.36e-03

Appendix

Table 8: Average test RMSE across 10 random seeds and 128 hyperparameter trials per model

	Linear	Ridge	Lasso	KNN	DT	AB	RF	XGB	SVR	MLP	DLN
Abalone	1.57	1.57	1.57	1.54	1.62	1.76	1.50	1.50	1.47	1.48	1.57
Airfoil	3.69	3.69	3.69	1.63	2.34	2.83	1.91	0.993	2.05	1.16	1.71
Bike	105	105	104	69.1	33.4	82.9	41.9	22.9	67.3	27.5	36.5
CCPP	3.61	3.61	3.61	2.48	3.04	3.76	2.95	2.19	2.95	2.87	3.09
Concrete	8.23	8.25	8.27	6.43	5.22	6.01	4.27	2.86	4.29	3.83	3.98
Electrical	0.0174	0.0174	0.0174	0.0129	0.0142	0.0126	0.0116	5.74e-03	4.52e-03	4.38e-03	7.14e-03

Appendix

Table 9: Average test MAE across 10 random seeds and 128 hyperparameter trials per model

Explore similar work

Learning Interpretable Differentiable Logic Networks for Time-Series Classification
Aug 24, 2025Chang Yue, Niraj K. JhaDifferentiable Logic Gate Networks Time-Series Classification
Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ 08544, USA
Neural Logic Networks for Interpretable Classification
Aug 11, 2025Vincent Perreault, Katsumi Inoue, Richard Labib +1Differentiable Logic Gate Networks Neural Network
Department of Mathematics and Industrial Engineering Polytechnique Montréal · National Institute of Informatics, Tokyo
Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks
Jul 10, 2026Wout Mommen, Lars Keuninckx, Matthias Hartmann +2Differentiable Logic Gate Networks Mnist
imec, Kapeldreef 75, Leuven, 3001, , Belgium · Vrije Universiteit Brussel, Peinlaan 2, Elsene, 1050, , Belgium

Aug 24, 2025, cs.LGJ/K move · Enter open · S save

Learning Interpretable Differentiable Logic Networks for Time-Series Classification

Chang Yue, Niraj K. Jha

Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ 08544, USA

Differentiable Logic Gate Networks Time-Series Classification Differentiable Optimization

Open paper AskPDF arXiv

Differentiable logic networks (DLNs) have shown promising results in tabular domains by combining accuracy, interpretability, and computational efficiency. In this work, we apply DLNs to the domain of TSC for the first time, focusing on univariate datasets. To enable DLN application in this context, we adopt feature-based representations relying on Catch22 and TSFresh, converting sequential time series into vectorized forms suitable for DLN classification. Unlike prior DLN studies that fix the training configuration and vary various settings in isolation via ablation, we integrate all such configurations into the hyperparameter search space, enabling the search process to select jointly optimal settings. We then analyze the distribution of selected configurations to better understand DLN training dynamics. We evaluate our approach on 51 publicly available univariate TSC benchmarks. The results confirm that classification DLNs maintain their core strengths in this new domain: they deliver competitive accuracy, retain low inference cost, and provide transparent, interpretable decision logic, thus aligning well with previous DLN findings in the realm of tabular classification and regression tasks.

Abstract

Explore similar work

Learning Interpretable Differentiable Logic Networks for Time-Series Classification

Neural Logic Networks for Interpretable Classification

Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks