Learning Interpretable Differentiable Logic Networks for Tabular Regression

Chang Yue; Niraj K. Jha

cs.LGMay 29, 2025

Learning Interpretable Differentiable Logic Networks for Tabular Regression

Authors: Chang Yue, Niraj K. Jha

Organizations: Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ 08544, USA

Abstract

Neural networks (NNs) achieve outstanding performance in many domains; however, their decision processes are often opaque and their inference can be computationally expensive in resource-constrained environments. We recently proposed Differentiable Logic Networks (DLNs) to address these issues for tabular classification based on relaxing discrete logic into a differentiable form, thereby enabling gradient-based learning of networks built from binary logic operations. DLNs offer interpretable reasoning and substantially lower inference cost. We extend the DLN framework to supervised tabular regression. We first redesign the final output layer (the SumLayer) to support continuous targets. More critically, we find the original two-phase training procedure used for classification is suboptimal for regression, and thus develop a unified, single-stage optimization procedure. We also demonstrate that temperature annealing of the network's differentiable relaxations is decisive for achieving stable convergence and high accuracy. We evaluate the resulting model on 15 public regression benchmarks, comparing it with modern neural networks and classical regression baselines. Regression DLNs match or exceed baseline accuracy while preserving interpretability and fast inference. Our results show that DLNs are a viable, cost-effective alternative for regression tasks, especially where model transparency and computational efficiency are important.

Figures & tables

Figure 1: A simplified regression DLN example. Continuous input features are first binarized by a ThresholdLayer, producing a binary vector. This vector is then processed by successive LogicLayers composed of two-input Boolean operators. The activations of the final LogicLayer are combined by a SumLayer that computes a weighted sum, yielding the real-valued prediction.

Figure 2: Training workflow for a regression DLN.

	ThresholdLayer	LogicLayer	SumLayer
Trainable parameters	bias $\mathbf{b}\in\mathbb{R}^{\text{in\_dim}}$ slope $\mathbf{s}\in\mathbb{R}^{\text{in\_dim}}$	logic fn weight $\mathbf{W}\in\mathbb{R}^{\text{out\_dim}\times 16}$ link $a$ weight $\mathbf{U}\in\mathbb{R}^{\text{out\_dim}\times\text{in\_dim}}$ link $b$ weight $\mathbf{V}\in\mathbb{R}^{\text{out\_dim}\times\text{in\_dim}}$	link weight $\mathbf{S}\in\mathbb{R}^{\text{in\_dim}}$ coefficient $\mathbf{C}\in\mathbb{R}^{\text{in\_dim}}$
Training	$\mathbf{y}_{i}=\mathrm{Sigmoid}\bigl(\mathbf{s}_{i}\cdot(\mathbf{x}_{i}-\mathbf{b}_{i})/\tau\bigr)$	$\mathbf{y}_{i}=\sum_{k=0}^{15}P_{k}\,\mathrm{SoftLogic}_{k}(a,b)$ $P=\mathrm{Softmax}(\mathbf{W}_{i,:}/\tau)$ $a=\sum_{j=0}^{\text{in\_dim}-1}[\mathrm{Softmax}(\mathbf{U}_{i,:}/\tau)]_{j}\cdot\mathbf{x}_{j}$ $b=\sum_{j=0}^{\text{in\_dim}-1}[\mathrm{Softmax}(\mathbf{V}_{i,:}/\tau)]_{j}\cdot\mathbf{x}_{j}$	$y=\sum_{j=0}^{\text{in\_dim}-1}\mathrm{Sigmoid}\bigl(\mathbf{S}_{j}/\tau\bigr)\cdot\mathbf{C}_{j}\cdot\mathbf{x}_{j}$
Inference	$\mathbf{y}_{i}=\mathrm{Heaviside}\bigl(\mathbf{s}_{i}\cdot(\mathbf{x}_{i}-\mathbf{b}_{i})\bigr)$	$\mathbf{y}_{i}=\text{HardLogic}_{k}(\mathbf{x}_{a_{i}},\mathbf{x}_{b_{i}})$ $k=\text{arg max}_{j}\mathbf{W}_{i,j}$ $a_{i}=\text{arg max}_{j}\mathbf{U}_{i,j}$ $b_{i}=\text{arg max}_{j}\mathbf{V}_{i,j}$	$y=\sum_{j=0}^{\text{in\_dim}-1}1_{\{\mathrm{Sigmoid}(\mathbf{S}_{j}/\tau)\,\geq\,\theta_{\mathrm{sum\text{-}th}}\}}\cdot\mathbf{C}_{j}\cdot\mathbf{x}_{j}$

Table 1: Summary of DLN trainable parameters and feedforward functions during unified training and during inference. We use

\mathbf{x}

to denote input and

\mathbf{y}

to denote the output of each layer.

Figure 3: Illustration of the training process. The network learns neuron functions and connections simultaneously: Key details are highlighted for clarity.

Dataset	# Train	# Test	# Cont.	# Cate.	Source	ID
Abalone	3132	1045	7	2	UCI	1
Airfoil	1127	376	5	0	UCI	291
Bike	13032	4345	6	14	UCI	275
CCPP	7145	2382	4	0	UCI	294
Concrete	753	252	8	0	UCI	165
Electrical	7500	2500	12	0	UCI	471

Table 2: Characteristics of the datasets after preprocessing

Figure 4: Comparison of

R^{2}

and the number of operations required for inference across models. We plot results for 32, 64, and 128 hyperparameter-search trials and draw the Pareto frontier for each case. DLN is Pareto-optimal in all three. XGBoost is the best-performing model in terms of accuracy. The vanilla DLN ensembles extend the Pareto frontier, outperforming random forest, a model that uses a similar ensembling framework, on both accuracy and inference cost.

	Linear	Ridge	Lasso	KNN	DT	AB	RF	XGB	SVR	MLP	DLN
Abalone	0.534 $\pm$ 0.024	0.534 $\pm$ 0.024	0.534 $\pm$ 0.024	0.517 $\pm$ 0.013	0.475 $\pm$ 0.018	0.463 $\pm$ 0.027	0.547 $\pm$ 0.019	0.554 $\pm$ 0.018	0.551 $\pm$ 0.018	0.570 $\pm$ 0.024	0.526 $\pm$ 0.026
Airfoil	0.516 $\pm$ 0.034	0.516 $\pm$ 0.033	0.516 $\pm$ 0.033	0.874 $\pm$ 0.026	0.793 $\pm$ 0.052	0.743 $\pm$ 0.020	0.868 $\pm$ 0.016	0.952 $\pm$ 7.4e-03	0.822 $\pm$ 0.024	0.943 $\pm$ 5.7e-03	0.889 $\pm$ 0.013
Bike	0.403 $\pm$ 0.012	0.403 $\pm$ 0.012	0.403 $\pm$ 0.012	0.659 $\pm$ 0.012	0.904 $\pm$ 0.011	0.660 $\pm$ 0.017	0.870 $\pm$ 7.1e-03	0.956 $\pm$ 3.2e-03	0.645 $\pm$ 0.014	0.939 $\pm$ 3.0e-03	0.916 $\pm$ 6.4e-03
CCPP	0.929 $\pm$ 2.4e-03	0.929 $\pm$ 2.4e-03	0.929 $\pm$ 2.4e-03	0.957 $\pm$ 2.0e-03	0.942 $\pm$ 2.8e-03	0.920 $\pm$ 3.8e-03	0.947 $\pm$ 4.6e-03	0.966 $\pm$ 1.3e-03	0.946 $\pm$ 2.2e-03	0.948 $\pm$ 2.4e-03	0.943 $\pm$ 2.3e-03
Concrete	0.593 $\pm$ 0.034	0.593 $\pm$ 0.034	0.593 $\pm$ 0.034	0.709 $\pm$ 0.038	0.795 $\pm$ 0.029	0.791 $\pm$ 0.015	0.877 $\pm$ 0.018	0.931 $\pm$ 0.010	0.869 $\pm$ 0.014	0.882 $\pm$ 0.022	0.888 $\pm$ 0.020
Electrical	0.647 $\pm$ 0.014	0.647 $\pm$ 0.014	0.647 $\pm$ 0.013	0.798 $\pm$ 4.0e-03	0.755 $\pm$ 0.012	0.826 $\pm$ 8.4e-03	0.843 $\pm$ 0.013	0.955 $\pm$ 2.7e-03	0.962 $\pm$ 1.7e-03	0.968 $\pm$ 2.0e-03	0.936 $\pm$ 3.6e-03

Table 3: Average test

R^{2}

across 10 random seeds and 128 hyperparameter trials per model

Figure 5: Distribution of

R^{2}

across datasets for each model, averaged over 10 random seeds and 128 hyperparameter-search trials.

Figure 6: Pearson correlation matrix of model

R^{2}

scores, averaged over 10 random seeds and 128 hyperparameter-search trials.

	32 trials	64 trials	128 trials
Linear	0.614	0.614 $\rightarrow$	0.614 $\rightarrow$
Ridge	0.615	0.615 $\rightarrow$	0.615 $\rightarrow$
Lasso	0.615	0.615 $\rightarrow$	0.615 $\rightarrow$
KNN	0.726	0.727 $\uparrow$	0.727 $\rightarrow$
DT	0.755	0.771 $\uparrow$	0.777 $\uparrow$
AB	0.722	0.723 $\uparrow$	0.725 $\uparrow$

Table 4: Mean

R^{2}

scores across HPO budgets

	Linear	Ridge	Lasso	KNN	DT	AB	RF	XGB	SVR	MLP	DLN
Abalone	11.2K	11.2K	10.7K	38.5M	1.19K	206K	240K	432K	383M	829K	41.2K
Airfoil	6.22K	6.22K	6.22K	9.66M	2.12K	295K	174K	853K	109M	1.71M	32.6K
Bike	24.9K	24.9K	21.9K	271M	2.85K	158K	177K	1.01M	1.30G	12.9M	50.0K
CCPP	4.98K	4.98K	4.98K	38.9M	2.01K	199K	201K	960K	504M	1.22M	22.5K
Concrete	9.95K	9.95K	8.71K	9.46M	2.05K	293K	161K	678K	67.6M	2.57M	60.6K
Electrical	14.9K	14.9K	13.3K	89.5M	2.36K	296K	191K	840K	1.03G	4.65M	91.1K

Table 5: Average number of basic gate-level logic operations required for inference across 10 random seeds and 128 hyperparameter trials, assuming FP16 for floating-point and INT16 for integer arithmetic

Batch Size	XGB	MLP	DLN
Batch = 1	227 $\mu\text{s}$	39.8 $\mu\text{s}$	12.6 $\mu\text{s}$
Batch = 32	13.8 $\mu\text{s}$	1.62 $\mu\text{s}$	1.11 $\mu\text{s}$

Table 6: Per-sample inference latency (in microseconds) on a single-threaded CPU. Values are the geometric mean (across 15 datasets) of the per-dataset average latencies (mean over 10 seeds). Lower is better.

Figure 7: Decision process learned by a DLN on the Insurance dataset. The model attains a test

R^{2}

of 0.866 and relies on two continuous features and two one-hot encoded categorical features, selected from an original set of two continuous and 12 one-hot features.

Figure 8: Decision process learned by a DLN on the Estate dataset. The model attains a test

R^{2}

of 0.758 and utilizes all six continuous input features.

Figure 9: Decision process learned by a DLN on the Yacht dataset. The model attains a test

R^{2}

of 0.997 and selects three of the six available continuous features.

	Orig	No $\tau$ sched.	Two phases	Subspace 16	Subspace 4	No concat
Abalone	0.525 $\pm$ 0.024	0.524 $\pm$ 0.024	0.526 $\pm$ 0.027	0.522 $\pm$ 0.025	0.520 $\pm$ 0.024	0.514 $\pm$ 0.015
Airfoil	0.871 $\pm$ 0.021	0.830 $\pm$ 0.024	0.864 $\pm$ 0.016	0.874 $\pm$ 0.014	0.866 $\pm$ 0.015	0.859 $\pm$ 0.018
Bike	0.902 $\pm$ 0.016	0.856 $\pm$ 0.037	0.900 $\pm$ 0.017	0.911 $\pm$ 0.016	0.880 $\pm$ 0.017	0.854 $\pm$ 0.039
CCPP	0.942 $\pm$ 2.5e-03	0.939 $\pm$ 2.8e-03	0.941 $\pm$ 3.1e-03	0.941 $\pm$ 3.0e-03	0.940 $\pm$ 2.9e-03	0.938 $\pm$ 2.9e-03
Concrete	0.887 $\pm$ 0.011	0.866 $\pm$ 0.023	0.876 $\pm$ 0.016	0.882 $\pm$ 0.012	0.885 $\pm$ 0.016	0.873 $\pm$ 0.028
Electrical	0.926 $\pm$ 7.2e-03	0.913 $\pm$ 0.015	0.932 $\pm$ 5.1e-03	0.930 $\pm$ 8.6e-03	0.919 $\pm$ 8.0e-03	0.918 $\pm$ 5.3e-03

Table 7: Average test

R^{2}

for the ablation studies across 10 random seeds and 32 hyperparameter trials

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

	Linear	Ridge	Lasso	KNN	DT	AB	RF	XGB	SVR	MLP	DLN
Abalone	2.16	2.16	2.16	2.20	2.29	2.32	2.13	2.11	2.12	2.07	2.18
Airfoil	4.77	4.78	4.78	2.42	3.10	3.48	2.49	1.50	2.89	1.64	2.28
Bike	140	140	140	106	55.9	105	65.3	37.9	108	44.6	52.5
CCPP	4.54	4.54	4.54	3.52	4.11	4.80	3.90	3.11	3.94	3.86	4.06
Concrete	10.3	10.3	10.3	8.70	7.31	7.38	5.65	4.23	5.84	5.53	5.40
Electrical	0.0220	0.0220	0.0220	0.0166	0.0183	0.0154	0.0147	7.86e-03	7.19e-03	6.57e-03	9.36e-03

Appendix

Table 8: Average test RMSE across 10 random seeds and 128 hyperparameter trials per model

	Linear	Ridge	Lasso	KNN	DT	AB	RF	XGB	SVR	MLP	DLN
Abalone	1.57	1.57	1.57	1.54	1.62	1.76	1.50	1.50	1.47	1.48	1.57
Airfoil	3.69	3.69	3.69	1.63	2.34	2.83	1.91	0.993	2.05	1.16	1.71
Bike	105	105	104	69.1	33.4	82.9	41.9	22.9	67.3	27.5	36.5
CCPP	3.61	3.61	3.61	2.48	3.04	3.76	2.95	2.19	2.95	2.87	3.09
Concrete	8.23	8.25	8.27	6.43	5.22	6.01	4.27	2.86	4.29	3.83	3.98
Electrical	0.0174	0.0174	0.0174	0.0129	0.0142	0.0126	0.0116	5.74e-03	4.52e-03	4.38e-03	7.14e-03

Appendix

Table 9: Average test MAE across 10 random seeds and 128 hyperparameter trials per model

Explore similar work

Aug 24, 2025cs.LG

Learning Interpretable Differentiable Logic Networks for Time-Series Classification

Differentiable logic networks (DLNs) have shown promising results in tabular domains by combining accuracy, interpretability, and computational efficiency. In this work, we apply DLNs to the domain of TSC for the first time, focusing on univariate datasets. To enable DLN application in this context, we adopt feature-based representations relying on Catch22 and TSFresh, converting sequential time series into vectorized forms suitable for DLN classification. Unlike prior DLN studies that fix the training configuration and vary various settings in isolation via ablation, we integrate all such configurations into the hyperparameter search space, enabling the search process to select jointly optimal settings. We then analyze the distribution of selected configurations to better understand DLN training dynamics. We evaluate our approach on 51 publicly available univariate TSC benchmarks. The results confirm that classification DLNs maintain their core strengths in this new domain: they deliver competitive accuracy, retain low inference cost, and provide transparent, interpretable decision logic, thus aligning well with previous DLN findings in the realm of tabular classification and regression tasks.

Chang Yue, Niraj K. Jha

Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ 08544, USA

Differentiable Logic Gate Networks Time-Series Classification Differentiable Optimization

Ask PDF

Aug 11, 2025cs.LG

Neural Logic Networks for Interpretable Classification

Traditional neural networks have an impressive classification performance, but what they learn cannot be inspected, verified or extracted. Neural Logic Networks on the other hand have an interpretable structure that enables them to learn a logical mechanism relating the inputs and outputs with AND and OR operations. We generalize these networks with NOT operations and biases that take into account unobserved data and develop a rigorous logical and probabilistic modeling in terms of concept combinations to motivate their use. We also propose a novel factorized IF-THEN rule structure for the model as well as a modified learning algorithm. Our method improves the state-of-the-art in Boolean networks discovery and is able to learn relevant, interpretable rules in tabular classification, notably on examples from the medical and industrial fields where interpretability has tangible value.

Vincent Perreault, Katsumi Inoue, Richard Labib +1

Department of Mathematics and Industrial Engineering Polytechnique Montréal · National Institute of Informatics, Tokyo

Differentiable Logic Gate Networks Neural Network Bayesian Networks Discovery

Ask PDF

Jul 10, 2026cs.LG

Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks

We introduce a novel method for both partial and full optimization of the connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs). Our training method utilizes a probability distribution over a set of connections per gate/lookup table (LUT) input pin, selecting the connection with highest merit, all whilst the optimal gate types or LUT-entries are learned in parallel. We show that the connection-optimized LGNs outperform standard fixed-connection LGNs on the Yin-Yang, MNIST Handwritten Digits and Fashion-MNIST benchmarks, while requiring only a fraction of the number of logic gates. We achieve 98.92% on the MNIST dataset with two layers of 8000 gates. With only one layer of 8000 gates, we obtain 98.45%, showing that our method requires almost 50 times fewer gates compared to fixed-connection LGNs. Training stability up to ten layers has been ensured by employing a high learning rate, straight-through estimators and trimming constant-output gate types. Additionally, we present a LUT neuron description that enables stable training with backpropagation, tested up to 6-layer deep networks. The model requires four times fewer trainable parameters and still achieves a higher accuracy compared to the fixed-connection LGN training algorithm. Our connection-training algorithm also works well for the LUTNs, achieving an accuracy of 98.88% for two layers of 2000 6-input LUTs.

Wout Mommen, Lars Keuninckx, Matthias Hartmann +2

imec, Kapeldreef 75, Leuven, 3001, , Belgium · Vrije Universiteit Brussel, Peinlaan 2, Elsene, 1050, , Belgium

Differentiable Logic Gate Networks Mnist Discontinuous Gating

Ask PDF