Task-Aware Discretization of Differentiable Logic Gate Networks

Thore Gerlach

cs.LGSep 27, 2026

Task-Aware Discretization of Differentiable Logic Gate Networks

Authors: Thore Gerlach

Organizations: Independent Researcher

Abstract

Differentiable logic gate networks (DLGNs) enable gradient-based training of highly efficient Boolean networks by relaxing discrete logic gates during training and discretizing them for inference. Standard approaches make this discretization decision locally, typically through argmax selection and confidence- or entropy-based convergence criteria. We show that local discretization can be task-suboptimal even for globally optimal relaxed solutions, with high gate confidence providing no general guarantee, and derive bounds relating task-aware gate selection to tractable interventions in the relaxed network. Motivated by these results, we study first-order downstream task information for progressive discretization and characterize when this local approximation is reliable. Experiments on convolutional DLGNs reveal a strong locality dependence: first-order scores become unreliable when directly optimized over nonlocal interventions, but accurately assess local argmax decisions for progressive freezing.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of the discrete optimality gap and our task-aware discretization framework. The top row decomposes the gap between the deployed argmax network and the task-optimal discrete network; the bottom row summarizes the theoretical motivation, task-aware gate selection, its efficient first-order approximation, and progressive discretization.

Figure 2: Predicting freeze effects. Measured loss change from freezing to the argmax gate versus gate entropy (left), from freezing to the score-selected gate $k_{i}^{*}=\argmin_{k}S_{i}(k)$ versus its first-order score (center), and from freezing to the argmax gate versus its first-order score (right). Score selection is poorly aligned with the realized effect, whereas the argmax freeze shock is strongly predictive.

	Compact-S			TreeLogicNet-S
Layer	Entropy	Score	Argmax	Entropy	Score	Argmax
Conv 1	$+0.301\pm 0.045$	$-0.576\pm 0.038$	$+0.742\pm 0.047$	$+0.450\pm 0.082$	$-0.356\pm 0.106$	$+0.583\pm 0.081$
Conv 2	$+0.370\pm 0.053$	$-0.625\pm 0.021$	$+0.792\pm 0.053$	$+0.257\pm 0.063$	$-0.291\pm 0.056$	$+0.766\pm 0.039$
Conv 3	$+0.281\pm 0.018$	$-0.304\pm 0.074$	$+0.868\pm 0.021$	$+0.076\pm 0.016$	$+0.042\pm 0.049$	$+0.762\pm 0.034$
Conv 4	$+0.048\pm 0.045$	$-0.242\pm 0.081$	$+0.991\pm 0.004$	$+0.024\pm 0.019$	$+0.454\pm 0.045$	$+0.676\pm 0.013$

Table 1: Layer-wise Spearman correlations between local discretization criteria and realized freeze effects. Values are mean

\pm

sample standard deviation across five seeds, after averaging checkpoints within each seed (four for Compact-S and five for TreeLogicNet-S).

Figure 3: Locality of first-order interventions. (a) Score selection produces substantially larger activation displacements than argmax, with median squared displacements of

0.42

and

2.9\times 10^{-6}

, respectively. (b) Approximation error increases with displacement for both first- and second-order scores, with median within-neuron Spearman correlations of

0.88

and

0.80

, respectively; second order substantially reduces the typical error. (c) The apparent advantage of selecting low-displacement task-aware interventions disappears when gate selection and evaluation use disjoint validation data.

Panel of Figure 4 — Figure 4: Progressive freezing dynamics on Compact-S. (a) Fraction of frozen neurons throughout training for different discretization criteria. (b) Layer-wise frozen fractions show that TaskFreeze progressively commits individual decisions, whereas EntropyFreeze induces a markedly different layer-dependent schedule.

	Compact-S		Compact-M	TreeLogicNet-S	TreeLogicNet-M
Method	Resampling	Stopped	Resampling
TaskFreeze (ours)	$66.25\pm 0.27$	$65.93\pm 0.15$	$69.34\pm 0.46$	$60.76\pm 0.46$	$71.31\pm 0.04$
EntropyFreeze (ours)	$64.11\pm 0.53$	$63.88\pm 0.76$	$68.47\pm 0.54$	$59.97\pm 0.84$	$70.95\pm 0.23$
GlobalFreeze (ours)	$66.41\pm 0.21$	$66.12\pm 0.37$	$69.26\pm 0.21$	$60.50\pm 0.52$	$71.15\pm 0.04$
LayerEntropyFreeze	$64.10\pm 0.45$	$65.99\pm 0.17$	$67.67\pm 0.93$	$60.63\pm 0.55$	$71.44\pm 0.01$
FinalArgmax	$64.63\pm 0.61$	$64.63\pm 0.61$	$67.67\pm 0.39$	$59.29\pm 0.78$	$70.49\pm 0.27$

Table 2: Final discrete CIFAR-10 test accuracy (%). Results are reported at the 400k discretization start for Compact models and the 25k start for TreeLogicNet models. For Compact-S, we compare continued and stopped connection resampling after discretization begins. Values are mean

\pm

sample standard deviation across available seeds.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Operation	Stride	Compact-S ( $k=128$ )	Compact-M ( $k=256$ )
Input	Thermometer encoding	–	$(9,32,32)$	$(21,32,32)$
Conv 1	$3\times 3$ logic conv.	$2$	$(128,16,16)$	$(256,16,16)$
Conv 2	$3\times 3$ logic conv.	$1$	$(128,16,16)$	$(256,16,16)$
Conv 3	$3\times 3$ logic conv.	$2$	$(512,8,8)$	$(1024,8,8)$
Conv 4	$3\times 3$ logic conv.	$1$	$(512,8,8)$	$(1024,8,8)$
Flatten	–	–	$32{,}768$	$65{,}536$

Appendix

Table 3: Compact CIFAR-10 architectures. Compact-S uses

k=128

and

N=3

input thresholds; Compact-M uses

k=256

and

N=7

. Convolutional layers use

3\times 3

receptive fields and padding

1

Stage	Operation	TreeLogicNet-S ( $k=32$ )	TreeLogicNet-M ( $k=256$ )
Input	Three-threshold encoding	$(9,32,32)$	$(9,32,32)$
Conv 1	$3\times 3$ , tree depth $3$	$32$ channels	$256$ channels
Pool 1	$2\times 2$ OR, stride $2$	$(32,16,16)$	$(256,16,16)$
Conv 2	$3\times 3$ , tree depth $3$	$128$ channels	$1024$ channels
Pool 2	$2\times 2$ OR, stride $2$	$(128,8,8)$	$(1024,8,8)$
Conv 3	$3\times 3$ , tree depth $3$	$512$ channels	$4096$ channels

Appendix

Table 4: TreeLogicNet CIFAR-10 architectures. Each convolutional kernel is a depth-

3

Boolean tree with seven trainable gates, followed by fixed

2\times 2

OR pooling. S and M correspond to

k=32

and

k=256

, respectively.

Parameter	Compact	TreeLogicNet
Optimizer	Adam	AdamW
Weight decay	$0$	$0.002$
Training steps	$600$ k	$200$ k
Batch size	$128$	$128$
Learning rate	$0.02$	$0.02$
LR schedule	Constant	Constant

Appendix

Table 5: Training and freezing hyperparameters.

H

denotes entropy convergence and

S

first-order freeze shock. Values are shared across the Compact and TreeLogicNet experiments unless shown otherwise.

Figure 5: Predicting freeze effects on TreeLogicNet-S. Measured loss change from freezing to the argmax gate versus gate entropy (left), from freezing to the task-aware gate $k_{i}^{*}=\argmin_{k}S_{i}(k)$ versus its first-order score (center), and from freezing to the argmax gate versus its first-order score (right). As for Compact-S, score-based selection is less reliably aligned with the realized effect, whereas the argmax freeze shock is strongly predictive.

Panel of Figure 6 — Figure 6: Progressive freezing dynamics on TreeLogicNet-S. (a) Fraction of frozen neurons throughout training for different discretization criteria. (b) Layer-wise frozen fractions show that TaskFreeze progressively commits individual decisions, whereas EntropyFreeze induces a markedly different layer-dependent schedule.

Method	200k	300k	400k	200k	300k	400k
	Continued resampling			Stopped resampling
TaskFreeze (ours)	$66.13\pm 0.20$	$66.52\pm 0.17$	$66.25\pm 0.27$	$65.55\pm 0.29$	$65.77\pm 0.15$	$65.93\pm 0.15$
EntropyFreeze (ours)	$64.30\pm 0.83$	$64.32\pm 0.72$	$64.11\pm 0.53$	$64.28\pm 0.48$	$64.00\pm 0.58$	$63.88\pm 0.76$
GlobalFreeze (ours)	$66.27\pm 0.51$	$66.17\pm 0.10$	$66.41\pm 0.21$	$65.84\pm 0.15$	$66.04\pm 0.26$	$66.12\pm 0.37$
LayerEntropyFreeze	$64.75\pm 0.80$	$65.24\pm 0.53$	$64.10\pm 0.45$	$65.94\pm 0.14$	$65.95\pm 0.26$	$65.99\pm 0.17$
FinalArgmax	$64.63\pm 0.61$	$64.63\pm 0.61$	$64.63\pm 0.61$	$64.63\pm 0.61$	$64.63\pm 0.61$	$64.63\pm 0.61$

Appendix

Table 6: Sensitivity to discretization start on Compact-S. Final discrete CIFAR-10 test accuracy (%) for discretization starting at 200k, 300k, or 400k training steps, with connection resampling either continued or stopped after discretization begins. Values are mean

\pm

sample standard deviation across available seeds.

Method	10k	25k	50k
TaskFreeze (ours)	$60.42\pm 0.63$	$60.76\pm 0.46$	$60.26\pm 0.80$
EntropyFreeze (ours)	$59.89\pm 0.45$	$59.97\pm 0.84$	$60.13\pm 0.57$
GlobalFreeze (ours)	$60.05\pm 0.82$	$60.50\pm 0.52$	$60.78\pm 0.28$
LayerEntropyFreeze	$60.72\pm 0.57$	$60.63\pm 0.55$	$60.81\pm 0.42$
FinalArgmax	$59.29\pm 0.78$	$59.29\pm 0.78$	$59.29\pm 0.78$
CAGE	$60.13\pm 0.49$	$59.98\pm 0.35$	$59.90\pm 0.71$

Appendix

Table 7: Sensitivity to discretization start on TreeLogicNet-S. Final discrete CIFAR-10 test accuracy (%) for discretization starting at 10k, 25k, or 50k training steps. Values are mean

\pm

sample standard deviation across five seeds.

Explore similar work

Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks
Jul 10, 2026Wout Mommen, Lars Keuninckx, Matthias Hartmann +2Differentiable Logic Gate Networks Mnist
imec, Kapeldreef 75, Leuven, 3001, , Belgium · Vrije Universiteit Brussel, Peinlaan 2, Elsene, 1050, , Belgium
On the Depth Scalability of Logic Gate Networks
Jul 19, 2026Taegun An, Dohun kim, Haebeom Lee +1Differentiable Logic Gate Networks Scalability
Jung Woonoh IT & General Education Center, Korea University 145, Anam-ro, Seongbuk-gu, Seoul 02841, Republic of Korea
Silicon Aware Neural Networks
Apr 21, 2026Sebastian Fieldhouse, Kea-Tiong TangDifferentiable Logic Gate Networks Field-Programmable Gate Arrays
College of Semiconductor Research National Tsing Hua University · Department of Electrical Engineering National Tsing Hua University

Jul 10, 2026, cs.LGJ/K move · Enter open · S save

Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks

Wout Mommen, Lars Keuninckx, Matthias Hartmann, Werner Van Leekwijck+1

imec, Kapeldreef 75, Leuven, 3001, , Belgium · Vrije Universiteit Brussel, Peinlaan 2, Elsene, 1050, , Belgium

Differentiable Logic Gate Networks Mnist Discontinuous Gating

Open paper AskPDF arXiv

We introduce a novel method for both partial and full optimization of the connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs). Our training method utilizes a probability distribution over a set of connections per gate/lookup table (LUT) input pin, selecting the connection with highest merit, all whilst the optimal gate types or LUT-entries are learned in parallel. We show that the connection-optimized LGNs outperform standard fixed-connection LGNs on the Yin-Yang, MNIST Handwritten Digits and Fashion-MNIST benchmarks, while requiring only a fraction of the number of logic gates. We achieve 98.92% on the MNIST dataset with two layers of 8000 gates. With only one layer of 8000 gates, we obtain 98.45%, showing that our method requires almost 50 times fewer gates compared to fixed-connection LGNs. Training stability up to ten layers has been ensured by employing a high learning rate, straight-through estimators and trimming constant-output gate types. Additionally, we present a LUT neuron description that enables stable training with backpropagation, tested up to 6-layer deep networks. The model requires four times fewer trainable parameters and still achieves a higher accuracy compared to the fixed-connection LGN training algorithm. Our connection-training algorithm also works well for the LUTNs, achieving an accuracy of 98.88% for two layers of 2000 6-input LUTs.

Abstract

Explore similar work

Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks

On the Depth Scalability of Logic Gate Networks

Silicon Aware Neural Networks