Task-Aware Discretization of Differentiable Logic Gate Networks

Thore Gerlach

cs.LGSep 27, 2026

Task-Aware Discretization of Differentiable Logic Gate Networks

Authors: Thore Gerlach

Organizations: Independent Researcher

Abstract

Differentiable logic gate networks (DLGNs) enable gradient-based training of highly efficient Boolean networks by relaxing discrete logic gates during training and discretizing them for inference. Standard approaches make this discretization decision locally, typically through argmax selection and confidence- or entropy-based convergence criteria. We show that local discretization can be task-suboptimal even for globally optimal relaxed solutions, with high gate confidence providing no general guarantee, and derive bounds relating task-aware gate selection to tractable interventions in the relaxed network. Motivated by these results, we study first-order downstream task information for progressive discretization and characterize when this local approximation is reliable. Experiments on convolutional DLGNs reveal a strong locality dependence: first-order scores become unreliable when directly optimized over nonlocal interventions, but accurately assess local argmax decisions for progressive freezing.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of the discrete optimality gap and our task-aware discretization framework. The top row decomposes the gap between the deployed argmax network and the task-optimal discrete network; the bottom row summarizes the theoretical motivation, task-aware gate selection, its efficient first-order approximation, and progressive discretization.

Figure 2: Predicting freeze effects. Measured loss change from freezing to the argmax gate versus gate entropy (left), from freezing to the score-selected gate $k_{i}^{*}=\argmin_{k}S_{i}(k)$ versus its first-order score (center), and from freezing to the argmax gate versus its first-order score (right). Score selection is poorly aligned with the realized effect, whereas the argmax freeze shock is strongly predictive.

	Compact-S			TreeLogicNet-S
Layer	Entropy	Score	Argmax	Entropy	Score	Argmax
Conv 1	$+0.301\pm 0.045$	$-0.576\pm 0.038$	$+0.742\pm 0.047$	$+0.450\pm 0.082$	$-0.356\pm 0.106$	$+0.583\pm 0.081$
Conv 2	$+0.370\pm 0.053$	$-0.625\pm 0.021$	$+0.792\pm 0.053$	$+0.257\pm 0.063$	$-0.291\pm 0.056$	$+0.766\pm 0.039$
Conv 3	$+0.281\pm 0.018$	$-0.304\pm 0.074$	$+0.868\pm 0.021$	$+0.076\pm 0.016$	$+0.042\pm 0.049$	$+0.762\pm 0.034$
Conv 4	$+0.048\pm 0.045$	$-0.242\pm 0.081$	$+0.991\pm 0.004$	$+0.024\pm 0.019$	$+0.454\pm 0.045$	$+0.676\pm 0.013$

Table 1: Layer-wise Spearman correlations between local discretization criteria and realized freeze effects. Values are mean

\pm

sample standard deviation across five seeds, after averaging checkpoints within each seed (four for Compact-S and five for TreeLogicNet-S).

Figure 3: Locality of first-order interventions. (a) Score selection produces substantially larger activation displacements than argmax, with median squared displacements of

0.42

and

2.9\times 10^{-6}

, respectively. (b) Approximation error increases with displacement for both first- and second-order scores, with median within-neuron Spearman correlations of

0.88

and

0.80

, respectively; second order substantially reduces the typical error. (c) The apparent advantage of selecting low-displacement task-aware interventions disappears when gate selection and evaluation use disjoint validation data.

Panel of Figure 4 — Figure 4: Progressive freezing dynamics on Compact-S. (a) Fraction of frozen neurons throughout training for different discretization criteria. (b) Layer-wise frozen fractions show that TaskFreeze progressively commits individual decisions, whereas EntropyFreeze induces a markedly different layer-dependent schedule.

	Compact-S		Compact-M	TreeLogicNet-S	TreeLogicNet-M
Method	Resampling	Stopped	Resampling
TaskFreeze (ours)	$66.25\pm 0.27$	$65.93\pm 0.15$	$69.34\pm 0.46$	$60.76\pm 0.46$	$71.31\pm 0.04$
EntropyFreeze (ours)	$64.11\pm 0.53$	$63.88\pm 0.76$	$68.47\pm 0.54$	$59.97\pm 0.84$	$70.95\pm 0.23$
GlobalFreeze (ours)	$66.41\pm 0.21$	$66.12\pm 0.37$	$69.26\pm 0.21$	$60.50\pm 0.52$	$71.15\pm 0.04$
LayerEntropyFreeze	$64.10\pm 0.45$	$65.99\pm 0.17$	$67.67\pm 0.93$	$60.63\pm 0.55$	$71.44\pm 0.01$
FinalArgmax	$64.63\pm 0.61$	$64.63\pm 0.61$	$67.67\pm 0.39$	$59.29\pm 0.78$	$70.49\pm 0.27$

Table 2: Final discrete CIFAR-10 test accuracy (%). Results are reported at the 400k discretization start for Compact models and the 25k start for TreeLogicNet models. For Compact-S, we compare continued and stopped connection resampling after discretization begins. Values are mean

\pm

sample standard deviation across available seeds.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Operation	Stride	Compact-S ( $k=128$ )	Compact-M ( $k=256$ )
Input	Thermometer encoding	–	$(9,32,32)$	$(21,32,32)$
Conv 1	$3\times 3$ logic conv.	$2$	$(128,16,16)$	$(256,16,16)$
Conv 2	$3\times 3$ logic conv.	$1$	$(128,16,16)$	$(256,16,16)$
Conv 3	$3\times 3$ logic conv.	$2$	$(512,8,8)$	$(1024,8,8)$
Conv 4	$3\times 3$ logic conv.	$1$	$(512,8,8)$	$(1024,8,8)$
Flatten	–	–	$32{,}768$	$65{,}536$

Appendix

Table 3: Compact CIFAR-10 architectures. Compact-S uses

k=128

and

N=3

input thresholds; Compact-M uses

k=256

and

N=7

. Convolutional layers use

3\times 3

receptive fields and padding

1

Stage	Operation	TreeLogicNet-S ( $k=32$ )	TreeLogicNet-M ( $k=256$ )
Input	Three-threshold encoding	$(9,32,32)$	$(9,32,32)$
Conv 1	$3\times 3$ , tree depth $3$	$32$ channels	$256$ channels
Pool 1	$2\times 2$ OR, stride $2$	$(32,16,16)$	$(256,16,16)$
Conv 2	$3\times 3$ , tree depth $3$	$128$ channels	$1024$ channels
Pool 2	$2\times 2$ OR, stride $2$	$(128,8,8)$	$(1024,8,8)$
Conv 3	$3\times 3$ , tree depth $3$	$512$ channels	$4096$ channels

Appendix

Table 4: TreeLogicNet CIFAR-10 architectures. Each convolutional kernel is a depth-

3

Boolean tree with seven trainable gates, followed by fixed

2\times 2

OR pooling. S and M correspond to

k=32

and

k=256

, respectively.

Parameter	Compact	TreeLogicNet
Optimizer	Adam	AdamW
Weight decay	$0$	$0.002$
Training steps	$600$ k	$200$ k
Batch size	$128$	$128$
Learning rate	$0.02$	$0.02$
LR schedule	Constant	Constant

Appendix

Table 5: Training and freezing hyperparameters.

H

denotes entropy convergence and

S

first-order freeze shock. Values are shared across the Compact and TreeLogicNet experiments unless shown otherwise.

Figure 5: Predicting freeze effects on TreeLogicNet-S. Measured loss change from freezing to the argmax gate versus gate entropy (left), from freezing to the task-aware gate $k_{i}^{*}=\argmin_{k}S_{i}(k)$ versus its first-order score (center), and from freezing to the argmax gate versus its first-order score (right). As for Compact-S, score-based selection is less reliably aligned with the realized effect, whereas the argmax freeze shock is strongly predictive.

Panel of Figure 6 — Figure 6: Progressive freezing dynamics on TreeLogicNet-S. (a) Fraction of frozen neurons throughout training for different discretization criteria. (b) Layer-wise frozen fractions show that TaskFreeze progressively commits individual decisions, whereas EntropyFreeze induces a markedly different layer-dependent schedule.

Method	200k	300k	400k	200k	300k	400k
	Continued resampling			Stopped resampling
TaskFreeze (ours)	$66.13\pm 0.20$	$66.52\pm 0.17$	$66.25\pm 0.27$	$65.55\pm 0.29$	$65.77\pm 0.15$	$65.93\pm 0.15$
EntropyFreeze (ours)	$64.30\pm 0.83$	$64.32\pm 0.72$	$64.11\pm 0.53$	$64.28\pm 0.48$	$64.00\pm 0.58$	$63.88\pm 0.76$
GlobalFreeze (ours)	$66.27\pm 0.51$	$66.17\pm 0.10$	$66.41\pm 0.21$	$65.84\pm 0.15$	$66.04\pm 0.26$	$66.12\pm 0.37$
LayerEntropyFreeze	$64.75\pm 0.80$	$65.24\pm 0.53$	$64.10\pm 0.45$	$65.94\pm 0.14$	$65.95\pm 0.26$	$65.99\pm 0.17$
FinalArgmax	$64.63\pm 0.61$	$64.63\pm 0.61$	$64.63\pm 0.61$	$64.63\pm 0.61$	$64.63\pm 0.61$	$64.63\pm 0.61$

Appendix

Table 6: Sensitivity to discretization start on Compact-S. Final discrete CIFAR-10 test accuracy (%) for discretization starting at 200k, 300k, or 400k training steps, with connection resampling either continued or stopped after discretization begins. Values are mean

\pm

sample standard deviation across available seeds.

Method	10k	25k	50k
TaskFreeze (ours)	$60.42\pm 0.63$	$60.76\pm 0.46$	$60.26\pm 0.80$
EntropyFreeze (ours)	$59.89\pm 0.45$	$59.97\pm 0.84$	$60.13\pm 0.57$
GlobalFreeze (ours)	$60.05\pm 0.82$	$60.50\pm 0.52$	$60.78\pm 0.28$
LayerEntropyFreeze	$60.72\pm 0.57$	$60.63\pm 0.55$	$60.81\pm 0.42$
FinalArgmax	$59.29\pm 0.78$	$59.29\pm 0.78$	$59.29\pm 0.78$
CAGE	$60.13\pm 0.49$	$59.98\pm 0.35$	$59.90\pm 0.71$

Appendix

Table 7: Sensitivity to discretization start on TreeLogicNet-S. Final discrete CIFAR-10 test accuracy (%) for discretization starting at 10k, 25k, or 50k training steps. Values are mean

\pm

sample standard deviation across five seeds.

Explore similar work

Jul 10, 2026cs.LG

Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks

We introduce a novel method for both partial and full optimization of the connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs). Our training method utilizes a probability distribution over a set of connections per gate/lookup table (LUT) input pin, selecting the connection with highest merit, all whilst the optimal gate types or LUT-entries are learned in parallel. We show that the connection-optimized LGNs outperform standard fixed-connection LGNs on the Yin-Yang, MNIST Handwritten Digits and Fashion-MNIST benchmarks, while requiring only a fraction of the number of logic gates. We achieve 98.92% on the MNIST dataset with two layers of 8000 gates. With only one layer of 8000 gates, we obtain 98.45%, showing that our method requires almost 50 times fewer gates compared to fixed-connection LGNs. Training stability up to ten layers has been ensured by employing a high learning rate, straight-through estimators and trimming constant-output gate types. Additionally, we present a LUT neuron description that enables stable training with backpropagation, tested up to 6-layer deep networks. The model requires four times fewer trainable parameters and still achieves a higher accuracy compared to the fixed-connection LGN training algorithm. Our connection-training algorithm also works well for the LUTNs, achieving an accuracy of 98.88% for two layers of 2000 6-input LUTs.

Wout Mommen, Lars Keuninckx, Matthias Hartmann +2

imec, Kapeldreef 75, Leuven, 3001, , Belgium · Vrije Universiteit Brussel, Peinlaan 2, Elsene, 1050, , Belgium

Differentiable Logic Gate Networks Mnist Discontinuous Gating

Ask PDF

Jul 19, 2026cs.LG

On the Depth Scalability of Logic Gate Networks

Logic Gate Networks (LGNs) compute through compositions of Boolean operations, yet existing LGNs do not reliably benefit from increased depth. We identify two causes: optimization collapse and topology-induced degradation of output-specific credit that persists even after skip-biased initialization and straight-through estimation stabilize training. We introduce Input-Anchored Logic Gate Networks (IALGNs), in which each gate combines a private hidden spine with a direct input anchor. This topology prevents output-path merging while retaining input access at every layer. Credit diagnostics show that random wiring dilutes or conflicts output-specific gradients, whereas IALGN maintains usable and coherent credit. Random-

k_x

relaxation improves anchor selection without relaxing the spine. Across MNIST, CIFAR-10, and CIFAR-100, IALGN exhibits consistent fixed-width depth--accuracy scaling up to 150 layers, while alternative topologies saturate or degrade. Linear probes, topology ablations, and operation-aware analysis show that trained IALGNs preserve private states and apply sparse anchor-conditioned updates. These results indicate that scalable LGN depth requires both stable optimization and credit-preserving information access.

Taegun An, Dohun kim, Haebeom Lee +1

Jung Woonoh IT & General Education Center, Korea University 145, Anam-ro, Seongbuk-gu, Seoul 02841, Republic of Korea

Differentiable Logic Gate Networks Scalability Step-Level Credit Degradation

Ask PDF

Apr 21, 2026cs.CV

Silicon Aware Neural Networks

Recent work in the machine learning literature has demonstrated that deep learning can train neural networks made of discrete logic gate functions to perform simple image classification tasks at very high speeds on CPU, GPU and FPGA platforms. By virtue of being formed by discrete logic gates, these Differentiable Logic Gate Networks (DLGNs) lend themselves naturally to implementation in custom silicon - in this work we present a method to map DLGNs in a one-to-one fashion to a digital CMOS standard cell library by converting the trained model to a gate-level netlist. We also propose a novel loss function whereby the DLGN can optimize the area, and indirectly power consumption, of the resulting circuit by minimizing the expected area per neuron based on the area of the standard cells in the target standard cell library. Finally, we also show for the first time an implementation of a DLGN as a silicon circuit in simulation, performing layout of a DLGN in the SkyWater 130nm process as a custom hard macro using a Cadence standard cell library and performing post-layout power analysis. We find that our custom macro can perform classification on MNIST with 97% accuracy 41.8 million times a second at a power consumption of 83.88 mW.

Sebastian Fieldhouse, Kea-Tiong Tang

College of Semiconductor Research National Tsing Hua University · Department of Electrical Engineering National Tsing Hua University

Differentiable Logic Gate Networks Field-Programmable Gate Arrays Printed Circuit Board Neural Network Deep Learning

Ask PDF