In machine learning, symbolic reasoning approaches often rely on static architectures that hinder integration into deep neural networks and gradient-based optimization. To address this, we introduce the Native Predicate Neuron, a differentiable logic-reasoning neural module that enriches standard architectures with symbolic reasoning capabilities via first-order logic quantization. The proposed neuron models relational logic by learning to alternate between operations using a single trainable parameter. It features a tailored gradient mechanism with a dual-nature formulation: executing hard logical operations during training while maintaining differentiability for backpropagation. By quantizing both its input and choice state, the Predicate Neuron approximates continuous and non-continuous functions, effectively resolving discontinuities that standard deep learning architectures struggle with. We evaluate the Predicate Neuron across Fully-Connected, Convolutional, Recurrent, and Transformer architectures on benchmarks like MNIST and Wikitext. Results show that incorporating Predicate Neurons accelerates convergence, boosts accuracy, and lowers loss in general, with particularly strong improvements in tasks requiring monotonic reasoning or handling discontinuities. Overall, the Predicate Neuron natively integrates into standard neural networks to enable fuzzy logic operations and is available in Keras and PyTorch.
Figures & tables
Figure 1. Heatmaps of the three logical operations, namely AND (left), OR (middle), and XOR (right).
Figure 2. A visual representation on how the Predicate Neuron switches between the three logical operations based on the value of the temperature hyperparameter.
Figure 3. Validation Loss of the Predicate Neuron (orange graph) and the Dense network (blue graph).
Figure 4. Validation loss of the proposed Logic Network with Predicate Layer (green graph), a Dense network with 441 trainable parameters (orange graph), and a Dense network with 5k parameters (blue graph). The bottom figure depicts the same graph as the top one, but zoomed in.
Model
Test Loss
Logic Network
9.8694∗10−7±∼1∗10−8
LTN
4.5123∗10−2±2.7∗10−4
LGN
2.6650∗10−6±∼1∗10−8
Table 1. Comparison between LTN, LGN and a Logic Network on the 20 -atom problem.
Figure 5. Validation loss of the proposed Logic Network with Predicate Layer (orange graph), a Dense network with 5k parameters (blue graph), an LTN network (purple graph) and an LGN network (green graph). The bottom figure depicts the same graph as the top one, but zoomed in.
Figure 6. The validation losses across 25000 epochs for: i) a Dense network without Logic Core (green), ii) a Dense network with Logic Core (purple), iii) a Dense model with Logic Core and ReLU activation in the second layer of its Logic Core (blue), and iv) the Logic Network incorporating Predicate Neurons in their Logic Core (orange).
Figure 7. Training loss of the Logic Network across different learning rate values. Figures (c) and (d) are the same as (a) and (b), but also depict one extra experiment (purple graph). The training loss is illustrated per-step, since usually the network converges within one epoch.
Model
Parameters
Training Time (Secs)
Test Loss
Test Accuracy
Logic CNN Network
241549
141.1219
0.03280±0.00053
0.9930±0.00119
Baseline CNN
225034
143.5178
0.03651±0.00061
0.9929±0.00130
Table 2. Results on MNIST dataset.
Model
Parameters
Training Time (Secs)
Test Loss
Test Accuracy
Logic RNN Network
84427
25151.5
0.3578±0.0057
0.9115±0.0046
Baseline RNN
67850
26119.5
0.3777±0.0071
0.9043±0.0053
Table 3. Results on Permuted MNIST.
Figure 8. Validation loss and accuracy of the RNN network with Predicate Layer (blue graph), and a Baseline RNN without a Logic Core (orange graph).
Figure 9. The architectures of the Baseline and Logic Transformer