cs.LGOct 27, 2026

Predicate Neuron: Differentiable Quantization of Predicate Relations in Gradient-based Learning

Authors: Ioannis Konstantoulas, Iliana Loi, Kyriakos Sgarbas

Organizations: Department of Electrical and Computer Engineering, University of Patras, Patras, Greece

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In machine learning, symbolic reasoning approaches often rely on static architectures that hinder integration into deep neural networks and gradient-based optimization. To address this, we introduce the Native Predicate Neuron, a differentiable logic-reasoning neural module that enriches standard architectures with symbolic reasoning capabilities via first-order logic quantization. The proposed neuron models relational logic by learning to alternate between operations using a single trainable parameter. It features a tailored gradient mechanism with a dual-nature formulation: executing hard logical operations during training while maintaining differentiability for backpropagation. By quantizing both its input and choice state, the Predicate Neuron approximates continuous and non-continuous functions, effectively resolving discontinuities that standard deep learning architectures struggle with. We evaluate the Predicate Neuron across Fully-Connected, Convolutional, Recurrent, and Transformer architectures on benchmarks like MNIST and Wikitext. Results show that incorporating Predicate Neurons accelerates convergence, boosts accuracy, and lowers loss in general, with particularly strong improvements in tasks requiring monotonic reasoning or handling discontinuities. Overall, the Predicate Neuron natively integrates into standard neural networks to enable fuzzy logic operations and is available in Keras and PyTorch.

Figures & tables

Figure 1. Heatmaps of the three logical operations, namely AND (left), OR (middle), and XOR (right).
Figure 2. A visual representation on how the Predicate Neuron switches between the three logical operations based on the value of the temperature hyperparameter.
Figure 3. Validation Loss of the Predicate Neuron (orange graph) and the Dense network (blue graph).
Figure 4. Validation loss of the proposed Logic Network with Predicate Layer (green graph), a Dense network with 441441 trainable parameters (orange graph), and a Dense network with 5k5k parameters (blue graph). The bottom figure depicts the same graph as the top one, but zoomed in.
ModelTest Loss
Logic Network9.8694∗10−7±∼1∗10−89.8694*10^{-7}\pm\sim 1*10^{-8}
LTN4.5123∗10−2±2.7∗10−44.5123*10^{-2}\pm 2.7*10^{-4}
LGN2.6650∗10−6±∼1∗10−82.6650*10^{-6}\pm\sim 1*10^{-8}
Table 1. Comparison between LTN, LGN and a Logic Network on the 2020 -atom problem.
Figure 5. Validation loss of the proposed Logic Network with Predicate Layer (orange graph), a Dense network with 5k5k parameters (blue graph), an LTN network (purple graph) and an LGN network (green graph). The bottom figure depicts the same graph as the top one, but zoomed in.
Figure 6. The validation losses across 2500025000 epochs for: i) a Dense network without Logic Core (green), ii) a Dense network with Logic Core (purple), iii) a Dense model with Logic Core and ReLU activation in the second layer of its Logic Core (blue), and iv) the Logic Network incorporating Predicate Neurons in their Logic Core (orange).
Figure 7. Training loss of the Logic Network across different learning rate values. Figures (c) and (d) are the same as (a) and (b), but also depict one extra experiment (purple graph). The training loss is illustrated per-step, since usually the network converges within one epoch.
ModelParametersTraining Time (Secs)Test LossTest Accuracy
Logic CNN Network241549241549141.1219141.12190.03280±0.000530.03280\pm 0.000530.9930±0.001190.9930\pm 0.00119
Baseline CNN225034225034143.5178143.51780.03651±0.000610.03651\pm 0.000610.9929±0.001300.9929\pm 0.00130
Table 2. Results on MNIST dataset.
ModelParametersTraining Time (Secs)Test LossTest Accuracy
Logic RNN Network844278442725151.525151.50.3578±0.00570.3578\pm 0.00570.9115±0.00460.9115\pm 0.0046
Baseline RNN678506785026119.526119.50.3777±0.00710.3777\pm 0.00710.9043±0.00530.9043\pm 0.0053
Table 3. Results on Permuted MNIST.
Figure 8. Validation loss and accuracy of the RNN network with Predicate Layer (blue graph), and a Baseline RNN without a Logic Core (orange graph).
Figure 9. The architectures of the Baseline and Logic Transformer
ModelParametersTraining Time (Secs)LossAccuracy
Logic Transformer Network2.917.7762.917.776313431343.8187±0.03443.8187\pm 0.03440.5335±0.00360.5335\pm 0.0036
Baseline Transformer2.834.9602.834.960247924793.8685±0.08893.8685\pm 0.08890.5261±0.00580.5261\pm 0.0058
Table 4. Results on Wikitext-103.
More in this topic

Differentiable Logic

See all 19 papers

Explore similar work

CardsList
Jul 23, 2026

Differentiable Logic Programming to Mitigate Reasoning Shortcuts in Neurosymbolic Systems

Neurosymbolic (NeSy) systems integrate neural networks with logical reasoning to achieve both generalization and interpretability, but recent work has shown they are susceptible to shortcut reasoning behaviors. We propose a novel method using matrix-based differentiable logic programming to mitigate reasoning shortcuts in two phenomena: constraint satisfaction shortcuts, where constraints are satisfied without achieving the intended task, and cognition shortcuts, where biased data leads to semantically incorrect concept mappings despite logically sound inference. Building on recent matrix-based logic programming semantics, we introduce design elements to mitigate shortcuts, including a unified encoding of rules and constraints in a single matrix. We also identify connections to fuzzy logic t-norms and empirically compare their gradient flow properties. Through carefully designed experiments on MNIST variants, we show that one-to-one grounding of neural outputs to logical atoms significantly reduces both shortcut types compared to previous methods that rely on soft probability distributions. We then confirm that architectural choices in coupling symbolic knowledge with neural learning play a critical role in shortcut mitigation.

Akihiro Takemura, Katsumi Inoue

National Institute of Informatics, Tokyo, Japan

Differentiable LogicDifferentiable ProgrammingNeuro-Symbolic Reasoning
AskPDF
May 13, 2026

Quantitative Linear Logic for Neuro-Symbolic Learning and Verification

Differentiable Logics are deployed in neuro-symbolic learning tasks as a way of embedding logical constraints in the training objective of neural networks. A differentiable logic consists of a syntax to write logical properties and a semantics to interpret them as real-valued functions to be folded in the loss function. A defining trade-off of the field is that between logical properties of the connectives, and analytic concerns for the semantics, with both aspects being relevant in applications. At one extreme we find fuzzy logics, that have well-established algebraic and proof-theoretic foundations, and at the other ad-hoc differentiable logics like Fischer's DL2, conceived for deep learning applications. However, no satisfactory foundation has emerged yet. We propose a resolution to this long-standing tension via a novel logic, Quantitative Linear Logic (QLL), with foundational ambitions. Our design is driven by naturality -- the idea that, since logical constraints are translated to losses, the semantics of the connectives should be pertinent operations used in ML practice (that is, sum and log-sum-exp) on additive quantities (like logits). We then judge the result on two aspects: logical adequacy -- that they satisfy most of the standard logical laws of Linear Logic; and empirical effectiveness -- test-time performance (as measured by adversarial attacks) is well-correlated to the actual verification of the logical constraints (as measured by off-the-shelf neural network verifiers), which makes QLL stand out among SoTA techniques.

Thomas Flinkow, Ekaterina Komendantskaya, Matteo Capucci +1

Maynooth University, Maynooth, IE · University of Southampton, Southampton, UK · University of Strathclyde, Glasgow, UK +1

Neural Network VerificationDifferentiable LogicNeuro-Symbolic ReasoningNeurosymbolic Learning
AskPDF
Mar 4, 2026

Continuous Modal Logical Neural Networks: Modal Reasoning via Stochastic Accessibility

We propose Fluid Logic, a paradigm in which modal logical reasoning, temporal, epistemic, doxastic, deontic, is lifted from discrete Kripke structures to continuous manifolds via Neural Stochastic Differential Equations (Neural SDEs). Each type of modal operator is backed by a dedicated Neural SDE, and nested formulas compose these SDEs in a single differentiable graph. A key instantiation is Logic-Informed Neural Networks (LINNs): analogous to Physics-Informed Neural Networks (PINNs), LINNs embed modal logical formulas such as (□\Box bounded) and (◊\Diamond visits_lobe) directly into the training loss, guiding neural networks to produce solutions that are structurally consistent with prescribed logical properties, without requiring knowledge of the governing equations. The resulting framework, Continuous Modal Logical Neural Networks (CMLNNs), yields several key properties: (i) stochastic diffusion prevents quantifier collapse (□\Box and ◊\Diamond differ), unlike deterministic ODEs; (ii) modal operators are entropic risk measures, sound with respect to risk-based semantics with explicit Monte Carlo concentration guarantees; (iii)SDE-induced accessibility provides structural correspondence with classical modal axioms; (iv) parameterizing accessibility through dynamics reduces memory from quadratic in world count to linear in parameters. Three case studies demonstrate that Fluid Logic and LINNs can guide neural networks to produce consistent solutions across diverse domains: epistemic/doxastic logic (multi-robot hallucination detection), temporal logic (recovering the Lorenz attractor geometry from logical constraints alone), and deontic logic (learning safe confinement dynamics from a logical specification).

Antonin Sulc

Lawrence Berkeley National Laboratory, Berkeley, CA, USA

Logical ReasoningNeuro-Symbolic ReasoningNeural Algorithmic ReasoningEpistemic Logic
AskPDF