Learning Probabilistic Logic Programs with Functional Gradient Guided Language Models

Saurabh Mathur; Sahil Sidheekh; Bhavan Vasu; Farbod Tavakkoli; Prasad Tadepalli; Kristian Kersting; Sriraam Natarajan

cs.AIOct 8, 2026

Learning Probabilistic Logic Programs with Functional Gradient Guided Language Models

Authors: Saurabh Mathur, Sahil Sidheekh, Bhavan Vasu, Farbod Tavakkoli, Prasad Tadepalli, Kristian Kersting, Sriraam Natarajan

Organizations: Technische Universität Darmstadt · The University of Texas at Dallas · AT&T CDO · Oregon state University

Abstract

Declarative logic programs offer a powerful and interpretable abstraction for encoding relational structure and neurosymbolic reasoning, by expressing dependencies as weighted compositional rules. However, inducing them from data remains fundamentally hard, bottlenecked by the combinatorial explosion of symbolic search spaces. LLMs have recently emerged as powerful hypothesis generators, but when used in isolation, they lack the capacity to do systematic inductive reasoning needed to reliably synthesize valid programs that fit complex relational distributions. We introduce grasp (Gradient-boosted Synthesis of Probabilistic logic programs), a neurosymbolic framework that casts relational structure learning as functional gradient boosting in which the weak learner is a first-order rule and the intractable inner search is delegated to an LLM proposal oracle. We evaluate grasp on four relational benchmarks spanning molecular toxicity prediction (Tox21), mutagenesis, and citation matching (Cora), and show that it improves over purely symbolic, neural, and LLM-based baselines, while producing interpretable weighted rule ensembles. By replacing combinatorial search with gradient-guided LLM hypothesis generation, grasp retains boosting guarantees without sacrificing the transparency of symbolic outputs.

Figures & tables

Figure 1: grasp Framework. grasp frames relational learning as a neurosymbolic functional gradient boosting problem. At each iteration $t$ , an LLM receives an example-centered relational context and proposes candidate symbolic rules. A logic solver evaluates each candidate against the relational data, assessing coverage and residual fit. The context updater computes pseudo-residuals $r_{i}=y_{i}-\sigma(F_{t-1}(x_{i}))$ and selects high-residual examples to steer the LLM toward patterns not yet explained by the ensemble. The accepted rule $\varphi_{t}$ is added with step size $\eta$ , yielding $F_{t}(x)=F_{t-1}(x)+\eta\varphi_{t}(x)$ . After $T$ iterations, grasp returns a weighted ensemble of interpretable probabilistic logic rules. This design replaces combinatorial symbolic search with gradient-guided LLM hypothesis generation, while retaining boosting convergence guarantees.

Citation and Mutagenesis domains
Dataset	BoostedRDN	NeuralLP	Grasp -Gem	Grasp -Cdx	One-shot
cora	$95.02{\pm}1.96$	$91.86{\pm}0.00$	$92.99{\pm}0.51$	$97.67{\pm}0.26$	$74.78{\pm}21.57$
mutagen	$82.92{\pm}2.37$	$79.08{\pm}2.44$	$88.36{\pm}1.72$	$88.41{\pm}1.82$	$82.72{\pm}2.85$
Tox21 domains
nr_ahr	$81.98{\pm}0.47$	$68.80{\pm}0.07$	$76.11{\pm}1.45$	$81.41{\pm}0.65$	$63.81{\pm}7.73$
nr_ar	$75.84{\pm}1.36$	$78.82{\pm}0.19$	$74.81{\pm}1.71$	$78.11{\pm}3.04$	$61.73{\pm}10.80$

Table 1: Predictive performance in terms of AUC-ROC across citation, mutagenesis, and Tox21 domains. Results are reported as percentages, with mean

\pm

std over three trials. The best completed result for each dataset is shown in blue. Overall, GRASP variants are competitive with or outperform classical relational baselines across most domains, while the one-shot LLM baseline is less stable. “–” denotes no usable result.

Citation and Mutagenesis domains
Dataset	BoostedRDN	NeuralLP	Grasp -Gem	Grasp -Cdx	One-shot
cora	$96.10{\pm}1.57$	$94.15{\pm}0.00$	$94.36{\pm}0.37$	$98.23{\pm}0.00$	$78.89{\pm}17.77$
mutagen	$86.53{\pm}1.47$	$89.53{\pm}1.48$	$94.36{\pm}0.65$	$93.65{\pm}0.77$	$90.23{\pm}2.51$
Tox21 domains
nr_ahr	$38.17{\pm}1.57$	$18.43{\pm}0.35$	$32.37{\pm}0.72$	$34.49{\pm}1.02$	$21.32{\pm}5.72$
nr_ar	$41.99{\pm}2.41$	$13.32{\pm}0.02$	$38.77{\pm}3.02$	$41.99{\pm}1.93$	$13.79{\pm}11.05$

Table 2: Predictive performance in terms of AUC-PR across citation, mutagenesis, and Tox21 domains. Results are reported as percentages, with mean

\pm

std over three trials. Since AUC-PR is more sensitive to class imbalance, these results highlight the ability of each method to recover positive examples in sparse relational prediction settings. The best result for each dataset is shown in blue bold. “–” denotes no usable result.

	Method	Top rule
mutagen	BoostedRDN	active(A) :- ind1(A,1).
	One-shot	active(A) :- lumo(A,B), B<-2.0, ind1(A,1.0)
	NeuralLP	active(A) :- bond(A,V0,E0,E1), bond(V0,V1,E2,E3)
	Grasp	active(A) :- num_atoms(A,N), N>25, lumo(A,L), L<-1.1 Exp: Large, low-LUMO molecules are mutagenic (size $+$ electronics).
nr_ar	BoostedRDN	nr_ar(A) :- fg(A,,ketone), fg(A,,allylic_oxidation_site).
	One-shot	nr_ar(A) :- fg(A,B,bicyclic), fg(A,C,allylic_oxidation_site)

Table 3: Representative top-ranked rule per method on three targets (baselines first, Grasp last and highlighted with a plain-language reading). Grasp invents high-level predicates ( mol_num_rings , mol_count_fg ), uses numeric thresholds, and recovers chemically meaningful patterns compared to other methods. Top rule per method ( Grasp :

|w_{1}-w_{0}|

; BoostedRDN : leaf value).

Figure 2: Evaluation of rule importance and ensemble efficiency via insertion (top) and deletion (bottom) curves, ordered by absolute discriminative weight. Gray dashed, and dotted lines denote random and reverse-ordering ablations for grasp (Codex). grasp models exhibit steep insertion asymptotes and rapid deletion degradation (outperforming baselines), confirming accurate weight calibration and demonstrating that a highly compact subset of rules captures the majority of predictive variance. Initial deletion robustness in grasp (Codex) reflects semantic redundancy among top-weighted rules.

Explore similar work

ANDRE: An Attention-based Neuro-symbolic Differentiable Rule Extractor for Inductive Logic Programming
May 5, 2026Iman Sharifi, Peng Wei, Saber FallahLogic Programming Differentiable Programming
Dept. of Mechanical and Aerospace Engineering, George Washington University, USA · Dept. of Mechanical Engineering Sciences, University of Surrey, UK
RLIE: Rule Generation with Logistic Regression, Iterative Refinement, and Evaluation for Large Language Models
Oct 22, 2025Yang Yang, Hua XU, Zhangyi Hu +1Logistic Regression Neuro-Symbolic Reasoning
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511400, China · Institute of Deep Perception Technology, JITRI, Wuxi 214000, China
ReaComp: Compiling LLM Reasoning into Symbolic Solvers for Efficient Program Synthesis
May 6, 2026Atharva Naik, Yash Mathur, Prakam +2Symbolic Reasoning LLM-Based Program Synthesis
Carnegie Mellon University · Independent Researcher

May 5, 2026, cs.AIJ/K move · Enter open · S save

ANDRE: An Attention-based Neuro-symbolic Differentiable Rule Extractor for Inductive Logic Programming

Iman Sharifi, Peng Wei, Saber Fallah

Dept. of Mechanical and Aerospace Engineering, George Washington University, USA · Dept. of Mechanical Engineering Sciences, University of Surrey, UK

Logic Programming Differentiable Programming Neuro-Symbolic Reasoning Inductive Logic Programming

Open paper AskPDF arXiv

Inductive Logic Programming (ILP) aims to learn interpretable first-order rules from data, but existing symbolic and neuro-symbolic approaches struggle to scale to noisy and probabilistic settings. Classical ILP relies on discrete combinatorial rule search and is brittle under uncertainty, while differentiable ILP methods typically depend on predefined rule templates or inaccurate fuzzy operators that suffer from vanishing gradients or poor approximation of logical structure when reasoning over probabilistic predicate valuations. This paper proposes an Attention-based Neuro-symbolic Differentiable Rule Extractor (ANDRE), a novel ILP framework that learns first-order logic programs by optimizing over a continuous rule space with attention-based logical operators. ANDRE replaces both rule templates and logical operators with fully differentiable, attention-driven conjunction and disjunction operators that approximate logical min-max semantics, enabling accurate, stable, and interpretable reasoning over probabilistic data. By softly selecting, negating, or excluding predicates within each rule, ANDRE supports flexible rule induction while preserving symbolic structure. Extensive experiments on classical ILP benchmarks, large-scale knowledge bases, and synthetic datasets with probabilistic predicates and noisy supervision demonstrate that ANDRE achieves competitive or superior predictive performance while reliably recovering correct symbolic rules under uncertainty. In particular, ANDRE remains robust to moderate label noise, substantially outperforming existing differentiable ILP methods in both rule extraction quality and stability.

Abstract

Explore similar work

ANDRE: An Attention-based Neuro-symbolic Differentiable Rule Extractor for Inductive Logic Programming

RLIE: Rule Generation with Logistic Regression, Iterative Refinement, and Evaluation for Large Language Models

ReaComp: Compiling LLM Reasoning into Symbolic Solvers for Efficient Program Synthesis