Interpretable Relational Inference with LLM-Guided Symbolic Dynamics Modeling

Xiaoxiao Liang; Juyuan Zhang; Liming Pan; Linyuan Lü

cs.LGApr 14, 2026

Interpretable Relational Inference with LLM-Guided Symbolic Dynamics Modeling

Authors: Xiaoxiao Liang, Juyuan Zhang, Liming Pan, Linyuan Lü

Organizations: School of Cyber Science and Technology, University of Science and Technology of China, Hefei, Anhui, China.

Abstract

Inferring latent interaction structures from observed dynamics is a fundamental inverse problem in many-body interacting systems. Most neural approaches rely on black-box surrogates over trainable graphs, achieving accuracy at the expense of mechanistic interpretability. Symbolic regression offers explicit dynamical equations and stronger inductive biases, but typically assumes known topology and a fixed function library. We propose COSINE (Co-Optimization of Symbolic Interactions and Network Edges), a differentiable framework that jointly discovers interaction graphs and sparse symbolic dynamics. To overcome the limitations of fixed symbolic libraries, COSINE further incorporates an outer-loop large language model that adaptively prunes and expands the hypothesis space using feedback from the inner optimization loop. Experiments on synthetic systems and large-scale real-world epidemic data demonstrate robust structural recovery and compact, mechanism-aligned dynamical expressions. Code: https://anonymous.4open.science/r/COSINE-6D43.

Figures & tables

Figure 1: Conceptual comparison of relational inference, symbolic regression, and COSINE. Relational inference recovers latent interaction graphs from dynamical data; symbolic regression discovers governing equations of dynamics given known structures; COSINE integrates sparse symbolic modeling with graph learning to jointly infer both structure and dynamics.

Figure 2: The COSINE architecture. (Left) LLM-based Reasoning refines the basis library $\Theta(\cdot)$ via performance feedback. (Middle) Graph-Guided Modeling co-optimizes latent edges $A_{ij}$ and coefficients $\mathbf{W}$ through differentiable symbolic message-passing. (Right) Symbolic Basis Library bridges high-level reasoning with numerical discovery of governing mechanisms.

Table 3

Figure 3: Results on COVID-19 epidemic data. Left: Mobility graph and inferred graph. Right: Learned weights of dominant terms in the Message and Update module across four U.S. states.

Figure 4: Abalation of LLM-guided evolution. COSINE vs. fixed library (w/o update) and threshold-based pruning (

\lambda

Figure 5: Hyperparameter sensitivity analysis of COSINE for Spring (a–b) and Kuramoto (c–d). (a, c) $\text{lr}_{a}$ vs. lr ; (b, d) $\beta_{\mathrm{KL}}$ vs. $\lambda_{W}$ . Performance: relational inference AUC.

Model	Params	Spr	Kura	MM	CMN
DeepSeek-R1	8B	1.00	1.00	0.53	0.51
Qwen3	8B	1.00	1.00	0.54	0.66
DeepSeek-R1	14B	1.00	0.99	0.88	1.00
GPT-OSS	20B	1.00	0.99	1.00	1.00
DeepSeek-R1	32B	1.00	1.00	0.94	1.00
Qwen3	32B	1.00	0.83	0.99	0.90

Table 3: Performance (AUC) of COSINE with different LLMs. Models are ordered by increasing parameter scale.

Figure 6: Computational efficiency and memory scaling comparison between COSINE and NRI (batch size 64, hidden dimension 256) across different network sizes

N\in\{5,10,50,100,200\}

. (a) Average epoch time, (b) Peak GPU memory usage. Note the logarithmic scale on the vertical axes.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Visual illustration of Michaelis–Menten kinetics. a , Node activities (with the trajectories of the first 5 nodes highlighted by colored lines). b , Adjacent matrix of the network.

Figure 8: Visual illustration of Diffusion dynamics. a , Node activities (with the trajectories of the first 5 nodes highlighted by colored lines). b , Adjacent matrix of the network.

Figure 9: Visual illustration of Spring dynamics. a , Node activities (with the trajectories of the first 5 nodes highlighted by colored lines). b , Adjacent matrix of the network.

Figure 10: Visual illustration of Kuramoto model. a , Node activities (with the trajectories of the first 5 nodes highlighted by colored lines). b , Adjacent matrix of the network. c , Natural frequencies $\omega_{i}$ of each node.

Figure 11: Visual illustration of Friedkin–Johnsen dynamics. a , Node activities (with the trajectories of the first 5 nodes highlighted by colored lines). b , Adjacent matrix of the network. c , Innate opinions $s_{i}$ of each node.

Figure 12: Visual illustration of Coupled map network. a , Node activities (with the trajectories of the first 5 nodes highlighted by colored lines). b , Adjacent matrix of the network.

Figure 13: Evolution of discovered basis terms for MM and CMN systems. Panels (a, b) show Message term weights, while (c, d) show Update term weights for the first (blue) and best (red) rounds.

Figure 14: Closed-loop evolution of the basis library in the Kuramoto system in the first 4 rounds (a-d). The first row displays the training loss (red) and structural inference AUC (blue) over four optimization rounds. The second and third rows illustrate the distribution of learned coefficients (importance) for candidate terms in the message and update libraries, respectively. Note the sharp rise in AUC following the introduction of the $\sin(x_{j}-x_{i})$ term and the subsequent pruning of redundant basis functions.

Dyn.	Graph	GC	MI	TE	GDP	NRI	RIVA	COSINE
MM	BA50	51.63	72.08	55.35	50.98	63.83	52.37	72.89
	ER50	54.65	64.90	53.84	52.85	59.84	52.31	73.21
	WS50	59.48	91.23	55.98	56.97	88.30	54.19	87.49
Diff	BA50	64.22	68.07	56.56	54.59	59.84	52.19	91.98
	ER50	58.65	56.18	53.13	63.52	81.57	52.69	81.72
	WS50	63.66	55.68	67.76	56.16	91.17	54.97	98.21

Appendix

Table 4: Relational inference performance (AUC %) under low-data regime (

10\times 10

samples) across different dynamics and networks.

Region	Component	Term 1	Term 2	Term 3	Term 4	Term 5	Term 6
Arizona	Message	$x_{i}\cdot x_{j}$ (0.90)	$x_{j}^{(1)}\cdot(x_{j}^{(0)}-x_{i}^{(0)})$ (0.56)	$\\|x_{j}-x_{i}\\|_{2}$ (0.10)	$\sigma(x_{j}-x_{i})$ (0.05)	$x_{j}-x_{i}$ (0.04)	-
Arizona	Update	$\tanh(x)$ (0.58)	$x\cdot h_{i}$ (0.13)	$\frac{h_{i}}{k_{i}+\epsilon}$ (0.06)	$x^{(2)}\cdot h_{i}^{(0)}$ (0.05)	$h_{i}^{(1)}$ (0.03)	-
Connecticut	Message	$(x_{j}-x_{i})^{2}$ (0.54)	$(x_{j}-x_{i})$ (0.10)	$x_{j}^{(0)}$ (0.07)	$\sigma(\\|x_{j}-x_{i}\\|_{2})$ (0.05)	$x_{j}^{(0)}\cdot\sigma((x_{j}^{(0)}-x_{i}^{(0)}))$ (0.04)	-
Connecticut	Update	$x^{(1)}$ (0.23)	$x^{(0)}$ (0.13)	$x^{(2)}$ (0.13)	$x^{(1)}\cdot x^{(2)}$ (0.10)	$\tanh(h_{i})$ (0.09)	$\frac{h_{i}}{k_{i}+\epsilon}$ (0.07)
Illinois	Message	$x_{i}\cdot x_{j}$ (0.52)	$\frac{(x_{i}\cdot(x_{j}-x_{i}))}{(1+\\|x_{j}-x_{i}\\|_{2})}$ (0.28)	$\\|x_{j}-x_{i}\\|_{2}$ (0.07)	$x_{j}^{(2)}-x_{i}^{(2)}$ (0.07)	$x_{j}^{(1)}-x_{i}^{(1)}$ (0.07)	$2\cdot(x_{j}-x_{i})$ (0.07)
Illinois	Update	$-0.1x$ (0.43)	$-0.2x^{(1)}$ (0.38)	$\frac{h_{i}}{k_{i}+\epsilon}$ (0.10)	$\tanh(h_{i})$ (0.09)	$x^{(0)}+0.5\cdot x^{(1)}$ (0.08)	$h_{i}$ (0.04)

Appendix

Table 5: Discovered terms from round 0 for COVID datasets. Terms are ordered by absolute coefficient magnitude. Each region has Message and Update rows.

x

: node state;

h_{i}

: aggregated message;

k_{i}

: degree;

\epsilon=10^{-6}

Figure 15: Epidemiological data for Arizona. (a) Geographic map and county-level interaction network. (b) Ridgeline plot of daily COVID-19 confirmed cases across counties from February 2020 to February 2023.

Figure 16: Epidemiological data for Connecticut. (a) Geographic map and county-level interaction network. (b) Ridgeline plot of daily COVID-19 confirmed cases across its 8 counties.

Figure 17: Epidemiological data for Illinois. (a) Geographic map and county-level interaction network. (b) Ridgeline plot showing the high-density case distribution across over 100 counties.

Figure 18: Epidemiological data for Michigan. (a) Geographic map and county-level interaction network. (b) Ridgeline plot of daily COVID-19 confirmed cases illustrating regional seasonal variations.

Explore similar work

Symbolic--KAN: Kolmogorov-Arnold Networks with Discrete Symbolic Structure for Interpretable Learning
Mar 25, 2026Salah A Faroughi, Farinaz Mostajeran, Amirhossein Arzani +1Interpretable ML Scientific ML
Energy & Intelligence Lab, Department of Chemical Engineering, University of Utah, Salt Lake City, Utah, 84112, USA · Department of Mechanical Engineering, University of Utah, Salt Lake City, Utah, 84112, USA · Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, Utah, 84112, USA +2
OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems
Jun 17, 2026Till Richter, Niki KilbertusSparse Identification of Nonlinear Dynamics Neural Network Interpretability
Technical University of Munich · Helmholtz Munich
SymTorch: Symbolic Distillation of Neural Networks
Feb 24, 2026Elizabeth S. Z. Tan, Adil Soubki, Miles CranmerNeural Network Interpretability Scientific Discovery
Department of Applied Mathematics and Theoretical Physics University of Cambridge

Mar 25, 2026J/K move · Enter open · S save

Symbolic--KAN: Kolmogorov-Arnold Networks with Discrete Symbolic Structure for Interpretable Learning

Salah A Faroughi, Farinaz Mostajeran, Amirhossein Arzani, Shirko Faroughi

Energy & Intelligence Lab, Department of Chemical Engineering, University of Utah, Salt Lake City, Utah, 84112, USA · Department of Mechanical Engineering, University of Utah, Salt Lake City, Utah, 84112, USA · Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, Utah, 84112, USA+2

Interpretable ML Scientific ML Kolmogorov-Arnold Networks Symbolic Regression PDE Discovery Physics-Informed ML

Open paper AskPDF arXiv

Symbolic discovery of governing equations is a long-standing goal in scientific machine learning, yet a fundamental trade-off persists between interpretability and scalable learning. Classical symbolic regression methods yield explicit analytic expressions but rely on combinatorial search, whereas neural networks scale efficiently with data and dimensionality but produce opaque representations. In this work, we introduce Symbolic Kolmogorov-Arnold Networks (Symbolic-KANs), a neural architecture that bridges this gap by embedding discrete symbolic structure directly within a trainable deep network. Symbolic-KANs represent multivariate functions as compositions of learned univariate primitives applied to learned scalar projections, guided by a library of analytic primitives, hierarchical gating, and symbolic regularization that progressively sharpens continuous mixtures into one-hot selections. After gated training and discretization, each active unit selects a single primitive and projection direction, yielding compact closed-form expressions without post-hoc symbolic fitting. Symbolic-KANs further act as scalable primitive discovery mechanisms, identifying the most relevant analytic components that can subsequently inform candidate libraries for sparse equation-learning methods. We demonstrate that Symbolic-KAN reliably recovers correct primitive terms and governing structures in data-driven regression and inverse dynamical systems. Moreover, the framework extends to forward and inverse physics-informed learning of partial differential equations, producing accurate solutions directly from governing constraints while constructing compact symbolic representations whose selected primitives reflect the true analytical structure of the underlying equations. These results position Symbolic-KAN as a step toward scalable, interpretable, and mechanistically grounded learning of governing laws.

Abstract

Explore similar work

Symbolic--KAN: Kolmogorov-Arnold Networks with Discrete Symbolic Structure for Interpretable Learning

OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems

SymTorch: Symbolic Distillation of Neural Networks