GENSR: Symbolic Regression Based in Equation Generative Space

Qian Li; Yuxiao Hu; Juncheng Liu; Yuntian Chen

cs.LGFeb 24, 2026

GENSR: Symbolic Regression Based in Equation Generative Space

Authors: Qian Li, Yuxiao Hu, Juncheng Liu, Yuntian Chen

Organizations: Shanghai Jiao Tong University, Shanghai, China · Eastern Institute of Technology, Ningbo, China · The Hong Kong Polytechnic University, Hong Kong, China · Imperial College London, London, England

Abstract

Symbolic Regression (SR) tries to reveal the hidden equations behind observed data. However, most methods search within a discrete equation space, where the structural modifications of equations rarely align with their numerical behavior, leaving fitting error feedback too noisy to guide exploration. To address this challenge, we propose GenSR, a generative latent space-based SR framework following the map construction -> coarse localization -> fine search'' paradigm. Specifically, GenSR first pretrains a dual-branch Conditional Variational Autoencoder (CVAE) to reparameterize symbolic equations into a generative latent space with symbolic continuity and local numerical smoothness. This space can be regarded as a well-structured map'' of the equation space, providing directional signals for search. At inference, the CVAE coarsely localizes the input data to promising regions in the latent space. Then, a modified CMA-ES refines the candidate region, leveraging smooth latent gradients. From a Bayesian perspective, GenSR reframes the SR task as maximizing the conditional distribution $p(\mathrm{Equ.} \mid \mathrm{Num.})$ , with CVAE training achieving this objective through the Evidence Lower Bound (ELBO). This new perspective provides a theoretical guarantee for the effectiveness of GenSR. Extensive experiments show that GenSR jointly optimizes predictive accuracy, expression simplicity, and computational efficiency, while remaining robust under noise.

Figures & tables

Figure 1: The overview of GenSR. During training, the dashed lines denote the prior branch, while the solid lines indicate the posterior branch. During inference, only the prior branch is used.

Panel of Figure 2 — Figure 2: Pareto front results on the Feynman dataset. The $x$ -axis shows the mean test $R^{2}$ rank, while the $y$ -axis shows equation complexity rank (left) and time complexity rank (right). Solid lines indicate the optimal Pareto front, and dashed lines show lower-ranked fronts from bottom-left to top-right.

Figure 3: Comparison on the Strogatz dataset under different noise levels. Subplots (left to right) report

R^{2}

score, time complexity (s), and equation complexity. Noise levels are represented by blue circles (0.000), orange squares (0.001), green triangles (0.01), and red diamonds (0.1), with error bars indicating standard deviations. Only methods whose mean

R^{2}

across noise settings exceeds 0.9 are included.

Figure 4: 2D t-SNE visualization of latent variables from E2ESR, SNIP, and GenSR. The legend distinguishes six categories, corresponding to equations from three representative function families, each evaluated under 2D and 5D input dimensionality, illustrating the clustering behavior of the learned latent spaces.

Figure 5: 2D t-SNE visualization of GenSR latent variables for equations from three function families (exponential, trigonometric, logarithmic) under 2D and 5D input settings, shown in subplots (a)–(f). Colors indicate the average of normalized $y$ values, as displayed in the accompanying color bar.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Pair 1: Expression 1: $c\exp(x)\cdot\sin(x-c)$ $\rightarrow$ Expression 2: $\log(x+c)\cdot\cos(x)$
$ratio$	Ours	E2ESR	SNIP
0	$c\exp(x)\sin(x-c)$	$c\exp(x)\sin(x-c)$	$c\exp(x)\sin(x-c)$
0.25	$c\exp(x)\sin(x-c)+c\sin(x)$	Invalid	$\exp(x+c)\sin(x)+c\sin(x)$
0.5	$c\exp(c\cdot c)\sin(x)+\log(c\cdot x)\cos(c)$	Invalid	$\exp(x)\cos(x)+c\cos(x+c)$
0.75	$\exp(c)+\log(x+c)\cos(x)$	$\log(x)\sin(x)\cdot\cos(x)$	Invalid
1	$\log(x+c)\cos(x)$	$\log(x+c)\cos(x)$	$\log(x+c)\cos(x)$

Appendix

Table 1: Linear interpolation between two expressions (1

\rightarrow

2) in latent space across methods. For each pair, we decode from

\bm{z}(ratio)=(1-ratio)\bm{z}_{1}+ratio\,\bm{z}_{2}

with

ratio\in\{0,0.25,0.5,0.75,1\}

. Expressions marked in red are syntactically invalid.

Dataset (ops, length, vars)	Posterior branch	Prior branch
LogExp-15 (vars $\leq$ 3)	0.094 $\;(\pm 0.023)$	0.106 $\;(\pm 0.031)$
LogExp-20 (vars $\leq$ 5)	0.095 $\;(\pm 0.027)$	0.112 $\;(\pm 0.029)$
LogExp-25 (vars $\leq$ 5)	0.107 $\;(\pm 0.018)$	0.120 $\;(\pm 0.022)$
Trig-15 (vars $\leq$ 3)	0.115 $\;(\pm 0.032)$	0.137 $\;(\pm 0.019)$
Trig-20 (vars $\leq$ 5)	0.113 $\;(\pm 0.026)$	0.142 $\;(\pm 0.043)$
Trig-25 (vars $\leq$ 5)	0.146 $\;(\pm 0.041)$	0.171 $\;(\pm 0.038)$

Appendix

Table 2: Reconstruction error on synthetic expression datasets. We report Levenshtein (edit) distance (mean

\pm

std; lower is better) between an expression and its reconstruction. Two operator sets are considered: (i) LogExp = {+,

-

\times

\div

\log

\exp

}; (ii) Trig = LogExp

\cup

{

\sin

\cos

\tan

}. Each dataset has 2k samples; the maximum expression length is shown in the name.

Figure 6: Effect of latent dimension in GenSR. The x-axis shows the latent dimension of the pretrained model (64, 128, 256, 512, 768), while the left and right y-axes report

R^{2}

and equation complexity. Results are reported under three noise levels: Noise-Free,

1\times 10^{-2}

, and

1\times 10^{-1}

Figure 7: Effect of

k

on CMA-ES performance.

Figure 8: Ablation results on CMA-ES hyperparameters. Left: effect of population size

s

R^{2}

and runtime. Middle: effect of initial step size

t

R^{2}

and equation complexity. Right: effect of weight

\omega

in the fitness function.

Figure 9: Accuracy comparison of different methods on the classification task. GenSR (Ours) achieves the highest accuracy (0.849), significantly outperforming E2ESR (0.661) and SNIP (0.723).

Method	Test $R^{2}$ ↑	Complexity ↓	Time (s) ↓
Operon	0.7945	65.69	2974
GP-GOMEA	0.7381	30.27	9636
MDL	0.6258	29.88	541.7
DSR	0.5625	9.465	36852
SBP-GP	0.7869	634	149344
FEAT	0.7621	82.49	6432

Appendix

Table 3: Complete experimental results of black-box dataset.

Method	Strogatz Dataset ( $\epsilon=0.0$ )			Feynman Dataset ( $\epsilon=0.0$ )
Method	$R^{2}$ ↑	Time (s)↓	Complx↓	$R^{2}$ ↑	Time (s)↓	Complx↓
FEAT	0.9210(±0.054)	1135(±970)	119(±16)	0.9190(±0.014)	2417(±1737)	205.3(±13)
NeurSR	0.5206(±0.029)	14.82(±1.7)	11.3(±0.51)	0.3958(±0.013)	24.18(±2.2)	13.28(±0.15)
E2ESR	0.5341(±0.043)	3.729(±2)	32.49(±2.9)	0.8570(±0.013)	4.062(±1.3)	36.02(±0.88)
SNIP	0.9952(±0.002)	509(±21)	29(±1.2)	0.985(±0.004)	1203.5(±145)	31.71(±1.8)
GPlearn	0.7689(±0.067)	1195(±592)	28.96(±2.8)	0.8809(±0.027)	3900(±1052)	72.43(±29)

Appendix

Table 4: Complete experimental results at a noise level

\epsilon=0.0

. The results include test set

R^{2}

, search time, and formula complexity of different methods in the Feynman and Strogatz datasets.

Method	Strogatz Dataset ( $\epsilon=0.001$ )			Feynman Dataset ( $\epsilon=0.001$ )
Method	$R^{2}$ ↑	Time (s)↓	Complx↓	$R^{2}$ ↑	Time (s)↓	Complx↓
FEAT	0.9244(±0.032)	594.4(±181)	106.7(±15)	0.9207(±0.006)	1726(±242)	196.5(±12)
NeurSR	0.5219(±0.031)	15.07(±1.7)	11.41(±0.31)	0.3979(±0.013)	24.51(±1.7)	13.24(±0.13)
E2ESR	0.5105(±0.060)	3.436(±0.75)	33.83(±4.4)	0.8585(±0.010)	3.894(±0.98)	35.85(±1.5)
SNIP	0.9667(±0.024)	277.9(±18.4)	27.57(±1.3)	0.9904(±0.002)	1365.07(±104)	31.43(±0.23)
GPlearn	0.7955(±0.067)	913.3(±121)	29.59(±2.5)	0.8902(±0.008)	3316(±540)	60.49(±12)

Appendix

Table 5: Complete experimental results at a noise level

\epsilon=0.001

. The results include test set

R^{2}

, search time, and formula complexity of different methods in the Feynman and Strogatz datasets.

Method	Strogatz Dataset ( $\epsilon=0.01$ )			Feynman Dataset ( $\epsilon=0.01$ )
Method	$R^{2}$ ↑	Time (s)↓	Complx↓	$R^{2}$ ↑	Time (s)↓	Complx↓
FEAT	0.9244(±0.043)	472.9(±90)	95.61(±16)	0.9212(±0.010)	1464(±365)	167.1(±6.5)
NeurSR	0.5179(±0.042)	15.48(±1.4)	11.63(±0.34)	0.3942(±0.011)	24.62(±1.7)	13.26(±0.12)
E2ESR	0.5031(±0.034)	3.392(±0.72)	35.94(±1.8)	0.8345(±0.007)	4.274(±0.58)	40.07(±0.90)
SNIP	0.9844(±0.025)	237.61(±32)	29.35(±1.8)	0.987(±0.007)	1210.1(±154)	32.62(±1.3)
GPlearn	0.7956(±0.059)	907.4(±110)	30.59(±4.3)	0.8890(±0.009)	3351(±437)	60.07(±19)

Appendix

Table 6: Complete experimental results at a noise level

\epsilon=0.01

. The results include test set

R^{2}

, search time, and formula complexity of different methods in the Feynman and Strogatz datasets.

Method	Strogatz Dataset ( $\epsilon=0.1$ )			Feynman Dataset ( $\epsilon=0.1$ )
Method	$R^{2}$ ↑	Time (s)↓	Complx↓	$R^{2}$ ↑	Time (s)↓	Complx↓
FEAT	0.9228(±0.027)	446.9(±73)	84.2(±15)	0.9195(±0.006)	777.9(±102)	99.48(±5.6)
NeurSR	0.5054(±0.058)	17.47(±1.6)	12.74(±0.37)	0.3823(±0.020)	25.81(±1.7)	13.54(±0.16)
E2ESR	0.5152(±0.038)	5.621(±4.9)	38.49(±2.7)	0.7714(±0.014)	6.183(±0.83)	44.09(±0.61)
SNIP	0.9187(±0.042)	5650.7(±35)	39.35(±2.3)	0.9917(±0.020)	6514.8(±821)	37.71(±2.4)
GPlearn	0.8228(±0.052)	894.6(±108)	25.84(±2.9)	0.8911(±0.007)	2938(±543)	48.83(±9.5)

Appendix

Table 7: Complete experimental results at a noise level

\epsilon=0.1

. The results include test set

R^{2}

, search time, and formula complexity of different methods in the Feynman and Strogatz datasets.

	Method	$R^{2}$ ↑	Time (s)↓	Complx↓
k=256	CMA	0.9872	192.3	22.94
k=256	standard BO	0.9274	295.8	21.37
k=128	CMA	0.9675	179.1	23.26
k=128	standard BO	0.9161	237.5	22.55
k=64	CMA	0.9388	173.2	23.93
k=64	standard BO	0.8974	165.6	23.07

Appendix

Table 8: Comparison of CMA and standard BO under different

k

with 512-dimensional latent space

	Method	$R^{2}$ ↑	Time (s)↓	Complx↓
k=256	CMA	0.6984	181.4	18.23
k=256	standard BO	0.6241	206.6	17.12
k=128	CMA	0.4935	174.5	19.08
k=128	standard BO	0.4797	178.3	18.67

Appendix

Table 9: Comparison of CMA and standard BO under different

k

with 256-dimensional latent space

Method	$R^{2}$ ↑	Complx↓
GenSR	0.921	19.04
GenSR w/o Posterior	0.752	26.41

Appendix

Table 10: Comparison between GenSR and its disrupted variant

Figure 10: The horizontal axis denotes the epoch, while the left and right vertical axes correspond to the KL term and the KL weight, respectively. The KL weight annealing strategy used here increases the weight from 0 to 1.0 during the first 50% of the training epoches and keeps it constant thereafter.

Figure 11: Estimated distributions of prior and posterior means and variances obtained from 5,000 randomly sampled inputs in GenSR.

Property	Generative latent space (GenSR)	Discriminative latent space (contrastive)
Learning Objective	Equation distribution	Pointwise alignment
Continuous latent sampling	Supported	Not supported
Suitability for search	High	Low: limited with invalid regions

Appendix

Table 11: Comparison between Generative latent space and Discriminative latent space

Explore similar work

Symbolic Regression via Latent Iterative Refinement
May 26, 2026Xieting Chu, Sriram Vishwanath, Vijay GaneshLatent Variable Models Amortized Inference
Georgia Institute of Technology
InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance
Aug 26, 2026Yating Ling, Wenjing Cun, Zhitang ChenSymbolic Regression Large Language Model-Guided Scientific Discovery
Neural Structure Embedding for Symbolic Regression via Continuous Structure Search and Coefficient Optimization
Mar 23, 2026Fateme Memar, Tao Zhe, Dongjie WangSymbolic Regression
Department of Electrical Engineering and Computer Science, University of Kansas USA

May 26, 2026J/K move · Enter open · S save

Symbolic Regression via Latent Iterative Refinement

Xieting Chu, Sriram Vishwanath, Vijay Ganesh

Georgia Institute of Technology

Latent Variable Models Amortized Inference Symbolic Regression

Open paper AskPDF arXiv

Symbolic regression (SR) seeks closed-form mathematical expressions that fit observed data. Neural SR methods amortize the search by training an encoder to map observations directly to expressions in a single pass, but this amortized inference leaves a residual amortization gap between its one-shot prediction and the true posterior. We propose Latent Equation Embedding (LEE), a framework that closes this gap through iterative amortized inference in a functionally grounded latent space. LEE learns a shared latent space Z equipped with three components: an encoder f_theta that jointly embeds symbolic tokens and numerical observations into a single latent vector z; an expression decoder g_expr that reconstructs formulas from z; and an evaluation decoder g_eval that predicts function values from z, explicitly grounding the latent space in functional behavior. At inference, LEE performs iterative refinement by re-encoding decoded expressions jointly with observations, progressively improving the latent estimate. LEE uses the encoder itself as a learned inference optimizer: each re-encoding step implicitly computes the mismatch between the candidate and the data. Because g_eval is differentiable in z, we additionally interleave continuous gradient descent with discrete re-encoding, yielding a hybrid iterative and gradient refinement procedure. On SRBench across three noise levels, against 19 baselines spanning genetic programming, symbolic-neural hybrids, and pre-trained Transformers, LEE produces expressions 2--10x simpler than the strongest accuracy-oriented baselines, including Operon, GP-GOMEA, TPSR, RAG-SR, and GenSR, with complexity 8--11 versus 20--90. These results advance the low-complexity region of the accuracy-complexity Pareto frontier and show graceful degradation as noise increases.

Abstract

Explore similar work

Symbolic Regression via Latent Iterative Refinement

InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance

Neural Structure Embedding for Symbolic Regression via Continuous Structure Search and Coefficient Optimization