SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery

Mansooreh Montazerin; Antonio Ortega; Ajitesh Srivastava

cs.LGSep 4, 2026

SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery

Authors: Mansooreh Montazerin, Antonio Ortega, Ajitesh Srivastava

Organizations: Department of Electrical and Computer Engineering University of Southern California

Abstract

Symbolic regression (SR) discovers closed-form mathematical expressions from data, offering interpretability beyond black-box models. Existing methods suffer from slow convergence in combinatorial search spaces and lack mechanisms to exploit compositional structure in the data. We introduce SMILE (Sine, Multiplication, Identity, Logarithm, Exponential), a hybrid framework that unifies continuous gradient-based optimization with discrete symbolic recovery through three stages: structural analysis of the data to identify the compositional hierarchy of the target expression, continuous optimization to learn parameters of a network that encodes the target expression using interpretable activations, and symbolic recovery through structured pruning, coefficient optimization, and rounding. This final stage distills the learned network into a compact expression with exact symbolic constants. We evaluate SMILE on SRBench across ground-truth and black-box datasets, with ablation studies validating each component. SMILE achieves the highest symbolic solution rate at the largest noise levels, demonstrating strong robustness where competing methods degrade substantially. It consistently lies on the Pareto front of accuracy versus complexity, recovering significantly simpler expressions in a fraction of the time required by the competing methods.

Figures & tables

Figure 1: End-to-end pipeline of the SMILE framework. The pipeline performs a log-linear $R^{2}$ scan to classify the problem into one of three cases: (C1) direct training, (C2) training on the inverted target, or (C3) decomposition into simpler subproblems. In all cases, the trained network is passed through a recovery pipeline consisting of greedy pruning, symbolic extraction, parametric optimization, and gradient-based coefficient rounding. HL denotes the number of hidden layers.

Figure 2: (a) Approximation of the Sigmoid function using a SMILE-based configuration. (b) Emulation of an

L

-layer MLP into an equivalent SMILE network using stacked SMILE layers and Sigmoid blocks.

Figure 3: Comparison of SMILE and baselines on the Feynman dataset in terms of SSR, accuracy solution rate ( $R^{2}>0.999$ ), and training time.

Figure 4: Pareto front of expression complexity versus accuracy on the Feynman dataset (right, accuracy solution rate) and the black-box dataset (left, median

R^{2}

Figure 5: SSR on the Feynman and Strogatz datasets across all evaluated baselines.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Parameter	Value
Architecture	Neurons per hidden layer	5 (Sin, Mul, Identity, Log, Exp)
	Hidden layers	1 or 2
	Residual connections	Fully dense, all layers
	Bias	Learnable constant 1 appended to input
	Gate initialization	0 ( $\sigma=0.5$ )
Training	Optimizer	Adam

Appendix

Table 1: SMILE hyperparameters and pipeline configuration.

Parameter	Value
CPU	Intel Xeon Gold 5420+
GPU	NVIDIA RTX A5000 (24 GB)
Framework	PyTorch
Avg. time per problem	43.9 s

Appendix

Table 2: Computational resources.

Figure 6: Comparison of all methods on the Feynman dataset in terms of median $R^{2}$ test at multiple precision levels, expression complexity, and training time.

Figure 7: Accuracy solution rate ( $R^{2}>0.999$ ) on the Feynman and Strogatz datasets across all evaluated methods.

Problem	Ground truth	SMILE output	$\boldsymbol{\sigma=0}$	$\boldsymbol{\sigma=0.001}$	$\boldsymbol{\sigma=0.01}$	$\boldsymbol{\sigma=0.1}$
I.6.2a	$\dfrac{e^{-\theta^{2}/2}}{\sqrt{2\pi}}$	$\dfrac{e^{-\theta^{2}/2}}{\sqrt{2\pi}}$	✓	$\times$	$\times$	$\times$
I.12.2	$\dfrac{q_{1}q_{2}r}{4\pi\epsilon r^{3}}$	$\dfrac{q_{1}q_{2}}{4\pi\epsilon r^{2}}$	✓	✓	✓	✓
I.18.14	$mrv\sin(\theta)$	$mrv\sin(\theta)$	✓	✓	✓	$\times$
I.40.1	$n_{0}e^{-mgx/(k_{b}T)}$	$n_{0}e^{-mgx/(k_{b}T)}$	✓	$\times$	$\times$	$\times$
I.43.43	$\dfrac{k_{b}v}{(\gamma-1)A}$	$\dfrac{k_{b}v}{(\gamma-1)A}$	✓	✓	$\times$	$\times$
II.15.4	$-\mu B\cos(\theta)$	$-\mu B\sin(\pi/2-\theta)$	✓	✓	✓	✓

Appendix

Table 3: Examples of expressions recovered by SMILE on the Feynman dataset under noise-free and noisy conditions.

Figure 8: SSR as a function of noise level for the top five methods on the Feynman and Strogatz benchmarks.

Figure 9: Ground-truth (left) and SMILE-recovered (right) expressions for six problems: three from the Jin et al. [ 38 ] benchmark and three from the Feynman benchmark. For three-variable problems, one variable is held fixed. Five expressions are recovered exactly ( $R^{2}=1$ ); the Gaussian PDF (bottom right) is a close approximation ( $R^{2}>0.99$ ).

Figure 10: Comparison of all methods on the black-box dataset in terms of median $R^{2}$ , expression complexity, and training time.

Figure 11: Ablation study on the Feynman dataset. Each configuration removes one pipeline component while keeping the rest unchanged. SSR, accuracy solution rate, and timeout error rate are reported.

Explore similar work

FePySR: A Neural Feature Extraction Framework for Efficient and Scalable Symbolic Regression
May 12, 2026Zhiming Yu, Wangtao Lu, Xin LaiSymbolic Regression Unified Path-Integral Update
School of Mathematical Sciences, Zhejiang University, Hangzhou, Zhejiang, China · Faculty of Medicine and Health Technology, Tampere University, Tampere, Finland
Symbolic Regression via Latent Iterative Refinement
May 26, 2026Xieting Chu, Sriram Vishwanath, Vijay GaneshSymbolic Regression Neuro-Symbolic Framework
Georgia Institute of Technology
DeepPySR -- A Symbolic Regression Framework with Dynamic Pruning, Pareto Selection, and Hierarchical Composition for Real-World Scientific Discovery
Jul 9, 2026Fuling Chen, Kevin Vinsen, Phillip Melton +1Symbolic Regression Unified Path-Integral Update
International Centre for Radio Astronomy Research (ICRAR), University of Western Australia, Perth, WA, Australia · The Raine Study, University of Western Australia, Perth, WA, Australia · Menzies Institute for Medical Research, University of Tasmania, Hobart, TAS, Australia +1

May 12, 2026, cs.SCJ/K move · Enter open · S save

FePySR: A Neural Feature Extraction Framework for Efficient and Scalable Symbolic Regression

Zhiming Yu, Wangtao Lu, Xin Lai

School of Mathematical Sciences, Zhejiang University, Hangzhou, Zhejiang, China · Faculty of Medicine and Health Technology, Tampere University, Tampere, Finland

Symbolic Regression Unified Path-Integral Update Scientific Discovery

Open paper AskPDF arXiv

A fundamental challenge in symbolic regression (SR) is efficiently recovering complex mathematical expressions from observational data. Although this problem is NP-hard, many expressions of practical interest decompose naturally into combinations of nonlinear feature modules, concentrating structural complexity into a small number of reusable components. Here, we introduce FePySR, a two-stage framework that reduces the SR search space by extracting valid features prior to equation search. FePySR first employs a heterogeneous neural network to constrain observational data to a set of candidate expressions, then performs structural optimization within this refined expression space using PySR. Across five standard benchmarks, FePySR outperforms state-of-the-art methods by achieving higher equation recovery rates. On a set of 75 highly complex synthesized equations, FePySR recovers 36 equations, while producing substantially smaller mean squared errors on the remaining unrecovered cases, with reduced computation time compared to PySR. FePySR's first stage also maintains consistent performance under varying numbers of selected top features and increasing levels of noise in the observational data. Applied to ordinary differential equations governing biological systems, FePySR successfully identifies governing equations in 24 out of 100 tests where PySR recovers none. Taken together, FePySR is a generalizable framework that can enhance the SR solvers, enabling the efficient and reliable recovery of symbolic expressions across scientific domains.

Abstract

Explore similar work

FePySR: A Neural Feature Extraction Framework for Efficient and Scalable Symbolic Regression

Symbolic Regression via Latent Iterative Refinement

DeepPySR -- A Symbolic Regression Framework with Dynamic Pruning, Pareto Selection, and Hierarchical Composition for Real-World Scientific Discovery