MorphingNet: Symbolic Regression Where the Network Is the Formula

Jincheng Gao; Jiahe Yi; Shuo Sun

cs.LGOct 14, 2026

MorphingNet: Symbolic Regression Where the Network Is the Formula

Authors: Jincheng Gao, Jiahe Yi, Shuo Sun

Organizations: Thrust of Financial Technology, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.

Abstract

We introduce MorphingNet, a network for symbolic regression (SR) whose units change their function continuously. Instead of searching over expressions built from a fixed operator library, MorphingNet trains the form of each function as a parameter, so SR becomes one continuous optimization problem. Its units are either pole-free rational functions or complex exponential--logarithm functions, which move between exponentials, sinusoids, damped oscillations, power laws and logarithms as their parameters change. The network is a readable formula at every step of training, and no operator library is needed to train it; a structure-recovery step afterwards rewrites the trained formula into short elementary expressions. Training is dense linear algebra, so it is GPU-friendly and scales to large data. MorphingNet achieves state-of-the-art accuracy on SRBench, recovers most generating formulas on synthetic benchmarks, and on large data is up to two orders of magnitude faster than the strongest search method. By turning the search for a formula into the training of a network, MorphingNet offers a new paradigm for SR and for scientific discovery from data.

Figures & tables

Panel of Figure 1 — Figure 1: MorphingNet at a glance. (a) A two-layer network. Each layer reads the whole bank through an affine map; its ports feed units (blue) and product ports (orange), and their outputs are appended to the bank. Dark lines show the part that survives training on $y=2\sin(x_{1})\,x_{2}+1$ ; gray parts were set to zero. (b) The complex exponential–logarithm unit $J$ at five settings of $(\alpha,\omega,\beta)$ , computed from its definition. One formula moves continuously between these shapes.

Rewrite	Action
Unit	Replace a unit by $A\,\phi(z)+\gamma z+d$ with $\phi$ from the library. A C unit with $\beta=0$ becomes $Ke^{\alpha z}\cos\omega z+Me^{\alpha z}\sin\omega z-K$ exactly.
Merge	Replace several terms that depend on the same direction by one function.
Quadratic	Collect products into a quadratic form and rewrite it with the fewest products by eigendecomposition.
Delete	Delete terms; absorb linear terms into the bias of a product factor.
Swap	Swap one elementary function for another if the error halves, e.g. $\tanh\to\exp$ .
Template	If the gradient outer product has rank two, fit $g(z_{1})+h(z_{2})$ or $g(z_{1})\,h(z_{2})$ .

Table 1: Rewrites used by structure recovery. Each is refitted jointly and kept only under the error bound.

Method	0	0.001	0.01	0.1
MorphingNet-R	97.0 $\pm$ 0.9	96.8 $\pm$ 1.1	96.3 $\pm$ 1.0	70.1 $\pm$ 2.5
MorphingNet-C	95.3 $\pm$ 1.5	94.8 $\pm$ 1.0	92.8 $\pm$ 1.6	66.4 $\pm$ 3.5
MRGP	88	87	83	3
Operon	85	86	85	68
SBP-GP	75	74	74	50
AI Feynman	74	73	69	12

Table 2: Fraction of SRBench ground-truth problems with test

R^{2}>0.999

, by noise level. MorphingNet: mean

\pm

standard deviation over ten splits. Other methods: published SRBench results; MRGP ( Arnaldo et al., 2014 ) , SBP-GP ( Virgolin et al., 2019 ) , AI Feynman 2.0 ( Udrescu et al., 2020 ) . Best in each column in bold, second underlined.

Unit	Variant	Reached	Ratio	Ops	Ratio	Ops
		Noise-free			Noisy
Rational	full	24/40	1	36	1	18
	first layer on axes	16/40	5.2	67	1.3	30
	no products	22/40	0.72	22	0.86	10
	one prune level	16/40	1.9	56	1.2	27
Confluent	full	24/40	1	60	1	17

Table 3: Ablations on the current engine, eight families

\times

five instances per condition. Reached: problems whose validation error is within

\tau=10^{-5}

. Ratio: median paired test-error ratio to the full model. Ops: median unit operations of the selected formula.

Family	Exact	Before	After	Operon
Response	36/36	26/175	7/9	16/18
Oscillation	29/34	26/250	7/8	17/14
Reuse	31/34	15/45	5/11	17/21
Special	0/34	21/115	13/23	13/14
All	96/138	23/107	7/10	16/16

Table 4: Structure recovery on 138 formulas. Exact: test nRMSE

<10^{-12}

. Medians of unit operations / constants, before and after recovery, and of Operon on the same data. Bold: shorter than Operon in both counts.

Rows	$10^{3}$	$10^{4}$	$10^{5}$	$10^{6}$
Wall time, CPU	1.0	3.3	4.9	11.4
Wall time, GPU	–	4.4	24.9	122.6
Test error	22	11	16	18

Table 5: Cost on 48 data sets. Medians of per-data-set ratios Operon / MorphingNet; above one favors MorphingNet. CPU at

10^{6}

rows has two pairs.

Method	Noise-free	Noisy	Ops / const.
MorphingNet-R	7.9e-6	4.0e-3	30/130
MorphingNet-C	1.3e-5	4.8e-3	34/130
Brush	3.9e-5	6.5e-3	20/17
Operon	9.7e-5	5.5e-3	13/16
PySR	2.4e-3	9.1e-3	20/13
ParFam	9.4e-4	4.5e-3	10/62

Table 6: Median test nRMSE on the common suite, eight problems per condition, and median unit operations / constants on noise-free data. Lowest error in bold, second underlined.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Layers	Units	Products	Degree $D$
1	6	2	3
2	6	2	3
3	12	4	5

Appendix

Table 7: Capacities trained in each run, from small to large. Units and products are counted per layer.

Setting	Value
$\ell_{1}$ weight $\lambda_{1}$ / group weight $\lambda_{g}$	$3\times 10^{-6}$ / $10^{-3}$
Warm-up without penalty	200 LM steps
Steps per fit and per refit	at most 2000
Control error	checked after every step
Stagnation	40 steps without a relative improvement of $10^{-5}$
Proximal gradient tolerance	$10^{-8}$

Appendix

Table 8: Training and selection settings.

Unit	Noise	All	Feynman	Strogatz	Plain rule	Ops / const.
Rational	0	97.0 $\pm$ 0.9	97.8 $\pm$ 0.4	90.0 $\pm$ 8.0	97.0 $\pm$ 0.9	59 / 216
	0.001	96.8 $\pm$ 1.1	97.8 $\pm$ 0.6	88.6 $\pm$ 9.1	96.8 $\pm$ 1.1	39 / 94
	0.01	96.3 $\pm$ 1.0	98.0 $\pm$ 0.4	82.1 $\pm$ 8.0	96.3 $\pm$ 1.0	36 / 83
	0.1	70.1 $\pm$ 2.5	76.5 $\pm$ 2.9	17.1 $\pm$ 5.7	33.2 $\pm$ 3.5	30 / 61
Confluent	0	95.3 $\pm$ 1.5	96.5 $\pm$ 1.0	85.7 $\pm$ 9.6	95.3 $\pm$ 1.5	50 / 196
	0.001	94.8 $\pm$ 1.0	95.7 $\pm$ 0.8	87.9 $\pm$ 6.4	94.8 $\pm$ 1.0	37 / 100

Appendix

Table 9: SRBench ground-truth problems: percentage with test

R^{2}>0.999

, mean

\pm

standard deviation over ten splits, for all 130 problems, the 116 Feynman and the 14 Strogatz problems, and with the plain

1.1\,e_{\min}

rule instead of the noise-aware rule. Last column: median unit operations / constants of the chosen formulas, before structure recovery.

Explore similar work

MOSAIC-SR: Transformer-Guided Symbolic Regression for Scientific Equation Recovery
Sep 17, 2026Peiyi Zheng, Yanming Kang, Hans De Sterck +1Symbolic Regression Scientific Discovery
University of Waterloo
Synthesizing Physics Formulae with Transformers
Oct 2, 2026Shuwei Wang, Vadim Bulitko, Michael Youngblood +5Symbolic Regression
Department of Computing Science, University of Alberta, 116 Street and 85 Avenue, Edmonton, AB, T6G 2R3, Canada · Filuta AI, 1606 Headway Circle, Austin, TX 78754, USA · Department of Computer Science, Data Science, University of British Columbia, 6200 University Blvd, Vancouver, BC Canada V6T 1Z4, Canada +2
Improving Neural Symbolic Regression with a Diverse Prior and Post-training Alignment
Oct 13, 2026David Otte, Jörg K. H. Franke, Frank HutterSymbolic Regression Synthetic Data Pretraining
LMU Munich · Munich Center for Machine Learning (MCML) · University of Freiburg +3

Sep 17, 2026J/K move · Enter open · S save

MOSAIC-SR: Transformer-Guided Symbolic Regression for Scientific Equation Recovery

Peiyi Zheng, Yanming Kang, Hans De Sterck, Giang Tran

University of Waterloo

Symbolic Regression Scientific Discovery

Open paper AskPDF arXiv

Symbolic regression aims to recover closed-form equations from observations, providing interpretable models for scientific discovery. Existing approaches struggle to combine flexible structural search with efficient inference. Search-based methods can refine expression structure but often rely on costly combinatorial optimization with random initialization. Pretrained neural models generate formulas almost instantly, but their predictions often contain symbolic errors. We introduce MOSAIC-SR, which uses a pretrained Transformer to propose multiple initial sketches. These sketches initialize searches in several promising regions, avoiding random starts in the vast expression space. Each search jointly recovers structure and constants through scale-aware constant optimization and local symbolic repair. We evaluate MOSAIC-SR on the SRSD-Feynman dataset with and without dummy variables and on six additional benchmarks. MOSAIC-SR obtains the highest symbolic solution rate on every dataset while ranking among the top two methods in predictive accuracy. This advantage persists in the presence of irrelevant dummy inputs. The results show that learned priors can focus search on promising equation structures, and that numerical optimization and symbolic repair are important for recovery.

Abstract

Explore similar work

MOSAIC-SR: Transformer-Guided Symbolic Regression for Scientific Equation Recovery

Synthesizing Physics Formulae with Transformers

Improving Neural Symbolic Regression with a Diverse Prior and Post-training Alignment