Mask-Based Deep Symbolic Regression

Zachary Bastiani; Robert M. Kirby; Jacob Hochhalter; Shandian Zhe

cs.LGMay 30, 2025

Mask-Based Deep Symbolic Regression

Authors: Zachary Bastiani, Robert M. Kirby, Jacob Hochhalter, Shandian Zhe

Organizations: Kahlert School of Computing University of Utah · Scientific Computing and Imaging Institute University of Utah · Department of Mechanical Engineering University of Utah

Abstract

Diffusion has emerged as a powerful framework for generative modeling, achieving remarkable success in applications such as image and audio synthesis. Enlightened by this progress, we propose a novel diffusion-based approach for symbolic regression. We construct a random mask-based diffusion and denoising process to generate diverse and high-quality equations. We integrate this generative processes with a token-wise Group Relative Policy Optimization (GRPO) method to conduct efficient reinforcement learning on the given measurement dataset. In addition, we introduce a long short-term risk-seeking policy to expand the pool of top-performing candidates, further enhancing performance. Extensive experiments and ablation studies have demonstrated the effectiveness of our approach.

Figures & tables

Figure 1: Visual representation of sampling methods where colored cells represent a sampled index, while white cells denote an unsampled or masked index.

Figure 2: MDSR’s performance on the SRBenchmark. Methods with * are taking from ParFam’s published results.

Dataset	MDSR		CADSR		MODSR		MMSR		SNIP		TPSR
Dataset	$R^{2}$	$\mathcal{C}$	$R^{2}$	$\mathcal{C}$	$R^{2}$	$\mathcal{C}$	$R^{2}$	$\mathcal{C}$	$R^{2}$	$\mathcal{C}$	$R^{2}$	$\mathcal{C}$
Feynman	0.991	17.7	0.934	16.5	0.788	16.4	0.991	20.8	0.985	31.6	0.993	57.2
Strogatz	0.994	13.0	0.980	15.4	0.970	15.8	0.982	21.6	0.995	29.3	0.965	56.1

Table 1: Performance comparison of different SR methods on SRBench datasets. The table shows mean

R^{2}

scores (higher is better) and average number of nodes

\mathcal{C}

(lower is better) for each method. All results are compiled from the method’s published benchmark results.

Figure 3: Ablations on the Strogatz dataset. Sampling-method ablations are shown in green and policy-optimization ablations in red; MDSR (blue) serves as the baseline for both. Dot size corresponds to mean expression complexity, so smaller dots indicate more interpretable expressions.

Figure 4: Comparison of the percentage of new expression found in each epoch. Shaded regions denote a 95% confidence region.

Panel of Figure 5 — Figure 5: Results for six symbolic regression methods across two real-world applications. Eight trials were ran for each method with access to a RTX 4090 GPU and Ryzen 9-7950X processor.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: MDSR’s architecture.

Hyperparameter	MDSR
Variables	{1, $c$ (Constant Token), $xB_{i}$ }
Unary Functions	{sin, cos, log, $\sqrt{(\cdot)}$ , exp}
Binary Functions	{+, -, *, /, $\hat{}$ }
Risk Seeking Percent ( $\alpha$ )	$5\%$
Learning Rate	5E-4
Max Depth	32

Appendix

Table 2: Hyperparameter settings of MDSR.

Figure 7: Breadth-first search ordering of the expression

\frac{x_{1}}{c_{1}}+\exp(x_{2})

	Symbolic Solution Rate (%)				Accuracy Rate (%)				Simplified Complexity				Time (m)
Algorithm	0.0	0.001	0.01	0.1	0.0	0.001	0.01	0.1	0.0	0.001	0.01	0.1	Median
AFP	22.3	19.9	16.9	12.9	43	42	40	41	29.4	28.4	29.0	30.8	58.1
AFP_FE	28.1	22.7	20.3	12.9	56	50	50	50	34.6	35.7	34.5	36.8	480.5
AIFeynman	61.8	31.9	12.6	0.9	74	74	68	10	83.3	88.7	99.3	110.5	493.2
Bingo	48.8	14.6	4.8	0.8	64	60	62	59	15.6	19.3	21.3	22.5	375.7
BSR	2.5	0.6	0.1	0.0	12	11	12	7	34.3	35.5	36.8	38.4	480.0

Appendix

Table 3: Performance on SRBench problems with known solutions.

Dataset	Method	Train $R^{2}$	Test $R^{2}$	Runtime (s)	Complexity
Water retention	MODSR	0.621	0.556	16.655	26.500
	E2E	0.000	-0.042	8.168	55.625
	SNIP	0.887	0.860	3558.539	35.250
	CADSR	0.689	0.622	6481.074	17.625
	MDSR	0.864	0.835	1869.911	23.875
	PySR	0.967	0.948	3462.261	31.500

Appendix

Table 4: Performance comparison on the water retention and woodrat datasets. We report the mean value across eight trials. Higher

R^{2}

is better; lower runtime and complexity are better. Bolded entries denote best performance in the given column.

Panel of Figure 8 — Figure 8: Pareto-frontier comparison of symbolic-regression methods, showing test $R^{2}$ against expression complexity. Methods nearer the upper-left offer stronger predictive performance with simpler expressions.

Model	Run	Train $R^{2}$	Test $R^{2}$	Best equation
MODSR	49529	0.724	0.792	(-1.35e4+(((1.94sin((50.4+(0.598PV))))-(-0.351Flow))-(((0.240Flow)-(-8.58e3arctan(((-5.43e3PV)-(741Size)))))-(-1.23Co))))
E2E	49529	0.004	0.117	(-0.005+(-0.06((1.6+(-8.6sin(((0.01+(0.2(1/(((-0.724+(0.704(-1.26+(0.378PV))))(-9+(-0.09(-1.16+(65.8Coductivity)))))))))+(8+(-0.6cos(((0.09+(0.9(-2.42+(0.557Flow))))+((0.02+(0.9(-17.3+(16.7Size))))+(0.02+(9(-1.63+(1.98Co))))))))))))*2))
SNIP	40320	0.972	0.975	(-6.16+((1/((-1.76e-4+(-8.67e-5Co))))((-2.28e-4Size)+((1.47e-3Coductivity)-((-2.15e-4arctan((-36.1+(8.76PV))))-((-3.87e-4arctan((-9.90+(12.5PV))))-(3.23e-5*Flow)))))))
PySR	49059	0.968	0.970	(1.09(Flow/Co0.853)+sin(1.83/(4.79-PV)))(sin(sin(2.36/PV)1.40)+0.602)-4.99
MDSR	8432	0.935	0.933	((c1(PV+(((c3+cos(PV))+(log((Co/Flow))+log(Co)))+(PVc2))))-c0)
CADSR	8432	0.917	0.889	(((square(((c5(PVPV))/c4))-(c3/Size))-(c1/(PV(PV/(c6Size)))))-((c2/PV)-c0))

Appendix

Table 5: Best expression found by each method on the mineral retention problem. Constants are rounded to three significant digits for readability.

Model	Run	Train $R^{2}$	Test $R^{2}$	Best equation
MODSR	49529	0.680	0.532	(-5.28+((1.04PC2)-(-0.371(PC1*(((-0.626Gsta1)-(0.0794Akr7a3))-(((-8.59sin(((-0.207Gsta6)-(4.61cos(((6.56(Cyp3a11PC2))-(-59.4Gsta3)))))))-(3.68Cyp2a3))+(-0.911Cyp2a5)))))))
E2E	40320	0.144	0.341	((-5.2+(0.36(0.002+(9.42PC1))))+((0.004+(-0.007cos(((-0.03+(9(-3.68+(0.902Cyp2a5))))+(0.9+(0.9(-4.09+(0.332Cyp3a11))))))))((0.01+(60cos(((0.9+(0.9(-5.62+(0.598Akr7a3))))((0.06+(0.9(0.0476+(9.52PC2))))+((0.01+(-0.9(-2.4+(0.92Gsta1))))+(0.6+(-2(-4.04+(1.8Cyp2a3))))))))))+((-0.01+(-0.01(-2.32+(0.401Gsta6))))+((-0.01+(-0.01(-2.31+(0.924Gsta3))))+(-0.5+(0.01(-6.08+(0.12Rsrc1)))))))))
SNIP	40320	0.687	0.504	(-5.23+(((2.86PC2)-((-0.271Cyp2a3)+((9.56e-7(((-2.46e5Cyp2a3)+(9.98(Cyp3a11((1.6e3Cyp2a3)-(87.5Akr7a3)))))Gsta1))-((-34.3((-0.0752+(0.00184Cyp2a5)))2)-(2.87e-9((-1.23e4+(-5.06e4PC2)))*2)))))((1/((-2.61e4+(2.32e4Gsta1))))(((-0.751(Cyp3a11Cyp3a11))+(5.68e3(1/((6.36e4+(3.42e3Cyp3a11))))))((412*(Akr7a3PC1))+(-6.49e3PC1))))))
PySR	49059	0.838	0.500	sin((PC1+Gsta1*(-9.091))(Cyp2a3*exp(sin(Gsta18.09)+cos(Rsrc1/0.595+0.011/PC1))+sin(Gsta18.090)))-5.229
MDSR	49059	0.452	0.539	(c0/(Cyp2a3**PC1))
CADSR	40320	0.475	0.541	((PC2+c0)+tan(exp(PC1)))

Appendix

Table 6: Best expression found by each method on the gene expression problem. Constants are rounded to three significant digits for readability.

Figure 9: Impact of the policy optimization method on expression diversity. Multi-step REINFORCE, PPO, and GRPO all reduce diversity relative to single-step REINFORCE, with PPO and GRPO reducing diversity more than multi-step REINFORCE. While multi-step REINFORCE reduced diversity compared the single-step variant, it improve symbolic recovery which could highlight multi-steps REINFORCE’s improvement to exploitation.

Symbolic Regression

Symbolic regression searches data for compact mathematical expressions that balance predictive accuracy with interpretability.

See all 144 papers

Fast Diffusion-GAN Model for Generating Symbolic Music with Emotions
Oct 21, 2023Jincheng Zhang, György Fazekas, Charalampos SaitisTimbre-Conditioned Music Generation Music Generation
Centre for Digital Music, Queen Mary University of London, UK
Data Enrichment for Symbolic Regression Using Diffusion Models
May 31, 2026Simon De Reuver, Tamas Kristof Toth, Teddy LazebnikPhysics-Informed Diffusion Models Physics-Informed Generative Modeling
Department of Computing, Jönköping University, Jönköping, Sweden · Department of Information Science, University of Haifa, Haifa, Israel
GENSR: Symbolic Regression Based in Equation Generative Space
Feb 24, 2026Qian Li, Yuxiao Hu, Juncheng Liu +1Symbolic Regression Latent Space Optimization
Shanghai Jiao Tong University, Shanghai, China · Eastern Institute of Technology, Ningbo, China · The Hong Kong Polytechnic University, Hong Kong, China +1

Oct 21, 2023J/K move · Enter open · S save

Fast Diffusion-GAN Model for Generating Symbolic Music with Emotions

Jincheng Zhang, György Fazekas, Charalampos Saitis

Centre for Digital Music, Queen Mary University of London, UK

Timbre-Conditioned Music Generation Music Generation Symbolic Music Generation Diffusion Models Controllable Music Generation Few-Step Diffusion Sampling

Open paper AskPDF arXiv

Diffusion models have shown promising results for a wide range of generative tasks with continuous data, such as image and audio synthesis. However, little progress has been made on using diffusion models to generate symbolic music since this class of generative models struggles with discrete data while its iterative sampling process is computationally expensive. In this work, we propose a diffusion model integrated with a Generative Adversarial Network, aiming to mitigate the slow sampling drawback of diffusion models applied to symbolic music generation. We enable the use of diffusion model by representing symbolic music as image-like pianorolls with velocity values instead of discrete event tokens and then used a Variational Autoencoder to obtain their latent representations to train the proposed diffusion model. Our results demonstrate the successful control of our diffusion model to generate symbolic music with a desired emotion. Our model uses merely 15 denoising diffusion steps to produce high-quality music, while most diffusion models for symbolic music generation require hundreds or thousands of steps.

Abstract

Symbolic Regression

Explore similar work

Fast Diffusion-GAN Model for Generating Symbolic Music with Emotions

Data Enrichment for Symbolic Regression Using Diffusion Models

GENSR: Symbolic Regression Based in Equation Generative Space