cs.NENov 12, 2026

TSGP-2: On the Non-Linearity of Symbolic Regression Problems and How to Exploit It with Model-based Semantic Genetic Programming

Authors: Philipp Anthes, Alina Geiger, Martin Briesch, Franz Rothlauf

Organizations: Johannes Gutenberg University, Mainz, Germany

Abstract

Model-based variation operators such as the one used in Transformer Semantic Genetic Programming (TSGP) are affected by possible biases of their training data. TSGP is trained, in particular, on programs generated by Genetic Programming (GP) runs on linear target functions, which leads to a bias toward linear offspring. This bias can hinder search performance on problems that require non-linear solutions. This work has two goals: first, it investigates the linearity of standard test problems such as the Feynman and real-world benchmark problems. While most of the structure of real-world problems can be well approximated by linear structures, the Feynman problems exhibit varying levels of non-linear structure. Thus, a proper variation operator of a search method should be able to create both types of programs, linear as well as non-linear ones. Motivated by this observation, we propose TSGP-2, which follows the functionality of TSGP but uses as training data programs from GP runs on both linear and non-linear target functions. As a result, the training data includes a lower share of programs that behave linearly (the share drops from 83% to 37%). With a higher share of non-linear structures, TSGP-2 outperforms TSGP on problems with non-linear structure, while performing similarly on linear problems. In addition, TSGP-2 is less biased toward linear offspring and is able to propose a higher number of meaningful non-linear offspring earlier during search, which coincides with better solutions. The findings underscore the importance of training data for a learned variation operator and emphasize that the biases of search operators should be aligned with the problems they are applied to.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Relocating Nonlinearity: How a Downstream Learner Reshapes What Genetic Programming Must Evolve

    Oct 7, 2026Nam H. LeGenetic Programming

  2. Evaluation of Population Initialization Methods for Genetic Programming-based Symbolic Regression

    Jun 30, 2026Lukas Kammerer, Gabriel Kronberger, Deaglan J. Bartlett +3Genetic ProgrammingSymbolic Regression

  3. On the Generalization Bounds of Symbolic Regression with Genetic Programming

    Apr 19, 2026Masahiro Nomura, Ryoki Hamano, Isao OnoGenetic ProgrammingStatistical Learning Theory