Synthetic Neurosymbolic Task Generation with Controlled Constraint Complexity
Organizations: Ukrainian Catholic University, Lviv, Ukraine · HUN-REN Alfr´ed R´enyi Institute of Mathematics, Budapest, Hungary · KU Leuven, Leuven, Belgium · University of Oxford, Oxford, UK
Abstract
We introduce NeSySy, a synthetic generator for neurosymbolic learning tasks that jointly produces logical constraints and consistent datasets. Unlike existing benchmarks, NeSySy provides fine-grained control over key properties such as constraint structure, tightness, input dependence, and data generation and alignment strategies. In particular, it allows the use of rich constraints over infinite domains, and uses solvers to ensure that the constraints are meaningful and that generated samples satisfy the constraints. We design NeSySy to support both learning with declarative constraints and learning from distant supervision. Across a range of constraint families, we provide evidence that its parameters induce systematic and largely independent effects on task difficulty. Using NeSySy, we study constraint-aware learning methods, identifying regimes where explicit constraint enforcement is beneficial and where expressive unconstrained models can instead learn constraint structure directly from data. Our results establish NeSySy as a controlled testbed for analysing the interaction between learning and logical structure. NeSySy is built around a pipeline that incorporates solvers. Satisfiability checkers (CNFgen and Z3) generate and certify constraints. Constrained optimizers (RC2 for MaxSAT, Z3 for OMT) compute optimal repairs that guarantee every generated sample is consistent.
Figures & tables
| Hidden | Accuracy ( ) | VR ( ) | MAE ( ) | Unique | |
|---|---|---|---|---|---|
| Filtering | |||||
| Fixing | |||||
| Accuracy (%, better) | VR (%, better) | ||||
|---|---|---|---|---|---|
| BaseCCN+ | BaseSolver | BaseSeq | Base | BaseSeq | |
| 1 | |||||
| 2 | |||||
| 3 | |||||
| 4 | |||||
| Training set size | |||||
|---|---|---|---|---|---|
| Method | 80 | 400 | 4000 | 8000 | 16000 |
| BaseCCN+ | |||||
| BaseSolver | |||||
| BaseSeq | |||||
| MAE ( better) | MCD ( better) | ||
|---|---|---|---|
| BaseDRL | BaseSolver | Base | |
| 1 | |||
| 2 | |||
| 3 | — | ||
| 4 | — | ||
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Synthetic Benchmark Generation
Synthetic benchmark generation creates controlled, verifiable evaluation tasks and datasets to test models across diverse settings.
See all 302 papers