GNBG-C: A Unified and Configurable Benchmark Generator for Constrained Continuous Optimization

Ali Ghasemzadeh; Erfan Babakhani; Armin Khosravi; Pariya Ghasemzadeh; Yong-Hong Kuo; Seyedali Mirjalili; Amir H. Gandomi

cs.NEOct 27, 2026

GNBG-C: A Unified and Configurable Benchmark Generator for Constrained Continuous Optimization

Authors: Ali Ghasemzadeh, Erfan Babakhani, Armin Khosravi, Pariya Ghasemzadeh, Yong-Hong Kuo, Seyedali Mirjalili, Amir H. Gandomi

Organizations: Department of Data and Systems Engineering, the University of Hong Kong, Hong Kong SAR, China · Independent Researcher, Isfahan, Iran · Centre for Artificial Intelligence Research and Optimisation, Torrens University Australia, Brisbane, Australia · Faculty of Electrical Engineering and Computer Science, VSB Technical University of Ostrava, Ostrava, Czech Republic ·

Abstract

Many real-world optimization tasks are governed not only by what should be optimized, but also by what is allowed: a solution must first satisfy physical, operational, safety, budgetary, or structural requirements before its objective value becomes meaningful. Yet benchmark design for constrained continuous optimization often treats these restrictions as fixed properties of predefined test instances. Existing generators and benchmark suites provide essential foundations for reproducible comparison, but they either focus primarily on configurable objective landscapes or expose constraint properties within limited, predefined regimes, making it difficult to ask targeted questions about constraint-induced difficulty: whether an optimizer struggles because feasible regions are small, boundaries are restrictive, variables are tightly coupled, or constraint information is poorly scaled. We introduce GNBG-C, a Generalized Numerical Benchmark Generator for Constrained Continuous Optimization. The central idea is to treat the feasible region itself as a configurable object rather than as an inseparable attachment to a fixed test function. By generating controlled constraint regimes around the known GNBG global minimizer, the generator enables systematic studies in which feasibility pressure can be varied while preserving that point as a global minimizer of the constrained problem. This allows researchers to examine how optimizers discover feasibility, handle constraint pressure, and improve objective quality under diverse and controllable feasibility conditions. GNBG-C therefore provides a flexible and auditable framework for probing the strengths and weaknesses of constrained continuous optimizers under targeted forms of feasibility-side difficulty.

Figures & tables

Figure 1: Core idea of GNBG-C. The objective landscape is kept fixed, while the generated constraint layer induces different feasible regions, such as loose, and strict regimes. This separates objective-landscape difficulty from feasible-region difficulty. The two-dimensional view is for interpretability; GNBG-C supports arbitrary user-specified dimensionality.

Figure 2: Conceptual view of GNBG-C as a configurable feasible-region layer attached to a GNBG objective landscape. Starting from a fixed objective landscape and a known GNBG optimum $x^{\star}$ , the constraint layer acts as a controllable shell whose size, activity at the optimum, boundary geometry, variable participation, numerical scale, and redundancy can be adjusted through $\kappa$ , $b_{j}$ , $\psi_{j}$ , $I_{j}$ , $\theta_{j}$ , and $\mathcal{R}$ , respectively.

Figure 3: Layered construction of a GNBG-C benchmark instance. The GNBG objective layer defines the base optimization landscape and the known reference optimum $x^{\star}$ [ 9 , 10 ] , while the GNBG-C constraint layer adds configurable feasibility-side structure around $x^{\star}$ . Their composition yields a constrained GNBG instance in which objective-side properties and feasibility-side properties can be configured, diagnosed, and inspected separately.

Element	Controlled benchmark property
$\kappa$	Shared calibration of feasible-region size through positive-slack constraints; zero-slack active constraints are unaffected.
$b_{j}$	Local activity and margin of constraint $j$ at $x^{\star}$ .
$\psi_{j}$	Boundary family and geometric complexity.
$I_{j}$	Sparse or dense variable participation in constraint $j$ .
$\theta_{j}$	Family parameters, coefficient magnitudes, and numerical scaling.
$\mathcal{R}$	Redundant constraints that increase constraint-handling burden without tightening the feasible set.

Table 1: Parameter map of the centered GNBG-C constraint layer and the benchmark property controlled by each element.

Family	Centered raw form $\psi_{j}(z;\theta_{j})$
Linear	$a_{j}^{\top}z$
Convex quadratic	$a_{j}^{\top}z+z^{\top}Q_{j}z,\;Q_{j}\succeq 0$
Nonconvex quadratic	$a_{j}^{\top}z+z^{\top}Q_{j}z,\;Q_{j}$ indefinite
Polynomial	$\sum_{r=1}^{R_{j}}c_{jr}(u_{jr}^{\top}z)^{p_{jr}}$
Transcendental	$\sum_{r=1}^{R_{j}}c_{jr}[\sin(\omega_{jr}u_{jr}^{\top}z+\phi_{jr})-\sin(\phi_{jr})]$

Table 2: Implemented GNBG-C constraint families through the centered function

\psi_{j}

Panel of Figure 4 — Figure 4: Two-dimensional views of feasibility-ratio calibration, active-set pressure, and redundancy. The first row varies the target feasible fraction, the second varies the number of constraints active at $x^{\star}$ while keeping the achieved fraction comparable, and the third adds implied constraints without changing sampled feasibility; $\mathrm{changed}=0$ indicates that no sampled classification changed.

Panel of Figure 5 — Figure 5: Effect of constraint-family choice on feasible-region geometry.

Panel of Figure 6 — Figure 6: Variable-participation patterns induced by different involvement rates. Filled cells indicate that variable $x_{i}$ participates in constraint $g_{j}$ , while the marginal bars summarize participation across variables.

Panel of Figure 7 — Figure 7: Numerical-scaling diagnostics under balanced and ill-scaled coefficient regimes. Panel (a) reports $\log_{10}(|g_{j}(x)|+\varepsilon)$ , whereas panel (b) reports $\log_{10}(\mathrm{CV}(x)+\varepsilon)$ . Here, $\varepsilon=10^{-12}$ is introduced only to avoid taking the logarithm of zero and does not affect feasibility classification.

Item	Value
Dimension and GNBG components	$d=10,\ o=3$
Feasibility-side regimes	8
Instances per regime	15
Runs per algorithm–instance	30
Runs per algorithm–regime	450
Algorithms	8

Table 3: Experimental protocol for the regime-wise optimizer study.

Algorithm	Backend	Base	Constraint handling
DE-FF	pymoo [ 2 ]	DE [ 29 ]	feasibility-first constrained comparison [ 6 ]
DE-EPS	pymoo	DE	adaptive $\epsilon$ -constraint handling [ 30 ]
DE-PEN	pymoo	DE	static penalty [ 5 ]
PSO-FF	pymoo	PSO [ 18 ]	feasibility-first constrained comparison
GA-FF	pymoo	GA [ 15 ]	feasibility-first constrained comparison
SRES	pymoo	SRES [ 24 ]	stochastic ranking

Table 4: Baseline optimizers and constraint-handling strategies.

Regime	Total (P+R)	Family mix	Support / $r_{\mathrm{inv}}$	Scaling	Active	Target $\rho$	Calib. tol.
Loose	10 (9+1)	2/2/2/2/1	2 / 0.20	Balanced	1	0.300	0.050
Medium	10 (9+1)	2/2/2/2/1	2 / 0.20	Balanced	2	0.050	0.025
Strict	10 (9+1)	2/2/2/2/1	7–10 / 0.70–1.00	Balanced	3	0.008	0.004
Active-high	10 (10+0)	2/2/2/2/2	2 / 0.20	Balanced	5	0.005	0.002
Mostly nonlinear	10 (9+1)	1/1/2/3/2	2 / 0.20	Balanced	2	0.020	0.010
Dense	12 (11+1)	3/2/2/2/2	7–10 / 0.70–1.00	Balanced	3	0.002	0.002

Table 5: Effective configurations of the eight regimes used in the baseline study.

Algorithm	Gap at 2,500 FE	Gap at 5,000 FE	Gap at 10,000 FE	Feasible runs	First feasible FE	AR25 (%)	AR50 (%)
DE-FF	89.30 (12.58)	67.72 (14.71)	49.13 (17.84)	450/450/450	8.34 (17.08)	3.6	58.7
DE-EPS	89.23 (12.96)	67.76 (14.08)	47.26 (17.32)	450/450/450	8.58 (17.65)	2.7	64.7
DE-PEN	86.46 (13.03)	66.18 (14.37)	48.31 (17.77)	450/450/450	9.52 (20.59)	4.9	61.6
PSO-FF	63.24 (24.94)	52.04 (28.68)	45.69 (31.79)	450/450/450	9.91 (22.08)	33.3	58.4
GA-FF	66.43 (15.74)	49.56 (20.75)	43.94 (23.50)	450/450/450	10.93 (30.38)	27.3	59.6
SRES	78.01 (13.84)	45.04 (17.16)	22.11 (21.84)	450/450/450	10.32 (24.25)	73.6	80.2

Table 6: Optimizer results for the loose regime.

Algorithm	Gap at 2,500 FE	Gap at 5,000 FE	Gap at 10,000 FE	Feasible runs	First feasible FE	AR25 (%)	AR50 (%)
DE-FF	89.51 (15.52)	66.98 (15.60)	47.67 (17.53)	450/450/450	22.21 (20.49)	4.4	70.2
DE-EPS	91.33 (16.97)	67.53 (17.56)	46.35 (19.68)	450/450/450	25.89 (28.13)	4.9	73.6
DE-PEN	87.14 (14.93)	66.15 (16.19)	48.09 (19.58)	450/450/450	23.97 (24.01)	2.2	72.9
PSO-FF	70.89 (30.27)	58.65 (33.58)	51.06 (36.46)	450/450/450	23.11 (24.21)	31.6	57.8
GA-FF	69.30 (18.59)	50.70 (21.43)	43.93 (23.84)	450/450/450	24.08 (25.36)	23.6	66.0
SRES	80.43 (15.09)	46.75 (16.88)	21.25 (21.17)	450/450/450	24.73 (26.53)	77.1	88.4

Table 7: Optimizer results for the medium regime.

Algorithm	Gap at 2,500 FE	Gap at 5,000 FE	Gap at 10,000 FE	Feasible runs	First feasible FE	AR25 (%)	AR50 (%)
DE-FF	106.58 (18.93)	84.85 (18.85)	66.55 (20.98)	450/450/450	244.08 (312.32)	0.0	25.8
DE-EPS	108.91 (21.66)	81.59 (20.31)	57.20 (21.58)	449/450/450	307.21 (472.34)	0.2	43.6
DE-PEN	106.30 (18.78)	84.73 (19.69)	66.50 (21.63)	450/450/450	244.52 (310.39)	0.0	27.3
PSO-FF	93.39 (33.42)	79.63 (38.03)	70.91 (42.23)	450/450/450	160.58 (145.04)	20.4	39.8
GA-FF	92.47 (21.64)	72.70 (27.11)	64.69 (30.58)	450/450/450	211.91 (240.62)	6.4	38.0
SRES	96.28 (17.67)	60.22 (20.29)	30.27 (27.74)	450/450/450	250.99 (354.97)	68.9	79.3

Table 8: Optimizer results for the strict regime.

Figure 8: Feasibility discovery in the loose, medium, and strict regimes. Each curve reports the fraction of runs that have identified a feasible solution by the indicated evaluation count.

Algorithm	Gap at 2,500 FE	Gap at 5,000 FE	Gap at 10,000 FE	Feasible runs	First feasible FE	AR25 (%)	AR50 (%)
DE-FF	108.50 (20.29)	83.46 (20.16)	63.65 (21.54)	450/450/450	436.33 (421.51)	2.4	30.4
DE-EPS	118.53 (27.44)	92.69 (26.38)	61.29 (21.10)	422/449/449	683.45 (943.68)	3.1	35.6
DE-PEN	105.08 (20.22)	80.42 (20.16)	62.48 (22.21)	449/450/450	416.89 (405.76)	3.8	33.3
PSO-FF	96.32 (31.68)	83.07 (35.83)	73.75 (39.58)	433/434/434	268.89 (234.30)	12.7	33.6
GA-FF	92.83 (24.89)	75.97 (29.01)	69.85 (31.32)	450/450/450	388.08 (321.85)	8.4	30.4
SRES	103.75 (22.24)	65.42 (21.70)	40.26 (30.00)	446/450/450	495.66 (586.27)	46.7	67.8

Table 9: Optimizer results for the active-high regime.

Algorithm	Gap at 2,500 FE	Gap at 5,000 FE	Gap at 10,000 FE	Feasible runs	First feasible FE	AR25 (%)	AR50 (%)
DE-FF	97.42 (18.31)	72.31 (18.17)	51.45 (18.02)	450/450/450	62.37 (62.73)	0.2	61.3
DE-EPS	101.22 (20.46)	73.87 (20.27)	48.55 (19.47)	450/450/450	65.49 (77.97)	1.8	69.3
DE-PEN	94.65 (18.44)	69.84 (18.34)	50.20 (19.57)	450/450/450	66.66 (68.51)	2.0	62.0
PSO-FF	82.46 (35.00)	71.42 (38.32)	64.67 (41.18)	450/450/450	56.31 (49.54)	23.6	43.6
GA-FF	77.65 (21.00)	58.09 (25.96)	51.33 (29.08)	450/450/450	60.37 (64.07)	20.7	56.9
SRES	87.29 (16.18)	49.23 (15.82)	21.17 (19.91)	450/450/450	57.52 (60.62)	82.4	88.4

Table 10: Optimizer results for the mostly nonlinear regime.

Algorithm	Gap at 2,500 FE	Gap at 5,000 FE	Gap at 10,000 FE	Feasible runs	First feasible FE	AR25 (%)	AR50 (%)
DE-FF	104.14 (18.25)	81.30 (18.83)	61.27 (20.33)	444/449/450	366.17 (571.13)	0.0	30.9
DE-EPS	105.94 (20.37)	75.64 (21.67)	51.89 (21.54)	447/450/450	361.79 (434.65)	0.0	66.4
DE-PEN	103.39 (19.63)	80.55 (19.63)	60.35 (19.98)	449/450/450	333.38 (374.65)	0.0	33.8
PSO-FF	90.62 (35.94)	76.41 (41.19)	67.22 (47.59)	429/434/444	376.46 (996.00)	22.4	50.2
GA-FF	90.75 (22.71)	69.64 (27.56)	60.39 (31.42)	443/443/444	300.81 (390.97)	9.6	47.6
SRES	90.98 (16.52)	52.20 (16.02)	17.47 (14.27)	450/450/450	283.52 (285.29)	88.0	97.3

Table 11: Optimizer results for the dense regime.

Algorithm	Gap at 2,500 FE	Gap at 5,000 FE	Gap at 10,000 FE	Feasible runs	First feasible FE	AR25 (%)	AR50 (%)
DE-FF	107.07 (20.58)	87.08 (23.29)	65.80 (22.84)	393/431/450	683.05 (1554.64)	0.0	28.7
DE-EPS	108.96 (22.38)	86.14 (23.75)	60.79 (22.52)	382/406/449	901.26 (1971.38)	0.0	39.3
DE-PEN	106.51 (19.36)	86.04 (22.05)	64.36 (20.55)	398/436/449	640.58 (1457.02)	0.0	28.9
PSO-FF	95.25 (32.79)	82.83 (36.25)	74.69 (40.66)	431/435/435	245.88 (508.30)	12.2	30.2
GA-FF	89.57 (23.82)	74.11 (29.47)	67.27 (32.25)	401/432/441	507.66 (1223.75)	9.3	35.1
SRES	100.44 (21.43)	65.89 (25.13)	36.76 (27.77)	393/445/449	572.97 (1216.40)	49.8	70.4

Table 12: Optimizer results for the ill-scaled regime.

Algorithm	Gap at 2,500 FE	Gap at 5,000 FE	Gap at 10,000 FE	Feasible runs	First feasible FE	AR25 (%)	AR50 (%)
DE-FF	98.44 (14.88)	74.40 (17.27)	52.83 (21.00)	450/450/450	35.42 (55.56)	0.4	63.6
DE-EPS	100.99 (16.13)	75.44 (18.14)	49.41 (20.60)	450/450/450	34.32 (46.19)	3.1	66.2
DE-PEN	95.17 (14.73)	72.07 (16.62)	51.32 (20.56)	450/450/450	35.37 (54.12)	1.6	63.3
PSO-FF	80.97 (32.71)	65.94 (38.28)	56.33 (42.98)	450/450/450	33.93 (48.78)	35.8	52.9
GA-FF	77.23 (18.82)	55.47 (25.56)	47.00 (29.34)	450/450/450	38.26 (60.53)	31.3	61.8
SRES	88.55 (14.77)	51.63 (18.57)	22.47 (24.78)	450/450/450	37.42 (63.07)	82.9	85.6

Table 13: Optimizer results for the high-redundancy regime.

Figure 9: Best-feasible-gap convergence across the eight generated regimes. The curves summarize feasible objective improvement over the evaluation budget; lower values indicate better progress.

Figure 10: Final median best-feasible gap across algorithms and generated regimes. Cell labels report the raw median best-feasible gaps, whereas the cell colours encode the logarithmically transformed values, $\log_{10}\!\left(\max\!\left\{\widetilde{\Delta f}_{\mathrm{bf}},\,10^{-12}\right\}\right)$ , where $\widetilde{\Delta f}_{\mathrm{bf}}$ denotes the median best-feasible gap for the corresponding algorithm–regime pair. The numerical floor of $10^{-12}$ is applied only to enable zero values to be displayed on the logarithmic colour scale. Lower raw values indicate better feasible objective quality at the end of the evaluation budget.

Regime	Target $\rho$	Achieved $\hat{\rho}$	Active req./real.	Target reached	Valid
Loose	0.300	0.2384 (0.1029)	1/1.0	12/15	15/15
Medium	0.050	0.0440 (0.0096)	2/2.0	15/15	15/15
Strict	0.008	0.0060 (0.0043)	3/3.0	10/15	15/15
Active-high	0.005	0.0027 (0.0025)	5/5.0	8/15	15/15
Mostly nonlinear	0.020	0.0166 (0.0052)	2/2.0	14/15	15/15
Dense	0.002	0.0016 (0.0009)	3/3.0	15/15	15/15

Table 14: Realized feasibility-ratio and active-set diagnostics for the

d=10

regimes; achieved ratios are mean (standard deviation) over 15 instances.

Synthetic Benchmark Generation

Synthetic benchmark generation creates controlled, verifiable evaluation tasks and datasets to test models across diverse settings.

See all 287 papers

ConstraintBench: Benchmarking LLM Constraint Reasoning on Direct Optimization
Feb 25, 2026Joseph Tso, Preston Schmittou, Quan Huynh +1Constraint-Guided Reasoning in Language Models Constrained Optimization
Haladir Research Team
Constraint-Bound Agnostic Bayesian Optimization: One Model for All Thresholds
Jul 26, 2026Jin Wang, Xi Lin, Handing WangBayesian Optimization Constrained Optimization
School of Artificial Intelligence, Xidian University, China · School of Mathematics and Statistics, Xi’an Jiaotong University, China
Constrained Bayesian Optimisation with Multiple Information Sources
Jul 1, 2026Hauke Maathuis, Roeland De Breuker, Saullo Castro +1Bayesian Optimization
TU Delft · University of Oxford

Feb 25, 2026J/K move · Enter open · S save

ConstraintBench: Benchmarking LLM Constraint Reasoning on Direct Optimization

Joseph Tso, Preston Schmittou, Quan Huynh, Jibran Hutchins

Haladir Research Team

Constraint-Guided Reasoning in Language Models Constrained Optimization LLM Reasoning Operations Research

Open paper AskPDF arXiv

Large language models are increasingly applied to operational decision-making where the underlying structure is constrained optimization. Existing benchmarks evaluate whether LLMs can formulate optimization problems as solver code, but leave open a complementary question. Can LLMs directly produce correct solutions to fully specified constrained optimization problems without access to a solver? We introduce ConstraintBench, a benchmark for evaluating LLMs on direct constrained optimization across 10 operations research domains, with all ground-truth solutions verified by the Gurobi solver. Each task presents a natural-language scenario with entities, constraints, and an optimization objective; the model must return a structured solution that a deterministic verifier checks against every constraint and the solver-proven optimum. We evaluate six frontier models on 200 tasks and find that feasibility, not optimality, is the primary bottleneck. The best model achieves only 65.0% feasibility, yet feasible solutions average 89 to 96% of the Gurobi-optimal objective. No model exceeds 30.5% on joint feasibility and optimality within 0.1% of the solver reference. Per-domain analysis shows large variation in difficulty, with average feasibility spanning from 85.0% in the facility location domain to 0.8% in the crew assignment domain. Further, systematic failure modes include duration constraint misunderstanding, entity hallucination, and a feasibility-optimality decoupling in facility location and vehicle routing where models achieve high feasibility but 0% optimality. ConstraintBench and all evaluation infrastructure will be publicly released.

Abstract

Synthetic Benchmark Generation

Explore similar work

ConstraintBench: Benchmarking LLM Constraint Reasoning on Direct Optimization

Constraint-Bound Agnostic Bayesian Optimization: One Model for All Thresholds

Constrained Bayesian Optimisation with Multiple Information Sources