GNBG-C: A Unified and Configurable Benchmark Generator for Constrained Continuous Optimization
Authors: Ali Ghasemzadeh, Erfan Babakhani, Armin Khosravi, Pariya Ghasemzadeh, Yong-Hong Kuo, Seyedali Mirjalili, Amir H. Gandomi
Organizations: Department of Data and Systems Engineering, the University of Hong Kong, Hong Kong SAR, China · Independent Researcher, Isfahan, Iran · Centre for Artificial Intelligence Research and Optimisation, Torrens University Australia, Brisbane, Australia · Faculty of Electrical Engineering and Computer Science, VSB Technical University of Ostrava, Ostrava, Czech Republic · Obuda University, Budapest, Hungary · University of Technology Sydney, Sydney, Australia
Many real-world optimization tasks are governed not only by what should be optimized, but also by what is allowed: a solution must first satisfy physical, operational, safety, budgetary, or structural requirements before its objective value becomes meaningful. Yet benchmark design for constrained continuous optimization often treats these restrictions as fixed properties of predefined test instances. Existing generators and benchmark suites provide essential foundations for reproducible comparison, but they either focus primarily on configurable objective landscapes or expose constraint properties within limited, predefined regimes, making it difficult to ask targeted questions about constraint-induced difficulty: whether an optimizer struggles because feasible regions are small, boundaries are restrictive, variables are tightly coupled, or constraint information is poorly scaled. We introduce GNBG-C, a Generalized Numerical Benchmark Generator for Constrained Continuous Optimization. The central idea is to treat the feasible region itself as a configurable object rather than as an inseparable attachment to a fixed test function. By generating controlled constraint regimes around the known GNBG global minimizer, the generator enables systematic studies in which feasibility pressure can be varied while preserving that point as a global minimizer of the constrained problem. This allows researchers to examine how optimizers discover feasibility, handle constraint pressure, and improve objective quality under diverse and controllable feasibility conditions. GNBG-C therefore provides a flexible and auditable framework for probing the strengths and weaknesses of constrained continuous optimizers under targeted forms of feasibility-side difficulty.
Figures & tables
Figure 1: Core idea of GNBG-C. The objective landscape is kept fixed, while the generated constraint layer induces different feasible regions, such as loose, and strict regimes. This separates objective-landscape difficulty from feasible-region difficulty. The two-dimensional view is for interpretability; GNBG-C supports arbitrary user-specified dimensionality.
Figure 2: Conceptual view of GNBG-C as a configurable feasible-region layer attached to a GNBG objective landscape. Starting from a fixed objective landscape and a known GNBG optimum x⋆ , the constraint layer acts as a controllable shell whose size, activity at the optimum, boundary geometry, variable participation, numerical scale, and redundancy can be adjusted through κ , bj , ψj , Ij , θj , and R , respectively.
Figure 3: Layered construction of a GNBG-C benchmark instance. The GNBG objective layer defines the base optimization landscape and the known reference optimum x⋆ [ 9 , 10 ] , while the GNBG-C constraint layer adds configurable feasibility-side structure around x⋆ . Their composition yields a constrained GNBG instance in which objective-side properties and feasibility-side properties can be configured, diagnosed, and inspected separately.
Element
Controlled benchmark property
κ
Shared calibration of feasible-region size through positive-slack constraints; zero-slack active constraints are unaffected.
bj
Local activity and margin of constraint j at x⋆ .
ψj
Boundary family and geometric complexity.
Ij
Sparse or dense variable participation in constraint j .
θj
Family parameters, coefficient magnitudes, and numerical scaling.
R
Redundant constraints that increase constraint-handling burden without tightening the feasible set.
Table 1: Parameter map of the centered GNBG-C constraint layer and the benchmark property controlled by each element.
Family
Centered raw form ψj(z;θj)
Linear
aj⊤z
Convex quadratic
aj⊤z+z⊤Qjz,Qj⪰0
Nonconvex quadratic
aj⊤z+z⊤Qjz,Qj indefinite
Polynomial
∑r=1Rjcjr(ujr⊤z)pjr
Transcendental
∑r=1Rjcjr[sin(ωjrujr⊤z+ϕjr)−sin(ϕjr)]
Table 2: Implemented GNBG-C constraint families through the centered function ψj .
Figure 4: Two-dimensional views of feasibility-ratio calibration, active-set pressure, and redundancy. The first row varies the target feasible fraction, the second varies the number of constraints active at x⋆ while keeping the achieved fraction comparable, and the third adds implied constraints without changing sampled feasibility; changed=0 indicates that no sampled classification changed.
Figure 5: Effect of constraint-family choice on feasible-region geometry.
Figure 6: Variable-participation patterns induced by different involvement rates. Filled cells indicate that variable xi participates in constraint gj , while the marginal bars summarize participation across variables.
Figure 7: Numerical-scaling diagnostics under balanced and ill-scaled coefficient regimes. Panel (a) reports log10(∣gj(x)∣+ε) , whereas panel (b) reports log10(CV(x)+ε) . Here, ε=10−12 is introduced only to avoid taking the logarithm of zero and does not affect feasibility classification.
Item
Value
Dimension and GNBG components
d=10,o=3
Feasibility-side regimes
8
Instances per regime
15
Runs per algorithm–instance
30
Runs per algorithm–regime
450
Algorithms
8
Table 3: Experimental protocol for the regime-wise optimizer study.
Algorithm
Backend
Base
Constraint handling
DE-FF
pymoo [ 2 ]
DE [ 29 ]
feasibility-first constrained comparison [ 6 ]
DE-EPS
pymoo
DE
adaptive ϵ -constraint handling [ 30 ]
DE-PEN
pymoo
DE
static penalty [ 5 ]
PSO-FF
pymoo
PSO [ 18 ]
feasibility-first constrained comparison
GA-FF
pymoo
GA [ 15 ]
feasibility-first constrained comparison
SRES
pymoo
SRES [ 24 ]
stochastic ranking
Table 4: Baseline optimizers and constraint-handling strategies.
Regime
Total (P+R)
Family mix
Support / rinv
Scaling
Active
Target ρ
Calib. tol.
Loose
10 (9+1)
2/2/2/2/1
2 / 0.20
Balanced
1
0.300
0.050
Medium
10 (9+1)
2/2/2/2/1
2 / 0.20
Balanced
2
0.050
0.025
Strict
10 (9+1)
2/2/2/2/1
7–10 / 0.70–1.00
Balanced
3
0.008
0.004
Active-high
10 (10+0)
2/2/2/2/2
2 / 0.20
Balanced
5
0.005
0.002
Mostly nonlinear
10 (9+1)
1/1/2/3/2
2 / 0.20
Balanced
2
0.020
0.010
Dense
12 (11+1)
3/2/2/2/2
7–10 / 0.70–1.00
Balanced
3
0.002
0.002
Table 5: Effective configurations of the eight regimes used in the baseline study.
Algorithm
Gap at 2,500 FE
Gap at 5,000 FE
Gap at 10,000 FE
Feasible runs
First feasible FE
AR25 (%)
AR50 (%)
DE-FF
89.30 (12.58)
67.72 (14.71)
49.13 (17.84)
450/450/450
8.34 (17.08)
3.6
58.7
DE-EPS
89.23 (12.96)
67.76 (14.08)
47.26 (17.32)
450/450/450
8.58 (17.65)
2.7
64.7
DE-PEN
86.46 (13.03)
66.18 (14.37)
48.31 (17.77)
450/450/450
9.52 (20.59)
4.9
61.6
PSO-FF
63.24 (24.94)
52.04 (28.68)
45.69 (31.79)
450/450/450
9.91 (22.08)
33.3
58.4
GA-FF
66.43 (15.74)
49.56 (20.75)
43.94 (23.50)
450/450/450
10.93 (30.38)
27.3
59.6
SRES
78.01 (13.84)
45.04 (17.16)
22.11 (21.84)
450/450/450
10.32 (24.25)
73.6
80.2
Table 6: Optimizer results for the loose regime.
Algorithm
Gap at 2,500 FE
Gap at 5,000 FE
Gap at 10,000 FE
Feasible runs
First feasible FE
AR25 (%)
AR50 (%)
DE-FF
89.51 (15.52)
66.98 (15.60)
47.67 (17.53)
450/450/450
22.21 (20.49)
4.4
70.2
DE-EPS
91.33 (16.97)
67.53 (17.56)
46.35 (19.68)
450/450/450
25.89 (28.13)
4.9
73.6
DE-PEN
87.14 (14.93)
66.15 (16.19)
48.09 (19.58)
450/450/450
23.97 (24.01)
2.2
72.9
PSO-FF
70.89 (30.27)
58.65 (33.58)
51.06 (36.46)
450/450/450
23.11 (24.21)
31.6
57.8
GA-FF
69.30 (18.59)
50.70 (21.43)
43.93 (23.84)
450/450/450
24.08 (25.36)
23.6
66.0
SRES
80.43 (15.09)
46.75 (16.88)
21.25 (21.17)
450/450/450
24.73 (26.53)
77.1
88.4
Table 7: Optimizer results for the medium regime.
Algorithm
Gap at 2,500 FE
Gap at 5,000 FE
Gap at 10,000 FE
Feasible runs
First feasible FE
AR25 (%)
AR50 (%)
DE-FF
106.58 (18.93)
84.85 (18.85)
66.55 (20.98)
450/450/450
244.08 (312.32)
0.0
25.8
DE-EPS
108.91 (21.66)
81.59 (20.31)
57.20 (21.58)
449/450/450
307.21 (472.34)
0.2
43.6
DE-PEN
106.30 (18.78)
84.73 (19.69)
66.50 (21.63)
450/450/450
244.52 (310.39)
0.0
27.3
PSO-FF
93.39 (33.42)
79.63 (38.03)
70.91 (42.23)
450/450/450
160.58 (145.04)
20.4
39.8
GA-FF
92.47 (21.64)
72.70 (27.11)
64.69 (30.58)
450/450/450
211.91 (240.62)
6.4
38.0
SRES
96.28 (17.67)
60.22 (20.29)
30.27 (27.74)
450/450/450
250.99 (354.97)
68.9
79.3
Table 8: Optimizer results for the strict regime.
Figure 8: Feasibility discovery in the loose, medium, and strict regimes. Each curve reports the fraction of runs that have identified a feasible solution by the indicated evaluation count.
Algorithm
Gap at 2,500 FE
Gap at 5,000 FE
Gap at 10,000 FE
Feasible runs
First feasible FE
AR25 (%)
AR50 (%)
DE-FF
108.50 (20.29)
83.46 (20.16)
63.65 (21.54)
450/450/450
436.33 (421.51)
2.4
30.4
DE-EPS
118.53 (27.44)
92.69 (26.38)
61.29 (21.10)
422/449/449
683.45 (943.68)
3.1
35.6
DE-PEN
105.08 (20.22)
80.42 (20.16)
62.48 (22.21)
449/450/450
416.89 (405.76)
3.8
33.3
PSO-FF
96.32 (31.68)
83.07 (35.83)
73.75 (39.58)
433/434/434
268.89 (234.30)
12.7
33.6
GA-FF
92.83 (24.89)
75.97 (29.01)
69.85 (31.32)
450/450/450
388.08 (321.85)
8.4
30.4
SRES
103.75 (22.24)
65.42 (21.70)
40.26 (30.00)
446/450/450
495.66 (586.27)
46.7
67.8
Table 9: Optimizer results for the active-high regime.
Algorithm
Gap at 2,500 FE
Gap at 5,000 FE
Gap at 10,000 FE
Feasible runs
First feasible FE
AR25 (%)
AR50 (%)
DE-FF
97.42 (18.31)
72.31 (18.17)
51.45 (18.02)
450/450/450
62.37 (62.73)
0.2
61.3
DE-EPS
101.22 (20.46)
73.87 (20.27)
48.55 (19.47)
450/450/450
65.49 (77.97)
1.8
69.3
DE-PEN
94.65 (18.44)
69.84 (18.34)
50.20 (19.57)
450/450/450
66.66 (68.51)
2.0
62.0
PSO-FF
82.46 (35.00)
71.42 (38.32)
64.67 (41.18)
450/450/450
56.31 (49.54)
23.6
43.6
GA-FF
77.65 (21.00)
58.09 (25.96)
51.33 (29.08)
450/450/450
60.37 (64.07)
20.7
56.9
SRES
87.29 (16.18)
49.23 (15.82)
21.17 (19.91)
450/450/450
57.52 (60.62)
82.4
88.4
Table 10: Optimizer results for the mostly nonlinear regime.
Algorithm
Gap at 2,500 FE
Gap at 5,000 FE
Gap at 10,000 FE
Feasible runs
First feasible FE
AR25 (%)
AR50 (%)
DE-FF
104.14 (18.25)
81.30 (18.83)
61.27 (20.33)
444/449/450
366.17 (571.13)
0.0
30.9
DE-EPS
105.94 (20.37)
75.64 (21.67)
51.89 (21.54)
447/450/450
361.79 (434.65)
0.0
66.4
DE-PEN
103.39 (19.63)
80.55 (19.63)
60.35 (19.98)
449/450/450
333.38 (374.65)
0.0
33.8
PSO-FF
90.62 (35.94)
76.41 (41.19)
67.22 (47.59)
429/434/444
376.46 (996.00)
22.4
50.2
GA-FF
90.75 (22.71)
69.64 (27.56)
60.39 (31.42)
443/443/444
300.81 (390.97)
9.6
47.6
SRES
90.98 (16.52)
52.20 (16.02)
17.47 (14.27)
450/450/450
283.52 (285.29)
88.0
97.3
Table 11: Optimizer results for the dense regime.
Algorithm
Gap at 2,500 FE
Gap at 5,000 FE
Gap at 10,000 FE
Feasible runs
First feasible FE
AR25 (%)
AR50 (%)
DE-FF
107.07 (20.58)
87.08 (23.29)
65.80 (22.84)
393/431/450
683.05 (1554.64)
0.0
28.7
DE-EPS
108.96 (22.38)
86.14 (23.75)
60.79 (22.52)
382/406/449
901.26 (1971.38)
0.0
39.3
DE-PEN
106.51 (19.36)
86.04 (22.05)
64.36 (20.55)
398/436/449
640.58 (1457.02)
0.0
28.9
PSO-FF
95.25 (32.79)
82.83 (36.25)
74.69 (40.66)
431/435/435
245.88 (508.30)
12.2
30.2
GA-FF
89.57 (23.82)
74.11 (29.47)
67.27 (32.25)
401/432/441
507.66 (1223.75)
9.3
35.1
SRES
100.44 (21.43)
65.89 (25.13)
36.76 (27.77)
393/445/449
572.97 (1216.40)
49.8
70.4
Table 12: Optimizer results for the ill-scaled regime.
Algorithm
Gap at 2,500 FE
Gap at 5,000 FE
Gap at 10,000 FE
Feasible runs
First feasible FE
AR25 (%)
AR50 (%)
DE-FF
98.44 (14.88)
74.40 (17.27)
52.83 (21.00)
450/450/450
35.42 (55.56)
0.4
63.6
DE-EPS
100.99 (16.13)
75.44 (18.14)
49.41 (20.60)
450/450/450
34.32 (46.19)
3.1
66.2
DE-PEN
95.17 (14.73)
72.07 (16.62)
51.32 (20.56)
450/450/450
35.37 (54.12)
1.6
63.3
PSO-FF
80.97 (32.71)
65.94 (38.28)
56.33 (42.98)
450/450/450
33.93 (48.78)
35.8
52.9
GA-FF
77.23 (18.82)
55.47 (25.56)
47.00 (29.34)
450/450/450
38.26 (60.53)
31.3
61.8
SRES
88.55 (14.77)
51.63 (18.57)
22.47 (24.78)
450/450/450
37.42 (63.07)
82.9
85.6
Table 13: Optimizer results for the high-redundancy regime.
Figure 9: Best-feasible-gap convergence across the eight generated regimes. The curves summarize feasible objective improvement over the evaluation budget; lower values indicate better progress.
Figure 10: Final median best-feasible gap across algorithms and generated regimes. Cell labels report the raw median best-feasible gaps, whereas the cell colours encode the logarithmically transformed values, log10(max{Δfbf,10−12}) , where Δfbf denotes the median best-feasible gap for the corresponding algorithm–regime pair. The numerical floor of 10−12 is applied only to enable zero values to be displayed on the logarithmic colour scale. Lower raw values indicate better feasible objective quality at the end of the evaluation budget.
Regime
Target ρ
Achieved ρ^
Active req./real.
Target reached
Valid
Loose
0.300
0.2384 (0.1029)
1/1.0
12/15
15/15
Medium
0.050
0.0440 (0.0096)
2/2.0
15/15
15/15
Strict
0.008
0.0060 (0.0043)
3/3.0
10/15
15/15
Active-high
0.005
0.0027 (0.0025)
5/5.0
8/15
15/15
Mostly nonlinear
0.020
0.0166 (0.0052)
2/2.0
14/15
15/15
Dense
0.002
0.0016 (0.0009)
3/3.0
15/15
15/15
Table 14: Realized feasibility-ratio and active-set diagnostics for the d=10 regimes; achieved ratios are mean (standard deviation) over 15 instances.
Large language models are increasingly applied to operational decision-making where the underlying structure is constrained optimization. Existing benchmarks evaluate whether LLMs can formulate optimization problems as solver code, but leave open a complementary question. Can LLMs directly produce correct solutions to fully specified constrained optimization problems without access to a solver? We introduce ConstraintBench, a benchmark for evaluating LLMs on direct constrained optimization across 10 operations research domains, with all ground-truth solutions verified by the Gurobi solver. Each task presents a natural-language scenario with entities, constraints, and an optimization objective; the model must return a structured solution that a deterministic verifier checks against every constraint and the solver-proven optimum. We evaluate six frontier models on 200 tasks and find that feasibility, not optimality, is the primary bottleneck. The best model achieves only 65.0% feasibility, yet feasible solutions average 89 to 96% of the Gurobi-optimal objective. No model exceeds 30.5% on joint feasibility and optimality within 0.1% of the solver reference. Per-domain analysis shows large variation in difficulty, with average feasibility spanning from 85.0% in the facility location domain to 0.8% in the crew assignment domain. Further, systematic failure modes include duration constraint misunderstanding, entity hallucination, and a feasibility-optimality decoupling in facility location and vehicle routing where models achieve high feasibility but 0% optimality. ConstraintBench and all evaluation infrastructure will be publicly released.
Expensive constrained optimization problems in real-world industry design often involve constraint thresholds that are difficult to determine in advance. Engineers may need to adjust constraint thresholds to explore different feasibility-performance trade-offs, requiring solutions under a wide range of threshold settings. However, existing constrained Bayesian optimization methods treat each threshold configuration independently, leading to repeated optimization and failing to exploit the shared relationship among continuously varying thresholds. To address this challenge, we propose constraint-bound agnostic Bayesian optimization (CBA-BO), a learning-based framework that learns a parametric constraint model mapping thresholds to optimal solutions. Once learned, CBA-BO directly predicts solutions for arbitrary unseen threshold configurations without additional optimization, with a one-step Bayesian optimization refinement further improving solution quality. Experiments on benchmark and engineering problems demonstrate that CBA-BO learns a transferable threshold-solution mapping, enabling efficient prediction and optimization for arbitrary threshold queries. An intent-guided constraint-bound recommendation mechanism is further developed to improve objective performance while satisfying user-specified constraint preferences.
Jin Wang, Xi Lin, Handing Wang
School of Artificial Intelligence, Xidian University, China · School of Mathematics and Statistics, Xi’an Jiaotong University, China
Bayesian Optimisation (BO) under unknown constraints is particularly challenging when feasible regions are small. In such settings, existing methods that typically rely solely on evaluations of the true objective and constraints struggle to efficiently explore the design space. However, many real-world applications offer auxiliary data sources (e.g. surrogate models or simplified simulations) that can support early exploration. Despite this potential, their integration into constrained BO remains largely unexplored. We propose a general multi-source framework that extends constrained Max-value Entropy Search, capturing inter-source correlation while balancing evaluation cost and information gain. Experiments on both synthetic and physics-based benchmarks show that our method efficiently identifies feasible and optimal solutions, even when auxiliary data are only weakly correlated. The proposed approach consistently outperforms existing methods, particularly in early-stage exploration.
Hauke Maathuis, Roeland De Breuker, Saullo Castro +1