In optimization problems, some variable subsets may have a joint non-linear or non-monotonical influence on the function value. Therefore, knowledge of variable dependencies may be crucial for effective optimization, and many state-of-the-art optimizers leverage it to improve performance. However, some real-world problem instances may be the subject of noise of various origins. In such a case, variable dependencies relevant to optimization may be hard or impossible to tell using dependency checks sufficient for problems without noise, making highly effective operators, e.g., Partition Crossover (PX), useless. Therefore, we use Statistical Linkage Learning (SLL) to decompose problems with noise and propose a new SLL-dedicated mask construction algorithm. We prove that if the quality of the SLL-based decomposition is sufficiently high, the proposed clustering algorithm yields masks equivalent to PX masks for the noise-free instances. The experiments show that the optimizer using the proposed mechanisms remains equally effective despite the noise level and outperforms state-of-the-art optimizers for the problems with high noise.
Figures & tables
x1
x2
x3
x4
x5
x6
x7
x8
x9
x1
X
1
1
1
0
0
0
0
0
x2
1
X
1
1
0
0
0
0
0
x3
1
1
X
1
0
0
0
0
0
x4
1
1
1
X
1
1
1
0
0
x5
0
0
0
1
X
1
1
0
0
x6
0
0
0
1
1
X
1
1
1
Table 1. VIG for fe1
x1
x2
x3
x4
x5
x6
x7
x8
x9
x1
X
0
0
0
0
0
0
0
0
x2
0
X
1
0
0
0
0
0
0
x3
0
1
X
0
0
0
0
0
0
x4
0
0
0
X
0
0
0
0
0
x5
0
0
0
0
X
1
0
0
0
x6
0
0
0
0
1
X
0
1
0
Table 2. Remaining VIG for the PX mask creation example
x1
x2
x3
x4
x5
x6
x7
x8
x9
x1
X
0.99
0.51
0.51
0.25
0.25
0.25
0.25
0.25
x2
0.99
X
0.51
0.51
0.25
0.25
0.25
0.25
0.25
x3
0.51
0.51
X
0.99
0.49
0.50
0.50
0.50
0.50
x4
0.51
0.51
0.99
X
0.60
0.98
0.73
0.50
0.50
x5
0.25
0.25
0.49
0.60
X
0.60
0.60
0.49
0.50
x6
0.25
0.25
0.50
0.98
0.60
X
0.73
0.73
0.55
Table 3. Exemplary D∗ for fe1
Figure 1. LTs for D∗ in Table 3
Figure 2. LTopWS strategy for choosing PX-LT nodes
Figure 3. Ranking scalability. X-axis represents Nsize as a percentage of the total length of the genotype, i.e., for 100%, Nsize=n
P3-PX-OM-LTopWS
P3-PX-OM-All
P3
Opt
FFE
Opt
FFE
Opt
FFE
Size
[%]
Med
Size
[%]
Med
Size
[%]
Med
nkLand
800
100
6.6E+6
200
100
4.7E+6
600
80
8.6E+6
ISG
841
100
5.1E+5
841
97
4.9E+6
841
100
5.7E+5
m3s
75
80
9.0E+4
50
93
3.1E+4
100
93
2.0E+5
Bim10
200
93
3.6E+6
100
90
9.8E+6
80
100
5.0E+6
Table 4. P3-based SLL-using optimizers comparison
Size
PX[%]
Size
PX[%]
Size
PX[%]
nkL
400
13.84
Bim10
100
10.96
nBim10
80
1.89
ISG
400
53.30
Bim10o1
72
8.29
nBim10o1
45
0.96
m3s
75
32.29
Bim10o2
72
7.73
nBim10o2
48
1.15
Dec5
200
37.33
Bim10o3
49
8.13
nBim10o3
49
2.26
Table 5. The median of the average percentage of PX masks in PX-LT trees for various problems in the PX-OM-LTopWS runs ( Nsize equal to 100% of the problem size).
We study the problem of black-box optimization of a function f of any dimension, given function evaluations perturbed by noise. The function is assumed to be locally smooth around one of its global optima, but this smoothness is unknown. Our contribution is an adaptive optimization algorithm, POO or parallel optimistic optimization, that is able to deal with this setting. POO performs almost as well as the best known algorithms requiring the knowledge of the smoothness. Furthermore, POO works for a larger class of functions than what was previously considered, especially for functions that are difficult to optimize, in a very precise sense. We provide a finite-time analysis of POO's performance, which shows that its error after n evaluations is at most a factor of sqrt(ln n) away from the error of the best known optimization algorithms using the knowledge of the smoothness.
Jean-Bastien Grill, Michal Valko, Rémi Munos
SequeL team, INRIA Lille - Nord Europe, France · Google DeepMind, UK∗
The rapid growth of nature-inspired metaheuristics has exposed a persistent gap between metaphorical novelty and genuine algorithmic advancement. Motivated by the biophysics of chromatin loop extrusion -- a well-characterized genome-folding process driven by SMC motor complexes and conditional barriers -- we introduce the Loop-Extrusion Linkage (LEL) operator, a structure-learning wrapper that combines online variable-interaction estimation, spectral seriation via the Fiedler vector, and adaptive interval-based subspace search. LEL constructs a sparse interaction graph from successful optimization steps, derives a heuristic one-dimensional variable ordering, and generates overlapping evaluation subsets through stochastic interval growth modulated by learned boundary-crossing probabilities. We evaluate LEL on six synthetic diagnostic functions at d=96 designed to probe specific structural hypotheses -- contiguous blocks, permuted blocks, overlapping windows, banded chains, separable controls, and dense rotated couplings -- across 10^4 and 5 x 10^4 evaluation budgets with 15 independent seeds. Results are assessed via the Wilcoxon signed-rank test with Holm-Bonferroni correction and Vargha-Delaney A12 effect sizes. At 10^4 evaluations, Full LEL achieves the best median log-gap on 3 of 6 functions significantly outperforming all ablations and jSO on the structured tasks. At 5 x 10^4 evaluations, simpler ablations and baselines often surpass the full method, indicating that the adaptive barrier mechanism may over-constrain late-stage search on uniformly partitioned landscapes. The strongest supported finding is that learned spectral ordering consistently improves over graph-only grouping and random variable ordering, suggesting that interaction-graph seriation is the most valuable component of the proposed framework.
We propose Dynamic Constraint Learning (DCL), a data-driven framework for constrained optimization when constraint functions are unknown and cannot be queried during optimization. At each iteration, the method learns a local surrogate from nearby data and solves a subproblem within a data-supported trust region. Compared with offline global constraint learning, the approach uses local surrogates that adapt to the data distribution during optimization and can achieve solution quality comparable to that of global models while using simpler local models and smaller optimization subproblems. We demonstrate the performance of DCL on a synthetic test problem and two case studies from the literature.
Ezgi Oztekin, Figen Oztoprak, S. Ilker Birbil
Department of Mathematics, Gebze Technical University · Department of Industrial Engineering, Gebze Technical University · Amsterdam Business School, University of Amsterdam