NeuroPareto: Calibrated Acquisition for Costly Many-Goal Search in Vast Parameter Spaces

Rong Fu; Chunlei Meng; Haoyu Zhao; Kun Liu; Simon James Fong

Figure 1: Overview of the NeuroPareto framework for high-dimensional, budget-constrained multi-objective optimization. The pipeline consists of three synergistic modules: The Bayesian Rank Classifier (

g_{\theta}

) screens a massive candidate pool

\mathcal{C}{\mathrm{rank}}

using temperature-calibrated softmax

\tilde{p}k

and adaptive MC dropout to quantify classifier epistemic uncertainty

u{\mathrm{ep}}^{\mathrm{clf}}

. The Complexity-Reduced Deep GP pipeline processes the filtered subset, utilizing sparse variational inference with inducing variables

\mathbf{u}

and Randomized Fourier Features (RFF) to output predictive objective means

\hat{\mathbf{f}}(\mathbf{x})

alongside decomposed epistemic

u{\mathrm{ep}}^{\mathrm{gp}}

and aleatoric

u_{\mathrm{al}}^{\mathrm{gp}}

variances. The History-Aware Acquisition Network (

a_{\psi}

) aggregates these signals with sliding window statistics (

\mu_{\Delta\mathrm{HV}},\sigma_{\Delta\mathrm{HV}}

) to predict hypervolume utility

\hat{s}{\mathrm{HV}}

and diversity

\hat{s}{\mathrm{div}}

. The framework is updated online via a bounded history buffer

B

, ensuring a calibrated exploration-exploitation trade-off while minimizing wall-clock overhead through staged screening and warm-started variational parameters.

Problem	$D$	GP-EI	RF-EI	GP-HV	CL-EGO	CPS-MOEA	K-RVEA	CSEA	EDN-ARM.	MCEA/D	CLMEA	NeuroPareto
DTLZ1	30	459.23	495.56	437.65	402.38	612.31	595.74	534.58	761.09	372.68	303.10	286.40
	50	878.25	948.44	836.92	753.62	1171.0	1285.9	1108.8	1406.6	686.36	534.12	455.36
	100	2069.9	2235.3	1971.9	1774.5	2759.9	3038.4	2926.3	3187.8	1941.8	1074.0	933.03
	200	4427.3	4780.8	4216.5	3795.4	5903.0	6785.1	6346.1	6766.7	4003.8	1990.6	1620.0
DTLZ2	30	0.9051	0.9773	0.8623	0.7761	1.2068	1.2689	0.84305	1.3224	0.43816	0.09704	0.02901
	50	1.4239	1.5375	1.3565	1.2209	1.8985	2.9998	1.7832	2.8303	0.62911	0.13942	0.08929

Table 1: Comparative IGD Performance on Bi-objective DTLZ Problems (Optimal Values Bolded)

Problem	$D$	GP-EI	RF-EI	GP-HV	CL-EGO	CPS-MOEA	K-RVEA	CSEA	EDN-ARM.	MCEA/D	CLMEA	NeuroPareto
DTLZ1	30	387.89	418.87	369.53	332.58	517.18	578.16	466.71	596.32	389.53	286.40	216.20
	50	739.76	798.61	696.86	627.17	986.34	1128.2	929.14	1160.5	725.70	455.36	353.19
	100	1763.0	1903.6	1903.1	1712.8	2350.7	2674.7	2537.5	2637.3	2226.6	933.03	791.11
	200	3678.2	3971.0	4116.5	3704.8	4904.3	5736.9	5488.6	5662.8	4272.5	2162.0	1791.5
DTLZ2	30	1.0089	1.0892	0.9609	0.8648	1.3452	1.5993	0.91154	1.5621	0.50009	0.29013	0.18035
	50	1.5999	1.7273	1.5089	1.3580	2.1332	2.9820	2.0117	2.9442	0.66442	0.89291	0.21626

Table 2: IGD Performance on Three-Objective DTLZ Problems

Problem	$D$	GP-EI	RF-EI	GP-HV	CL-EGO	CPS-MOEA	K-RVEA	CSEA	EDN-ARM.	MCEA/D	CLMEA	NeuroPareto
ZDT1	30	1.5651	1.6899	1.4906	1.3415	2.0868	0.05732	1.0050	0.76113	1.9462	0.17209	0.02901
	50	1.6418	1.7729	1.5639	1.4075	2.1891	0.14374	1.2874	1.1993	2.3084	0.25621	0.08929
	100	1.8677	2.0168	1.7789	1.6010	2.4903	0.71883	2.0480	1.9585	2.4053	0.65192	0.17226
	200	1.9555	2.1115	1.8624	1.6762	2.6073	2.5471	2.2855	2.4627	2.5974	1.1857	0.31795
ZDT2	30	2.4419	2.6368	2.3259	2.0933	3.2559	0.07016	2.1445	1.5338	2.9519	0.00994	0.00631
	50	2.7863	3.0084	2.6536	2.3882	3.7150	0.13021	2.7810	2.1001	3.4710	0.01495	0.00893

Table 3: IGD Performance on Bi-objective ZDT Problems

Component	Parameter	Values examined
Bayesian classifier	Number of hidden layers	{1, 2}
	Hidden width	{64, 128}
	Dropout probability	{0.1, 0.2}
Deep GP surrogate	Inducing points $M_{\mathrm{ind}}$	{100, 200, 400, 800}
	Random Fourier feature dimension	{1024, 2048}
	ELBO validation threshold (%)	{1.0, 2.0}

Table 4: Hyper-parameter grids explored for NeuroPareto components. The final set was selected on DTLZ2-100D and applied unchanged across benchmarks.

Figure 2: Sensitivity of final hypervolume to inducing point count

M_{\mathrm{ind}}

on DTLZ2-100D. Error bars indicate standard error across five random seeds.

Figure 3: Convergence curves on Type A problems.

Figure 4: Convergence curves on 100D bi-objective problems. The results indicate faster convergence and higher final hypervolume across different problem instances.

Figure 5: Geothermal optimization results: (a) approximated Pareto fronts; (b) hypervolume progression over optimization iterations.

Figure 6: Evolution of average uncertainty during optimization on a 100D problem. Epistemic uncertainty decreases as the model learns, while aleatoric uncertainty reflects intrinsic problem variability.

Figure 7: Computational overhead distribution per iteration.

Figure 8: Convergence curves (IGD) over function evaluations. NeuroPareto demonstrates faster convergence and lower final IGD compared to the baselines.

Figure 9: Ablation study on 100D DTLZ2: contribution of each component to final IGD. Removing key modules degrades performance, showing their individual importance.

Figure 10: High-dimensional scaling: final IGD as a function of available budget (log-scale). The method remains stable up to 500D and large budgets (e.g., 5M evaluations).

Figure 11: Reliability diagram comparing temperature scaling and MC-Dropout. The closer the curve to the diagonal, the better calibrated the predictive uncertainties.

Figure 12: Pareto front approximation (2D): comparison between the true Pareto front and found solutions.

Configuration	IGD
Complete NeuroPareto	1.7226
Excluding Bayesian uncertainty quantification	2.4124
Substituting Deep GP with standard GP	3.1546
Without learned acquisition function	2.0342
Removing temperature scaling	1.8943
Original CLMEA [ 2 ]	3.8858

Table 5: Ablation on 100D DTLZ2 under 300-evaluation budget; w/o complexity uses full Deep-GP every iteration (no RFF/screening) while keeping sample budget fixed.

Pool size	Median HV	Standard error
200	0.836	0.009
500	0.851	0.008
1,000	$\mathbf{0.873}$	0.007
2,000	0.870	0.008

Table 6: Median hypervolume (HV) after 300 evaluations on 100-D DTLZ2 for different candidate pool sizes. The number of expensive full GP evaluations is fixed.

Figure 13: Relationship between proxy ranking and the true hypervolume contribution for a large candidate pool. The uncertainty-penalized proxy achieves substantially higher alignment with the true contribution.

Figure 14: Validation negative log predictive density (NLPD) across iterations. Selective full-GP evaluation mitigates posterior drift and maintains better calibration than evaluating all candidates.

Figure 15: Predictive uncertainty decomposition on a 1-D slice of 100-D DTLZ2. The epistemic band expands away from observed points whereas the aleatoric band grows near regions of high local variability.

Surrogate variant	Median HV	Relative change
Heteroscedastic Deep GP (ours)	$\mathbf{0.873}$	—
Homoscedastic sparse GP	0.818	$-6.3\%$
Shallow RBF GP (no depth)	0.791	$-9.4\%$

Table 7: Median hypervolume after 300 evaluations on 100-D DTLZ2 for different surrogate variants.

Figure 16: Median hypervolume achieved as a function of the number of rank categories

K

on 100-D DTLZ2 with

B=300

evaluations. Error bars indicate the standard error across 20 runs.

Method	100-D DTLZ2	200-D DTLZ4
Independent Deep GP	$\mathbf{0.873\pm 0.011}$	$\mathbf{0.836\pm 0.013}$
LMC Deep GP ( $R=M$ )	$0.851\pm 0.015$	$0.818\pm 0.017$

Table 8: Median hypervolume (mean ± standard error) after 300 evaluations for independent Deep GP and LMC Deep GP with

R=M

. Results are aggregated over multiple independent trials.

Figure 17: Predictive correlation matrix among objectives on 100-D DTLZ2, estimated by the LMC Deep GP after 300 evaluations. Color scale denotes Pearson correlation.

Scenario	Transfer	Refine (optional)	Total overhead
Original exhaustive offline grid	–	–	3,000 (100%)
New task with similar characteristics	15 (5%)	9 (3%)	24 (8%)
New task with 500-eval budget	15 (3%)	9 (1.8%)	24 (4.8%)

Table 9: Estimated re-tuning cost under a 300-evaluation budget. Transfer denotes Stage 1; Refine denotes the optional Stage 2 local refinement. Numbers are expressed as absolute evaluations and as percentages of the 300-evaluation budget.

Figure 18: Dropout-based uncertainty index (vertical) versus high-fidelity reference information measure (horizontal) on a small regression problem. Each point is a test input; the diagonal indicates perfect agreement.

Benchmark	Dimension	$\rho$
DTLZ2	100D	0.84
DTLZ2	200D	0.81
ZDT3	100D	0.83
Median	–	0.82

Table 10: Empirical fidelity factor

\rho

across benchmarks.

Problem	GP-EI	RF-EI	GP-HV	CL-EGO	CPS-MOEA	K-RVEA	CSEA	EDN-ARM.	MCEA/D	CLMEA	NeuroPareto
DTLZ1	38.72	41.83	36.91	32.45	34.28	33.91	35.62	32.87	31.00	30.00	28.13
DTLZ2	29.45	31.67	28.93	26.78	27.45	27.13	28.94	26.92	25.50	24.50	23.41
DTLZ3	42.18	45.92	40.67	37.24	38.97	38.45	40.28	37.86	36.00	34.50	32.86
DTLZ4	27.89	30.14	26.95	24.63	25.87	25.42	27.15	24.98	23.50	22.50	21.05
DTLZ5	24.37	26.81	23.64	21.97	22.83	22.46	24.17	22.15	21.50	20.50	19.22
DTLZ6	35.42	38.75	34.18	31.05	32.76	32.28	34.09	31.42	30.50	29.50	27.89

Table 11: IGD comparison on 500-D, 5-objective DTLZ problems (300 evaluations). Lower is better.

Acquisition strategy	HV (mean $\pm$ std)	Relative gain
Static-EHVI (best weights found by grid search)	$0.701\pm 0.013$	—
Learned acquisition network (Module 3)	$0.863\pm 0.011$	$\mathbf{+23.1\%}$

Table 12: Final hypervolume (HV) on DTLZ2-100D after 300 evaluations. The learned acquisition network obtains substantially higher HV than the best static weighted rule.

Figure 19: Hypervolume progression on DTLZ2-100D under the static weighted rule and the learned acquisition network. Shaded bands indicate standard error across 20 runs.

Method	Type	IGD (↓)	HV (↑)	IGD (↓)	HV (↑)	IGD (↓)	HV (↑)
		DTLZ2 (100D)		ZDT3 (100D)		DTLZ2 (200D)
MORBO	Local Regions	0.154 ±0.012	0.832 ±0.018	0.238 ±0.021	0.761 ±0.024	0.243 ±0.019	0.794 ±0.016
EGBO	Evolution-Guided	0.142 ±0.011	0.846 ±0.015	0.225 ±0.018	0.773 ±0.022	0.231 ±0.017	0.808 ±0.014
CDM-PSL	Diffusion Models	0.138 ±0.010	0.851 ±0.014	0.219 ±0.016	0.782 ±0.020	0.226 ±0.015	0.815 ±0.013
NeuroPareto	Rank-Based + Uncertainty	0.121 ±0.008	0.873 ±0.012	0.198 ±0.014	0.801 ±0.018	0.208 ±0.013	0.836 ±0.011

Table 13: Performance comparison of NeuroPareto against state-of-the-art MOBO methods on 100D and 200D test problems with 300 function evaluations. Best results are highlighted in bold.

Method	DTLZ2 (100D)		ZDT3 (100D)		DTLZ2 (200D)
	Total Time	Time/Iter	Total Time	Time/Iter	Total Time	Time/Iter
CPS-MOEA	$1245.2\pm 38.7$	$4.15\pm 0.13$	$1189.6\pm 42.3$	$3.97\pm 0.14$	$2356.8\pm 67.9$	$7.86\pm 0.23$
K-RVEA	$893.4\pm 31.5$	$2.98\pm 0.11$	$856.2\pm 29.8$	$2.85\pm 0.10$	$1678.3\pm 58.4$	$5.59\pm 0.19$
CSEA	$1567.8\pm 45.2$	$5.23\pm 0.15$	$1498.3\pm 43.6$	$4.99\pm 0.15$	$2894.1\pm 82.7$	$9.65\pm 0.28$
EDN-ARMOEA	$2045.6\pm 62.8$	$6.82\pm 0.21$	$1967.4\pm 59.3$	$6.56\pm 0.20$	$3789.2\pm 114.5$	$12.63\pm 0.38$
MCEA/D	$1789.3\pm 53.4$	$5.96\pm 0.18$	$1712.8\pm 51.7$	$5.71\pm 0.17$	$3324.6\pm 98.3$	$11.08\pm 0.33$

Table 14: Wall-clock time comparison (seconds) across algorithms and problem scales. Results are averaged over 20 runs; “Speedup” is defined relative to the fastest non-NeuroPareto baseline in the same column.

Component	DTLZ2 (100D)	ZDT3 (100D)	DTLZ2 (200D)
Classifier inference	$18.2\%\pm 1.3\%$	$19.5\%\pm 1.4\%$	$15.8\%\pm 1.1\%$
Deep GP training	$35.6\%\pm 2.1\%$	$33.8\%\pm 2.0\%$	$38.2\%\pm 2.3\%$
Acquisition network	$12.4\%\pm 0.9\%$	$13.1\%\pm 1.0\%$	$11.7\%\pm 0.8\%$
Candidate evaluation (surrogate preds)	$28.3\%\pm 1.8\%$	$27.6\%\pm 1.7\%$	$29.8\%\pm 1.9\%$
Overhead	$5.5\%\pm 0.4\%$	$6.0\%\pm 0.5\%$	$4.5\%\pm 0.3\%$

Table 15: Component-wise time distribution of NeuroPareto (percentage of total time). Values are mean

\pm

standard error across runs.

NeuroPareto: Calibrated Acquisition for Costly Many-Goal Search in Vast Parameter Spaces

Abstract

Figures & tables

Multi-Objective Optimization

Expected Hypervolume Maximization for Multiobjective Optimization under Uncertainties

NeurGO: Learning to Generate Elite Candidates for Meta-Black-Box Expensive Optimization

ParetoPilot: Zero-Surrogate Offline Multi-Objective Optimization via Infer-Perturb-Guide Diffusion

Abstract

Multi-Objective Optimization

Explore similar work

Expected Hypervolume Maximization for Multiobjective Optimization under Uncertainties

NeurGO: Learning to Generate Elite Candidates for Meta-Black-Box Expensive Optimization

ParetoPilot: Zero-Surrogate Offline Multi-Objective Optimization via Infer-Perturb-Guide Diffusion