Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models

Yuan Lu; Esha Singh; Yi-An Ma; Yusu Wang

cs.LGSep 30, 2026

Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models

Authors: Yuan Lu, Esha Singh, Yi-An Ma, Yusu Wang

Organizations: Halıcıoğlu Data Science Institute, UC San Diego · Department of Computer Science & Engineering, UC San Diego

Abstract

Offline multi-objective optimization (MOO) seeks solutions with better objective trade-offs using only a fixed dataset, without querying the objectives. Diffusion models trained on such data have emerged as a promising approach, but their samples are not inherently better than the data and must be steered toward the Pareto front. Existing methods guide or condition every sampling step. We instead act on the initial noise and leave the sampling process unchanged. Across Off-MOO-Bench, we observe that the objectives, as functions of the noise, are sensitive to only a few directions. We estimate these directions once per task via a Recursive Feature Machine using function values alone, and a small cache serves every trade-off, so each candidate costs one noise displacement and one ODE solve. We prove that this displacement increases the learned scalarized objective in expectation, and that sweeping trade-offs recovers the flow's attainable front up to proxy and steering errors. With additional guidance, for which we introduce novel data-adaptive and Pareto-aware operators, our method attains the best average hypervolume rank among generative methods on 47 tasks, at comparable or lower sampling cost. Steering alone outranks the best prior generative method at a fraction of its sampling cost.

Figures & tables

Figure 1 : The noise-space objective is low rank. (a) Spectrum of

{\bm{M}}_{\bm{w}}

on IN-1K/MOP8. A handful of directions carry nearly all the sensitivity. (b) Displacing along them raises the true score. A trailing or random direction of equal norm does not.

Figure 2 : Noise-space RFM steering. (Left, once per task.) Random noise is decoded by the flow

\Phi_{\theta}

and scored by the proxies

\hat{{\bm{f}}}

, and the RFM recovers from these pairs a metric whose few leading directions are the ones that move the objectives. (Right, per trade off

{\bm{w}}

) . A noise vector is displaced along the cached directions for

{\bm{w}}

and decoded by the unmodified flow in one ODE solve, with optional late guidance (Section 4 ).

Methods	DTLZ (7)	ZDT (5)	C-10/MOP (9)	IN-1K/MOP (9)	MORL (2)	RE (15)	All Tasks (47)
$\mathcal{D}$ (best)	12.43 $\pm$ 0.32	13.00 $\pm$ 0.45	10.58 $\pm$ 0.51	6.38 $\pm$ 0.10	4.10 $\pm$ 0.89	16.80 $\pm$ 0.27	12.02 $\pm$ 0.13
E2E	3.34 $\pm$ 0.61	5.16 $\pm$ 0.62	12.41 $\pm$ 0.55	11.38 $\pm$ 1.06	17.40 $\pm$ 1.95	8.37 $\pm$ 1.20	9.01 $\pm$ 0.59
E2E + GradNorm	9.71 $\pm$ 0.76	13.32 $\pm$ 0.59	13.80 $\pm$ 1.24	17.60 $\pm$ 0.41	7.70 $\pm$ 3.78	13.04 $\pm$ 0.67	13.37 $\pm$ 0.46
E2E + PcGrad	4.97 $\pm$ 1.19	7.56 $\pm$ 1.82	11.94 $\pm$ 0.36	12.42 $\pm$ 0.75	14.90 $\pm$ 3.45	6.05 $\pm$ 0.22	8.78 $\pm$ 0.33
MH	6.00 $\pm$ 0.96	4.88 $\pm$ 1.23	12.26 $\pm$ 1.68	13.58 $\pm$ 0.38	11.20 $\pm$ 3.56	6.59 $\pm$ 0.30	8.94 $\pm$ 0.43
MH + GradNorm	9.91 $\pm$ 0.45	11.36 $\pm$ 0.55	15.56 $\pm$ 1.42	16.47 $\pm$ 0.56	8.10 $\pm$ 6.18	11.37 $\pm$ 0.69	12.79 $\pm$ 0.56

Table 1 : Average rank (

\downarrow

) per task family (number of tasks in parentheses), based on 100th-percentile hypervolume (Section 5.1 ); mean

\pm

std over 5 seeds. Best per column in bold, second best underlined. The last two rows count tasks on which RFM-Cone wins, ties, or loses (W/T/L) against PCD and against ParetoFlow ⋆ , the better of ParetoFlow and ParetoFlow

+

on each task, with ties within

5\times 10^{-4}

in hypervolume.

Method	DTLZ2	ZDT4	C-10/MOP1	MO-Hopper	RE21	RE61	Mean
RFM precomputation (one-off, shared by all RFM variants)	7.6 $\pm$ 0.1	7.4 $\pm$ 0.0	7.5 $\pm$ 0.1	22.7 $\pm$ 0.1	7.4 $\pm$ 0.1	7.5 $\pm$ 0.0	10.0
ParetoFlow	7.4 $\pm$ 0.1	8.9 $\pm$ 2.0	7.9 $\pm$ 0.1	10.8 $\pm$ 0.2	8.6 $\pm$ 0.3	11.8 $\pm$ 0.2	9.2
ParetoFlow+	12.4 $\pm$ 0.2	12.2 $\pm$ 0.2	11.1 $\pm$ 0.2	16.2 $\pm$ 0.5	13.5 $\pm$ 0.1	19.3 $\pm$ 0.1	14.1
PCD	6.8 $\pm$ 1.5	7.6 $\pm$ 2.3	22.2 $\pm$ 5.3	149.4 $\pm$ 30.0	22.6 $\pm$ 4.9	25.3 $\pm$ 4.2	39.0
RFM-AN (ours)	1.3 $\pm$ 0.0	1.3 $\pm$ 0.1	2.0 $\pm$ 0.0	6.3 $\pm$ 0.2	1.2 $\pm$ 0.4	1.2 $\pm$ 0.0	2.2
RFM-APG (ours)	3.3 $\pm$ 0.2	4.2 $\pm$ 0.4	8.2 $\pm$ 0.2	31.0 $\pm$ 1.3	2.8 $\pm$ 0.1	3.7 $\pm$ 0.1	8.9

Table 2 : Sampling wall-clock time in seconds (mean

\pm

std over 5 seeds) to produce the final 256 candidates, same hardware for all methods (see § A.4 ). The first row is the one-off per-task RFM precomputation, cached and shared within seeds and variants and excluded from the sampling times. Best per column in bold.

Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

Task	$d$	$m$	$\lvert{\mathcal{D}}\rvert$	Space	Reference point ${\bm{r}}$
DTLZ
DTLZ1	7	3	60,000	Cont.	$(-2.2,\ -2.2,\ -2.2)$
DTLZ2	10	3	60,000	Cont.	$(-2.07826,\ -2.44443,\ -2.45812)$
DTLZ3	10	3	60,000	Cont.	$(-2.22594,\ -2.07323,\ -2.1453)$
DTLZ4	10	3	60,000	Cont.	$(-2.29461,\ -2.91967,\ -2.63431)$
DTLZ5	10	3	60,000	Cont.	$(-2.74136,\ -2.99806,\ -2.6272)$

Appendix

Table 3 : Task metadata and hypervolume reference points for all

47

tasks. Reference points are in normalized maximization coordinates (negated from the evaluator’s minimization convention).

d

is the number of design variables (categorical variables for NAS, whose logits have dimension

D

\lvert{\mathcal{D}}\rvert

the offline dataset size, and Cont. / Cat. continuous/categorical.

Flow and Proxy Training			Noise-Space Steering and Sampling
Parameter	Flow ${\bm{v}}_{\theta}$	Proxy $\hat{f}_{a}$	Parameter	Value(s)
Architecture	Residual MLP	MLP	Kernel	Laplace (Mahalanobis)
Depth	4 residual blocks	2 hidden layers	Bandwidth $\ell$	20
Width	512	2048	Ridge $\lambda$	$10^{-3}$
Activation	SiLU	LeakyReLU	RFM samples $n$	10,000 (5,000)
Normalization	LayerNorm	—	AGOP iterations $T$	7 (4)

Appendix

Table 4 : Hyperparameters for flow and proxy training (left) and noise-space steering and sampling (right), shared by all tasks. MORL values are given in parentheses where they differ.

Component	RFM / ParetoFlow+	PCD
Generative model	Rectified flow	EDM diffusion
Backbone	Residual MLP	Residual MLP
Hidden width	512	512
Residual blocks	4	4
Normalization	LayerNorm	LayerNorm
Input projection layers	1	2

Appendix

Table 5: Architecture comparison with the released PCD implementation. Main-path layer counts exclude the separate time/noise embedding MLP.

Figure 3 : Effective rank Eq. ( 13 ) of $\widehat{\bm{M}}_{\bm{w}}$ at ${\bm{w}}=\mathbf{1}/m$ against noise dimension $D$ for all 47 tasks, log-log axes. The dashed line is a least squares fit, $r_{\mathrm{eff}}\propto D^{0.42}$ , and the dotted line is $r_{\mathrm{eff}}=D$ .

Task	$D$	$r_{\max}$	$r^{\ast}$	$\gamma^{\ast}$	seed s.d. at anchor
RE36	4	4	3	20	0.01
ZDT4	10	10	1	2.5	$<0.01$
C-10/MOP1	31	31	1	10	0.02–0.03
IN-1K/MOP8	174	174	4	20	0.01
MO-Hopper	10,184	300	100	10	0.03–0.04

Appendix

Table 6 : Per-task anchor settings for the sensitivity studies.

D

is the noise dimension and

r_{\max}

the number of available directions.

\gamma^{\ast}

is the largest steering strength within one seed standard deviation of the task’s best HV in preliminary sweeps, and

r^{\ast}

the rank with the highest mean HV under soft weighting, with ties resolved toward the smaller rank. On MO-Hopper both are the main-result defaults. The last column is the seed standard deviation of normalized HV at these settings.

Figure 4 : Steering strength sweep. Normalized HV against the steering strength

\gamma

(top) and against the per-coordinate displacement

\|\Delta{\bm{z}}\|_{2}/\sqrt{D}

(bottom), mean

\pm

s.d. over five seeds. The dotted line is the offline data. On MO-Hopper, solid and dashed lines are the RFM fits with

r=100

n=10{,}000

and

r=300

n=40{,}000

Figure 5 : Retained rank sweep for RFM-AN (top) and RFM-Cone (bottom) , mean

\pm

s.d. over five seeds. The vertical line marks

r^{\ast}

, the anchor rank of Table 6 , and the horizontal dotted line the offline data. Indicator weights lose HV as

r

grows on every task with room to add directions, while soft and geometric weights stay flat.

Figure 6 : Step length versus shape. Paired difference in normalized HV, soft weighting minus each hard mode, for RFM-AN, mean

\pm

s.d. over five seeds. Positive values favor soft weighting, so a higher curve means that hard mode loses more. Solid lines use the original step, dashed lines rescale the hard step to the length of the soft step.

Figure 7 : Soft exponent. Top, normalized HV against

\alpha

r=r^{\ast}

(solid) and

r=r_{\max}

(dashed), mean

\pm

s.d. over five seeds. Bottom, the number of directions the weighting uses,

r_{\mathrm{eff}}

of Eq. ( 13 ), at

r=r_{\max}

. It falls from nearly the full basis at

\alpha=0.1

to a single direction at

\alpha=3

Figure 8 : Estimating the noise-space metric. Normalized HV against the sample budget

n

, the AGOP iterations

T

, the bandwidth

\ell

, and the ridge

\lambda_{\mathrm{stat}}

(rows), mean

\pm

s.d. over five seeds. The dotted line is the offline data. On MO-Hopper, solid and dashed lines are the fits with

r=100

and

r=300

Figure 9 : Steering ascends under every scalarization. True scalarized objective against step size on ZDT2 (mean

\pm

s.d. over five seeds, same

{\bm{w}}

and base noise). In every panel the leading arms climb, the controls stay flat, and the reversed arm descends, and all four reach the same HV. Only the soft arm under the non-smooth Tch and aug-Tch becomes unreliable at large steps.

Figure 10 : One precomputation serves every scalarization at a fraction of the cost of refitting. (a) HV of cache-based steering minus that of refit-based steering on ZDT2 (mean and 95% CI over five runs, grey band

\pm 0.02

). The cache matches or beats the refit at the balanced weight and trails it by a few percent elsewhere, similarly for all four scalarizations. (b) Cost per weight on a log scale. The precomputation runs once, a refit takes

0.56

s per weight, and cache assembly takes about a millisecond. Neither path runs the flow to build

\widehat{\bm{M}}_{\bm{w}}

. (c) Difference between the cache and refit matrices. With features held fixed, the refit reproduces the weighted-sum assembly to about

10^{-4}

, so Eq. ( 8 ) is exact. A free refit differs by

6\%

24\%

for every scalarization, because it learns new features.

Figure 11 : Every scalarization reaches the same front quality. Hypervolume relative to the best offline designs (top) and IGD to the analytic front (bottom) at two steering budgets, mean and 95% CI over five seeds. The four scalarizations agree within the

2\%

tolerance on every task. More steering helps on the ZDT tasks, whose optima lie on the design boundary, and hurts IGD on DTLZ2, whose optima lie in its interior.

Figure 12 : Steering beyond the offline front on ZDT1. Best offline points, RFM-AN samples under weighted-sum scalarization at steering budgets

\|\Delta{\bm{z}}\|_{2}/\sqrt{D}=0.3

and

1

, and the true Pareto front. Both objectives are minimized. The larger step moves the generated frontier further beyond the offline points and toward the true front.

Figure 13 : Guidance adds to steering and grows with its strength. Change in normalized HV over steering alone against the guidance strength, relative to its default (dotted line), on the five tasks where steering leaves headroom, mean and 95% CI over five seeds. The remaining three tasks are in Figure 14 .

Figure 14 : Guidance strength on all eight tasks. As Figure 13 , including ZDT4, IN-1K/MOP8, and MO-Swimmer.

Figure 15 : Guidance strength at the

50

th percentile. As Figure 14 , for the HV of the better half of the returned set.

Figure 16 : Guidance improves most runs. Fraction of task and seed pairs in which guidance raises HV over steering alone, against the guidance strength. The dashed line is chance.

Methods	MO-Hopper	MO-Swimmer
$\mathcal{D}$ (best)	6.33	3.70
E2E	5.56 $\pm$ 0.46	2.99 $\pm$ 0.10
E2E + GradNorm	6.17 $\pm$ 0.15	3.55 $\pm$ 0.30
E2E + PcGrad	5.65 $\pm$ 0.53	3.31 $\pm$ 0.17
MH	6.03 $\pm$ 0.22	3.43 $\pm$ 0.14
MH + GradNorm	6.07 $\pm$ 0.57	3.61 $\pm$ 0.15

Appendix

Table 7 : 100th-percentile hypervolume results for MORL. For each task, methods within one standard deviation (or

5\times 10^{-4}